back to indexChris Lattner: The Future of Computing and Programming Languages | Lex Fridman Podcast #131
link |
The following is a conversation with Chris Latner,
link |
his second time on the podcast.
link |
He's one of the most brilliant engineers
link |
in modern computing, having created
link |
LLVM compiler infrastructure project,
link |
the Clang compiler, the Swift programming language,
link |
a lot of key contributions to TensorFlow and TPUs
link |
as part of Google.
link |
He served as vice president of autopilot software at Tesla,
link |
was a software innovator and leader at Apple,
link |
and now is at SciFive as senior vice president
link |
of platform engineering, looking to revolutionize
link |
chip design to make it faster, better, and cheaper.
link |
Quick mention of each sponsor, followed by some thoughts
link |
related to the episode.
link |
First sponsor is Blinkist, an app that summarizes
link |
key ideas from thousands of books.
link |
I use it almost every day to learn new things
link |
or to pick which books I want to read or listen to next.
link |
Second is Neuro, the maker of functional sugar free gum
link |
and mints that I use to supercharge my mind
link |
with caffeine, altheanine, and B vitamins.
link |
Third is Masterclass, online courses from the best people
link |
in the world on each of the topics covered,
link |
from rockets, to game design, to poker,
link |
to writing, and to guitar.
link |
And finally, Cash App, the app I use to send money
link |
to friends for food, drinks, and unfortunately, lost bets.
link |
Please check out the sponsors in the description
link |
to get a discount and to support this podcast.
link |
As a side note, let me say that Chris has been
link |
an inspiration to me on a human level
link |
because he is so damn good as an engineer
link |
and leader of engineers, and yet he's able to stay humble,
link |
especially humble enough to hear the voices
link |
of disagreement and to learn from them.
link |
He was supportive of me and this podcast
link |
from the early days, and for that, I'm forever grateful.
link |
To be honest, most of my life, no one really believed
link |
that I would amount to much.
link |
So when another human being looks at me,
link |
it makes me feel like I might be someone special,
link |
it can be truly inspiring.
link |
That's a lesson for educators.
link |
The weird kid in the corner with a dream
link |
is someone who might need your love and support
link |
in order for that dream to flourish.
link |
If you enjoy this thing, subscribe on YouTube,
link |
review it with five stars on Apple Podcast,
link |
follow on Spotify, support on Patreon,
link |
or connect with me on Twitter at Lex Friedman.
link |
And now, here's my conversation with Chris Latner.
link |
What are the strongest qualities of Steve Jobs,
link |
Elon Musk, and the great and powerful Jeff Dean
link |
since you've gotten the chance to work with each?
link |
You're starting with an easy question there.
link |
These are three very different people.
link |
I guess you could do maybe a pairwise comparison
link |
between them instead of a group comparison.
link |
So if you look at Steve Jobs and Elon,
link |
I worked a lot more with Elon than I did with Steve.
link |
They have a lot of commonality.
link |
They're both visionary in their own way.
link |
They're both very demanding in their own way.
link |
My sense is Steve is much more human factor focused
link |
where Elon is more technology focused.
link |
What does human factor mean?
link |
Steve's trying to build things that feel good,
link |
that people love, that affect people's lives, how they live.
link |
He's looking into the future a little bit
link |
in terms of what people want.
link |
Where I think that Elon focuses more on
link |
learning how exponentials work and predicting
link |
the development of those.
link |
Steve worked with a lot of engineers.
link |
That was one of the things that are reading the biography.
link |
How can a designer essentially talk to engineers
link |
and get their respect?
link |
I think, so I did not work very closely with Steve.
link |
I'm not an expert at all.
link |
My sense is that he pushed people really hard,
link |
but then when he got an explanation that made sense to him,
link |
then he would let go.
link |
And he did actually have a lot of respect for engineering,
link |
but he also knew when to push.
link |
And when you can read people well,
link |
you can know when they're holding back
link |
and when you can get a little bit more out of them.
link |
And I think he was very good at that.
link |
I mean, if you compare the other folks,
link |
so Jeff Dean, right?
link |
Jeff Dean's an amazing guy.
link |
He's super smart, as are the other guys.
link |
Jeff is a really, really, really nice guy, well meaning.
link |
He's a classic Googler.
link |
He wants people to be happy.
link |
He combines it with brilliance
link |
so he can pull people together in a really great way.
link |
He's definitely not a CEO type.
link |
I don't think he would even want to be that.
link |
Do you know if he still programs?
link |
Oh yeah, he definitely programs.
link |
Jeff is an amazing engineer today, right?
link |
And that has never changed.
link |
So it's really hard to compare Jeff to either of those two.
link |
I think that Jeff leads through technology
link |
and building it himself and then pulling people in
link |
and inspiring them.
link |
And so I think that that's one of the amazing things
link |
But each of these people, with their pros and cons,
link |
all are really inspirational
link |
and have achieved amazing things.
link |
So I've been very fortunate to get to work with these guys.
link |
For yourself, you've led large teams,
link |
you've done so many incredible,
link |
difficult technical challenges.
link |
Is there something you've picked up from them
link |
about how to lead?
link |
Yeah, so I mean, I think leadership is really hard.
link |
It really depends on what you're looking for there.
link |
I think you really need to know what you're talking about.
link |
So being grounded on the product, on the technology,
link |
on the business, on the mission is really important.
link |
Understanding what people are looking for,
link |
why they're there.
link |
One of the most amazing things about Tesla
link |
is the unifying vision, right?
link |
People are there because they believe in clean energy
link |
and electrification, all these kinds of things.
link |
The other is to understand what really motivates people,
link |
how to get the best people,
link |
how to build a plan that actually can be executed, right?
link |
There's so many different aspects of leadership
link |
and it really depends on the time, the place, the problems.
link |
There's a lot of issues that don't need to be solved.
link |
And so if you focus on the right things and prioritize well,
link |
that can really help move things.
link |
Two interesting things you mentioned.
link |
One is you really have to know what you're talking about.
link |
How you've worked on your business,
link |
you've worked on a lot of very challenging technical things.
link |
So I kind of assume you were born technically savvy,
link |
but assuming that's not the case,
link |
how did you develop technical expertise?
link |
Like even at Google you worked on,
link |
I don't know how many projects,
link |
but really challenging, very varied.
link |
Compilers, TPUs, hardware, cloud stuff,
link |
bunch of different things.
link |
The thing that I've become comfortable
link |
as I've more comfortable with as I've gained experience
link |
is being okay with not knowing.
link |
And so a major part of leadership is actually,
link |
it's not about having the right answer,
link |
it's about getting the right answer.
link |
And so if you're working in a team of amazing people, right?
link |
And many of these places, many of these companies
link |
all have amazing people.
link |
It's the question of how do you get people together?
link |
How do you build trust?
link |
How do you get people to open up?
link |
How do you get people to be vulnerable sometimes
link |
with an idea that maybe isn't good enough,
link |
but it's the start of something beautiful?
link |
How do you provide an environment
link |
where you're not just like top down,
link |
thou shalt do the thing that I tell you to do, right?
link |
But you're encouraging people to be part of the solution
link |
and providing a safe space
link |
where if you're not doing the right thing,
link |
they're willing to tell you about it, right?
link |
So you're asking dumb questions?
link |
Yeah, dumb questions are my specialty, yeah.
link |
Well, so I've been in the hardware realm recently
link |
and I don't know much at all about how chips are designed.
link |
I know a lot about using them.
link |
I know some of the principles
link |
and the art's technical level of this,
link |
but it turns out that if you ask a lot of dumb questions,
link |
you get smarter really, really quick.
link |
And when you're surrounded by people that want to teach
link |
and learn themselves, it can be a beautiful thing.
link |
So let's talk about programming languages, if it's okay.
link |
At the highest absurd philosophical level,
link |
Don't get romantic on me, Lex.
link |
I will forever get romantic and torture you, I apologize.
link |
Why do programming languages even matter?
link |
Okay, well, thank you very much.
link |
You're saying why should you care
link |
about any one programming language
link |
or why do we care about programming computers or?
link |
No, why do we care about programming language design,
link |
creating effective programming languages,
link |
choosing one programming languages
link |
such as another programming language,
link |
why we keep struggling and improving
link |
through the evolution of these programming languages.
link |
Okay, so I mean, I think you have to come back
link |
to what are we trying to do here, right?
link |
So we have these beasts called computers
link |
that are very good at specific kinds of things
link |
and we think it's useful to have them do it for us, right?
link |
Now you have this question of how best to express that
link |
because you have a human brain still
link |
that has an idea in its head
link |
and you want to achieve something, right?
link |
So, well, there's lots of ways of doing this.
link |
You can go directly to the machine
link |
and speak assembly language
link |
and then you can express directly
link |
what the computer understands, that's fine.
link |
You can then have higher and higher and higher levels
link |
of abstraction up until machine learning
link |
and you're designing a neural net to do the work for you.
link |
The question is where along this way do you want to stop
link |
and what benefits do you get out of doing so?
link |
And so programming languages in general,
link |
you have C, you have Fortran, Java and Ada, Pascal, Swift,
link |
you have lots of different things.
link |
They'll have different trade offs
link |
and they're tackling different parts of the problems.
link |
Now, one of the things that most programming languages do
link |
is they're trying to make it
link |
so that you have pretty basic things
link |
like portability across different hardware.
link |
So you've got, I'm gonna run on an Intel PC,
link |
I'm gonna run on a RISC 5 PC,
link |
I'm gonna run on a ARM phone or something like that, fine.
link |
I wanna write one program and have it portable.
link |
And this is something that assembly doesn't do.
link |
Now, when you start looking
link |
at the space of programming languages,
link |
this is where I think it's fun
link |
because programming languages all have trade offs
link |
and most people will walk up to them
link |
and they look at the surface level of syntax and say,
link |
oh, I like curly braces or I like tabs
link |
or I like semi colons or not or whatever, right?
link |
Subjective, fairly subjective, very shallow things.
link |
But programming languages when done right
link |
can actually be very powerful.
link |
And the benefit they bring is expression.
link |
Okay, and if you look at programming languages,
link |
there's really kind of two different levels to them.
link |
One is the down in the dirt, nuts and bolts
link |
of how do you get the computer to be efficient,
link |
stuff like that, how they work,
link |
type systems, compiler stuff, things like that.
link |
The other is the UI.
link |
And the UI for programming language
link |
is really a design problem
link |
and a lot of people don't think about it that way.
link |
And the UI, you mean all that stuff with the braces and?
link |
Yeah, all that stuff's the UI and what it is
link |
and UI means user interface.
link |
And so what's really going on is
link |
it's the interface between the guts and the human.
link |
And humans are hard, right?
link |
Humans have feelings, they have things they like,
link |
they have things they don't like.
link |
And a lot of people treat programming languages
link |
as though humans are just kind of abstract creatures
link |
that cannot be predicted.
link |
But it turns out that actually there is better and worse.
link |
Like people can tell when a programming language is good
link |
or when it was an accident, right?
link |
And one of the things with Swift in particular
link |
is that a tremendous amount of time
link |
by a tremendous number of people
link |
have been put into really polishing and making it feel good.
link |
But it also has really good nuts and bolts underneath it.
link |
You said that Swift makes a lot of people feel good.
link |
How do you get to that point?
link |
So how do you predict that tens of thousands,
link |
hundreds of thousands of people are going to enjoy
link |
using this user experience of this programming language?
link |
Well, you can look at it in terms of better and worse, right?
link |
So if you have to write lots of boilerplate
link |
or something like that, you will feel unproductive.
link |
And so that's a bad thing.
link |
You can look at it in terms of safety.
link |
If like C for example,
link |
is what's called a memory unsafe language.
link |
And so you get dangling pointers
link |
and you get all these kinds of bugs
link |
that then you have spent tons of time debugging
link |
and it's a real pain in the butt and you feel unproductive.
link |
And so by subtracting these things from the experience,
link |
you get happier people.
link |
But again, keep interrupting.
link |
I'm sorry, but so hard to deal with.
link |
If you look at the people that are most productive
link |
on Stack Overflow, they have a set of priorities
link |
that may not always correlate perfectly
link |
with the experience of the majority of users.
link |
If you look at the most upvoted,
link |
quote unquote, correct answer on Stack Overflow,
link |
it usually really sort of prioritizes
link |
like safe code, proper code, stable code,
link |
you know, that kind of stuff.
link |
As opposed to like,
link |
if I wanna use go to statements in my basic, right?
link |
I wanna use go to statements.
link |
Like what if 99% of people wanna use go to statements?
link |
So you use completely improper, you know, unsafe syntax.
link |
I don't think that people actually,
link |
like if you boil it down and you get below
link |
the surface level, people don't actually care
link |
about go tos or if statements or things like this.
link |
They care about achieving a goal, right?
link |
So the real question is I wanna set up a web server
link |
and I wanna do a thing, whatever.
link |
Like how quickly can I achieve that, right?
link |
And so from a programming language perspective,
link |
there's really two things that matter there.
link |
One is what libraries exist
link |
and then how quickly can you put it together
link |
and what are the tools around that look like, right?
link |
And when you wanna build a library that's missing,
link |
Okay, now this is where you see huge divergence
link |
in the force between worlds, okay?
link |
And so you look at Python, for example.
link |
Python is really good at assembling things,
link |
but it's not so great at building all the libraries.
link |
And so what you get because of performance reasons,
link |
other things like this,
link |
is you get Python layered on top of C, for example,
link |
and that means that doing certain kinds of things
link |
well, it doesn't really make sense to do in Python.
link |
Instead you do it in C and then you wrap it
link |
and then you have, you're living in two worlds
link |
and two worlds never is really great
link |
because tooling and the debugger doesn't work right
link |
and like all these kinds of things.
link |
Can you clarify a little bit what you mean
link |
by Python is not good at building libraries,
link |
meaning it doesn't make it conducive.
link |
Certain kinds of libraries.
link |
No, but just the actual meaning of the sentence,
link |
meaning like it's not conducive to developers
link |
to come in and add libraries
link |
or is it the duality of the,
link |
it's a dance between Python and C and...
link |
Well, so Python's amazing.
link |
Python's a great language.
link |
I did not mean to say that Python is bad for libraries.
link |
What I meant to say is there are libraries
link |
that Python's really good at that you can write in Python,
link |
but there are other things,
link |
like if you wanna build a machine learning framework,
link |
you're not gonna build a machine learning framework
link |
in Python because of performance, for example,
link |
or you want GPU acceleration or things like this.
link |
Instead, what you do is you write a bunch of C
link |
or C++ code or something like that,
link |
and then you talk to it from Python, right?
link |
And so this is because of decisions
link |
that were made in the Python design
link |
and those decisions have other counterbalancing forces.
link |
But the trick when you start looking at this
link |
from a programming language perspective,
link |
you start to say, okay, cool.
link |
How do I build this catalog of libraries
link |
that are really powerful?
link |
And how do I make it so that then they can be assembled
link |
into ways that feel good
link |
and they generally work the first time?
link |
Because when you're talking about building a thing,
link |
you have to include the debugging, the fixing,
link |
the turnaround cycle, the development cycle,
link |
all that kind of stuff
link |
into the process of building the thing.
link |
It's not just about pounding out the code.
link |
And so this is where things like catching bugs
link |
at compile time is valuable, for example.
link |
But if you dive into the details in this,
link |
Swift, for example, has certain things like value semantics,
link |
which is this fancy way of saying
link |
that when you treat a variable like a value,
link |
it acts like a mathematical object would.
link |
Okay, so you have used PyTorch a little bit.
link |
In PyTorch, you have tensors.
link |
Tensors are n dimensional grid of numbers, very simple.
link |
You can do plus and other operators on them.
link |
It's all totally fine.
link |
But why do you need to clone a tensor sometimes?
link |
Have you ever run into that?
link |
Okay, and so why is that?
link |
Why do you need to clone a tensor?
link |
It's the usual object thing that's in Python.
link |
So in Python, and just like with Java
link |
and many other languages, this isn't unique to Python.
link |
In Python, it has a thing called reference semantics,
link |
which is the nerdy way of explaining this.
link |
And what that means is you actually have a pointer
link |
do a thing instead of the thing, okay?
link |
Now, this is due to a bunch of implementation details
link |
that you don't want to go into.
link |
But in Swift, you have this thing called value semantics.
link |
And so when you have a tensor in Swift, it is a value.
link |
If you copy it, it looks like you have a unique copy.
link |
And if you go change one of those copies,
link |
then it doesn't update the other one
link |
because you just made a copy of this thing, right?
link |
So that's like highly error prone
link |
in at least computer science, math centric disciplines
link |
about Python, that like the thing you would expect
link |
to behave like math.
link |
Like math, it doesn't behave like math.
link |
And in fact, quietly it doesn't behave like math
link |
and then can ruin the entirety of your math thing.
link |
Well, and then it puts you in debugging land again.
link |
Right now, you just want to get something done
link |
and you're like, wait a second, where do I need to put clone?
link |
And what level of the stack, which is very complicated,
link |
which I thought I was reusing somebody's library
link |
and now I need to understand it
link |
to know where to clone a thing, right?
link |
And hard to debug, by the way.
link |
And so this is where programming languages really matter.
link |
Right, and so in Swift having value semantics
link |
so that both you get the benefit of math,
link |
working like math, right?
link |
But also the efficiency that comes with certain advantages
link |
there, certain implementation details there
link |
really benefit you as a programmer, right?
link |
Can you clarify the value semantics?
link |
Like how do you know that a thing should be treated
link |
Yeah, so Swift has a pretty strong culture
link |
and good language support for defining values.
link |
And so if you have an array,
link |
so tensors are one example that the machine learning folks
link |
Just think about arrays, same thing,
link |
where you have an array, you create an array,
link |
you put two or three or four things into it,
link |
and then you pass it off to another function.
link |
What happens if that function adds some more things to it?
link |
Well, you'll see it on the side that you pass it in, right?
link |
This is called reference semantics.
link |
Now, what if you pass an array off to a function,
link |
it scrolls it away in some dictionary
link |
or some other data structure somewhere, right?
link |
Well, it thought that you just handed it that array,
link |
then you return back and that reference to that array
link |
still exists in the caller,
link |
and they go and put more stuff in it, right?
link |
The person you handed it off to
link |
may have thought they had the only reference to that,
link |
and so they didn't know that this was gonna change
link |
underneath the covers.
link |
And so this is where you end up having to do clone.
link |
So like I was passed a thing,
link |
I'm not sure if I have the only version of it,
link |
so now I have to clone it.
link |
So what value semantics does is it allows you to say,
link |
hey, I have a, so in Swift, it defaults to value semantics.
link |
Oh, so it defaults to value semantics,
link |
and then because most things
link |
should end up being like values,
link |
then it makes sense for that to be the default.
link |
And one of the important things about that
link |
is that arrays and dictionaries
link |
and all these other collections
link |
that are aggregations of other things
link |
also have value semantics.
link |
And so when you pass this around
link |
to different parts of your program,
link |
you don't have to do these defensive copies.
link |
And so this is great for two sides, right?
link |
It's great because you define away the bug,
link |
which is a big deal for productivity,
link |
the number one thing most people care about,
link |
but it's also good for performance
link |
because when you're doing a clone,
link |
so you pass the array down to the thing,
link |
it's like, I don't know if anybody else has it,
link |
I have to clone it.
link |
Well, you just did a copy of a bunch of data.
link |
And then it could be that the thing that called you
link |
is not keeping track of the old thing.
link |
So you just made a copy of it,
link |
and you may not have had to.
link |
And so the way the value semantics work in Swift
link |
is it uses this thing called copy on write,
link |
which means that you get the benefit of safety
link |
And it has another special trick
link |
because if you think certain languages like Java,
link |
for example, they have immutable strings.
link |
And so what they're trying to do
link |
is they provide value semantics
link |
by having pure immutability.
link |
Functional languages have pure immutability
link |
in lots of different places,
link |
and this provides a much safer model
link |
and it provides value semantics.
link |
The problem with this is if you have immutability,
link |
everything is expensive.
link |
Everything requires a copy.
link |
For example, in Java, if you have a string X
link |
and a string Y, you append them together,
link |
we have to allocate a new string to hold X, Y.
link |
If they're immutable.
link |
Well, strings in Java are immutable.
link |
And if there's optimizations for short ones,
link |
it's complicated, but generally think about them
link |
as a separate allocation.
link |
And so when you append them together,
link |
you have to go allocate a third thing
link |
because somebody might have a pointer
link |
to either of the other ones, right?
link |
And you can't go change them.
link |
So you have to go allocate a third thing.
link |
Because of the beauty of how the Swift value semantics
link |
system works out, if you have a string in Swift
link |
and you say, hey, put in X, right?
link |
And they say, append on Y, Z, W,
link |
it knows that there's only one reference to that.
link |
And so it can do an in place update.
link |
And so you're not allocating tons of stuff on the side.
link |
You don't have all those problems.
link |
When you pass it off,
link |
you can know you have the only reference.
link |
If you pass it off to multiple different people,
link |
but nobody changes it, they can all share the same thing.
link |
So you get a lot of the benefit of purely immutable design.
link |
And so you get a really nice sweet spot
link |
that I haven't seen in other languages.
link |
Yeah, that's interesting.
link |
I thought there was going to be a philosophical narrative
link |
here that you're gonna have to pay a cost for it.
link |
Cause it sounds like, I think value semantics
link |
is beneficial for easing of debugging
link |
or minimizing the risk of errors,
link |
like bringing the errors closer to the source,
link |
bringing the symptom of the error closer
link |
to the source of the error, however you say that.
link |
But you're saying there's not a performance cost either
link |
if you implement it correctly.
link |
Well, so there's trade offs with everything.
link |
And so if you are doing very low level stuff,
link |
then sometimes you can notice a cost,
link |
but then what you're doing is you're saying,
link |
what is the right default?
link |
So coming back to user interface,
link |
when you talk about programming languages,
link |
one of the major things that Swift does
link |
that makes people love it,
link |
that is not obvious when it comes to designing a language
link |
is this UI principle of progressive disclosure
link |
Okay, so Swift, like many languages is very powerful.
link |
The question is, when do you have to learn
link |
the power as a user?
link |
So Swift, like Python, allows you to start with like,
link |
print hello world, right?
link |
Certain other languages start with like,
link |
public static void main class,
link |
like all the ceremony, right?
link |
And so you go to teach a new person,
link |
hey, welcome to this new thing.
link |
Let's talk about public access control classes.
link |
Wait, what's that?
link |
String system.out.println, like packages,
link |
And so instead, if you take this and you say,
link |
hey, we need packages, modules,
link |
we need powerful things like classes,
link |
we need data structures, we need like all these things.
link |
The question is, how do you factor the complexity?
link |
And how do you make it so that the normal case scenario
link |
is you're dealing with things that work the right way
link |
in the right way, give you good performance
link |
by default, but then as a power user,
link |
if you want to dive down to it,
link |
you have full C performance, full control
link |
over low level pointers.
link |
You can call malloc if you want to call malloc.
link |
This is not recommended on the first page of every tutorial,
link |
but it's actually really important
link |
when you want to get work done, right?
link |
And so being able to have that is really the design
link |
in programming language design,
link |
and design is really, really hard.
link |
It's something that I think a lot of people kind of,
link |
outside of UI, again, a lot of people just think
link |
is subjective, like there's nothing,
link |
you know, it's just like curly braces or whatever.
link |
It's just like somebody's preference,
link |
but actually good design is something that you can feel.
link |
And how many people are involved with good design?
link |
So if we looked at Swift, but look at historically,
link |
I mean, this might touch like,
link |
it's almost like a Steve Jobs question too,
link |
like how much dictatorial decision making is required
link |
versus collaborative, and we'll talk about
link |
how all that can go wrong or right, but.
link |
Yeah, well, Swift, so I can't speak to in general,
link |
all design everywhere.
link |
So the way it works with Swift is that there's a core team,
link |
and so a core team is six or seven people ish,
link |
something like that, that is people that have been working
link |
with Swift since very early days, and so.
link |
And by early days is not that long ago.
link |
Okay, yeah, so it became public in 2014,
link |
so it's been six years public now,
link |
but so that's enough time that there's a story arc there.
link |
Okay, yeah, and there's mistakes have been made
link |
that then get fixed, and you learn something,
link |
and then you, you know, and so what the core team does
link |
is it provides continuity, and so you wanna have a,
link |
okay, well, there's a big hole that we wanna fill.
link |
We know we wanna fill it, so don't do other things
link |
that invade that space until we fill the hole, right?
link |
There's a boulder that's missing here,
link |
we wanna do, we will do that boulder,
link |
even though it's not today, keep out of that space.
link |
And the whole team remembers the myth of the boulder
link |
Yeah, yeah, there's a general sense
link |
of what the future looks like in broad strokes,
link |
and a shared understanding of that,
link |
combined with a shared understanding of what has happened
link |
in the past that worked out well and didn't work out well.
link |
The next level out is you have the,
link |
what's called the Swift evolution community,
link |
and you've got, in that case, hundreds of people
link |
that really care passionately about the way Swift evolves,
link |
and that's like an amazing thing to, again,
link |
the core team doesn't necessarily need to come up
link |
with all the good ideas.
link |
You got hundreds of people out there
link |
that care about something,
link |
and they come up with really good ideas too,
link |
and that provides this rock tumbler for ideas.
link |
And so the evolution process is,
link |
a lot of people in a discourse forum,
link |
they're like hashing it out and trying to talk about,
link |
okay, well, should we go left or right,
link |
or if we did this, what would be good?
link |
And here you're talking about hundreds of people,
link |
so you're not gonna get consensus, necessarily,
link |
not obvious consensus, and so there's a proposal process
link |
that then allows the core team and the community
link |
to work this out, and what the core team does
link |
is it aims to get consensus out of the community
link |
and provide guardrails, but also provide long term,
link |
make sure we're going the right direction kind of things.
link |
So does that group represent like the,
link |
how much people will love the user interface?
link |
Like, do you think they're able to capture that?
link |
Well, I mean, it's something we talk about a lot,
link |
it's something we care about.
link |
How well we do that's up for debate,
link |
but I think that we've done pretty well so far.
link |
Is the beginner in mind?
link |
Yeah. Like, because you said
link |
the progressive disclosure complexity.
link |
Yeah, so we care a lot about that,
link |
a lot about power, a lot about efficiency,
link |
a lot about, there are many factors to good design,
link |
and you have to figure out a way
link |
to kind of work your way through that, and.
link |
So if you think about, like the language I love is Lisp,
link |
probably still because I use Emacs,
link |
but I haven't done anything, any serious work in Lisp,
link |
but it has a ridiculous amount of parentheses.
link |
I've also, you know, with Java and C++, the braces,
link |
you know, I like, I enjoyed the comfort
link |
of being between braces, you know?
link |
Yeah, yeah, well, let's talk.
link |
And then Python is, sorry to interrupt,
link |
just like, and last thing to me, as a designer,
link |
if I was a language designer, God forbid,
link |
is I would be very surprised that Python with no braces
link |
would nevertheless somehow be comforting also.
link |
So like, I could see arguments for all of this.
link |
But look at this, this is evidence
link |
that it's not about braces versus tabs.
link |
Right, exactly, you're good, that's a good point.
link |
Right, so like, you know, there's evidence that.
link |
But see, like, it's one of the most argued about things.
link |
Oh yeah, of course, just like tabs and spaces,
link |
which it doesn't, I mean, there's one obvious right answer,
link |
but it doesn't actually matter.
link |
Let's not, like, come on, we're friends.
link |
Like, come on, what are you trying to do to me here?
link |
People are gonna, yeah, half the people are gonna tune out.
link |
Yeah, so these, so you're able to identify things
link |
that don't really matter for the experience.
link |
Well, no, no, no, it's always a really hard,
link |
so the easy decisions are easy, right?
link |
I mean, fine, those are not the interesting ones.
link |
The hard ones are the ones that are most interesting, right?
link |
The hard ones are the places where,
link |
hey, we wanna do a thing, everybody agrees we should do it,
link |
there's one proposal on the table,
link |
but it has all these bad things associated with it.
link |
Well, okay, what are we gonna do about that?
link |
Do we just take it?
link |
Do we say, hey, well, maybe there's this other feature
link |
that if we do that first, this will work out better.
link |
How does this, if we do this,
link |
are we paying ourselves into a corner, right?
link |
And so this is where, again,
link |
you're having that core team of people
link |
that has some continuity and has perspective,
link |
has some of the historical understanding,
link |
is really valuable because you get,
link |
it's not just like one brain,
link |
you get the power of multiple people coming together
link |
to make good decisions,
link |
and then you get the best out of all these people,
link |
and you also can harness the community around it.
link |
And what about the decision of whether in Python
link |
having one type or having strict typing?
link |
Yeah, let's talk about this.
link |
So I like how you put that, by the way.
link |
So many people would say that Python doesn't have types.
link |
Doesn't have types, yeah.
link |
I haven't listened to you enough to where,
link |
I'm a fan of yours and I've listened to way too many
link |
podcasts and videos of you talking about this stuff.
link |
Oh yeah, so I would argue that Python has one type.
link |
And so like when you import Python into Swift,
link |
which by the way works really well,
link |
you have everything comes in as a Python object.
link |
Now here are their trade offs because
link |
it depends on what you're optimizing for.
link |
And Python is a super successful language
link |
for a really good reason.
link |
Because it has one type,
link |
you get duck typing for free and things like this.
link |
But also you're pushing,
link |
you're making it very easy to pound out code on one hand,
link |
but you're also making it very easy to introduce
link |
complicated bugs that you have to debug.
link |
And you pass a string into something that expects an integer
link |
and it doesn't immediately die.
link |
It goes all the way down the stack trace
link |
and you find yourself in the middle of some code
link |
that you really didn't want to know anything about.
link |
And it blows up and you're just saying,
link |
well, what did I do wrong, right?
link |
And so types are good and bad and they have trade offs.
link |
They're good for performance and certain other things
link |
depending on where you're coming from,
link |
but it's all about trade offs.
link |
And so this is what design is, right?
link |
Design is about weighing trade offs
link |
and trying to understand the ramifications
link |
of the things that you're weighing,
link |
like types or not, or one type or many types.
link |
But also within many types,
link |
how powerful do you make that type system
link |
is another very complicated question
link |
with lots of trade offs.
link |
It's very interesting by the way,
link |
but that's like one dimension and there's a bunch
link |
of other dimensions, JIT compiled versus static compiled,
link |
garbage collected versus reference counted
link |
versus manual memory management versus,
link |
like in like all these different trade offs
link |
and how you balance them
link |
are what make a program language good.
link |
So in all those things, I guess,
link |
when you're designing the language,
link |
you also have to think of how that's gonna get
link |
all compiled down to.
link |
If you care about performance, yeah.
link |
Well, and go back to Lisp, right?
link |
So Lisp also, I would say JavaScript
link |
is another example of a very simple language, right?
link |
And so one of the, so I also love Lisp.
link |
I don't use it as much as maybe you do or you did.
link |
No, I think we're both, everyone who loves Lisp,
link |
it's like, you love, it's like, I don't know,
link |
I love Frank Sinatra,
link |
but like how often do I seriously listen to Frank Sinatra?
link |
But you look at that or you look at JavaScript,
link |
which is another very different,
link |
but relatively simple language.
link |
And there are certain things that don't exist
link |
but there is inherent complexity to the problems
link |
that we're trying to model.
link |
And so what happens to the complexity?
link |
In the case of both of them, for example,
link |
you say, well, what about large scale software development?
link |
Okay, well, you need something like packages.
link |
Neither language has a language affordance for packages.
link |
And so what you get is patterns.
link |
You get things like NPN.
link |
You get things like these ecosystems that get built around.
link |
And I'm a believer that if you don't model
link |
at least the most important inherent complexity
link |
then what ends up happening
link |
is that complexity gets pushed elsewhere.
link |
And when it gets pushed elsewhere,
link |
sometimes that's great because often building things
link |
as libraries is very flexible and very powerful
link |
and allows you to evolve and things like that.
link |
But often it leads to a lot of unnecessary divergence
link |
in the force and fragmentation.
link |
And when that happens, you just get kind of a mess.
link |
And so the question is, how do you balance that?
link |
Don't put too much stuff in the language
link |
because that's really expensive
link |
and it makes things complicated.
link |
But how do you model enough of the inherent complexity
link |
of the problem that you provide the framework
link |
and the structure for people to think about?
link |
Well, so the key thing to think about
link |
with programming languages,
link |
and you think about what a programming language is therefore,
link |
is it's about making a human more productive, right?
link |
And so there's an old, I think it's Steve Jobs quote
link |
about it's a bicycle for the mind, right?
link |
You can definitely walk,
link |
but you'll get there a lot faster
link |
if you can bicycle on your way.
link |
A programming language is a bicycle for the mind?
link |
Is it basically, wow,
link |
that's really interesting way to think about it.
link |
By raising the level of abstraction,
link |
now you can fit more things in your head.
link |
By being able to just directly leverage somebody's library,
link |
you can now get something done quickly.
link |
In the case of Swift, Swift UI is this new framework
link |
that Apple has released recently for doing UI programming.
link |
And it has this declarative programming model,
link |
which defines away entire classes of bugs.
link |
It builds on value semantics and many other nice Swift things.
link |
And what this does is it allows you to just get way more done
link |
with way less code.
link |
And now your productivity as a developer is much higher,
link |
And so that's really what programming languages
link |
is it's not about tabs versus spaces
link |
or curly braces or whatever.
link |
It's about how productive do you make the person.
link |
And you can only see that when you have libraries
link |
that were built with the right intention
link |
that the language was designed for.
link |
And with Swift, I think we're still a little bit early,
link |
but Swift UI and many other things that are coming out now
link |
are really showing that.
link |
And I think that they're opening people's eyes.
link |
It's kind of interesting to think about like how
link |
that the knowledge of something,
link |
of how good the bicycle is,
link |
how people learn about that.
link |
now this is not going to be a trash talking session
link |
about C++, but I used C++ for a really long time.
link |
You can go there if you want, I have the scars.
link |
I feel like I spent many years without realizing
link |
like there's languages that could,
link |
for my particular lifestyle, brain style, thinking style,
link |
there's languages that could make me a lot more productive
link |
in the debugging stage, in just the development stage
link |
and thinking like the bicycle for the mind
link |
that I can fit more stuff into my...
link |
Python's a great example of that, right?
link |
I mean, a machine learning framework in Python
link |
is a great example of that.
link |
It's just very high abstraction level.
link |
And so you can be thinking about things
link |
on a like very high level algorithmic level
link |
instead of thinking about, okay, well,
link |
am I copying this tensor to a GPU or not, right?
link |
It's not what you want to be thinking about.
link |
And as I was telling you, I mean,
link |
I guess the question I had is,
link |
how does a person like me or in general people
link |
discover more productive languages?
link |
Like how I was, as I've been telling you offline,
link |
I've been looking for like a project to work on in Swift
link |
so I can really try it out.
link |
I mean, my intuition was like doing a hello world
link |
is not going to get me there
link |
to get me to experience the power of language.
link |
You need a few weeks to change your metabolism.
link |
Exactly, beautifully put.
link |
That's one of the problems with people with diets,
link |
like I'm actually currently, to go in parallel,
link |
but in a small tangent is I've been recently
link |
eating only meat, okay?
link |
And okay, so most people are like,
link |
they think that's horribly unhealthy or whatever,
link |
you have like a million, whatever the science is,
link |
it just doesn't sound right.
link |
Well, so back when I was in college,
link |
we did the Atkins diet, that was a thing.
link |
Similar, but you have to always give these things a chance.
link |
I mean, with dieting, I was not dieting,
link |
but it's just the things that you like.
link |
If I eat, personally, if I eat meat,
link |
just everything, I can be super focused
link |
or more focused than usual.
link |
I just feel great.
link |
I mean, I've been running a lot,
link |
doing pushups and pull ups and so on.
link |
I mean, Python is similar in that sense for me.
link |
Where are you going with this?
link |
I mean, literally, I just felt I had like a stupid smile
link |
on my face when I first started using Python.
link |
I could code up really quick things.
link |
Like I would see the world,
link |
I would be empowered to write a script
link |
to do some basic data processing,
link |
to rename files on my computer.
link |
Like Perl didn't do that for me,
link |
And again, none of these are about which is best
link |
or something like that,
link |
but there's definitely better and worse here.
link |
But it clicks, right?
link |
If you look at Perl, for example,
link |
you get bogged down in scalars versus arrays
link |
versus hashes versus type globs
link |
and like all that kind of stuff.
link |
And Python's like, yeah, let's not do this.
link |
And some of it is debugging.
link |
Like everyone has different priorities.
link |
But for me, it's, can I create systems for myself
link |
that empower me to debug quickly?
link |
Like I've always been a big fan,
link |
even just crude like asserts,
link |
like always stating things that should be true,
link |
which in Python, I found in myself doing more
link |
because of type, all these kinds of stuff.
link |
Well, you could think of types in a programming language
link |
as being kind of assert.
link |
They could check the compile time, right?
link |
So how do you learn a new thing?
link |
Well, so this, or how do people learn new things, right?
link |
People don't like to change.
link |
People generally don't like change around them either.
link |
And so we're all very slow to adapt and change.
link |
And usually there's a catalyst that's required
link |
to force yourself over this.
link |
So for learning a programming language,
link |
it really comes down to finding an excuse,
link |
like build a thing that the language is actually good for,
link |
that the ecosystem's ready for.
link |
And so if you were to write an iOS app, for example,
link |
that'd be the easy case.
link |
Obviously you would use Swift for that, right?
link |
There are other...
link |
So Swift runs on Android.
link |
Yeah, Swift runs in lots of places.
link |
How does that work?
link |
Okay, so Swift is built on top of LLVM.
link |
LLVM runs everywhere.
link |
LLVM, for example, builds the Android kernel.
link |
I didn't realize this.
link |
Yeah, so Swift is very portable, runs on Windows.
link |
There's, it runs on lots of different things.
link |
And Swift, sorry to interrupt, Swift UI,
link |
and then there's a thing called UI Kit.
link |
So can I build an app with Swift?
link |
Well, so that's the thing,
link |
is the ecosystem is what matters there.
link |
So Swift UI and UI Kit are Apple technologies.
link |
And so they happen to,
link |
like Swift UI happens to be written in Swift,
link |
but it's an Apple proprietary framework
link |
that Apple loves and wants to keep on its platform,
link |
which makes total sense.
link |
You go to Android and you don't have that library, right?
link |
And so Android has a different ecosystem of things
link |
that hasn't been built out
link |
and doesn't work as well with Swift.
link |
And so you can totally use Swift to do like arithmetic
link |
and things like this,
link |
but building UI with Swift on Android
link |
is not a great experience right now.
link |
So if I wanted to learn Swift, what's the,
link |
I mean, the one practical different version of that
link |
is Swift for TensorFlow, for example.
link |
And one of the inspiring things for me
link |
with both TensorFlow and PyTorch
link |
is how quickly the community can like switch
link |
from different libraries, like you could see
link |
some of the communities switching to PyTorch now,
link |
but it's very easy to see.
link |
And then TensorFlow is really stepping up its game.
link |
And then there's no reason why,
link |
I think the way it works is basically
link |
it has to be one GitHub repo,
link |
like one paper steps up.
link |
It gets people excited.
link |
It gets people excited and they're like,
link |
ah, I have to learn this Swift for,
link |
what's Swift again?
link |
And then they learn and they fall in love with it.
link |
I mean, that's what happened, PyTorch has it.
link |
There has to be a reason, a catalyst.
link |
And so, and there, I mean, people don't like change,
link |
but it turns out that once you've worked
link |
with one or two programming languages,
link |
the basics are pretty similar.
link |
And so one of the fun things
link |
about learning programming languages,
link |
even maybe Lisp, I don't know if you agree with this,
link |
is that when you start doing that,
link |
you start learning new things.
link |
Cause you have a new way to do things
link |
and you're forced to do them.
link |
And that forces you to explore
link |
and it puts you in learning mode.
link |
And when you get in learning mode,
link |
your mind kind of opens a little bit
link |
and you can see things in a new way,
link |
even when you go back to the old place.
link |
Yeah, it's totally, well Lisp is functional.
link |
But yeah, I wish there was a kind of window,
link |
maybe you can tell me if there is, there you go.
link |
This is a question to ask,
link |
what is the most beautiful feature
link |
in a programming language?
link |
Before I ask it, let me say like with Python,
link |
I remember I saw Lisp comprehensions.
link |
Was like, when I like really took it in.
link |
I don't know, I just loved it.
link |
It was like fun to do, like it was fun to do that kind of,
link |
something about it to be able to filter through a list
link |
and to create a new list all in a single line was elegant.
link |
I could all get into my head
link |
and it just made me fall in love with the language.
link |
So is there, let me ask you a question.
link |
Is there, what do you use the most beautiful feature
link |
in a programming languages that you've ever encountered
link |
in Swift maybe and then outside of Swift?
link |
I think the thing that I like the most
link |
from a programming language.
link |
So I think the thing you have to think about
link |
with the programming language, again, what is the goal?
link |
You're trying to get people to get things done quickly.
link |
And so you need libraries, you need high quality libraries
link |
and then you need a user base around them
link |
that can assemble them and do cool things with them, right?
link |
And so to me, the question is
link |
what enables high quality libraries?
link |
And there's a huge divide in the world
link |
between libraries who enable high quality libraries
link |
versus the ones that put special stuff in the language.
link |
So programming languages that enable high quality libraries?
link |
So, and what I mean by that is expressive libraries
link |
that then feel like a natural integrated part
link |
of the language itself.
link |
So an example of this in Swift is that int and float
link |
and also array and string, things like this,
link |
these are all part of the library.
link |
Like int is not hard coded into Swift.
link |
And so what that means is that
link |
because int is just a library thing
link |
defined in the standard library,
link |
along with strings and arrays and all the other things
link |
that come with the standard library.
link |
Well, hopefully you do like int,
link |
but anything that any language features
link |
that you needed to define int,
link |
you can also use in your own types.
link |
So if you wanted to find a quaternion
link |
or something like this, right?
link |
Well, it doesn't come in the standard library.
link |
There's a very special set of people
link |
that care a lot about this,
link |
but those people are also important.
link |
It's not about classism, right?
link |
It's not about the people who care about ints and floats
link |
are more important than the people who care about quaternions.
link |
And so to me, the beautiful things
link |
about programming languages is when you allow
link |
those communities to build high quality libraries,
link |
They feel like they're built into the compiler
link |
without having to be.
link |
What does it mean for the int to be part
link |
of not hard coded in?
link |
So is it like how, so what is an int?
link |
Okay, int is just a integer.
link |
In this case, it's like a 64 bit integer
link |
or something like this.
link |
But so like the 64 bit is hard coded or no?
link |
No, none of that's hard coded.
link |
So int, if you go look at how it's implemented,
link |
it's just a struct in Swift.
link |
And so it's a struct.
link |
And then how do you add two structs?
link |
Well, you define plus.
link |
And so you can define plus on int.
link |
Well, you can define plus on your thing too.
link |
You can define, int is an odd method
link |
or something like that on it.
link |
And so yeah, you can add methods on the things.
link |
So you can define operators, like how it behaves.
link |
That's just beautiful when there's something
link |
about the language which enables others
link |
to create libraries which are not hacky.
link |
Yeah, they feel native.
link |
And so one of the best examples of this is Lisp, right?
link |
Because in Lisp, like all the libraries
link |
are basically part of the language, right?
link |
You write, turn, rewrite systems and things like this.
link |
Can you as a counter example provide
link |
what makes it difficult to write a library that's native?
link |
Is it the Python C?
link |
Well, so one example, I'll give you two examples.
link |
Java and C++, there's Java and C.
link |
They both allow you to define your own types,
link |
but int is hard code in the language.
link |
Well, in Java, for example, coming back
link |
to this whole reference semantic value semantic thing,
link |
int gets passed around by value.
link |
But if you make like a pair or something like that,
link |
a complex number, right, it's a class in Java.
link |
And now it gets passed around by reference, by pointer.
link |
And so now you lose value semantics, right?
link |
You lost math, okay.
link |
Well, that's not great, right?
link |
If you can do something with int,
link |
why can't I do it with my type, right?
link |
So that's the negative side of the thing I find beautiful
link |
is when you can solve that,
link |
when you can have full expressivity,
link |
where you as a user of the language
link |
have as much or almost as much power
link |
as the people who implemented
link |
all the standard built in stuff,
link |
because what that enables
link |
is that enables truly beautiful libraries.
link |
You know, it's kind of weird
link |
because I've gotten used to that.
link |
That's one, I guess, other aspect
link |
of program language design.
link |
You have to think, you know,
link |
the old first principles thinking,
link |
like, why are we doing it this way?
link |
By the way, I mean, I remember,
link |
because I was thinking about the walrus operator
link |
and I'll ask you about it later,
link |
but it hit me that like the equal sign for assignment.
link |
Like, why are we using the equal sign for assignment?
link |
And that's not the only solution, right?
link |
So if you look at Pascal,
link |
they use colon equals for assignment
link |
and equals for equality.
link |
And they use like less than greater than
link |
instead of the not equal thing.
link |
Like, there are other answers here.
link |
So, but like, and yeah, like I ask you all,
link |
but how do you then decide to break convention
link |
to say, you know what, everybody's doing it wrong.
link |
We're gonna do it right.
link |
So it's like an ROI,
link |
like return on investment trade off, right?
link |
So if you do something weird,
link |
let's just say like not like colon equal
link |
instead of equal for assignment,
link |
that would be weird with today's aesthetic, right?
link |
And so you'd say, cool, this is theoretically better,
link |
but is it better in which ways?
link |
Like, what do I get out of that?
link |
Do I define away class of bugs?
link |
Well, one of the class of bugs that C has
link |
is that you can use like, you know,
link |
if X equals without equals equals X equals Y, right?
link |
Well, turns out you can solve that problem in lots of ways.
link |
Clang, for example, GCC, all these compilers
link |
will detect that as a likely bug, produce a warning.
link |
I feel like they didn't.
link |
It's like one of the important things
link |
about programming language design is like,
link |
you're literally creating suffering in the world.
link |
Like, I feel like, I mean, one way to see it
link |
is the bicycle for the mind,
link |
but the other way is to like minimizing suffering.
link |
Well, you have to decide if it's worth it, right?
link |
And so let's come back to that.
link |
But if you look at this, and again,
link |
this is where there's a lot of detail
link |
that goes into each of these things.
link |
Equal and C returns a value.
link |
That allows you to say X equals Y equals Z,
link |
like that works in C.
link |
You know, most people think it's messed up, I think.
link |
It is very, by messed up, what I mean is
link |
it is very rarely used for good,
link |
and it's often used for bugs.
link |
That's a good definition of messed up, yeah.
link |
You could use, you know, in hindsight,
link |
this was not such a great idea, right?
link |
One of the things with Swift that is really powerful
link |
and one of the reasons it's actually good
link |
versus it being full of good ideas
link |
is that when we launched Swift 1,
link |
we announced that it was public,
link |
people could use it, people could build apps,
link |
but it was gonna change and break, okay?
link |
When Swift 2 came out, we said, hey, it's open source,
link |
and there's this open process
link |
which people can help evolve and direct the language.
link |
So the community at large, like Swift users,
link |
can now help shape the language as it is.
link |
And what happened as part of that process is
link |
a lot of really bad mistakes got taken out.
link |
So for example, Swift used to have the C style plus plus
link |
and minus minus operators.
link |
Like, what does it mean when you put it before
link |
versus after, right?
link |
Well, that got cargo culted from C into Swift early on.
link |
What's cargo culted?
link |
Cargo culted means brought forward
link |
without really considering it.
link |
This is maybe not the most PC term, but.
link |
You have to look it up in Urban Dictionary, yeah.
link |
Yeah, so it got pulled into C without,
link |
or it got pulled into Swift
link |
without very good consideration.
link |
And we went through this process,
link |
and one of the first things got ripped out
link |
was plus plus and minus minus,
link |
because they lead to confusion.
link |
They have very low value over saying x plus equals one,
link |
and x plus equals one is way more clear.
link |
And so when you're optimizing for teachability and clarity
link |
and bugs and this multidimensional space
link |
that you're looking at,
link |
things like that really matter.
link |
And so being first principles on where you're coming from
link |
and what you're trying to achieve
link |
and being anchored on the objective is really important.
link |
Well, let me ask you about the most,
link |
sort of this podcast isn't about information,
link |
Let me talk to you about some drama.
link |
So you mentioned Pascal and colon equals,
link |
there's something that's called the Walrus operator.
link |
And Python in Python 3.8 added the Walrus operator.
link |
And the reason I think it's interesting
link |
is not just because of the feature,
link |
it has the same kind of expression feature
link |
you can mention to see that it returns
link |
the value of the assignment.
link |
And then maybe you can comment on that in general,
link |
but on the other side of it,
link |
it's also the thing that toppled the dictator.
link |
So it finally drove Guido
link |
to step down from BDFL, the toxicity of the community.
link |
So maybe what do you think about the Walrus operator
link |
Is there an equivalent thing in Swift
link |
that really stress tested the community?
link |
And then on the flip side,
link |
what do you think about Guido stepping down over it?
link |
Yeah, well, if I look past the details
link |
of the Walrus operator,
link |
one of the things that makes it most polarizing
link |
is that it's syntactic sugar.
link |
What do you mean by syntactic sugar?
link |
It means you can take something
link |
that already exists in the language
link |
and you can express it in a more concise way.
link |
So, okay, I'm going to play dollars advocate.
link |
Is that a objective or subjective statement?
link |
Like, can you argue that basically anything
link |
isn't syntactic sugar or not?
link |
No, not everything is syntactic sugar.
link |
So for example, the type system,
link |
like can you have classes versus,
link |
like, do you have types or not, right?
link |
So one type versus many types
link |
is not something that affects syntactic sugar.
link |
And so if you say,
link |
I want to have the ability to define types,
link |
I have to have all this like language mechanics
link |
to define classes.
link |
And oh, now I have to have inheritance.
link |
And I have like, I have all this stuff
link |
that's just making the language more complicated.
link |
That's not about sugaring it.
link |
Swift has the sugar.
link |
So like Swift has this thing called if let,
link |
and it has a lot of different types
link |
and it has various operators
link |
that are used to concisify specific use cases.
link |
So the problem with syntactic sugar,
link |
when you're talking about,
link |
hey, I have a thing that takes a lot to write
link |
and I have a new way to write it.
link |
You have this like horrible trade off,
link |
which becomes almost completely subjective,
link |
which is how often does this happen and does it matter?
link |
And one of the things that is true about human psychology,
link |
particularly when you're talking about introducing
link |
a new thing is that people overestimate
link |
the burden of learning something.
link |
And so it looks foreign when you haven't gotten used to it.
link |
But if it was there from the beginning,
link |
of course it's just part of Python.
link |
Like unquestionably, like this is just the thing I know.
link |
And it's not a new thing that you're worried about learning.
link |
It's just part of the deal.
link |
Now with Guido, I don't know Guido well.
link |
Yeah, have you passed cross much?
link |
Yeah, I've met him a couple of times,
link |
but I don't know Guido well.
link |
But the sense that I got out of that whole dynamic
link |
was that he had put the,
link |
not just the decision maker weight on his shoulders,
link |
but it was so tied to his personal identity
link |
that he took it personally and he felt the need
link |
and he kind of put himself in the situation
link |
of being the person,
link |
instead of building a base of support around him.
link |
I mean, this is probably not quite literally true,
link |
but by too much concentrated on him, right?
link |
And that can wear you down.
link |
Well, yeah, particularly because people then say,
link |
Guido, you're a horrible person.
link |
I hate this thing, blah, blah, blah, blah, blah, blah, blah.
link |
And sure, it's like maybe 1% of the community
link |
that's doing that, but Python's got a big community.
link |
And 1% of millions of people is a lot of hate mail.
link |
And that just from human factor will just wear on you.
link |
Well, to clarify, it looked from just what I saw
link |
in the messaging for the,
link |
let's not look at the million Python users,
link |
but at the Python core developers,
link |
it feels like the majority, the big majority
link |
on a vote were opposed to it.
link |
Okay, I'm not that close to it, so I don't know.
link |
Okay, so the situation is like literally,
link |
yeah, I mean, the majority of the core developers
link |
Were opposed to it.
link |
So, and they weren't even like against it.
link |
It was, there was a few, well, they were against it,
link |
but the against it wasn't like, this is a bad idea.
link |
They were more like, we don't see why this is a good idea.
link |
And what that results in is there's a stalling feeling,
link |
like you just slow things down.
link |
Now, from my perspective, that you could argue this,
link |
and I think it's very interesting
link |
if we look at politics today and the way Congress works,
link |
it's slowed down everything.
link |
Yeah, it's a dampener, but like,
link |
that's a dangerous thing too,
link |
because if it dampens things like, you know,
link |
if the dampening results.
link |
What are you talking about?
link |
Like, it's a low pass filter,
link |
but if you need billions of dollars
link |
injected into the economy or trillions of dollars,
link |
then suddenly stuff happens, right?
link |
So you're talking about.
link |
I'm not defending our political situation,
link |
But you're talking about like a global pandemic.
link |
I was hoping we could fix like the healthcare system
link |
and the education system, like, you know.
link |
I'm not a politics person.
link |
When it comes to languages,
link |
the community's kind of right in terms of,
link |
it's a very high burden to add something to a language.
link |
So as soon as you add something,
link |
you have a community of people building on it
link |
and you can't remove it, okay?
link |
And if there's a community of people
link |
that feel really uncomfortable with it,
link |
then taking it slow, I think, is an important thing to do.
link |
And there's no rush, particularly if it was something
link |
that's 25 years old and is very established.
link |
And, you know, it's not like coming into its own.
link |
What about features?
link |
Well, so I think that the issue with Guido
link |
is that maybe this is a case
link |
where he realized it had outgrown him
link |
and it went from being the language.
link |
So Python, I mean, Guido's amazing,
link |
but Python isn't about Guido anymore.
link |
It's about the users.
link |
And to a certain extent, the users own it.
link |
And, you know, Guido spent years of his life,
link |
a significant fraction of his career on Python.
link |
And from his perspective, I imagine he's like,
link |
well, this is my thing.
link |
I should be able to do the thing I think is right.
link |
But you can also understand the users
link |
where they feel like, you know, this is my thing.
link |
I use this, like, and I don't know, it's a hard thing.
link |
But what, if we could talk about leadership in this,
link |
because it's so interesting to me.
link |
I'm gonna make, I'm gonna work.
link |
Hopefully somebody makes it.
link |
If not, I'll make it a Walrus Operator shirt,
link |
because I think it represents, to me,
link |
maybe it's my Russian roots or something.
link |
But, you know, it's the burden of leadership.
link |
Like, I feel like to push back,
link |
I feel like progress can only,
link |
like most difficult decisions, just like you said,
link |
there'll be a lot of divisiveness over,
link |
especially in a passionate community.
link |
It just feels like leaders need to take
link |
those risky decisions that if you like listen,
link |
that with some nonzero probability,
link |
maybe even a high probability would be the wrong decision.
link |
But they have to use their gut and make that decision.
link |
Well, this is like one of the things
link |
where you see amazing founders.
link |
The founders understand exactly what's happened
link |
and how the company got there and are willing to say,
link |
we have been doing thing X the last 20 years,
link |
but today we're gonna do thing Y.
link |
And they make a major pivot for the whole company.
link |
The company lines up behind them,
link |
they move and it's the right thing.
link |
But then when the founder dies,
link |
the successor doesn't always feel that agency
link |
to be able to make those kinds of decisions.
link |
Even though they're a CEO,
link |
they could theoretically do whatever.
link |
There's two reasons for that, in my opinion,
link |
or in many cases, it's always different.
link |
But one of which is they weren't there
link |
for all the decisions that were made.
link |
And so they don't know the principles
link |
in which those decisions were made.
link |
And once the principles change,
link |
you should be obligated to change what you're doing
link |
and change direction, right?
link |
And so if you don't know how you got to where you are,
link |
it just seems like gospel
link |
and you're not gonna question it.
link |
You may not understand
link |
that it really is the right thing to do,
link |
so you just may not see it.
link |
That's so brilliant.
link |
I never thought of it that way.
link |
Like it's so much higher burden
link |
when as a leader you step into a thing
link |
that's already worked for a long time.
link |
Well, and if you change it and it doesn't work out,
link |
now you're the person who screwed it up.
link |
People always second guess it.
link |
And the second thing is that
link |
even if you decide to make a change,
link |
even if you're theoretically in charge,
link |
you're just a person that thinks they're in charge.
link |
Meanwhile, you have to motivate the troops.
link |
You have to explain it to them
link |
in terms they'll understand.
link |
You have to get them to buy into it and believe in it,
link |
because if they don't,
link |
then they're not gonna be able to make the turn
link |
even if you tell them their bonuses are gonna be curtailed.
link |
They're just not gonna like buy into it, you know?
link |
And so there's only so much power you have as a leader,
link |
and you have to understand what those limitations are.
link |
Are you still BDFL?
link |
You've been a BDFL of some stuff.
link |
You're very heavy on the B,
link |
the benevolent, benevolent dictator for life.
link |
Yeah, so I still lead the LLVM world.
link |
I mean, what's the role of,
link |
so then on Swift you said that there's a group of people.
link |
Yeah, so if you contrast Python with Swift, right,
link |
one of the reasons,
link |
so everybody on the core team takes the role
link |
really seriously, and I think we all really care
link |
about where Swift goes,
link |
but you're almost delegating the final decision making
link |
to the wisdom of the group,
link |
and so it doesn't become personal.
link |
And also, when you're talking with the community,
link |
so yeah, some people are very annoyed
link |
as certain decisions get made.
link |
There's a certain faith in the process,
link |
because it's a very transparent process,
link |
and when a decision gets made,
link |
a full rationale is provided, things like this.
link |
These are almost defense mechanisms
link |
to help both guide future discussions
link |
and provide case law, kind of like Supreme Court does
link |
about this decision was made for this reason,
link |
and here's the rationale
link |
and what we want to see more of or less of.
link |
But it's also a way to provide a defense mechanism,
link |
so that when somebody's griping about it,
link |
they're not saying that person did the wrong thing.
link |
They're saying, well, this thing sucks,
link |
and later they move on and they get over it.
link |
Yeah, the analogy of the Supreme Court,
link |
I think, is really good.
link |
But then, okay, not to get personal on the SWIFT team,
link |
but it just seems like it's impossible
link |
for division not to emerge.
link |
Well, each of the humans on the SWIFT Core Team,
link |
for example, are different,
link |
and the membership of the SWIFT Core Team
link |
changes slowly over time, which is, I think, a healthy thing.
link |
And so each of these different humans
link |
have different opinions.
link |
Trust me, it's not a singular consciousness
link |
by any stretch of the imagination.
link |
You've got three major organizations,
link |
including Apple, Google, and SciFive,
link |
all kind of working together.
link |
And it's a small group of people, but you need high trust.
link |
You need, again, it comes back to the principles
link |
of what you're trying to achieve
link |
and understanding what you're optimizing for.
link |
And I think that starting with strong principles
link |
and working towards decisions
link |
is always a good way to both make wise decisions in general
link |
but then be able to communicate them to people
link |
so that they can buy into them.
link |
And so you mentioned LLVM.
link |
LLVM is gonna be 20 years old this December,
link |
so it's showing its own age.
link |
Do you have like a dragon cake plan?
link |
No, I should definitely do that.
link |
Yeah, if we can have a pandemic cake.
link |
Everybody gets a slice of cake
link |
and it gets sent through email.
link |
But LLVM has had tons of its own challenges
link |
over time too, right?
link |
And one of the challenges that the LLVM community has,
link |
in my opinion, is that it has a whole bunch of people
link |
that have been working on LLVM for 10 years, right?
link |
Because this happens somehow.
link |
And LLVM has always been one way,
link |
but it needs to be a different way, right?
link |
And they've worked on it for like 10 years.
link |
It's a long time to work on something.
link |
And you suddenly can't see the faults
link |
in the thing that you're working on.
link |
And LLVM has lots of problems and we need to address them
link |
and we need to make it better.
link |
And if we don't make it better,
link |
then somebody else will come up with a better idea, right?
link |
And so it's just kind of of that age
link |
where the community is like in danger
link |
of getting too calcified.
link |
And so I'm happy to see new projects joining
link |
and new things mixing it up.
link |
Fortran is now a new thing in the LLVM community,
link |
which is hilarious and good.
link |
I've been trying to find, on a little tangent,
link |
find people who program in Cobalt or Fortran,
link |
Fortran especially, to talk to, they're hard to find.
link |
Yeah, look to the scientific community.
link |
They still use Fortran quite a bit.
link |
Well, interesting thing you kind of mentioned with LLVM,
link |
or just in general, that as something evolves,
link |
you're not able to see the faults.
link |
So do you fall in love with the thing over time?
link |
Or do you start hating everything
link |
about the thing over time?
link |
Well, so my personal folly is that I see,
link |
maybe not all, but many of the faults,
link |
and they grate on me, and I don't have time to go fix them.
link |
Yeah, and they get magnified over time.
link |
Well, and they may not get magnified,
link |
but they never get fixed.
link |
And it's like sand underneath,
link |
you know, it's just like grating against you.
link |
And it's like sand underneath your fingernails or something.
link |
It's just like, you know it's there,
link |
you can't get rid of it.
link |
And so the problem is that if other people don't see it,
link |
like I don't have time to go write the code
link |
and fix it anymore,
link |
but then people are resistant to change.
link |
And so you say, hey, we should go fix this thing.
link |
They're like, oh yeah, that sounds risky.
link |
It's like, well, is it the right thing or not?
link |
Are the challenges the group dynamics,
link |
or is it also just technical?
link |
I mean, some of these features like,
link |
I think as an observer, it's almost like a fan
link |
in the, you know, as a spectator of the whole thing,
link |
I don't often think about, you know,
link |
some things might actually be
link |
technically difficult to implement.
link |
An example of this is we built this new compiler framework
link |
MLIR is a whole new framework.
link |
It's not, many people think it's about machine learning.
link |
The ML stands for multi level
link |
because compiler people can't name things very well,
link |
Do we dig into what MLIR is?
link |
Yeah, so when you look at compilers,
link |
compilers have historically been solutions for a given space.
link |
So LLVM is a, it's really good for dealing with CPUs,
link |
let's just say, at a high level.
link |
You look at Java, Java has a JVM.
link |
The JVM is very good for garbage collected languages
link |
that need dynamic compilation,
link |
and it's very optimized for a specific space.
link |
And so hotspot is one of the compilers
link |
that gets used in that space,
link |
and that compiler is really good at that kind of stuff.
link |
Usually when you build these domain specific compilers,
link |
you end up building the whole thing from scratch
link |
So what's the scope of a domain?
link |
Well, so here I would say, like, if you look at Swift,
link |
there's several different parts to the Swift compiler,
link |
one of which is covered by the LLVM part of it.
link |
There's also a high level piece that's specific to Swift,
link |
and there's a huge amount of redundancy
link |
between those two different infrastructures
link |
and a lot of re implemented stuff
link |
that is similar but different.
link |
What does LLVM define?
link |
LLVM is effectively an infrastructure.
link |
So you can mix and match it in different ways.
link |
It's built out of libraries.
link |
You can use it for different things,
link |
but it's really good at CPUs and GPUs.
link |
CPUs and like the tip of the iceberg on GPUs.
link |
It's not really great at GPUs.
link |
But it turns out. A bunch of languages that.
link |
That then use it to talk to CPUs.
link |
And so it turns out there's a lot of hardware out there
link |
that is custom accelerators.
link |
So machine learning, for example.
link |
There are a lot of matrix multiply accelerators
link |
and things like this.
link |
There's a whole world of hardware synthesis.
link |
So we're using MLIR to build circuits.
link |
And so you're compiling for a domain of transistors.
link |
And so what MLIR does is it provides
link |
a tremendous amount of compiler infrastructure
link |
that allows you to build these domain specific compilers
link |
in a much faster way and have the result be good.
link |
If we're thinking about the future,
link |
now we're talking about like ASICs.
link |
So if we project into the future,
link |
it's very possible that the number of these kinds of ASICs,
link |
very specific infrastructure architecture things
link |
like multiplies exponentially.
link |
So what MLIR does is it allows you
link |
to build these compilers very efficiently.
link |
Right now, one of the things that coming back
link |
to the LLVM thing, and then we'll go to hardware,
link |
is LLVM is a specific compiler for a specific domain.
link |
MLIR is now this very general, very flexible thing
link |
that can solve lots of different kinds of problems.
link |
So LLVM is a subset of what MLIR does.
link |
So MLIR is, I mean, it's an ambitious project then.
link |
Yeah, it's a very ambitious project, yeah.
link |
And so to make it even more confusing,
link |
MLIR has joined the LLVM Umbrella Project.
link |
So it's part of the LLVM family.
link |
But where this comes full circle is now folks
link |
that work on the LLVM part,
link |
the classic part that's 20 years old,
link |
aren't aware of all the cool new things
link |
that have been done in the new thing,
link |
that MLIR was built by me and many other people
link |
that knew a lot about LLVM,
link |
and so we fixed a lot of the mistakes that lived in LLVM.
link |
And so now you have this community dynamic
link |
where it's like, well, there's this new thing,
link |
but it's not familiar, nobody knows it,
link |
it feels like it's new, and so let's not trust it.
link |
And so it's just really interesting
link |
to see the cultural social dynamic that comes out of that.
link |
And I think it's super healthy
link |
because we're seeing the ideas percolate
link |
and we're seeing the technology diffusion happen
link |
as people get more comfortable with it,
link |
they start to understand things in their own terms.
link |
And this just gets to the,
link |
it takes a while for ideas to propagate,
link |
even though they may be very different
link |
than what people are used to.
link |
So maybe let's talk about that a little bit,
link |
the world of Asics.
link |
Actually, you have a new role at SciFive.
link |
What's that place about?
link |
What is the vision for their vision
link |
for, I would say, the future of computer?
link |
Yeah, so I lead the engineering and product teams at SciFive.
link |
SciFive is a company who was founded
link |
with this architecture called RISC5.
link |
RISC5 is a new instruction set.
link |
Instruction sets are the things inside of your computer
link |
that tell it how to run things.
link |
X86 from Intel and ARM from the ARM company
link |
and things like this are other instruction sets.
link |
I've talked to, sorry to interrupt,
link |
I've talked to Dave Patterson,
link |
who's super excited about RISC5.
link |
Yeah, he's brilliant, yeah.
link |
The RISC5 is distinguished by not being proprietary.
link |
And so X86 can only be made by Intel and AMD.
link |
ARM can only be made by ARM.
link |
They sell licenses to build ARM chips to other companies,
link |
MIPS is another instruction set
link |
that is owned by the MIPS company, now Wave.
link |
And then it gets licensed out, things like that.
link |
And so RISC5 is an open standard
link |
that anybody can build chips for.
link |
And so SciFive was founded by three of the founders
link |
of RISC5 that designed and built it in Berkeley,
link |
working with Dave.
link |
And so that was the genesis of the company.
link |
SciFive today has some of the world's best RISC5 cores
link |
and we're selling them and that's really great.
link |
They're going to tons of products, it's very exciting.
link |
So they're taking this thing that's open source
link |
and just trying to be or are the best in the world
link |
at building these things.
link |
Yeah, so here it's the specifications open source.
link |
It's like saying TCP IP is an open standard
link |
or C is an open standard,
link |
but then you have to build an implementation
link |
And so SciFive, on the one hand, pushes forward
link |
and defined and pushes forward the standard.
link |
On the other hand, we have implementations
link |
that are best in class for different points in the space,
link |
depending on if you want a really tiny CPU
link |
or if you want a really big, beefy one that is faster,
link |
but it uses more area and things like this.
link |
What about the actual manufacturer chips?
link |
So like, where does that all fit?
link |
I'm going to ask a bunch of dumb questions.
link |
That's okay, this is how we learn, right?
link |
And so the way this works is that there's generally
link |
a separation of the people who designed the circuits
link |
and then people who manufacture them.
link |
And so you'll hear about fabs like TSMC and Samsung
link |
and things like this that actually produce the chips,
link |
but they take a design coming in
link |
and that design specifies how the,
link |
you turn code for the chip into little rectangles
link |
that then use photolithography to make mask sets
link |
and then burn transistors onto a chip
link |
or onto a, onto silicon rather.
link |
So, and we're talking about mass manufacturing, so.
link |
Yeah, they're talking about making hundreds of millions
link |
of parts and things like that, yeah.
link |
And so the fab handles the volume production,
link |
But when you look at this problem,
link |
the interesting thing about the space when you look at it
link |
is that these, the steps that you go from designing a chip
link |
and writing the quote unquote code for it
link |
and things like Verilog and languages like that,
link |
down to what you hand off to the fab
link |
is a really well studied, really old problem, okay?
link |
Tons of people have worked on it.
link |
Lots of smart people have built systems and tools.
link |
These tools then have generally gone through acquisitions.
link |
And so they've ended up at three different major companies
link |
that build and sell these tools.
link |
They're called the EDA tools
link |
like for electronic design automation.
link |
The problem with this is you have huge amounts
link |
of fragmentation, you have loose standards
link |
and the tools don't really work together.
link |
So you have tons of duct tape
link |
and you have tons of loss productivity.
link |
Now these are, these are tools for designing.
link |
So the RISC 5 is a instruction.
link |
Like what is RISC 5?
link |
Like how deep does it go?
link |
How much does it touch the hardware?
link |
How much does it define how much of the hardware is?
link |
Yeah, so RISC 5 is all about given a CPU.
link |
So the processor and your computer,
link |
how does the compiler like Swift compiler,
link |
the C compiler, things like this, how does it make it work?
link |
So it's, what is the assembly code?
link |
And so you write RISC 5 assembly
link |
instead of XA6 assembly, for example.
link |
But it's a set of instructions
link |
as opposed to instructions.
link |
Why do you say it tells you how the compiler works?
link |
Sorry, it's what the compiler talks to.
link |
And then the tooling you mentioned
link |
that the disparate tools are for what?
link |
For when you're building a specific chip.
link |
So RISC 5. In hardware.
link |
In hardware, yeah.
link |
So RISC 5, you can buy a RISC 5 core from SciFive
link |
and say, hey, I want to have a certain number of,
link |
run a certain number of gigahertz.
link |
I want it to be this big.
link |
I want it to be, have these features.
link |
I want to have like, I want floating point or not,
link |
And then what you get is you get a description
link |
of a CPU with those characteristics.
link |
Now, if you want to make a chip,
link |
you want to build like an iPhone chip
link |
or something like that, right?
link |
You have to take both the CPU,
link |
but then you have to talk to memory.
link |
You have to have timers, IOs, a GPU, other components.
link |
And so you need to pull all those things together
link |
into what's called an ASIC,
link |
an Application Specific Integrated Circuit.
link |
And then you take that design
link |
and then you have to transform it into something
link |
that the fabs, like TSMC, for example,
link |
know how to take to production.
link |
So, but yeah, okay.
link |
And so that process, I will,
link |
I can't help but see it as, is a big compiler.
link |
It's a whole bunch of compilers written
link |
without thinking about it through that lens.
link |
Isn't the universe a compiler?
link |
Yeah, compilers do two things.
link |
They represent things and transform them.
link |
And so there's a lot of things that end up being compilers.
link |
But this is a space where we're talking about design
link |
and usability and the way you think about things,
link |
the way things compose correctly, it matters a lot.
link |
And so SciFi is investing a lot into that space.
link |
And we think that there's a lot of benefit
link |
that can be made by allowing people to design chips faster,
link |
get them to market quicker and scale out
link |
because at the alleged end of Moore's law,
link |
you've got this problem of you're not getting
link |
free performance just by waiting another year
link |
And so you have to find performance in other ways.
link |
And one of the ways to do that is with custom accelerators
link |
and other things and hardware.
link |
And so, well, we'll talk a little more about ASICs,
link |
but do you see that a lot of people,
link |
a lot of companies will try to have
link |
different sets of requirements
link |
that this whole process to go for?
link |
So like almost different car companies might use different
link |
and like different PC manufacturers.
link |
So is RISC 5 in this whole process,
link |
is it potentially the future of all computing devices?
link |
Yeah, I think that, so if you look at RISC 5
link |
and step back from the Silicon side of things,
link |
RISC 5 is an open standard.
link |
And one of the things that has happened
link |
over the course of decades,
link |
if you look over the long arc of computing,
link |
somehow became decades old.
link |
Is that you have companies that come and go
link |
and you have instruction sets that come and go.
link |
Like one example of this out of many is Sun with Spark.
link |
Yeah, it's on one way.
link |
Spark still lives on at Fujitsu,
link |
but we have HP had this instruction set called PA RISC.
link |
So PA RISC was this big server business
link |
and had tons of customers.
link |
They decided to move to this architecture
link |
called Itanium from Intel.
link |
This didn't work out so well.
link |
Right, and so you have this issue of
link |
you're making many billion dollar investments
link |
on instruction sets that are owned by a company.
link |
And even companies as big as Intel
link |
don't always execute as well as they could.
link |
They even have their own issues.
link |
HP, for example, decided that it wasn't
link |
in their best interest to continue investing in the space
link |
because it was very expensive.
link |
And so they make technology decisions
link |
or they make their own business decisions.
link |
And this means that as a customer, what do you do?
link |
You've sunk all this time, all this engineering,
link |
all this software work, all these,
link |
you've built other products around them
link |
and now you're stuck, right?
link |
What RISC 5 does is provide you more optionality
link |
in the space because if you buy an implementation
link |
of RISC 5 from SciFive, and you should,
link |
they're the best ones.
link |
But if something bad happens to SciFive in 20 years, right?
link |
Well, great, you can turn around
link |
and buy a RISC 5 core from somebody else.
link |
And there's an ecosystem of people
link |
that are all making different RISC 5 cores
link |
with different trade offs, which means that
link |
if you have more than one requirement,
link |
if you have a family of products,
link |
you can probably find something in the RISC 5 space
link |
that fits your needs.
link |
Whereas with, if you're talking about XA6, for example,
link |
it's Intel's only gonna bother
link |
to make certain classes of devices, right?
link |
I see, so maybe a weird question,
link |
but like if SciFive is like infinitely successful
link |
in the next 20, 30 years, what does the world look like?
link |
So like how does the world of computing change?
link |
So too much diversity in hardware instruction sets,
link |
Like we have a lot of people that are using
link |
lots of different instruction sets,
link |
particularly in the embedded,
link |
the like very tiny microcontroller space,
link |
the thing in your toaster that are just weird
link |
and different for historical reasons.
link |
And so the compilers and the tool chains
link |
and the languages on top of them aren't there.
link |
And so the developers for that software
link |
have to use really weird tools
link |
because the ecosystem that supports is not big enough.
link |
So I expect that will change, right?
link |
People will have better tools and better languages,
link |
better features everywhere
link |
that then can serve as many different points in the space.
link |
And I think RISC5 will progressively
link |
eat more of the ecosystem because it can scale up,
link |
it can scale down, sideways, left, right.
link |
It's very flexible and very well considered
link |
and well designed instruction set.
link |
I think when you look at SciFive tackling silicon
link |
and how people build chips,
link |
which is a very different space,
link |
that's where you say,
link |
I think we'll see a lot more custom chips.
link |
And that means that you get much more battery life,
link |
you get better tuned solutions for your IoT thingy.
link |
You get people that move faster,
link |
you get the ability to have faster time to market,
link |
So how many custom...
link |
So first of all, on the IoT side of things,
link |
do you see the number of smart toasters
link |
increasing exponentially?
link |
So, and if you do,
link |
like how much customization per toaster is there?
link |
Do all toasters in the world run the same silicon,
link |
like the same design,
link |
or is it different companies have different design?
link |
Like how much customization is possible here?
link |
Well, a lot of it comes down to cost, right?
link |
And so the way that chips work is you end up paying by the...
link |
One of the factors is the size of the chip.
link |
And so what ends up happening
link |
just from an economic perspective is
link |
there's only so many chips that get made in a year
link |
of a given design.
link |
And so often what customers end up having to do
link |
is they end up having to pick up a chip that exists
link |
that was built for somebody else
link |
so that they can then ship their product.
link |
And the reason for that
link |
is they don't have the volume of the iPhone.
link |
They can't afford to build a custom chip.
link |
However, what that means is they're now buying
link |
an off the shelf chip that isn't really good,
link |
isn't a perfect fit for their needs.
link |
And so they're paying a lot of money for it
link |
because they're buying silicon that they're not using.
link |
Well, if you now reduce the cost of designing the chip,
link |
now you get a lot more chips.
link |
And the more you reduce it,
link |
the easier it is to design chips.
link |
The more the world keeps evolving
link |
and we get more AI accelerators,
link |
we get more other things,
link |
we get more standards to talk to,
link |
You get changes in the world
link |
that you wanna be able to talk to these different things.
link |
There's more diversity in the cross product of features
link |
And that drives differentiated chips
link |
in another direction.
link |
And so nobody really knows what the future looks like,
link |
but I think that there's a lot of silicon in the future.
link |
Speaking of the future,
link |
you said Moore's law allegedly is dead.
link |
So do you agree with Dave Patterson and many folks
link |
that Moore's law is dead?
link |
Or do you agree with Jim Keller,
link |
who's standing at the helm of the pirate ship
link |
saying it's still alive?
link |
Well, so I agree with what they're saying
link |
and different people are interpreting
link |
the end of Moore's law in different ways.
link |
there's another thousand X left in physics
link |
and we can continue to squeeze the stone
link |
and make it faster and smaller and smaller geometries
link |
and all that kind of stuff.
link |
So Jim is absolutely right
link |
that there's a ton of progress left
link |
and we're not at the limit of physics yet.
link |
That's not really what Moore's law is though.
link |
If you look at what Moore's law is,
link |
is that it's a very simple evaluation of,
link |
okay, well you look at the cost per,
link |
I think it was cost per area
link |
and the most economic point in that space.
link |
And if you go look at the now quite old paper
link |
that describes this,
link |
Moore's law has a specific economic aspect to it
link |
and I think this is something
link |
that Dave and others often point out.
link |
And so on a technicality, that's right.
link |
I look at it from,
link |
so I can acknowledge both of those viewpoints.
link |
They're both right.
link |
They're both right.
link |
I'll give you a third wrong viewpoint
link |
that may be right in its own way,
link |
which is single threaded performance
link |
doesn't improve like it used to.
link |
And it used to be back when you got a,
link |
you know, a Pentium 66 or something
link |
and the year before you had a Pentium 33
link |
and now it's twice as fast, right?
link |
Well, it was twice as fast at doing exactly the same thing.
link |
Okay, like literally the same program ran twice as fast.
link |
You just wrote a check and waited a year, year and a half.
link |
Well, so that's what a lot of people think about Moore's law
link |
and I think that is dead.
link |
And so what we're seeing instead is we're pushing,
link |
we're pushing people to write software in different ways.
link |
And so we're pushing people to write CUDA
link |
so they can get GPU compute
link |
and the thousands of cores on GPU.
link |
We're talking about C programmers having to use P threads
link |
because they now have, you know,
link |
a hundred threads or 50 cores in a machine
link |
or something like that.
link |
You're now talking about machine learning accelerators
link |
that are now domain specific.
link |
And when you look at these kinds of use cases,
link |
you can still get performance
link |
and Jim will come up with cool things
link |
that utilize the silicon in new ways for sure,
link |
but you're also gonna change the programming model.
link |
And now when you start talking about changing
link |
the programming model,
link |
that's when you come back to languages
link |
and things like this too,
link |
because often what you see is like you take
link |
the C programming language, right?
link |
The C programming language is designed for CPUs.
link |
And so if you want to talk to a GPU,
link |
now you're talking to its cousin CUDA, okay?
link |
CUDA is a different thing with a different set of tools,
link |
a different world, a different way of thinking.
link |
And we don't have one world that scales.
link |
And I think that we can get there.
link |
We can have one world that scales in a much better way.
link |
And a small tangent then,
link |
I think most programming languages are designed for CPUs,
link |
for single core, even just in their spirit,
link |
even if they allow for parallelization.
link |
So what does it look like for a programming language
link |
to have parallelization or massive parallelization
link |
as it's like first principle?
link |
So the canonical example of this
link |
is the hardware design world.
link |
So Verilog, VHDL, these kinds of languages,
link |
they're what's called a high level synthesis language.
link |
This is the thing people design chips in.
link |
And when you're designing a chip,
link |
it's kind of like a brain where you have infinite parallelism.
link |
Like you're like laying down transistors.
link |
Transistors are always running, okay?
link |
And so you're not saying run this transistor,
link |
then this transistor, then this transistor.
link |
It's like your brain,
link |
like your neurons are always just doing something.
link |
They're not clocked, right?
link |
They're just doing their thing.
link |
And so when you design a chip or when you design a CPU,
link |
when you design a GPU, when you design,
link |
when you're laying down the transistors,
link |
similarly, you're talking about,
link |
well, okay, well, how do these things communicate?
link |
And so these languages exist.
link |
Verilog is a kind of mixed example of that.
link |
None of these languages are really great.
link |
You have a very low level, yeah.
link |
Yeah, they're very low level
link |
and abstraction is necessary here.
link |
And there's different approaches with that.
link |
And it's itself a very complicated world,
link |
but it's implicitly parallel.
link |
And so having that as the domain that you program towards
link |
makes it so that by default, you get parallel systems.
link |
If you look at CUDA,
link |
CUDA is a point halfway in the space where in CUDA,
link |
when you write a CUDA kernel for your GPU,
link |
it feels like you're writing a scalar program.
link |
So you're like, you have ifs, you have for loops,
link |
stuff like this, you're just writing normal code.
link |
But what happens outside of that in your driver
link |
is that it actually is running you
link |
on like a thousand things at once, right?
link |
And so it's parallel,
link |
but it has pulled it out of the programming model.
link |
And so now you as a programmer are working in a simpler world
link |
and it's solved that for you, right?
link |
How do you take the language like Swift?
link |
If we think about GPUs, but also ASICs,
link |
maybe if we can dance back and forth
link |
between hardware and software.
link |
How do you design for these features
link |
to be able to program and get a first class citizen
link |
to be able to do like Swift for TensorFlow
link |
to be able to do machine learning on current hardware,
link |
but also future hardware like TPUs
link |
and all kinds of ASICs
link |
that I'm sure will be popping up more and more.
link |
Yeah, well, so a lot of this comes down
link |
to this whole idea of having the nuts and bolts
link |
underneath the covers that work really well.
link |
So you need, if you're talking to TPUs,
link |
you need MLIR or XLA or one of these compilers
link |
that talks to TPUs to build on top of, okay?
link |
And if you're talking to circuits,
link |
you need to figure out how to lay down the transistors
link |
and how to organize it and how to set up clocking
link |
and like all the domain problems
link |
that you get with circuits.
link |
Then you have to decide how to explain it to a human.
link |
What is ZY, right?
link |
And if you do it right, that's a library problem,
link |
not a language problem.
link |
And that works if you have a library or a language
link |
which allows your library to write things
link |
that feel native in the language by implementing libraries,
link |
because then you can innovate in programming models
link |
without having to change your syntax again.
link |
Like you have to invent new code formatting tools
link |
and like all the other things that languages come with.
link |
And this gets really interesting.
link |
And so if you look at the space,
link |
the interesting thing once you separate out syntax
link |
becomes what is that programming model?
link |
And so do you want the CUDA style?
link |
I write one program and it runs many places.
link |
Do you want the implicitly parallel model?
link |
How do you reason about that?
link |
How do you give developers, chip architects,
link |
the ability to express their intent?
link |
And that comes into this whole design question
link |
of how do you detect bugs quickly?
link |
So you don't have to tape out a chip
link |
to find out it's wrong, ideally, right?
link |
How do you, and this is a spectrum,
link |
how do you make it so that people feel productive?
link |
So their turnaround time is very quick.
link |
All these things are really hard problems.
link |
And in this world, I think that not a lot of effort
link |
has been put into that design problem
link |
and thinking about the layering in other pieces.
link |
Well, you've, on the topic of concurrency,
link |
you've written the Swift concurrency manifest.
link |
I think it's kind of interesting.
link |
Anything that has the word manifest on it
link |
is very interesting.
link |
Can you summarize the key ideas of each of the five parts
link |
you've written about?
link |
So what is a manifesto?
link |
How about, we start there.
link |
So in the Swift community, we have this problem,
link |
which is on the one hand,
link |
you wanna have relatively small proposals
link |
that you can kind of fit in your head,
link |
you can understand the details at a very fine grain level
link |
that move the world forward.
link |
But then you also have these big arcs, okay?
link |
And often when you're working on something
link |
that is a big arc, but you're tackling it in small pieces,
link |
you have this question of,
link |
how do I know I'm not doing a random walk?
link |
Where are we going?
link |
How does this add up?
link |
Furthermore, when you start the first small step,
link |
what terminology do you use?
link |
How do we think about it?
link |
What is better and worse in the space?
link |
What are the principles?
link |
What are we trying to achieve?
link |
And so what a manifesto in the Swift community does
link |
is it starts to say,
link |
hey, well, let's step back from the details of everything.
link |
Let's paint a broad picture to talk about
link |
what we're trying to achieve.
link |
Let's give an example design point.
link |
Let's try to paint the big picture
link |
so that then we can zero in on the individual steps
link |
and make sure that we're making good progress.
link |
And so the Swift concurrency manifesto
link |
is something I wrote three years ago.
link |
It's been a while, maybe more.
link |
Trying to do that for Swift and concurrency.
link |
It starts with some fairly simple things
link |
like making the observation that
link |
when you have multiple different computers
link |
and multiple different threads that are communicating,
link |
it's best for them to be asynchronous.
link |
And so you need things to be able to run separately
link |
and then communicate with each other.
link |
And this means asynchrony.
link |
And this means that you need a way
link |
to modeling asynchronous communication.
link |
Many languages have features like this.
link |
Async await is a popular one.
link |
And so that's what I think is very likely in Swift.
link |
But as you start building this tower of abstractions,
link |
it's not just about how do you write this,
link |
you then reach into the how do you get memory safety
link |
because you want correctness,
link |
you want debuggability and sanity for developers.
link |
And how do you get that memory safety into the language?
link |
So if you take a language like Go or C
link |
or any of these languages,
link |
you get what's called a race condition
link |
when two different threads or Go routines or whatever
link |
touch the same point in memory, right?
link |
This is a huge like maddening problem to debug
link |
because it's not reproducible generally.
link |
And so there's tools,
link |
there's a whole ecosystem of solutions
link |
that built up around this,
link |
but it's a huge problem
link |
when you're writing concurrent code.
link |
And so with Swift,
link |
this whole value semantics thing is really powerful there
link |
because it turns out that math and copies actually work
link |
even in concurrent worlds.
link |
And so you get a lot of safety just out of the box,
link |
but there are also some hard problems.
link |
And it talks about some of that.
link |
When you start building up to the next level up
link |
and you start talking beyond memory safety,
link |
you have to talk about what is the programmer model?
link |
How does a human think about this?
link |
So a developer that's trying to build a program
link |
and it proposes a really old model with a new spin
link |
Actors are about saying,
link |
we have islands of single threadedness logically.
link |
So you write something that feels like
link |
it's one program running in a unit,
link |
and then it communicates asynchronously with other things.
link |
And so making that expressive and natural feel good
link |
be the first thing you reach for and being safe by default
link |
is a big part of the design of that proposal.
link |
When you start going beyond that,
link |
now you start to say, cool,
link |
well, these things that communicate asynchronously,
link |
they don't have to share memory.
link |
Well, if they don't have to share memory
link |
and they're sending messages to each other,
link |
why do they have to be in the same process?
link |
These things should be able to be in different processes
link |
And why just processes?
link |
Well, why not different machines?
link |
And so now you have a very nice gradual transition
link |
towards distributed programming.
link |
And of course, when you start talking about the big future,
link |
the manifesto doesn't go into it,
link |
but accelerators are things you talk to asynchronously
link |
by sending messages to them.
link |
And how do you program those?
link |
Well, that gets very interesting.
link |
That's not in the proposal.
link |
And how much do you wanna make that explicit
link |
like the control of that whole process
link |
explicit to the program?
link |
Yeah, good question.
link |
So when you're designing any of these kinds of features
link |
or language features or even libraries,
link |
you have this really hard trade off you have to make,
link |
which is how much is it magic
link |
or how much is it in the human's control?
link |
How much can they predict and control it?
link |
What do you do when the default case is the wrong case?
link |
And so when you're designing a system,
link |
and so when you're designing a system, I won't name names,
link |
but there are systems where it's really easy to get started
link |
and then you jump.
link |
So let's pick like logo.
link |
Okay, so something like this.
link |
So it's really easy to get started.
link |
It's really designed for teaching kids,
link |
but as you get into it, you hit a ceiling
link |
and then you can't go any higher.
link |
And then what do you do?
link |
Well, you have to go switch to a different world
link |
and rewrite all your code.
link |
And this logo is a silly example here.
link |
This exists in many other languages.
link |
With Python, you would say like concurrency, right?
link |
So Python has the global interpreter block.
link |
So threading is challenging in Python.
link |
And so if you start writing a large scale application
link |
in Python, and then suddenly you need concurrency,
link |
you're kind of stuck with a series of bad trade offs, right?
link |
There's other ways to go where you say like,
link |
foist all the complexity on the user all at once, right?
link |
And that's also bad in a different way.
link |
And so what I prefer is building a simple model
link |
that you can explain that then has an escape hatch.
link |
So you get in, you have guardrails,
link |
memory safety works like this in Swift,
link |
where you can start with, like by default,
link |
if you use all the standard things, it's memory safe,
link |
you're not gonna shoot your foot off.
link |
But if you wanna get a C level pointer to something,
link |
you can explicitly do that.
link |
But by default, there's guardrails.
link |
There's guardrails.
link |
Okay, so but like, whose job is it to figure out
link |
which part of the code is parallelizable?
link |
So in the case of the proposal, it is the human's job.
link |
So they decide how to architect their application.
link |
And then the runtime in the compiler is very predictable.
link |
And so this is in contrast to like,
link |
there's a long body of work, including on Fortran
link |
for auto parallelizing compilers.
link |
And this is an example of a bad thing in my,
link |
so as a compiler person, I can drag on compiler people.
link |
Often compiler people will say,
link |
cool, since I can't change the code,
link |
I'm gonna write my compiler that then takes
link |
this unmodified code and makes go way faster on this machine.
link |
Okay, application, and so it does pattern matching.
link |
It does like really deep analysis.
link |
Compiler people are really smart.
link |
And so they like wanna like do something
link |
really clever and tricky.
link |
And you get like 10X speed up by taking
link |
like an array of structures and turn it
link |
into a structure of arrays or something,
link |
because it's so much better for memory.
link |
Like there's bodies, like tons of tricks.
link |
They love optimization.
link |
Yeah, you love optimization.
link |
Everyone loves optimization.
link |
Everyone loves it.
link |
Well, and it's this promise of build with my compiler
link |
and your thing goes fast, right?
link |
But here's the problem, Lex, you write a program,
link |
you run it with my compiler, it goes fast.
link |
You're very happy.
link |
Wow, it's so much faster than the other compiler.
link |
Then you go and you add a feature to your program
link |
or you refactor some code.
link |
And suddenly you got a 10X loss in performance.
link |
What just happened there?
link |
What just happened there is the heuristic,
link |
the pattern matching, the compiler,
link |
whatever analysis it was doing just got defeated
link |
because you didn't inline a function or something, right?
link |
As a user, you don't know, you don't wanna know.
link |
That was the whole point.
link |
You don't wanna know how the compiler works.
link |
You don't wanna know how the memory hierarchy works.
link |
You don't wanna know how it got parallelized
link |
across all these things.
link |
You wanted that abstracted away from you,
link |
but then the magic is lost as soon as you did something
link |
and you fall off a performance cliff.
link |
And now you're in this funny position
link |
where what do I do?
link |
I don't change my code.
link |
I don't fix that bug.
link |
It costs 10X performance.
link |
Well, this is the problem with unpredictable performance.
link |
If you care about performance,
link |
predictability is a very important thing.
link |
And so what the proposal does is it provides
link |
architectural patterns for being able to lay out your code,
link |
gives you full control over that,
link |
makes it really simple so you can explain it.
link |
And then if you wanna scale out in different ways,
link |
you have full control over that.
link |
So in your sense, the intuition is for a compiler,
link |
it's too hard to do automated parallelization.
link |
Cause the compilers do stuff automatically
link |
that's incredibly impressive for other things,
link |
but for parallelization, we're not close to there.
link |
Well, it depends on the programming model.
link |
So there's many different kinds of compilers.
link |
And so if you talk about like a C compiler
link |
or Swift compiler or something like that,
link |
where you're writing imperative code,
link |
parallelizing that and reasoning about all the pointers
link |
and stuff like that is a very difficult problem.
link |
Now, if you switch domains,
link |
so there's this cool thing called machine learning, right?
link |
So machine learning nerds among other endearing things
link |
like solving cat detectors and other things like that
link |
have done this amazing breakthrough
link |
of producing a programming model,
link |
operations that you compose together
link |
that has raised levels of abstraction high enough
link |
that suddenly you can have auto parallelizing compilers.
link |
You can write a model using a TensorFlow
link |
and have it run on 1024 nodes of a TPU.
link |
Yeah, that's true.
link |
I didn't even think about like,
link |
cause there's so much flexibility
link |
in the design of architectures that ultimately boil down
link |
to a graph that's parallelized for you.
link |
And if you think about it, that's pretty cool.
link |
That's pretty cool, yeah.
link |
And you think about batching, for example,
link |
as a way of being able to exploit more parallelism.
link |
Like that's a very simple thing that now is very powerful.
link |
That didn't come out of the programming language nerds,
link |
those people, like that came out of people
link |
that are just looking to solve a problem
link |
and use a few GPUs and organically developed
link |
by the community of people focusing on machine learning.
link |
And it's an incredibly powerful abstraction layer
link |
that enables the compiler people to go and exploit that.
link |
And now you can drive supercomputers from Python.
link |
Well, that's pretty cool.
link |
So just to pause on that,
link |
cause I'm not sufficiently low level,
link |
I forget to admire the beauty and power of that,
link |
but maybe just to linger on it,
link |
like what does it take to run a neural network fast?
link |
Like how hard is that compilation?
link |
So we just skipped,
link |
you said like, it's amazing that that's a thing,
link |
but yeah, how hard is that of a thing?
link |
It's hard and I would say that not all of the systems
link |
are really great, including the ones I helped build.
link |
So there's a lot of work left to be done there.
link |
Is it the compiler nerds working on that
link |
or is it a whole new group of people?
link |
Well, it's a full stack problem,
link |
including compiler people, including APIs,
link |
so like Keras and the module API and PyTorch and Jax.
link |
And there's a bunch of people pushing
link |
on all the different parts of these things,
link |
because when you look at it as it's both,
link |
how do I express the computation?
link |
Do I stack up layers?
link |
Well, cool, like setting up a linear sequence of layers
link |
is great for the simple case,
link |
but how do I do the hard case?
link |
How do I do reinforcement learning?
link |
Well, now I need to integrate my application logic in this.
link |
Then it's the next level down of,
link |
how do you represent that for the runtime?
link |
How do you get hardware abstraction?
link |
And then you get to the next level down of saying like,
link |
forget about abstraction,
link |
how do I get the peak performance out of my TPU
link |
or my iPhone accelerator or whatever, right?
link |
And all these different things.
link |
And so this is a layered problem
link |
with a lot of really interesting design and work
link |
going on in the space
link |
and a lot of really smart people working on it.
link |
Machine learning is a very well funded area
link |
of investment right now.
link |
And so there's a lot of progress being made.
link |
So how much innovation is there on the lower level,
link |
so closer to the ASIC,
link |
so redesigning the hardware
link |
or redesigning concurrently compilers with that hardware?
link |
Is that like, if you were to predict the biggest,
link |
the equivalent of Moore's law improvements
link |
in the inference and the training of neural networks
link |
and just all of that,
link |
where is that gonna come from, you think?
link |
Sure, you get scalability of different things.
link |
And so you get Jim Keller shrinking process technology,
link |
you get three nanometer instead of five or seven or 10
link |
or 28 or whatever.
link |
And so that marches forward and that provides improvements.
link |
You get architectural level performance.
link |
And so the TPU with a matrix multiply unit
link |
and a systolic array is much more efficient
link |
than having a scalar core doing multiplies and adds
link |
and things like that.
link |
You then get system level improvements.
link |
So how you talk to memory,
link |
how you talk across a cluster of machines,
link |
how you scale out,
link |
how you have fast interconnects between machines.
link |
You then get system level programming models.
link |
So now that you have all this hardware, how to utilize it.
link |
You then have algorithmic breakthroughs where you say,
link |
Instead of training in a resonant 50 in a week,
link |
I'm now training it in 25 seconds.
link |
And it's a combination of new optimizers
link |
and new just training regimens
link |
and different approaches to train.
link |
And all of these things come together
link |
to push the world forward.
link |
That was a beautiful exposition.
link |
But if you were to force to bet all your money
link |
Why do we have to?
link |
Unfortunately, we have people working on all this.
link |
It's an exciting time, right?
link |
So, I mean, OpenAI did this little paper
link |
showing the algorithmic improvement you can get.
link |
It's been improving exponentially.
link |
I haven't quite seen the same kind of analysis
link |
on other layers of the stack.
link |
I'm sure it's also improving significantly.
link |
I just, it's a nice intuition builder.
link |
I mean, there's a reason why Moore's Law,
link |
that's the beauty of Moore's Law is
link |
somebody writes a paper that makes a ridiculous prediction.
link |
And it becomes reality in a sense.
link |
There's something about these narratives
link |
when you, when Chris Ladner on a silly little podcast
link |
makes, bets all his money on a particular thing,
link |
somehow it can have a ripple effect
link |
of actually becoming real.
link |
That's an interesting aspect of it.
link |
Cause like it might've been,
link |
we focus with Moore's Law,
link |
most of the computing industry
link |
really, really focused on the hardware.
link |
I mean, software innovation,
link |
I don't know how much software innovation
link |
there was in terms of efficient.
link |
What Intel Giveth Bill takes away, right?
link |
Yeah, I mean, compilers improved significantly also.
link |
So actually, I mean, so I'm joking
link |
about how software has gotten slower
link |
pretty much as fast as hardware got better,
link |
at least through the nineties.
link |
There's another joke, another law in compilers,
link |
which is called, I think it's called Probstine's Law,
link |
which is compilers double the performance
link |
of any given code every 18 years.
link |
So they move slowly.
link |
Well, yeah, it's exponential also.
link |
Yeah, you're making progress,
link |
but there again, it's not about,
link |
the power of compilers is not just about
link |
how do you make the same thing go faster?
link |
It's how do you unlock the new hardware?
link |
A new chip came out, how do you utilize it?
link |
You say, oh, the programming model,
link |
how do we make people more productive?
link |
How do we have better error messages?
link |
Even such mundane things like how do I generate
link |
a very specific error message about your code
link |
actually makes people happy
link |
because then they know how to fix it, right?
link |
And it comes back to how do you help people
link |
get their job done.
link |
Yeah, and yeah, and then in this world
link |
of exponentially increasing smart toasters,
link |
how do you expand computing to all these kinds of devices?
link |
Do you see this world where just everything
link |
is a computing surface?
link |
You see that possibility?
link |
Just everything is a computer?
link |
Yeah, I don't see any reason
link |
that that couldn't be achieved.
link |
It turns out that sand goes into glass
link |
and glass is pretty useful too.
link |
So very important question then,
link |
if we're living in a simulation
link |
and the simulation is running a computer,
link |
like what's the architecture of that computer, do you think?
link |
So you're saying is it a quantum system?
link |
Yeah, like this whole quantum discussion, is it needed?
link |
Or can we run it with a RISC 5 architecture,
link |
I think it comes down to the right tool for the job.
link |
And what's the compiler?
link |
Yeah, exactly, that's my question.
link |
Did I get that job?
link |
Feed the universe compiler.
link |
And so there, as far as we know,
link |
quantum systems are the bottom of the pile of turtles
link |
And so we don't know efficient ways
link |
to implement quantum systems without using quantum computers.
link |
Yeah, and that's totally outside
link |
of everything we've talked about.
link |
But who runs that quantum computer?
link |
Right, so if we really are living in a simulation,
link |
then is it bigger quantum computers?
link |
Is it different ones?
link |
Like how does that work out?
link |
How does that scale?
link |
Well, it's the same size.
link |
It's the same size.
link |
But then the thought of the simulation
link |
is that you don't have to run the whole thing,
link |
that we humans are cognitively very limited.
link |
We do checkpoints.
link |
We do checkpoints, yeah.
link |
And if we, the point at which we human,
link |
so you basically do minimal amount of,
link |
what is it, Swift does on right, copy on right.
link |
So you only adjust the simulation.
link |
Parallel universe theories, right?
link |
And so every time a decision's made,
link |
somebody opens the short end of your box,
link |
then there's a fork.
link |
And then this could happen.
link |
And then, thank you for considering the possibility.
link |
But yeah, so it may not require the entirety
link |
of the universe to simulate it.
link |
But it's interesting to think about
link |
as we create this higher and higher fidelity systems.
link |
But I do wanna ask on the quantum computer side,
link |
because everything we've talked about,
link |
whether you work with SciFive, with compilers,
link |
none of that includes quantum computers, right?
link |
So have you ever thought about this whole
link |
serious engineering work of quantum computers
link |
looks like of compilers, of architectures,
link |
all of that kind of stuff?
link |
So I've looked at it a little bit.
link |
I know almost nothing about it,
link |
which means that at some point,
link |
I will have to find an excuse to get involved,
link |
because that's how it works.
link |
But do you think that's a thing to be,
link |
like with your little tingly senses of the timing
link |
of when to be involved, is it not yet?
link |
Well, so the thing I do really well
link |
is I jump into messy systems
link |
and figure out how to make them,
link |
figure out what the truth in the situation is,
link |
try to figure out what the unifying theory is,
link |
how to like factor the complexity,
link |
how to find a beautiful answer to a problem
link |
that has been well studied
link |
and lots of people have bashed their heads against it.
link |
I don't know that quantum computers are mature enough
link |
and accessible enough to be figured out yet, right?
link |
And I think the open question with quantum computers is,
link |
is there a useful problem
link |
that gets solved with a quantum computer
link |
that makes it worth the economic cost
link |
of like having one of these things
link |
and having legions of people that set it up?
link |
You go back to the fifties, right?
link |
And there's the projections
link |
of the world will only need seven computers, right?
link |
Well, and part of that was that people hadn't figured out
link |
what they're useful for.
link |
What are the algorithms we wanna run?
link |
What are the problems that get solved?
link |
And this comes back to how do we make the world better,
link |
either economically or making somebody's life better
link |
or like solving a problem that wasn't solved before,
link |
And I think that just we're a little bit too early
link |
in that development cycle
link |
because it's still like literally a science project,
link |
not a negative connotation, right?
link |
It's literally a science project
link |
and the progress there is amazing.
link |
And so I don't know if it's 10 years away,
link |
if it's two years away,
link |
exactly where that breakthrough happens,
link |
but you look at machine learning,
link |
we went through a few winners
link |
before the AlexNet transition
link |
and then suddenly it had its breakout moment.
link |
And that was the catalyst
link |
that then drove the talent flocking into it.
link |
That's what drove the economic applications of it.
link |
That's what drove the technology to go faster
link |
because you now have more minds thrown at the problem.
link |
This is what caused like a serious knee in deep learning
link |
and the algorithms that we're using.
link |
And so I think that's what quantum needs to go through.
link |
And so right now it's in that formidable finding itself,
link |
getting the like literally the physics figured out.
link |
And then it has to figure out the application
link |
that makes this useful.
link |
Yeah, but I'm not skeptical that I think that will happen.
link |
I think it's just 10 years away, something like that.
link |
what programming language do you think
link |
the simulation is written in?
link |
Ooh, probably Lisp.
link |
Like if you're a Tibet, I'll just leave it at that.
link |
So, I mean, we've mentioned that you worked
link |
with all these companies,
link |
we've talked about all these projects.
link |
It's kind of like if we just step back and zoom out
link |
about the way you did that work.
link |
And we look at COVID times,
link |
this pandemic we're living through that may,
link |
if I look at the way Silicon Valley folks
link |
are talking about it, the way MIT is talking about it,
link |
this might last for a long time.
link |
Not just the virus, but the remote nature.
link |
The economic impact.
link |
I mean, all of it, yeah.
link |
Yeah, it's gonna be a mess.
link |
Do you think, what's your prediction?
link |
I mean, from sci fi to Google,
link |
to just all the places you worked in,
link |
just Silicon Valley, you're in the middle of it.
link |
What do you think is,
link |
how is this whole place gonna change?
link |
Yeah, so, I mean, I really can only speak
link |
to the tech perspective.
link |
I am in that bubble.
link |
I think it's gonna be really interesting
link |
because the Zoom culture of being remote
link |
and on video chat all the time
link |
has really interesting effects on people.
link |
So on the one hand, it's a great normalizer.
link |
It's a normalizer that I think will help communities
link |
of people that have traditionally been underrepresented
link |
because now you're taking, in some cases, a face off
link |
because you don't have to have a camera going, right?
link |
And so you can have conversations
link |
without physical appearance being part of the dynamic,
link |
which is pretty powerful.
link |
You're taking remote employees
link |
that have already been remote,
link |
and you're saying you're now on the same level
link |
and footing as everybody else.
link |
Nobody gets whiteboards.
link |
You're not gonna be the one person
link |
that doesn't get to be participating
link |
in the whiteboard conversation,
link |
and that's pretty powerful.
link |
You've got, you're forcing people to think asynchronously
link |
in some cases because it's harder to just get people
link |
physically together, and the bumping into each other
link |
forces people to find new ways to solve those problems.
link |
And I think that that leads to more inclusive behavior,
link |
On the other hand, it's also, it just sucks, right?
link |
And so the actual communication just sucks
link |
being not with people on a daily basis
link |
and collaborating with them.
link |
Yeah, all of that, right?
link |
I mean, everything, this whole situation is terrible.
link |
What I meant primarily was the,
link |
I think that most humans
link |
like working physically with humans.
link |
I think this is something that not everybody,
link |
but many people are programmed to do.
link |
And I think that we get something out of that
link |
that is very hard to express, at least for me.
link |
And so maybe this isn't true of everybody.
link |
But, and so the question to me is,
link |
when you get through that time of adaptation,
link |
you get out of March and April,
link |
and you get into December,
link |
you get into next March, if it's not changed, right?
link |
It's already terrifying.
link |
Well, you think about that,
link |
and you think about what is the nature of work?
link |
And how do we adapt?
link |
And humans are very adaptable species, right?
link |
We can learn things when we're forced to,
link |
and there's a catalyst to make that happen.
link |
And so what is it that comes out of this,
link |
and are we better or worse off?
link |
I think that you look at the Bay Area,
link |
housing prices are insane.
link |
Well, there's a high incentive to be physically located,
link |
because if you don't have proximity,
link |
you end up paying for it and commute, right?
link |
And there has been huge social pressure
link |
in terms of you will be there for the meeting, right?
link |
Or whatever scenario it is.
link |
And I think that's gonna be way better.
link |
I think it's gonna be much more the norm
link |
to have remote employees,
link |
and I think this is gonna be really great.
link |
Do you have friends, or do you hear of people moving?
link |
I know one family friend that moved.
link |
They moved back to Michigan,
link |
and they were a family with three kids
link |
living in a small apartment,
link |
and we're going insane, right?
link |
And they're in tech, husband works for Google.
link |
So first of all, friends of mine
link |
are in the process of, or have already lost the business.
link |
The thing that represents their passion, their dream,
link |
it could be small entrepreneurial projects,
link |
but it could be large businesses,
link |
like people that run gyms.
link |
Restaurants, tons of things, yeah.
link |
But also, people look at themselves in the mirror
link |
and ask the question of, what do I wanna do in life?
link |
For some reason, they haven't done it until COVID.
link |
They really ask that question,
link |
and that results often in moving or leaving the company
link |
with starting your own business
link |
or transitioning to a different company.
link |
Do you think we're gonna see that a lot?
link |
Well, I can't speak to that.
link |
I mean, we're definitely gonna see it
link |
at a higher frequency than we did before,
link |
just because I think what you're trying to say
link |
is there are decisions that you make yourself,
link |
big life decisions that you make yourself,
link |
and I'm gonna quit my job and start a new thing.
link |
There's also decisions that get made for you.
link |
I got fired from my job, what am I gonna do, right?
link |
And that's not a decision that you think about,
link |
but you're forced to act, okay?
link |
And so I think that those, you're forced to act
link |
kind of moments where global pandemic
link |
comes and wipes out the economy,
link |
and now your business doesn't exist.
link |
I think that does lead to more reflection, right?
link |
Because you're less anchored on what you have,
link |
and it's not a, what do I have to lose
link |
versus what do I have to gain, A, B, comparison.
link |
It's more of a fresh slate.
link |
Cool, I could do anything now.
link |
Do I wanna do the same thing I was doing?
link |
Did that make me happy?
link |
Is this now time to go back to college
link |
and take a class and learn a new skill?
link |
Is this a time to spend time with family
link |
if you can afford to do that?
link |
Is this time to literally move in with parents, right?
link |
I mean, all these things that were not normative before
link |
suddenly become, I think, very, the value systems change.
link |
And I think that's actually a good thing
link |
in the short term, at least, because it leads to,
link |
there's kind of been an overoptimization
link |
along one set of priorities for the world,
link |
and now maybe we'll get to a more balanced
link |
and more interesting world
link |
where people are doing different things.
link |
I think it could be good.
link |
I think there could be more innovation
link |
that comes out of it, for example.
link |
What do you think about all the social chaos
link |
we're in the middle of?
link |
You think it's, let me ask you a whole,
link |
you think it's all gonna be okay?
link |
Well, I think humanity will survive.
link |
The, from an existential,
link |
like we're not all gonna kill, yeah, well.
link |
Yeah, I don't think the virus is gonna kill all the humans.
link |
I don't think all the humans are gonna kill all the humans.
link |
I think that's unlikely.
link |
But I look at it as progress requires a catalyst, right?
link |
So you need a reason for people to be willing
link |
to do things that are uncomfortable.
link |
I think that the US, at least,
link |
but I think the world in general
link |
is a pretty unoptimal place to live in for a lot of people.
link |
And I think that what we're seeing right now
link |
is we're seeing a lot of unhappiness.
link |
And because of all the pressure,
link |
because of all the badness in the world
link |
that's coming together,
link |
it's really kind of igniting some of that debate
link |
that should have happened a long time ago, right?
link |
I mean, I think that we'll see more progress.
link |
You're asking about, offline you're asking about politics
link |
and wouldn't it be great if politics moved faster
link |
because there's all these problems in the world
link |
and we can move it.
link |
Well, people are intentionally, are inherently conservative.
link |
And so if you're talking about conservative people,
link |
particularly if they have heavy burdens on their shoulders
link |
because they represent literally thousands of people,
link |
it makes sense to be conservative.
link |
But on the other hand, when you need change,
link |
how do you get it?
link |
The global pandemic will probably lead to some change.
link |
And it's not a directed, it's not a directed plan,
link |
but I think that it leads to people
link |
asking really interesting questions.
link |
And some of those questions
link |
should have been asked a long time ago.
link |
Well, let me know if you've observed this as well.
link |
Something that's bothered me in the machine learning
link |
community, I'm guessing it might be prevalent
link |
in other places, is something that feels like in 2020
link |
increase the level of toxicity.
link |
Like people are just quicker to pile on,
link |
to just be, they're just harsh on each other,
link |
to like mob, pick a person that screwed up
link |
and like make it a big thing.
link |
And is there something that we can like,
link |
yeah, have you observed that in other places?
link |
Is there some way out of this?
link |
I think there's an inherent thing in humanity
link |
that's kind of an us versus them thing,
link |
which is that you wanna succeed and how do you succeed?
link |
Well, it's relative to somebody else.
link |
And so what's happening in, at least in some part
link |
is that with the internet and with online communication,
link |
the world's getting smaller, right?
link |
And so we're having some of the social ties
link |
of like my town versus your town's football team, right?
link |
Turn into much larger and yet shallower problems.
link |
And people don't have time, the incentives,
link |
the clickbait and like all these things
link |
kind of really feed into this machine.
link |
And I don't know where that goes.
link |
Yeah, I mean, the reason I think about that,
link |
I mentioned to you this offline a little bit,
link |
but I have a few difficult conversations scheduled,
link |
some of them political related,
link |
some of them within the community,
link |
difficult personalities that went through some stuff.
link |
I mean, one of them I've talked before,
link |
I will talk again is Yann LeCun.
link |
He got a little bit of crap on Twitter
link |
for talking about a particular paper
link |
and the bias within a data set.
link |
And then there's been a huge, in my view,
link |
and I'm willing, comfortable saying it,
link |
irrational, over exaggerated pile on his comments
link |
because he made pretty basic comments about the fact that
link |
if there's bias in the data,
link |
there's going to be bias in the results.
link |
So we should not have bias in the data,
link |
but people piled on to him because he said
link |
he trivialized the problem of bias.
link |
Like it's a lot more than just bias in the data,
link |
but like, yes, that's a very good point,
link |
but that's not what he was saying.
link |
That's not what he was saying.
link |
And the response, like the implied response
link |
that he's basically sexist and racist
link |
is something that completely drives away
link |
the possibility of nuanced discussion.
link |
One nice thing about like a pocket long form of conversation
link |
is you can talk it out.
link |
You can lay your reasoning out.
link |
And even if you're wrong,
link |
you can still show that you're a good human being
link |
You know, your point about
link |
you can't have a productive discussion.
link |
Well, how do you get to the point where people can turn?
link |
They can learn, they can listen, they can think,
link |
they can engage versus just being a shallow like,
link |
and then keep moving, right?
link |
And I don't think that progress really comes from that,
link |
And I don't think that one should expect that.
link |
I think that you'd see that as reinforcing
link |
individual circles and the us versus them thing.
link |
And I think that's fairly divisive.
link |
Yeah, I think there's a big role in,
link |
like the people that bother me most on Twitter
link |
when I observe things is not the people
link |
who get very emotional, angry, like over the top.
link |
It's the people who like prop them up.
link |
It's all the, it's this,
link |
I think what should be the,
link |
we should teach each other is to be sort of empathetic.
link |
The thing that it's really easy to forget,
link |
particularly on like Twitter or the internet or an email,
link |
is that sometimes people just have a bad day, right?
link |
You have a bad day or you're like,
link |
I've been in the situation where it's like between meetings,
link |
like fire off a quick response to an email
link |
because I want to like help get something unblocked,
link |
phrase it really objectively wrong.
link |
And suddenly this is now something that sticks with people.
link |
And it's not because they're bad.
link |
It's not because you're bad.
link |
Just psychology of like, you said a thing,
link |
it sticks with you.
link |
You didn't mean it that way,
link |
but it really impacted somebody
link |
because the way they interpret it.
link |
And this is just an aspect of working together as humans.
link |
And I have a lot of optimism in the long term,
link |
the very long term about what we as humanity can do.
link |
But I think that's going to be,
link |
it's just always a rough ride.
link |
And you came into this by saying like,
link |
what does COVID and all the social strife
link |
that's happening right now mean?
link |
And I think that it's really bad in the short term,
link |
but I think it'll lead to progress.
link |
And for that, I'm very thankful.
link |
Yeah, painful in the short term though.
link |
I mean, people are out of jobs.
link |
Like some people can't eat.
link |
Like it's horrible.
link |
And, but you know, it's progress.
link |
So we'll see what happens.
link |
I mean, the real question is when you look back 10 years,
link |
20 years, a hundred years from now,
link |
how do we evaluate the decisions are being made right now?
link |
I think that's really the way you can frame that
link |
And you say, you know,
link |
you integrate across all the short term horribleness
link |
that's happening and you look at what that means
link |
and is the improvement across the world
link |
or the regression across the world significant enough
link |
to make it a good or a bad thing?
link |
I think that's the question.
link |
And for that, it's good to study history.
link |
I mean, one of the big problems for me right now
link |
is I'm reading The Rise and Fall of the Third Reich.
link |
So it's everything is just,
link |
I just see parallels and it means it's,
link |
you have to be really careful not to overstep it,
link |
but just the thing that worries me the most
link |
is the pain that people feel when a few things combine,
link |
which is like economic depression,
link |
which is quite possible in this country.
link |
And then just being disrespected in some kind of way,
link |
which the German people were really disrespected
link |
by most of the world, like in a way that's over the top,
link |
that something can build up
link |
and then all you need is a charismatic leader
link |
to go either positive or negative and both work
link |
as long as they're charismatic.
link |
It's taking advantage of, again,
link |
that inflection point that the world's in
link |
and what they do with it could be good or bad.
link |
And so it's a good way to think about times now,
link |
like on an individual level,
link |
what we decide to do is when history is written,
link |
30 years from now, what happened in 2020,
link |
probably history is gonna remember 2020.
link |
Either for good or bad,
link |
and it's up to us to write it so it's good.
link |
Well, one of the things I've observed
link |
that I find fascinating is most people act
link |
as though the world doesn't change.
link |
You make decision knowingly, right?
link |
You make a decision where you're predicting the future
link |
based on what you've seen in the recent past.
link |
And so if something's always been,
link |
it's rained every single day,
link |
then of course you expect it to rain today too, right?
link |
On the other hand, the world changes all the time.
link |
Constantly, like for better and for worse.
link |
And so the question is,
link |
if you're interested in something that's not right,
link |
what is the inflection point that led to a change?
link |
And you can look to history for this.
link |
Like what is the catalyst that led to that explosion
link |
that led to that bill that led to the,
link |
like you can kind of work your way backwards from that.
link |
And maybe if you pull together the right people
link |
and you get the right ideas together,
link |
you can actually start driving that change
link |
and doing it in a way that's productive
link |
and hurts fewer people.
link |
Yeah, like a single person,
link |
a single event can turn all of history.
link |
Absolutely, everything starts somewhere.
link |
And often it's a combination of multiple factors,
link |
but yeah, these things can be engineered.
link |
That's actually the optimistic view that.
link |
I'm a longterm optimist on pretty much everything
link |
We can look to all the negative things
link |
that humanity has, all the pettiness
link |
and all the like self servingness
link |
and the just the cruelty, right?
link |
The biases, the just humans can be very horrible.
link |
But on the other hand, we're capable of amazing things.
link |
And the progress across 100 year chunks
link |
And even across decades, we've come a long ways
link |
and there's still a long ways to go,
link |
but that doesn't mean that we've stopped.
link |
Yeah, the kind of stuff we've done in the last 100 years
link |
It's kind of scary to think what's gonna happen
link |
in this 100 years.
link |
It's scary, like exciting.
link |
Like scary in a sense that it's kind of sad
link |
that the kind of technology is gonna come out
link |
in 10, 20, 30 years.
link |
We're probably too old to really appreciate
link |
if you don't grow up with it.
link |
It'll be like kids these days with their virtual reality
link |
and their TikToks and stuff like this.
link |
Like, how does this thing and like,
link |
come on, give me my static photo.
link |
Okay, sorry, we kind of skipped over.
link |
Let me ask on, the machine learning world
link |
has been kind of inspired, their imagination captivated
link |
with GPT3 and these language models.
link |
I thought it'd be cool to get your opinion on it.
link |
What's your thoughts on this exciting world of,
link |
it connects to computation actually,
link |
is of language models that are huge
link |
and take many, many computers, not just to train,
link |
but to also do inference on.
link |
Well, I mean, it depends on what you're speaking to there.
link |
But I mean, I think that there's been
link |
a pretty well understood maximum in deep learning
link |
that if you make the model bigger
link |
and you shove more data into it,
link |
assuming you train it right
link |
and you have a good model architecture,
link |
that you'll get a better model out.
link |
And so on one hand, GPT3 was not that surprising.
link |
On the other hand, a tremendous amount of engineering
link |
went into making it possible.
link |
The implications of it are pretty huge.
link |
I think that when GPT2 came out,
link |
there was a very provocative blog post from OpenAI
link |
talking about, we're not gonna release it
link |
because of the social damage it could cause
link |
I think that's still a concern.
link |
I think that we need to look at how technology is applied
link |
and well meaning tools can be applied in very horrible ways
link |
and they can have very profound impact on that.
link |
I think that GPT3 is a huge technical achievement.
link |
And what will GPT4 be?
link |
Well, it'll probably be bigger, more expensive to train.
link |
Really cool architectural tricks.
link |
What do you think, is there,
link |
I don't know how much thought you've done
link |
on distributed computing.
link |
Is there some technical challenges that are interesting
link |
that you're hopeful about exploring
link |
in terms of a system that,
link |
like a piece of code that with GPT4 that might have,
link |
I don't know, hundreds of trillions of parameters
link |
which have to run on thousands of computers.
link |
Is there some hope that we can make that happen?
link |
Yeah, well, I mean, today you can write a check
link |
and get access to a thousand TPU cores
link |
and do really interesting large scale training
link |
and inference and things like that in Google Cloud,
link |
for example, right?
link |
And so I don't think it's a question about scale,
link |
it's a question about utility.
link |
And when I look at the transformer series of architectures
link |
that the GPT series is based on,
link |
it's really interesting to look at that
link |
because they're actually very simple designs.
link |
They're not recurrent.
link |
The training regimens are pretty simple.
link |
And so they don't really reflect like human brains, right?
link |
But they're really good at learning language models
link |
and they're unrolled enough
link |
that you can simulate some recurrence, right?
link |
And so the question I think about is,
link |
where does this take us?
link |
Like, so we can just keep scaling it,
link |
have more parameters, more data, more things,
link |
we'll get a better result for sure.
link |
But are there architectural techniques
link |
that can lead to progress at a faster pace, right?
link |
This is when, you know, how do you get,
link |
instead of just like making it a constant time bigger,
link |
how do you get like an algorithmic improvement
link |
out of this, right?
link |
And whether it be a new training regimen,
link |
if it becomes sparse networks, for example,
link |
the human brain is sparse, all these networks are dense.
link |
The connectivity patterns can be very different.
link |
I think this is where I get very interested
link |
and I'm way out of my league
link |
on the deep learning side of this.
link |
But I think that could lead to big breakthroughs.
link |
When you talk about large scale networks,
link |
one of the things that Jeff Dean likes to talk about
link |
and he's given a few talks on
link |
is this idea of having a sparsely gated mixture of experts
link |
kind of a model where you have, you know,
link |
different nets that are trained
link |
and are really good at certain kinds of tasks.
link |
And so you have this distributor across a cluster
link |
and so you have a lot of different computers
link |
that end up being kind of locally specialized
link |
in different demands.
link |
And then when a query comes in,
link |
you gate it and you use learn techniques
link |
to route to different parts of the network.
link |
And then you utilize the compute resources
link |
of the entire cluster by having specialization within it.
link |
And I don't know where that goes
link |
or when it starts to work,
link |
but I think things like that
link |
could be really interesting as well.
link |
And on the data side too,
link |
if you can think of data selection
link |
as a kind of programming.
link |
I mean, essentially, if you look at it,
link |
like Karpathy talked about software 2.0,
link |
I mean, in a sense, data is the program.
link |
So let me try to summarize Andre's position really quick
link |
before I disagree with it.
link |
So Andre Karpathy is amazing.
link |
So this is nothing personal with him.
link |
He's an amazing engineer.
link |
And also a good blog post writer.
link |
Yeah, well, he's a great communicator.
link |
You know, he's just an amazing person.
link |
He's also really sweet.
link |
So his basic premise is that software is suboptimal.
link |
I think we can all agree to that.
link |
He also points out that deep learning
link |
and other learning based techniques are really great
link |
because you can solve problems
link |
in more structured ways with less like ad hoc code
link |
that people write out and don't write test cases for
link |
And so they don't even know if it works in the first place.
link |
And so if you start replacing systems of imperative code
link |
with deep learning models, then you get a better result.
link |
And I think that he argues that software 2.0
link |
is a pervasively learned set of models
link |
and you get away from writing code.
link |
And he's given talks where he talks about, you know,
link |
swapping over more and more and more parts of the code
link |
to being learned and driven that way.
link |
I think that works.
link |
And if you're predisposed to liking machine learning,
link |
then I think that that's definitely a good thing.
link |
I think this is also good for accessibility in many ways
link |
because certain people are not gonna write C code
link |
And so having a data driven approach to do
link |
this kind of stuff, I think can be very valuable.
link |
On the other hand, there are huge trade offs.
link |
It's not clear to me that software 2.0 is the answer.
link |
And probably Andre wouldn't argue that it's the answer
link |
for every problem either.
link |
But I look at machine learning as not a replacement
link |
I look at it as a new programming paradigm.
link |
And so programming paradigms, when you look across demands,
link |
is structured programming where you go from go tos
link |
to if, then, else, or functional programming from Lisp.
link |
And you start talking about higher order functions
link |
and values and things like this.
link |
Or you talk about object oriented programming.
link |
You're talking about encapsulation,
link |
subclassing, inheritance.
link |
You start talking about generic programming
link |
where you start talking about code reuse
link |
through specialization and different type instantiations.
link |
When you start talking about differentiable programming,
link |
something that I am very excited about
link |
in the context of machine learning,
link |
talking about taking functions and generating variants,
link |
like the derivative of another function.
link |
Like that's a programming paradigm that's very useful
link |
for solving certain classes of problems.
link |
Machine learning is amazing
link |
at solving certain classes of problems.
link |
Like you're not gonna write a cat detector
link |
or even a language translation system by writing C code.
link |
That's not a very productive way to do things anymore.
link |
And so machine learning is absolutely
link |
the right way to do that.
link |
In fact, I would say that learned models
link |
are really one of the best ways to work
link |
with the human world in general.
link |
And so anytime you're talking about sensory input
link |
of different modalities,
link |
anytime that you're talking about generating things
link |
in a way that makes sense to a human,
link |
I think that learned models are really, really useful.
link |
And that's because humans are very difficult
link |
to characterize, okay?
link |
And so this is a very powerful paradigm
link |
for solving classes of problems.
link |
But on the other hand, imperative code is too.
link |
You're not gonna write a bootloader for your computer
link |
with a deep learning model.
link |
Deep learning models are very hardware intensive.
link |
They're very energy intensive
link |
because you have a lot of parameters
link |
and you can provably implement any function
link |
with a learned model, like this has been shown,
link |
but that doesn't make it efficient.
link |
And so if you're talking about caring about a few orders
link |
of magnitudes worth of energy usage,
link |
then it's useful to have other tools in the toolbox.
link |
There's also robustness too.
link |
All the problems of dealing with data and bias in data,
link |
all the problems of software 2.0.
link |
And one of the great things that Andre is arguing towards,
link |
which I completely agree with him,
link |
is that when you start implementing things
link |
with deep learning, you need to learn from software 1.0
link |
in terms of testing, continuous integration,
link |
how you deploy, how do you validate,
link |
all these things and building systems around that
link |
so that you're not just saying like,
link |
oh, it seems like it's good, ship it, right?
link |
Well, what happens when I regress something?
link |
What happens when I make a classification that's wrong
link |
and now I hurt somebody, right?
link |
I mean, all these things you have to reason about.
link |
Yeah, but at the same time,
link |
the bootloader that works for us humans
link |
looks awfully a lot like a neural network, right?
link |
It's messy and you can cut out
link |
different parts of the brain.
link |
There's a lot of this neuroplasticity work
link |
that shows that it's gonna adjust.
link |
It's a really interesting question,
link |
how much of the world's programming
link |
could be replaced by software 2.0?
link |
Oh, well, I mean, it's provably true
link |
that you could replace all of it.
link |
Right, so then it's a question of the trade offs.
link |
Anything that's a function, you can.
link |
So it's not a question about if.
link |
I think it's a economic question.
link |
It's a, what kind of talent can you get?
link |
What kind of trade offs in terms of maintenance, right?
link |
Those kinds of questions, I think.
link |
What kind of data can you collect?
link |
I think one of the reasons that I'm most interested
link |
in machine learning as a programming paradigm
link |
is that one of the things that we've seen
link |
across computing in general is that
link |
being laser focused on one paradigm
link |
often puts you in a box that's not super great.
link |
And so you look at object oriented programming,
link |
like it was all the rage in the early 80s
link |
and like everything has to be objects.
link |
And people forgot about functional programming
link |
even though it came first.
link |
And then people rediscovered that,
link |
hey, if you mix functional and object oriented
link |
in structure, like you mix these things together,
link |
you can provide very interesting tools
link |
that are good at solving different problems.
link |
And so the question there is how do you get
link |
the best way to solve the problems?
link |
It's not about whose tribe should win, right?
link |
It's not about, you know, that shouldn't be the question.
link |
The question is how do you make it
link |
so that people can solve those problems the fastest
link |
and they have the right tools in their box
link |
to build good libraries and they can solve these problems.
link |
And when you look at that, that's like, you know,
link |
you look at reinforcement learning
link |
as one really interesting subdomain of this.
link |
Reinforcement learning, often you have to have
link |
the integration of a learned model
link |
combined with your Atari or whatever the other scenario
link |
it is that you're working in.
link |
You have to combine that thing with the robot control
link |
for the arm, right?
link |
And so now it's not just about that one paradigm,
link |
it's about integrating that with all the other systems
link |
that you have, including often legacy systems
link |
and things like this, right?
link |
And so to me, I think that the interesting thing to say
link |
is like, how do you get the best out of this domain
link |
and how do you enable people to achieve things
link |
that they otherwise couldn't do
link |
without excluding all the good things
link |
we already know how to do?
link |
Right, but okay, this is a crazy question,
link |
but we talked a little bit about GPT3,
link |
but do you think it's possible that these language models
link |
that in essence, in the language domain,
link |
Software 2.0 could replace some aspect of compilation,
link |
for example, or do program synthesis,
link |
replace some aspect of programming?
link |
So I think that learned models in general
link |
are extremely powerful,
link |
and I think that people underestimate them.
link |
Maybe you can suggest what I should do.
link |
So if I have access to the GPT3 API,
link |
would I be able to generate Swift code, for example?
link |
Do you think that could do something interesting
link |
So GPT3 is probably not trained on the right corpus,
link |
so it probably has the ability to generate some Swift.
link |
It's probably not gonna generate a large enough body
link |
of Swift to be useful,
link |
but take it a next step further.
link |
Like if you had the goal of training something like GPT3
link |
and you wanted to train it to generate source code, right?
link |
It could definitely do that.
link |
Now the question is, how do you express the intent
link |
of what you want filled in?
link |
You can definitely write scaffolding of code
link |
and say, fill in the hole,
link |
and sort of put in some for loops,
link |
or put in some classes or whatever.
link |
And the power of these models is impressive,
link |
but there's an unsolved question, at least unsolved to me,
link |
which is, how do I express the intent of what to fill in?
link |
And kind of what you'd really want to have,
link |
and I don't know that these models are up to the task,
link |
is you wanna be able to say,
link |
here's the scaffolding,
link |
and here are the assertions at the end.
link |
And the assertions always pass.
link |
And so you want a generative model on the one hand, yes.
link |
Oh, that's fascinating, yeah.
link |
Right, but you also want some loop back,
link |
some reinforcement learning system or something,
link |
where you're actually saying like,
link |
I need to hill climb towards something
link |
that is more correct.
link |
And I don't know that we have that.
link |
So it would generate not only a bunch of the code,
link |
but like the checks that do the testing.
link |
It would generate the tests.
link |
I think the humans would generate the tests, right?
link |
But it would be fascinating if...
link |
Well, the tests are the requirements.
link |
Yes, but the, okay, so...
link |
Because you have to express to the model
link |
what you want to...
link |
You don't just want gibberish code.
link |
Look at how compelling this code looks.
link |
You want a story about four horned unicorns or something.
link |
Well, okay, so exactly.
link |
But that's human requirements.
link |
But then I thought it's a compelling idea
link |
that the GPT4 model could generate checks
link |
that are more high fidelity that check for correctness.
link |
Because the code it generates,
link |
like say I ask it to generate a function
link |
that gives me the Fibonacci sequence.
link |
So decompose the problem, right?
link |
So you have two things.
link |
You have, you need the ability to generate
link |
syntactically correct Swift code that's interesting, right?
link |
I think GPT series of model architectures can do that.
link |
But then you need the ability to add the requirements.
link |
So generate Fibonacci.
link |
The human needs to express that goal.
link |
We don't have that language that I know of.
link |
No, I mean, it can generate stuff.
link |
Have you seen what GPT3 can generate?
link |
You can say, I mean, there's a interface stuff
link |
like it can generate HTML.
link |
It can generate basic for loops that give you like...
link |
Right, but pick HTML.
link |
How do I say I want google.com?
link |
Well, no, you could say...
link |
Or not literally google.com.
link |
How do I say I want a webpage
link |
that's got a shopping cart and this and that?
link |
So just, I don't know if you've seen these demonstrations
link |
but you type in, I want a red button
link |
with the text that says hello.
link |
And you type that in natural language
link |
and it generates the correct HTML.
link |
I've done this demo.
link |
It's kind of compelling.
link |
So you have to prompt it with similar kinds of mappings.
link |
Of course, it's probably handpicked.
link |
I got to experiment that probably,
link |
but the fact that you can do that once
link |
even out of like 20 is quite impressive.
link |
Again, that's very basic.
link |
Like the HTML is kind of messy and bad.
link |
But yes, the intent is...
link |
The idea is the intent is specified in natural language.
link |
Yeah, so I have not seen that.
link |
That's really cool.
link |
Yeah, but the question is the correctness of that.
link |
Like visually you can check, oh, the button is red,
link |
but for more complicated functions
link |
where the intent is harder to check.
link |
This goes into like NP completeness kind of things.
link |
Like I want to know that this code is correct
link |
and generates a giant thing
link |
that does some kind of calculation.
link |
It seems to be working.
link |
It's interesting to think like,
link |
should the system also try to generate checks
link |
for itself for correctness?
link |
Yeah, I don't know.
link |
And this is way beyond my experience.
link |
The thing that I think about is that
link |
there doesn't seem to be a lot of
link |
equational reasoning going on.
link |
There's a lot of pattern matching and filling in
link |
and kind of propagating patterns
link |
that have been seen before into the future
link |
and into the generator result.
link |
And so if you want to get correctness,
link |
you kind of need theorem proving kind of things
link |
and like higher level logic.
link |
And I don't know that...
link |
You could talk to Jan about that
link |
and see what the bright minds
link |
are thinking about right now,
link |
but I don't think the GPT is in that vein.
link |
It's still really cool.
link |
Yeah, and surprisingly, who knows,
link |
maybe reasoning is...
link |
Yeah, is overrated.
link |
Right, I mean, do we reason?
link |
How do you tell, right?
link |
Are we just pattern matching based on what we have
link |
and then reverse justifying to ourselves?
link |
Exactly, the reverse.
link |
So like I think what the neural networks are missing
link |
and I think GPT4 might have
link |
is to be able to tell stories to itself about what it did.
link |
Well, that's what humans do, right?
link |
I mean, you talk about like network explainability, right?
link |
And we give, no, that's a hard time about this,
link |
but humans don't know why we make decisions.
link |
We have this thing called intuition
link |
and then we try to like say,
link |
this feels like the right thing, but why, right?
link |
And you wrestle with that
link |
when you're making hard decisions
link |
and is that science?
link |
Let me ask you about a few high level questions, I guess.
link |
Because you've done a million things in your life
link |
and been very successful.
link |
A bunch of young folks listen to this,
link |
ask for advice from successful people like you.
link |
If you were to give advice to somebody,
link |
you know, another graduate student
link |
or some high school student
link |
about pursuing a career in computing
link |
or just advice about life in general,
link |
is there some words of wisdom you can give them?
link |
So I think you come back to change
link |
and profound leaps happen
link |
because people are willing to believe
link |
that change is possible
link |
and that the world does change
link |
and are willing to do the hard thing
link |
that it takes to make change happen.
link |
And whether it be implementing a new programming language
link |
or employing a new system
link |
or employing a new research paper,
link |
designing a new thing,
link |
moving the world forward in science
link |
and philosophy, whatever,
link |
it really comes down to somebody
link |
who's willing to put in the work, right?
link |
And you have, the work is hard
link |
for a whole bunch of different reasons.
link |
One of which is, it's work, right?
link |
And so you have to have the space in your life
link |
in which you can do that work,
link |
which is why going to grad school
link |
can be a beautiful thing for certain people.
link |
But also there's a self doubt that happens.
link |
Like you're two years into a project,
link |
is it going anywhere, right?
link |
Well, what do you do?
link |
Do you just give up because it's hard?
link |
No, no, I mean, some people like suffering.
link |
And so you plow through it.
link |
The secret to me is that you have to love what you're doing
link |
and follow that passion
link |
because when you get to the hard times,
link |
that's when, if you love what you're doing,
link |
you're willing to kind of push through.
link |
And this is really hard
link |
because it's hard to know what you will love doing
link |
until you start doing a lot of things.
link |
And so that's why I think that,
link |
particularly early in your career,
link |
it's good to experiment.
link |
Do a little bit of everything.
link |
Go take the survey class on
link |
the first half of every class in your upper division lessons
link |
and just get exposure to things
link |
because certain things will resonate with you
link |
and you'll find out, wow, I'm really good at this.
link |
I'm really smart at this.
link |
Well, it's just because it works with the way your brain.
link |
And when something jumps out,
link |
I mean, that's one of the things
link |
that people often ask about is like,
link |
well, I think there's a bunch of cool stuff out there.
link |
Like how do I pick the thing?
link |
Like how do you hook, in your life,
link |
how did you just hook yourself in and stuck with it?
link |
Well, I got lucky, right?
link |
I mean, I think that many people forget
link |
that a huge amount of it or most of it is luck, right?
link |
So let's not forget that.
link |
So for me, I fell in love with computers early on
link |
because they spoke to me, I guess.
link |
What language did they speak?
link |
But then it was just kind of following
link |
a set of logical progressions,
link |
but also deciding that something that was hard
link |
was worth doing and a lot of fun, right?
link |
And so I think that that is also something
link |
that's true for many other domains,
link |
which is if you find something that you love doing
link |
that's also hard, if you invest yourself in it
link |
and add value to the world,
link |
then it will mean something generally, right?
link |
And again, that can be a research paper,
link |
that can be a software system,
link |
that can be a new robot,
link |
that can be, there's many things that that can be,
link |
but a lot of it is like real value
link |
comes from doing things that are hard.
link |
And that doesn't mean you have to suffer, but.
link |
I mean, you don't often hear that message.
link |
We talked about it last time a little bit,
link |
but it's one of my, not enough people talk about this.
link |
It's beautiful to hear a successful person.
link |
Well, and self doubt and imposter syndrome,
link |
these are all things that successful people
link |
suffer with as well,
link |
particularly when they put themselves
link |
in a point of being uncomfortable,
link |
which I like to do now and then
link |
just because it puts you in learning mode.
link |
Like if you wanna grow as a person,
link |
put yourself in a room with a bunch of people
link |
that know way more about whatever you're talking about
link |
than you do and ask dumb questions.
link |
Smart people love to teach often,
link |
not always, but often.
link |
And if you listen, if you're prepared to listen,
link |
if you're prepared to grow,
link |
if you're prepared to make connections,
link |
you can do some really interesting things.
link |
And I think a lot of progress is made by people
link |
who kind of hop between domains now and then,
link |
because they bring a perspective into a field
link |
that nobody else has,
link |
if people have only been working in that field themselves.
link |
We mentioned that the universe is kind of like a compiler.
link |
The entirety of it, the whole evolution
link |
is kind of a kind of compilation.
link |
Maybe us human beings are kind of compilers.
link |
Let me ask the old sort of question
link |
that I didn't ask you last time,
link |
which is what's the meaning of it all?
link |
Is there a meaning?
link |
Like if you asked a compiler why,
link |
what would a compiler say?
link |
What's the meaning of life?
link |
What's the meaning of life?
link |
I'm prepared for it not to mean anything.
link |
Here we are, all biological things programmed to survive
link |
and propagate our DNA.
link |
And maybe the universe is just a computer
link |
and you just go until entropy takes over the universe
link |
and then you're done.
link |
I don't think that's a very productive way
link |
to live your life, if so.
link |
And so I prefer to bias towards the other way,
link |
which is saying the universe has a lot of value.
link |
And I take happiness out of other people.
link |
And a lot of times part of that's having kids,
link |
but also the relationships you build with other people.
link |
And so the way I try to live my life is like,
link |
what can I do that has value?
link |
How can I move the world forward?
link |
How can I take what I'm good at
link |
and bring it into the world?
link |
And I'm one of these people that likes to work really hard
link |
and be very focused on the things that I do.
link |
And so if I'm gonna do that,
link |
how can it be in a domain that actually will matter?
link |
Because a lot of things that we do,
link |
we find ourselves in the cycle of like,
link |
okay, I'm doing a thing.
link |
I'm very familiar with it.
link |
I've done it for a long time.
link |
I've never done anything else,
link |
but I'm not really learning, right?
link |
I'm not really, I'm keeping things going,
link |
but there's a younger generation
link |
that can do the same thing,
link |
maybe even better than me, right?
link |
Maybe if I actually step out of this
link |
and jump into something I'm less comfortable with,
link |
But on the other hand,
link |
it gives somebody else a new opportunity.
link |
It also then puts you back in learning mode,
link |
and that can be really interesting.
link |
And one of the things I've learned is that
link |
when you go through that,
link |
that first you're deep into imposter syndrome,
link |
but when you start working your way out,
link |
you start to realize,
link |
hey, well, there's actually a method to this.
link |
And now I'm able to add new things
link |
because I bring different perspective.
link |
And this is one of the good things
link |
about bringing different kinds of people together.
link |
Diversity of thought is really important.
link |
And if you can pull together people
link |
that are coming at things from different directions,
link |
you often get innovation.
link |
And I love to see that, that aha moment
link |
where you're like, oh, we've really cracked this.
link |
This is something nobody's ever done before.
link |
And then if you can do it in the context
link |
where it adds value, other people can build on it,
link |
it helps move the world,
link |
then that's what really excites me.
link |
So that kind of description
link |
of the magic of the human experience,
link |
do you think we'll ever create that in an AGI system?
link |
Do you think we'll be able to create,
link |
give AI systems a sense of meaning
link |
where they operate in this kind of world
link |
exactly in the way you've described,
link |
which is they interact with each other,
link |
they interact with us humans.
link |
Well, so, I mean, why are you being so a speciest, right?
link |
All right, so AGI versus Bionets,
link |
or something like that versus biology, right?
link |
You know, what are we but machines, right?
link |
We're just programmed to run our,
link |
we have our objective function
link |
that we were optimized for, right?
link |
And so we're doing our thing,
link |
we think we have purpose, but do we really, right?
link |
I'm not prepared to say that those newfangled AGI's
link |
have no soul just because we don't understand them, right?
link |
And I think that would be, when they exist,
link |
that would be very premature to look at a new thing
link |
through your own lens without fully understanding it.
link |
You might be just saying that
link |
because AI systems in the future
link |
will be listening to this and then.
link |
You don't wanna say anything.
link |
Please be nice to me, you know,
link |
when Skynet kills everybody, please spare me.
link |
So wise look ahead thinking.
link |
Yeah, but I mean, I think that people will spend
link |
a lot of time worrying about this kind of stuff,
link |
and I think that what we should be worrying about
link |
is how do we make the world better?
link |
And the thing that I'm most scared about with AGI's
link |
is not that necessarily the Skynet
link |
will start shooting everybody with lasers
link |
and stuff like that to use us for our calories.
link |
The thing that I'm worried about is that
link |
humanity, I think, needs a challenge.
link |
And if we get into a mode of not having a personal challenge,
link |
not having a personal contribution,
link |
whether that be like, you know, your kids
link |
and seeing what they grow into and helping guide them,
link |
whether it be your community that you're engaged in,
link |
you're driving forward, whether it be your work
link |
and the things that you're doing
link |
and the people you're working with
link |
and the products you're building and the contribution there,
link |
if people don't have a objective,
link |
I'm afraid what that means.
link |
And I think that this would lead to a rise
link |
of the worst part of people, right?
link |
Instead of people striving together
link |
and trying to make the world better,
link |
it could degrade into a very unpleasant world.
link |
I mean, we hopefully have a long ways to go
link |
before we discover that.
link |
And fortunately, we have pretty on the ground problems
link |
with the pandemic right now,
link |
and so I think we should be focused on that as well.
link |
Yeah, ultimately, just as you said, you're optimistic.
link |
I think it helps for us to be optimistic.
link |
So that's fake it until you make it.
link |
Yeah, well, and why not?
link |
What's the other side, right?
link |
So, I mean, I'm not personally a very religious person,
link |
but I've heard people say like,
link |
oh yeah, of course I believe in God.
link |
Of course I go to church, because if God's real,
link |
you know, I wanna be on the right side of that.
link |
If it's not real, it doesn't matter.
link |
Yeah, it doesn't matter.
link |
And so, you know, that's a fair way to do it.
link |
Yeah, I mean, the same thing with nuclear deterrence,
link |
all, you know, global warming, all these things,
link |
all these threats, natural engineer pandemics,
link |
all these threats we face.
link |
I think it's paralyzing to be terrified
link |
of all the possible ways we could destroy ourselves.
link |
I think it's much better or at least productive
link |
to be hopeful and to engineer defenses
link |
against these things, to engineer a future
link |
where like, you know, see like a positive future
link |
and engineer that future.
link |
Yeah, well, and I think that's another thing
link |
to think about as, you know, a human,
link |
particularly if you're young and trying to figure out
link |
what it is that you wanna be when you grow up, like I am.
link |
I'm always looking for that.
link |
The question then is, how do you wanna spend your time?
link |
And right now there seems to be a norm
link |
of being a consumption culture.
link |
Like I'm gonna watch the news and revel
link |
in how horrible everything is right now.
link |
I'm going to go find out about the latest atrocity
link |
and find out all the details of like the terrible thing
link |
that happened and be outraged by it.
link |
You can spend a lot of time watching TV
link |
and watching the news at home or whatever
link |
people watch these days, I don't know.
link |
But that's a lot of hours, right?
link |
And those are hours that if you're turned
link |
to being productive, learning, growing,
link |
experiencing, you know, when the pandemic's over,
link |
going exploring, right, it leads to more growth.
link |
And I think it leads to more optimism and happiness
link |
because you're building, right?
link |
You're building yourself, you're building your capabilities,
link |
you're building your viewpoints,
link |
you're building your perspective.
link |
And I think that a lot of the consuming
link |
of other people's messages leads
link |
to kind of a negative viewpoint,
link |
which you need to be aware of what's happening
link |
because that's also important,
link |
but there's a balance that I think focusing
link |
on creation is a very valuable thing to do.
link |
Yeah, so what you're saying is people should focus
link |
on working on the sexiest field of them all,
link |
which is compiler design.
link |
Hey, you could go work on machine learning
link |
and be crowded out by the thousands of graduates
link |
popping out of school that all want to do the same thing,
link |
or you could work in the place that people overpay you
link |
because there's not enough smart people working in it.
link |
And here at the end of Moore's Law,
link |
according to some people,
link |
actually the software is the hard part too.
link |
I mean, optimization is truly, truly beautiful.
link |
And also on the YouTube side or education side,
link |
it'd be nice to have some material
link |
that shows the beauty of compilers.
link |
So that's a call for people to create
link |
that kind of content as well.
link |
Chris, you're one of my favorite people to talk to.
link |
It's such a huge honor that you would waste your time
link |
I've always appreciated it.
link |
Thank you so much for talking to me.
link |
The truth of it is you spent a lot of time talking to me
link |
just on walks and other things like that,
link |
so it's great to catch up with.
link |
Thanks for listening to this conversation
link |
with Chris Latner, and thank you to our sponsors.
link |
Blinkist, an app that summarizes key ideas
link |
from thousands of books.
link |
Neuro, which is a maker of functional gum and mints
link |
that supercharge my mind.
link |
Masterclass, which are online courses from world experts.
link |
And finally, Cash App, which is an app
link |
for sending money to friends.
link |
Please check out these sponsors in the description
link |
to get a discount and to support this podcast.
link |
If you enjoy this thing, subscribe on YouTube,
link |
review it with five stars on Apple Podcast,
link |
follow on Spotify, support it on Patreon,
link |
connect with me on Twitter at Lex Friedman.
link |
And now, let me leave you with some words from Chris Latner.
link |
So much of language design is about tradeoffs,
link |
and you can't see those tradeoffs
link |
unless you have a community of people
link |
that really represent those different points.
link |
Thank you for listening, and hope to see you next time.