back to index

Guido van Rossum: Python | Lex Fridman Podcast #6


small model | large model

link |
00:00:00.000
The following is a conversation with Guido van Rossum, creator of Python, one of the most popular
link |
00:00:05.680
programming languages in the world, used in almost any application that involves computers
link |
00:00:11.120
from web back end development to psychology, neuroscience, computer vision, robotics, deep
link |
00:00:17.760
learning, natural language processing, and almost any subfield of AI. This conversation is part of
link |
00:00:24.560
MIT course on artificial general intelligence and the artificial intelligence podcast.
link |
00:00:29.280
If you enjoy it, subscribe on YouTube, iTunes, or your podcast provider of choice, or simply connect
link |
00:00:36.080
with me on Twitter at Lex Friedman, spelled F R I D. And now, here's my conversation with Guido van
link |
00:00:44.720
Rossum. You were born in the Netherlands in 1956. Your parents and the world around you was deeply
link |
00:00:53.120
deeply impacted by World War Two, as was my family from the Soviet Union. So with that context,
link |
00:01:02.000
what is your view of human nature? Are some humans inherently good,
link |
00:01:07.360
and some inherently evil? Or do we all have both good and evil within us?
link |
00:01:12.240
Guido van Rossum Ouch, I did not expect such a deep one. I, I guess we all have good and evil
link |
00:01:24.880
potential in us. And a lot of it depends on circumstances and context.
link |
00:01:31.440
Peter Bell out of that world, at least on the Soviet Union side in Europe, sort of out of
link |
00:01:38.800
suffering, out of challenge, out of that kind of set of traumatic events, often emerges beautiful
link |
00:01:46.480
art, music, literature. In an interview I read or heard, you said you enjoyed Dutch literature
link |
00:01:54.320
when you were a child. Can you tell me about the books that had an influence on you in your
link |
00:01:59.760
childhood? Guido van Rossum
link |
00:02:01.520
Well, with as a teenager, my favorite writer was my favorite Dutch author was a guy named Willem
link |
00:02:09.120
Frederik Hermans, who's writing, certainly his early novels were all about sort of
link |
00:02:19.440
ambiguous things that happened during World War Two. I think he was a young adult during that time.
link |
00:02:31.600
And he wrote about it a lot, and very interesting, very good books, I thought, I think.
link |
00:02:40.800
Peter Bell In a nonfiction way?
link |
00:02:42.560
Guido van Rossum No, it was all fiction, but it was
link |
00:02:46.400
very much set in the ambiguous world of resistance against the Germans,
link |
00:02:54.560
where often you couldn't tell whether someone was truly in the resistance or really a spy for the
link |
00:03:03.840
Germans. And some of the characters in his novels sort of crossed that line, and you never really
link |
00:03:11.280
find out what exactly happened.
link |
00:03:13.840
Peter Bell And in his novels, there's always a
link |
00:03:16.880
good guy and a bad guy, the nature of good and evil. Is it clear there's a hero?
link |
00:03:22.160
Guido van Rossum No, his heroes are often more,
link |
00:03:25.120
his main characters are often anti heroes. And so they're not very heroic. They're often,
link |
00:03:36.640
they fail at some level to accomplish their lofty goals.
link |
00:03:40.800
Peter Bell And looking at the trajectory
link |
00:03:43.040
through the rest of your life, has literature, Dutch or English or translation had an impact
link |
00:03:50.560
outside the technical world that you existed in?
link |
00:03:54.160
Guido van Rossum I still read novels.
link |
00:04:00.640
I don't think that it impacts me that much directly.
link |
00:04:05.200
Peter Bell It doesn't impact your work.
link |
00:04:07.280
Guido van Rossum It's a separate world.
link |
00:04:10.080
My work is highly technical and sort of the world of art and literature doesn't really
link |
00:04:17.440
directly have any bearing on it.
link |
00:04:19.120
Peter Bell You don't think there's a creative element
link |
00:04:22.400
to the design? You know, some would say design of a language is art.
link |
00:04:26.880
Guido van Rossum I'm not disagreeing with that.
link |
00:04:32.160
I'm just saying that sort of I don't feel direct influences from more traditional art
link |
00:04:39.360
on my own creativity.
link |
00:04:40.880
Peter Bell Right. Of course, you don't feel doesn't mean
link |
00:04:43.280
it's not somehow deeply there in your subconscious.
link |
00:04:46.000
Guido van Rossum Who knows?
link |
00:04:48.240
Peter Bell Who knows? So let's go back to your early
link |
00:04:51.200
teens. Your hobbies were building electronic circuits, building mechanical models.
link |
00:04:57.440
What if you can just put yourself back in the mind of that young Guido 12, 13, 14, was
link |
00:05:06.080
that grounded in a desire to create a system? So to create something? Or was it more just
link |
00:05:12.240
tinkering? Just the joy of puzzle solving?
link |
00:05:14.720
Guido van Rossum I think it was more the latter, actually.
link |
00:05:18.720
I maybe towards the end of my high school period, I felt confident enough that that
link |
00:05:29.920
I designed my own circuits that were sort of interesting somewhat. But a lot of that
link |
00:05:39.120
time, I literally just took a model kit and follow the instructions, putting the things
link |
00:05:46.000
together. I mean, I think the first few years that I built electronics kits, I really did
link |
00:05:51.680
not have enough understanding of sort of electronics to really understand what I was doing. I mean,
link |
00:05:59.760
I could debug it, and I could sort of follow the instructions very carefully, which has
link |
00:06:06.480
always stayed with me. But I had a very naive model of, like, how do I build a circuit?
link |
00:06:14.560
Of, like, how a transistor works? And I don't think that in those days, I had any understanding
link |
00:06:22.800
of coils and capacitors, which actually sort of was a major problem when I started to build
link |
00:06:32.560
more complex digital circuits, because I was unaware of the sort of the analog part of
link |
00:06:39.840
the – how they actually work. And I would have things that – the schematic looked
link |
00:06:50.080
– everything looked fine, and it didn't work. And what I didn't realize was that
link |
00:06:57.440
there was some megahertz level oscillation that was throwing the circuit off, because
link |
00:07:02.720
I had a sort of – two wires were too close, or the switches were kind of poorly built.
link |
00:07:13.360
But through that time, I think it's really interesting and instructive to think about,
link |
00:07:19.280
because echoes of it are in this time now. So in the 1970s, the personal computer was
link |
00:07:24.600
being born. So did you sense, in tinkering with these circuits, did you sense the encroaching
link |
00:07:33.200
revolution in personal computing? So if at that point, we would sit you down and ask
link |
00:07:39.320
you to predict the 80s and the 90s, do you think you would be able to do so successfully
link |
00:07:46.040
to unroll the process that's happening? No, I had no clue. I remember, I think, in
link |
00:07:55.560
the summer after my senior year – or maybe it was the summer after my junior year – well,
link |
00:08:03.060
at some point, I think, when I was 18, I went on a trip to the Math Olympiad in Eastern
link |
00:08:11.600
Europe, and there was like – I was part of the Dutch team, and there were other nerdy
link |
00:08:16.920
kids that sort of had different experiences, and one of them told me about this amazing
link |
00:08:23.040
thing called a computer. And I had never heard that word. My own explorations in electronics
link |
00:08:31.840
were sort of about very simple digital circuits, and I had sort of – I had the idea that
link |
00:08:40.420
I somewhat understood how a digital calculator worked. And so there is maybe some echoes
link |
00:08:49.760
of computers there, but I never made that connection. I didn't know that when my parents
link |
00:08:56.440
were paying for magazine subscriptions using punched cards, that there was something called
link |
00:09:03.520
a computer that was involved that read those cards and transferred the money between accounts.
link |
00:09:08.260
I was also not really interested in those things. It was only when I went to university
link |
00:09:15.880
to study math that I found out that they had a computer, and students were allowed to use
link |
00:09:23.120
it.
link |
00:09:24.120
And there were some – you're supposed to talk to that computer by programming it.
link |
00:09:27.800
What did that feel like, finding –
link |
00:09:29.920
Yeah, that was the only thing you could do with it. The computer wasn't really connected
link |
00:09:35.440
to the real world. The only thing you could do was sort of – you typed your program
link |
00:09:41.400
on a bunch of punched cards. You gave the punched cards to the operator, and an hour
link |
00:09:47.840
later the operator gave you back your printout. And so all you could do was write a program
link |
00:09:55.520
that did something very abstract. And I don't even remember what my first forays into programming
link |
00:10:04.080
were, but they were sort of doing simple math exercises and just to learn how a programming
link |
00:10:13.440
language worked.
link |
00:10:15.560
Did you sense, okay, first year of college, you see this computer, you're able to have
link |
00:10:21.680
a program and it generates some output. Did you start seeing the possibility of this,
link |
00:10:29.420
or was it a continuation of the tinkering with circuits? Did you start to imagine that
link |
00:10:34.920
one, the personal computer, but did you see it as something that is a tool, like a word
link |
00:10:42.460
processing tool, maybe for gaming or something? Or did you start to imagine that it could
link |
00:10:47.160
be going to the world of robotics, like the Frankenstein picture that you could create
link |
00:10:53.860
an artificial being? There's like another entity in front of you. You did not see the
link |
00:10:59.640
computer.
link |
00:11:00.640
I don't think I really saw it that way. I was really more interested in the tinkering.
link |
00:11:05.840
It's maybe not a sort of a complete coincidence that I ended up sort of creating a programming
link |
00:11:14.920
language which is a tool for other programmers. I've always been very focused on the sort
link |
00:11:20.360
of activity of programming itself and not so much what happens with the program you
link |
00:11:28.920
write.
link |
00:11:29.920
Right.
link |
00:11:30.920
I do remember, and I don't remember, maybe in my second or third year, probably my second
link |
00:11:37.800
actually, someone pointed out to me that there was this thing called Conway's Game of Life.
link |
00:11:46.680
You're probably familiar with it. I think –
link |
00:11:50.480
In the 70s, I think is when they came up with it.
link |
00:11:53.200
So there was a Scientific American column by someone who did a monthly column about
link |
00:12:00.840
mathematical diversions. I'm also blanking out on the guy's name. It was very famous
link |
00:12:06.580
at the time and I think up to the 90s or so. And one of his columns was about Conway's
link |
00:12:12.440
Game of Life and he had some illustrations and he wrote down all the rules and sort of
link |
00:12:18.160
there was the suggestion that this was philosophically interesting, that that was why Conway had
link |
00:12:23.720
called it that. And all I had was like the two pages photocopy of that article. I don't
link |
00:12:31.480
even remember where I got it. But it spoke to me and I remember implementing a version
link |
00:12:40.200
of that game for the batch computer we were using where I had a whole Pascal program that
link |
00:12:49.000
sort of read an initial situation from input and read some numbers that said do so many
link |
00:12:56.480
generations and print every so many generations and then out would come pages and pages of
link |
00:13:05.960
sort of things.
link |
00:13:08.480
I remember much later I've done a similar thing using Python but that original version
link |
00:13:18.360
I wrote at the time I found interesting because I combined it with some trick I had learned
link |
00:13:27.700
during my electronics hobbyist times. I essentially first on paper I designed a simple circuit
link |
00:13:36.000
built out of logic gates that took nine bits of input which is sort of the cell and its
link |
00:13:45.780
neighbors and produced a new value for that cell and it's like a combination of a half
link |
00:13:54.040
adder and some other clipping. It's actually a full adder. And so I had worked that out
link |
00:14:01.040
and then I translated that into a series of Boolean operations on Pascal integers where
link |
00:14:10.520
you could use the integers as bitwise values. And so I could basically generate 60 bits
link |
00:14:21.740
of a generation in like eight instructions or so.
link |
00:14:28.800
Nice.
link |
00:14:29.800
So I was proud of that.
link |
00:14:32.560
It's funny that you mentioned, so for people who don't know Conway's Game of Life, it's
link |
00:14:38.120
a cellular automata where there's single compute units that kind of look at their neighbors
link |
00:14:44.840
and figure out what they look like in the next generation based on the state of their
link |
00:14:50.080
neighbors and this is deeply distributed system in concept at least. And then there's simple
link |
00:14:57.840
rules that all of them follow and somehow out of this simple rule when you step back
link |
00:15:04.400
and look at what occurs, it's beautiful. There's an emergent complexity. Even though the underlying
link |
00:15:13.160
rules are simple, there's an emergent complexity. Now the funny thing is you've implemented
link |
00:15:17.440
this and the thing you're commenting on is you're proud of a hack you did to make it
link |
00:15:23.660
run efficiently. When you're not commenting on, it's a beautiful implementation, you're
link |
00:15:30.800
not commenting on the fact that there's an emergent complexity that you've coded a simple
link |
00:15:36.780
program and when you step back and you print out the following generation after generation,
link |
00:15:42.960
that's stuff that you may have not predicted would happen is happening.
link |
00:15:48.400
And is that magic? I mean, that's the magic that all of us feel when we program. When
link |
00:15:53.600
you create a program and then you run it and whether it's Hello World or it shows something
link |
00:15:59.240
on screen, if there's a graphical component, are you seeing the magic in the mechanism
link |
00:16:03.840
of creating that?
link |
00:16:05.200
I think I went back and forth. As a student, we had an incredibly small budget of computer
link |
00:16:14.440
time that we could use. It was actually measured. I once got in trouble with one of my professors
link |
00:16:20.280
because I had overspent the department's budget. It's a different story.
link |
00:16:29.640
I actually wanted the efficient implementation because I also wanted to explore what would
link |
00:16:36.900
happen with a larger number of generations and a larger size of the board. Once the implementation
link |
00:16:48.560
was flawless, I would feed it different patterns and then I think maybe there was a follow
link |
00:16:57.000
up article where there were patterns that were like gliders, patterns that repeated
link |
00:17:03.620
themselves after a number of generations but translated one or two positions to the right
link |
00:17:13.200
or up or something like that. I remember things like glider guns. Well, you can Google Conway's
link |
00:17:21.720
Game of Life. People still go aww and ooh over it.
link |
00:17:27.560
For a reason because it's not really well understood why. I mean, this is what Stephen
link |
00:17:32.680
Wolfram is obsessed about. We don't have the mathematical tools to describe the kind of
link |
00:17:40.240
complexity that emerges in these kinds of systems. The only way you can do is to run
link |
00:17:45.120
it.
link |
00:17:47.120
I'm not convinced that it's sort of a problem that lends itself to classic mathematical
link |
00:17:55.720
analysis.
link |
00:17:59.920
One theory of how you create an artificial intelligence or artificial being is you kind
link |
00:18:05.120
of have to, same with the Game of Life, you kind of have to create a universe and let
link |
00:18:10.120
it run. That creating it from scratch in a design way, coding up a Python program that
link |
00:18:17.520
creates a fully intelligent system may be quite challenging. You might need to create
link |
00:18:22.760
a universe just like the Game of Life.
link |
00:18:27.120
You might have to experiment with a lot of different universes before there is a set
link |
00:18:33.200
of rules that doesn't essentially always just end up repeating itself in a trivial
link |
00:18:41.480
way.
link |
00:18:42.480
Yeah, and Stephen Wolfram works with these simple rules, says that it's kind of surprising
link |
00:18:49.840
how quickly you find rules that create interesting things. You shouldn't be able to, but somehow
link |
00:18:55.280
you do. And so maybe our universe is laden with rules that will create interesting things
link |
00:19:02.120
that might not look like humans, but emergent phenomena that's interesting may not be as
link |
00:19:07.440
difficult to create as we think.
link |
00:19:09.440
Sure.
link |
00:19:10.440
But let me sort of ask, at that time, some of the world, at least in popular press, was
link |
00:19:17.440
kind of captivated, perhaps at least in America, by the idea of artificial intelligence, that
link |
00:19:25.680
these computers would be able to think pretty soon. And did that touch you at all? In science
link |
00:19:33.240
fiction or in reality in any way?
link |
00:19:37.800
I didn't really start reading science fiction until much, much later. I think as a teenager
link |
00:19:49.000
I read maybe one bundle of science fiction stories.
link |
00:19:54.520
Was it in the background somewhere, like in your thoughts?
link |
00:19:57.960
That sort of the using computers to build something intelligent always felt to me, because
link |
00:20:04.720
I felt I had so much understanding of what actually goes on inside a computer. I knew
link |
00:20:12.920
how many bits of memory it had and how difficult it was to program. And sort of, I didn't believe
link |
00:20:22.880
at all that you could just build something intelligent out of that, that would really
link |
00:20:30.560
sort of satisfy my definition of intelligence. I think the most influential thing that I
link |
00:20:40.600
read in my early twenties was Gödel Escherbach. That was about consciousness, and that was
link |
00:20:48.680
a big eye opener in some sense.
link |
00:20:54.040
In what sense? So, on your own brain, did you at the time or do you now see your own
link |
00:21:00.760
brain as a computer? Or is there a total separation of the way? So yeah, you're very pragmatically
link |
00:21:07.720
practically know the limits of memory, the limits of this sequential computing or weakly
link |
00:21:14.600
paralyzed computing, and you just know what we have now, and it's hard to see how it creates.
link |
00:21:21.000
But it's also easy to see, it was in the 40s, 50s, 60s, and now at least similarities between
link |
00:21:29.920
the brain and our computers.
link |
00:21:31.680
Oh yeah, I mean, I totally believe that brains are computers in some sense. I mean, the rules
link |
00:21:43.200
they use to play by are pretty different from the rules we can sort of implement in our
link |
00:21:51.200
current hardware, but I don't believe in, like, a separate thing that infuses us with
link |
00:22:02.960
intelligence or consciousness or any of that. There's no soul, I've been an atheist
link |
00:22:10.480
probably from when I was 10 years old, just by thinking a bit about math and the universe,
link |
00:22:18.800
and well, my parents were atheists. Now, I know that you could be an atheist and still
link |
00:22:26.640
believe that there is something sort of about intelligence or consciousness that cannot
link |
00:22:34.080
possibly emerge from a fixed set of rules. I am not in that camp. I totally see that,
link |
00:22:44.560
sort of, given how many millions of years evolution took its time, DNA is a particular
link |
00:22:53.680
machine that sort of encodes information and an unlimited amount of information in chemical
link |
00:23:07.040
form and has figured out a way to replicate itself.
link |
00:23:12.320
I thought that that was, maybe it's 300 million years ago, but I thought it was closer
link |
00:23:16.880
to half a billion years ago, that that's sort of originated and it hasn't really changed,
link |
00:23:25.120
that the sort of the structure of DNA hasn't changed ever since. That is like our binary
link |
00:23:32.040
code that we have in hardware. I mean...
link |
00:23:35.200
The basic programming language hasn't changed, but maybe the programming itself...
link |
00:23:39.760
Obviously, it did sort of, it happened to be a set of rules that was good enough to
link |
00:23:48.320
sort of develop endless variability and sort of the idea of self replicating molecules
link |
00:23:59.520
competing with each other for resources and one type eventually sort of always taking
link |
00:24:05.360
over. That happened before there were any fossils, so we don't know how that exactly
link |
00:24:12.320
happened, but I believe it's clear that that did happen.
link |
00:24:17.920
Can you comment on consciousness and how you see it? Because I think we'll talk about
link |
00:24:25.360
programming quite a bit. We'll talk about, you know, intelligence connecting to programming
link |
00:24:30.080
fundamentally, but consciousness is this whole other thing. Do you think about it often as
link |
00:24:38.080
a developer of a programming language and as a human?
link |
00:24:45.440
Those are pretty sort of separate topics. Sort of my line of work working with programming
link |
00:24:55.000
does not involve anything that goes in the direction of developing intelligence or consciousness,
link |
00:25:02.720
but sort of privately as an avid reader of popular science writing, I have some thoughts
link |
00:25:13.880
which is mostly that I don't actually believe that consciousness is an all or nothing thing.
link |
00:25:25.680
I have a feeling that, and I forget what I read that influenced this, but I feel that
link |
00:25:35.960
if you look at a cat or a dog or a mouse, they have some form of intelligence. If you
link |
00:25:41.400
look at a fish, it has some form of intelligence, and that evolution just took a long time,
link |
00:25:54.040
but I feel that the sort of evolution of more and more intelligence that led to sort of
link |
00:26:01.320
the human form of intelligence followed the evolution of the senses, especially the visual
link |
00:26:12.920
sense. I mean, there is an enormous amount of processing that's needed to interpret
link |
00:26:20.480
a scene, and humans are still better at that than computers are.
link |
00:26:28.240
And I have a feeling that there is a sort of, the reason that like mammals in particular
link |
00:26:39.660
developed the levels of consciousness that they have and that eventually sort of going
link |
00:26:47.960
from intelligence to self awareness and consciousness has to do with sort of being a robot that
link |
00:26:55.360
has very highly developed senses.
link |
00:26:58.920
Has a lot of rich sensory information coming in, so that's a really interesting thought
link |
00:27:04.760
that whatever that basic mechanism of DNA, whatever that basic building blocks of programming,
link |
00:27:14.200
if you just add more abilities, more high resolution sensors, more sensors, you just
link |
00:27:21.080
keep stacking those things on top that this basic programming in trying to survive develops
link |
00:27:26.760
very interesting things that start to us humans to appear like intelligence and consciousness.
link |
00:27:35.000
As far as robots go, I think that the self driving cars have that sort of the greatest
link |
00:27:42.280
opportunity of developing something like that, because when I drive myself, I don't just
link |
00:27:50.400
pay attention to the rules of the road.
link |
00:27:53.800
I also look around and I get clues from that, oh, this is a shopping district, oh, here's
link |
00:28:01.220
an old lady crossing the street, oh, here is someone carrying a pile of mail, there's
link |
00:28:08.960
a mailbox, I bet you they're going to cross the street to reach that mailbox.
link |
00:28:14.040
And I slow down, and I don't even think about that.
link |
00:28:17.520
And so, there is so much where you turn your observations into an understanding of what
link |
00:28:25.780
other consciousnesses are going to do, or what other systems in the world are going
link |
00:28:32.680
to be, oh, that tree is going to fall.
link |
00:28:37.400
I see sort of, I see much more of, I expect somehow that if anything is going to become
link |
00:28:46.800
unconscious, it's going to be the self driving car and not the network of a bazillion computers
link |
00:28:55.520
in a Google or Amazon data center that are all networked together to do whatever they
link |
00:29:03.160
do.
link |
00:29:04.160
So, in that sense, so you actually highlight, because that's what I work in Thomas Vehicles,
link |
00:29:09.640
you highlight the big gap between what we currently can't do and what we truly need
link |
00:29:15.600
to be able to do to solve the problem.
link |
00:29:18.500
Under that formulation, then consciousness and intelligence is something that basically
link |
00:29:24.600
a system should have in order to interact with us humans, as opposed to some kind of
link |
00:29:30.020
abstract notion of a consciousness.
link |
00:29:35.280
Consciousness is something that you need to have to be able to empathize, to be able to
link |
00:29:39.200
fear, understand what the fear of death is, all these aspects that are important for interacting
link |
00:29:47.440
with pedestrians, you need to be able to do basic computation based on our human desires
link |
00:29:56.160
and thoughts.
link |
00:29:57.160
And if you sort of, yeah, if you look at the dog, the dog clearly knows, I mean, I'm
link |
00:30:02.280
not the dog owner, but I have friends who have dogs, the dogs clearly know what the
link |
00:30:07.340
humans around them are going to do, or at least they have a model of what those humans
link |
00:30:11.400
are going to do and they learn.
link |
00:30:14.160
Some dogs know when you're going out and they want to go out with you, they're sad when
link |
00:30:19.060
you leave them alone, they cry, they're afraid because they were mistreated when they were
link |
00:30:26.080
younger.
link |
00:30:31.040
We don't assign sort of consciousness to dogs, or at least not all that much, but I also
link |
00:30:39.280
don't think they have none of that.
link |
00:30:42.500
So I think it's consciousness and intelligence are not all or nothing.
link |
00:30:50.360
The spectrum is really interesting.
link |
00:30:52.780
But in returning to programming languages and the way we think about building these
link |
00:30:58.760
kinds of things, about building intelligence, building consciousness, building artificial
link |
00:31:03.260
beings.
link |
00:31:04.260
So I think one of the exciting ideas came in the 17th century and with Leibniz, Hobbes,
link |
00:31:10.920
Descartes, where there's this feeling that you can convert all thought, all reasoning,
link |
00:31:18.520
all the thing that we find very special in our brains, you can convert all of that into
link |
00:31:24.480
logic.
link |
00:31:25.480
So you can formalize it, formal reasoning, and then once you formalize everything, all
link |
00:31:30.400
of knowledge, then you can just calculate and that's what we're doing with our brains
link |
00:31:34.400
is we're calculating.
link |
00:31:35.400
So there's this whole idea that this is possible, that this we can actually program.
link |
00:31:40.240
But they weren't aware of the concept of pattern matching in the sense that we are aware of
link |
00:31:46.520
it now.
link |
00:31:47.640
They sort of thought they had discovered incredible bits of mathematics like Newton's calculus
link |
00:31:57.640
and their sort of idealism, their sort of extension of what they could do with logic
link |
00:32:06.840
and math sort of went along those lines and they thought there's like, yeah, logic.
link |
00:32:18.000
There's like a bunch of rules and a bunch of input.
link |
00:32:22.020
They didn't realize that how you recognize a face is not just a bunch of rules but is
link |
00:32:28.600
a shit ton of data plus a circuit that sort of interprets the visual clues and the context
link |
00:32:39.160
and everything else and somehow can massively parallel pattern match against stored rules.
link |
00:32:49.400
I mean, if I see you tomorrow here in front of the Dropbox office, I might recognize you.
link |
00:32:56.320
Even if I'm wearing a different shirt, yeah, but if I see you tomorrow in a coffee shop
link |
00:33:01.320
in Belmont, I might have no idea that it was you or on the beach or whatever.
link |
00:33:06.640
I make those kind of mistakes myself all the time.
link |
00:33:10.160
I see someone that I only know as like, oh, this person is a colleague of my wife's and
link |
00:33:16.320
then I see them at the movies and I didn't recognize them.
link |
00:33:20.860
But do you see those, you call it pattern matching, do you see that rules is unable
link |
00:33:29.320
to encode that?
link |
00:33:32.380
Everything you see, all the pieces of information you look around this room, I'm wearing a black
link |
00:33:36.320
shirt, I have a certain height, I'm a human, all these, there's probably tens of thousands
link |
00:33:41.720
of facts you pick up moment by moment about this scene.
link |
00:33:45.680
You take them for granted and you aggregate them together to understand the scene.
link |
00:33:50.000
You don't think all of that could be encoded to where at the end of the day, you can just
link |
00:33:53.800
put it all on the table and calculate?
link |
00:33:57.440
I don't know what that means.
link |
00:33:58.840
I mean, yes, in the sense that there is no actual magic there, but there are enough layers
link |
00:34:08.680
of abstraction from the facts as they enter my eyes and my ears to the understanding of
link |
00:34:17.640
the scene that I don't think that AI has really covered enough of that distance.
link |
00:34:29.240
It's like if you take a human body and you realize it's built out of atoms, well, that
link |
00:34:37.800
is a uselessly reductionist view, right?
link |
00:34:41.960
The body is built out of organs, the organs are built out of cells, the cells are built
link |
00:34:46.380
out of proteins, the proteins are built out of amino acids, the amino acids are built
link |
00:34:53.240
out of atoms and then you get to quantum mechanics.
link |
00:34:58.040
So that's a very pragmatic view.
link |
00:34:59.920
I mean, obviously as an engineer, I agree with that kind of view, but you also have
link |
00:35:03.720
to consider the Sam Harris view of, well, intelligence is just information processing.
link |
00:35:13.160
Like you said, you take in sensory information, you do some stuff with it and you come up
link |
00:35:17.320
with actions that are intelligent.
link |
00:35:20.760
That makes it sound so easy.
link |
00:35:22.480
I don't know who Sam Harris is.
link |
00:35:24.280
Oh, well, it's a philosopher.
link |
00:35:26.400
So like this is how philosophers often think, right?
link |
00:35:29.680
And essentially that's what Descartes was, is wait a minute, if there is, like you said,
link |
00:35:33.760
no magic, so he basically says it doesn't appear like there's any magic, but we know
link |
00:35:39.320
so little about it that it might as well be magic.
link |
00:35:44.280
So just because we know that we're made of atoms, just because we know we're made
link |
00:35:47.800
of organs, the fact that we know very little how to get from the atoms to organs in a way
link |
00:35:53.280
that's recreatable means that you shouldn't get too excited just yet about the fact that
link |
00:36:00.400
you figured out that we're made of atoms.
link |
00:36:02.240
Right, and the same about taking facts as our sensory organs take them in and turning
link |
00:36:11.920
that into reasons and actions, that sort of, there are a lot of abstractions that we haven't
link |
00:36:19.820
quite figured out how to deal with those.
link |
00:36:23.960
I mean, sometimes, I don't know if I can go on a tangent or not, so if I take a simple
link |
00:36:37.440
program that parses, say I have a compiler that parses a program, in a sense the input
link |
00:36:45.640
routine of that compiler, of that parser, is a sensing organ, and it builds up a mighty
link |
00:36:55.640
complicated internal representation of the program it just saw, it doesn't just have
link |
00:37:01.960
a linear sequence of bytes representing the text of the program anymore, it has an abstract
link |
00:37:08.200
syntax tree, and I don't know how many of your viewers or listeners are familiar with
link |
00:37:15.480
compiler technology, but there's…
link |
00:37:18.680
Fewer and fewer these days, right?
link |
00:37:21.880
That's also true, probably.
link |
00:37:24.920
People want to take a shortcut, but there's sort of, this abstraction is a data structure
link |
00:37:30.360
that the compiler then uses to produce outputs that is relevant, like a translation of that
link |
00:37:37.480
program to machine code that can be executed by hardware, and then that data structure
link |
00:37:47.880
gets thrown away.
link |
00:37:50.600
When a fish or a fly sees, sort of gets visual impulses, I'm sure it also builds up some
link |
00:38:02.560
data structure, and for the fly that may be very minimal, a fly may have only a few, I
link |
00:38:10.000
mean, in the case of a fly's brain, I could imagine that there are few enough layers of
link |
00:38:17.680
abstraction that it's not much more than when it's darker here than it is here, well
link |
00:38:24.040
it can sense motion, because a fly sort of responds when you move your arm towards it,
link |
00:38:29.880
so clearly its visual processing is intelligent, well, not intelligent, but it has an abstraction
link |
00:38:39.240
for motion, and we still have similar things in, but much more complicated in our brains,
link |
00:38:46.440
I mean, otherwise you couldn't drive a car if you couldn't, if you didn't have an
link |
00:38:50.400
incredibly good abstraction for motion.
link |
00:38:53.480
Yeah, in some sense, the same abstraction for motion is probably one of the primary
link |
00:38:59.160
sources of our, of information for us, we just know what to do, I think we know what
link |
00:39:05.080
to do with that, we've built up other abstractions on top.
link |
00:39:08.280
We build much more complicated data structures based on that, and we build more persistent
link |
00:39:14.120
data structures, sort of after some processing, some information sort of gets stored in our
link |
00:39:20.320
memory pretty much permanently, and is available on recall, I mean, there are some things that
link |
00:39:27.240
you sort of, you're conscious that you're remembering it, like, you give me your phone
link |
00:39:34.040
number, I, well, at my age I have to write it down, but I could imagine, I could remember
link |
00:39:39.560
those seven numbers, or ten digits, and reproduce them in a while, if I sort of repeat them
link |
00:39:46.240
to myself a few times, so that's a fairly conscious form of memorization.
link |
00:39:53.320
On the other hand, how do I recognize your face, I have no idea.
link |
00:39:57.800
My brain has a whole bunch of specialized hardware that knows how to recognize faces,
link |
00:40:04.080
I don't know how much of that is sort of coded in our DNA, and how much of that is
link |
00:40:10.200
trained over and over between the ages of zero and three, but somehow our brains know
link |
00:40:17.960
how to do lots of things like that, that are useful in our interactions with other humans,
link |
00:40:26.000
without really being conscious of how it's done anymore.
link |
00:40:29.880
Right, so our actual day to day lives, we're operating at the very highest level of abstraction,
link |
00:40:36.200
we're just not even conscious of all the little details underlying it.
link |
00:40:39.760
There's compilers on top of, it's like turtles on top of turtles, or turtles all the way
link |
00:40:43.360
down, there's compilers all the way down, but that's essentially, you say that there's
link |
00:40:48.200
no magic, that's what I, what I was trying to get at, I think, is with Descartes started
link |
00:40:54.920
this whole train of saying that there's no magic, I mean, there's all this beforehand.
link |
00:40:59.600
Well didn't Descartes also have the notion though that the soul and the body were fundamentally
link |
00:41:06.120
separate?
link |
00:41:07.120
Separate, yeah, I think he had to write in God in there for political reasons, so I don't
link |
00:41:11.800
know actually, I'm not a historian, but there's notions in there that all of reasoning, all
link |
00:41:17.880
of human thought can be formalized.
link |
00:41:20.120
I think that continued in the 20th century with Russell and with Gadot's incompleteness
link |
00:41:28.480
theorem, this debate of what are the limits of the things that could be formalized, that's
link |
00:41:33.120
where the Turing machine came along, and this exciting idea, I mean, underlying a lot of
link |
00:41:37.960
computing that you can do quite a lot with a computer.
link |
00:41:43.160
You can encode a lot of the stuff we're talking about in terms of recognizing faces and so
link |
00:41:47.640
on, theoretically, in an algorithm that can then run on a computer.
link |
00:41:53.960
And in that context, I'd like to ask programming in a philosophical way, what does it mean
link |
00:42:05.040
to program a computer?
link |
00:42:06.480
So you said you write a Python program or compiled a C++ program that compiles to some
link |
00:42:13.360
byte code, it's forming layers, you're programming a layer of abstraction that's higher, how
link |
00:42:21.200
do you see programming in that context?
link |
00:42:24.920
Can it keep getting higher and higher levels of abstraction?
link |
00:42:29.800
I think at some point the higher levels of abstraction will not be called programming
link |
00:42:35.960
and they will not resemble what we call programming at the moment.
link |
00:42:44.720
There will not be source code, I mean, there will still be source code sort of at a lower
link |
00:42:52.080
level of the machine, just like there are still molecules and electrons and sort of
link |
00:42:59.320
proteins in our brains, but, and so there's still programming and system administration
link |
00:43:09.120
and who knows what, to keep the machine running, but what the machine does is a different level
link |
00:43:15.960
of abstraction in a sense, and as far as I understand the way that for the last decade
link |
00:43:23.060
or more people have made progress with things like facial recognition or the self driving
link |
00:43:28.440
cars is all by endless, endless amounts of training data where at least as a lay person,
link |
00:43:38.200
and I feel myself totally as a lay person in that field, it looks like the researchers
link |
00:43:47.420
who publish the results don't necessarily know exactly how their algorithms work, and
link |
00:43:57.400
I often get upset when I sort of read a sort of a fluff piece about Facebook in the newspaper
link |
00:44:04.840
or social networks and they say, well, algorithms, and that's like a totally different interpretation
link |
00:44:12.680
of the word algorithm, because for me, the way I was trained or what I learned when I
link |
00:44:19.240
was eight or ten years old, an algorithm is a set of rules that you completely understand
link |
00:44:25.920
that can be mathematically analyzed and you can prove things.
link |
00:44:30.720
You can like prove that Aristotelian sieve produces all prime numbers and only prime
link |
00:44:37.840
numbers.
link |
00:44:38.840
Yeah.
link |
00:44:39.840
So I don't know if you know who Andrej Karpathy is, I'm afraid not.
link |
00:44:44.360
So he's a head of AI at Tesla now, but he was at Stanford before and he has this cheeky
link |
00:44:51.980
way of calling this concept software 2.0.
link |
00:44:56.480
So let me disentangle that for a second.
link |
00:45:00.120
So kind of what you're referring to is the traditional, the algorithm, the concept of
link |
00:45:06.080
an algorithm, something that's there, it's clear, you can read it, you understand it,
link |
00:45:09.560
you can prove it's functioning as kind of software 1.0.
link |
00:45:14.800
And what software 2.0 is, is exactly what you described, which is you have neural networks,
link |
00:45:21.920
which is a type of machine learning that you feed a bunch of data and that neural network
link |
00:45:26.600
learns to do a function.
link |
00:45:30.200
All you specify is the inputs and the outputs you want and you can't look inside.
link |
00:45:35.220
You can't analyze it.
link |
00:45:37.040
All you can do is train this function to map the inputs to the outputs by giving a lot
link |
00:45:41.920
of data.
link |
00:45:42.920
And that's as programming becomes getting a lot of data.
link |
00:45:47.040
That's what programming is.
link |
00:45:48.920
Well, that would be programming 2.0.
link |
00:45:52.120
To programming 2.0.
link |
00:45:53.800
I wouldn't call that programming.
link |
00:45:55.600
It's just a different activity.
link |
00:45:57.480
Just like building organs out of cells is not called chemistry.
link |
00:46:02.640
Well, so let's just step back and think sort of more generally, of course.
link |
00:46:09.680
But you know, it's like as a parent teaching your kids, things can be called programming.
link |
00:46:18.080
In that same sense, that's how programming is being used.
link |
00:46:22.720
You're providing them data, examples, use cases.
link |
00:46:27.080
So imagine writing a function not by, not with for loops and clearly readable text,
link |
00:46:36.680
but more saying, well, here's a lot of examples of what this function should take.
link |
00:46:42.760
And here's a lot of examples of when it takes those functions, it should do this.
link |
00:46:47.860
And then figure out the rest.
link |
00:46:50.280
So that's the 2.0 concept.
link |
00:46:52.640
And so the question I have for you is like, it's a very fuzzy way.
link |
00:46:58.560
This is the reality of a lot of these pattern recognition systems and so on.
link |
00:47:01.680
It's a fuzzy way of quote unquote programming.
link |
00:47:05.400
What do you think about this kind of world?
link |
00:47:09.160
Should it be called something totally different than programming?
link |
00:47:13.640
If you're a software engineer, does that mean you're designing systems that are very, can
link |
00:47:21.000
be systematically tested, evaluated, they have a very specific specification and then this
link |
00:47:28.140
other fuzzy software 2.0 world, machine learning world, that's something else totally?
link |
00:47:33.520
Or is there some intermixing that's possible?
link |
00:47:41.000
Well the question is probably only being asked because we don't quite know what that software
link |
00:47:48.600
2.0 actually is.
link |
00:47:51.400
And I think there is a truism that every task that AI has tackled in the past, at some point
link |
00:48:02.960
we realized how it was done and then it was no longer considered part of artificial intelligence
link |
00:48:09.160
because it was no longer necessary to use that term.
link |
00:48:15.200
It was just, oh now we know how to do this.
link |
00:48:21.600
And a new field of science or engineering has been developed and I don't know if sort
link |
00:48:30.320
of every form of learning or sort of controlling computer systems should always be called programming.
link |
00:48:39.000
So I don't know, maybe I'm focused too much on the terminology.
link |
00:48:43.720
But I expect that there just will be different concepts where people with sort of different
link |
00:48:56.200
education and a different model of what they're trying to do will develop those concepts.
link |
00:49:07.920
I guess if you could comment on another way to put this concept is, I think the kind of
link |
00:49:17.240
functions that neural networks provide is things as opposed to being able to upfront
link |
00:49:23.480
prove that this should work for all cases you throw at it.
link |
00:49:28.720
All you're able, it's the worst case analysis versus average case analysis.
link |
00:49:32.320
All you're able to say is it seems on everything we've tested to work 99.9% of the time, but
link |
00:49:39.800
we can't guarantee it and it fails in unexpected ways.
link |
00:49:44.160
We can't even give you examples of how it fails in unexpected ways, but it's like really
link |
00:49:48.080
good most of the time.
link |
00:49:50.120
Is there no room for that in current ways we think about programming?
link |
00:50:00.720
programming 1.0 is actually sort of getting to that point too, where the sort of the ideal
link |
00:50:11.080
of a bug free program has been abandoned long ago by most software developers.
link |
00:50:21.120
We only care about bugs that manifest themselves often enough to be annoying.
link |
00:50:30.120
And we're willing to take the occasional crash or outage or incorrect result for granted
link |
00:50:40.680
because we can't possibly, we don't have enough programmers to make all the code bug free
link |
00:50:47.600
and it would be an incredibly tedious business.
link |
00:50:50.200
And if you try to throw formal methods at it, it becomes even more tedious.
link |
00:50:56.320
So every once in a while the user clicks on a link and somehow they get an error and the
link |
00:51:05.520
average user doesn't panic.
link |
00:51:07.360
They just click again and see if it works better the second time, which often magically
link |
00:51:14.840
it does, or they go up and they try some other way of performing their tasks.
link |
00:51:21.600
So that's sort of an end to end recovery mechanism and inside systems there is all
link |
00:51:29.880
sorts of retries and timeouts and fallbacks and I imagine that that sort of biological
link |
00:51:39.120
systems are even more full of that because otherwise they wouldn't survive.
link |
00:51:46.320
Do you think programming should be taught and thought of as exactly what you just said?
link |
00:51:54.160
I come from this kind of, you're always denying that fact always.
link |
00:52:01.560
In sort of basic programming education, the sort of the programs you're having students
link |
00:52:12.680
write are so small and simple that if there is a bug you can always find it and fix it.
link |
00:52:23.480
Because the sort of programming as it's being taught in some, even elementary, middle schools,
link |
00:52:29.720
in high school, introduction to programming classes in college typically, it's programming
link |
00:52:36.680
in the small.
link |
00:52:38.920
Very few classes sort of actually teach software engineering, building large systems.
link |
00:52:47.560
Every summer here at Dropbox we have a large number of interns.
link |
00:52:51.360
Every tech company on the West Coast has the same thing.
link |
00:52:56.720
These interns are always amazed because this is the first time in their life that they
link |
00:53:02.520
see what goes on in a really large software development environment.
link |
00:53:12.920
Everything they've learned in college was almost always about a much smaller scale and
link |
00:53:20.280
somehow that difference in scale makes a qualitative difference in how you do things and how you
link |
00:53:27.840
think about it.
link |
00:53:29.600
If you then take a few steps back into decades, 70s and 80s, when you were first thinking
link |
00:53:36.300
about Python or just that world of programming languages, did you ever think that there would
link |
00:53:41.840
be systems as large as underlying Google, Facebook, and Dropbox?
link |
00:53:46.720
Did you, when you were thinking about Python?
link |
00:53:51.440
I was actually always caught by surprise by sort of this, yeah, pretty much every stage
link |
00:53:57.520
of computing.
link |
00:53:59.680
So maybe just because you've spoken in other interviews, but I think the evolution of programming
link |
00:54:07.280
languages are fascinating and it's especially because it leads from my perspective towards
link |
00:54:13.080
greater and greater degrees of intelligence.
link |
00:54:15.640
I learned the first programming language I played with in Russia was with the Turtle
link |
00:54:21.880
logo.
link |
00:54:22.880
Logo, yeah.
link |
00:54:24.840
And if you look, I just have a list of programming languages, all of which I've now played with
link |
00:54:29.960
a little bit.
link |
00:54:30.960
I mean, they're all beautiful in different ways from Fortran, Cobalt, Lisp, Algol 60,
link |
00:54:36.640
Basic, Logo again, C, as a few, the object oriented came along in the 60s, Simula, Pascal,
link |
00:54:46.160
Smalltalk.
link |
00:54:47.560
All of that leads.
link |
00:54:48.560
They're all the classics.
link |
00:54:49.560
The classics.
link |
00:54:50.560
Yeah.
link |
00:54:51.560
The classic hits, right?
link |
00:54:52.560
Steam, that's built on top of Lisp.
link |
00:54:58.280
On the database side, SQL, C++, and all of that leads up to Python, Pascal too, and that's
link |
00:55:05.900
before Python, MATLAB, these kind of different communities, different languages.
link |
00:55:10.960
So can you talk about that world?
link |
00:55:13.240
I know that sort of Python came out of ABC, which I actually never knew that language.
link |
00:55:18.680
I just, having researched this conversation, went back to ABC and it looks remarkably,
link |
00:55:24.400
it has a lot of annoying qualities, but underneath those, like all caps and so on, but underneath
link |
00:55:31.240
that, there's elements of Python that are quite, they're already there.
link |
00:55:35.720
That's where I got all the good stuff.
link |
00:55:37.540
All the good stuff.
link |
00:55:38.540
So, but in that world, you're swimming these programming languages, were you focused on
link |
00:55:41.580
just the good stuff in your specific circle, or did you have a sense of what is everyone
link |
00:55:48.080
chasing?
link |
00:55:49.080
You said that every programming language is built to scratch an itch.
link |
00:55:57.000
Were you aware of all the itches in the community?
link |
00:55:59.920
And if not, or if yes, I mean, what itch were you trying to scratch with Python?
link |
00:56:05.080
Well, I'm glad I wasn't aware of all the itches because I would probably not have been able
link |
00:56:12.040
to do anything.
link |
00:56:14.040
I mean, if you're trying to solve every problem at once, you'll solve nothing.
link |
00:56:19.760
Well, yeah, it's too overwhelming.
link |
00:56:23.880
And so I had a very, very focused problem.
link |
00:56:28.360
I wanted a programming language that sat somewhere in between shell scripting and C. And now,
link |
00:56:41.880
arguably, there is like, one is higher level, one is lower level.
link |
00:56:48.720
And Python is sort of a language of an intermediate level, although it's still pretty much at
link |
00:56:56.760
the high level end.
link |
00:57:00.560
I was thinking about much more about, I want a tool that I can use to be more productive
link |
00:57:11.200
as a programmer in a very specific environment.
link |
00:57:16.640
And I also had given myself a time budget for the development of the tool.
link |
00:57:22.280
And that was sort of about three months for both the design, like thinking through what
link |
00:57:29.340
are all the features of the language syntactically and semantically, and how do I implement the
link |
00:57:38.900
whole pipeline from parsing the source code to executing it.
link |
00:57:43.680
So I think both with the timeline and the goals, it seems like productivity was at the
link |
00:57:51.440
core of it as a goal.
link |
00:57:54.040
So like, for me in the 90s, and the first decade of the 21st century, I was always doing
link |
00:58:01.280
machine learning, AI programming for my research was always in C++.
link |
00:58:07.620
And then the other people who are a little more mechanical engineering, electrical engineering,
link |
00:58:14.240
are MATLABby.
link |
00:58:15.240
They're a little bit more MATLAB focused.
link |
00:58:18.520
Those are the world, and maybe a little bit Java too.
link |
00:58:21.200
But people who are more interested in emphasizing the object oriented nature of things.
link |
00:58:29.160
So within the last 10 years or so, especially with the oncoming of neural networks and these
link |
00:58:34.920
packages that are built on Python to interface with neural networks, I switched to Python
link |
00:58:41.360
and it's just, I've noticed a significant boost that I can't exactly, because I don't
link |
00:58:47.120
think about it, but I can't exactly put into words why I'm just much, much more productive.
link |
00:58:52.840
Just being able to get the job done much, much faster.
link |
00:58:56.400
So how do you think, whatever that qualitative difference is, I don't know if it's quantitative,
link |
00:59:01.880
it could be just a feeling, I don't know if I'm actually more productive, but how
link |
00:59:07.280
do you think about...
link |
00:59:08.280
You probably are.
link |
00:59:09.280
Yeah.
link |
00:59:10.280
Well, that's right.
link |
00:59:11.880
I think there's elements, let me just speak to one aspect that I think that was affecting
link |
00:59:15.400
my productivity is C++ was, I really enjoyed creating performant code and creating a beautiful
link |
00:59:26.160
structure where everything that, you know, this kind of going into this, especially with
link |
00:59:31.000
the newer and newer standards of templated programming of just really creating this beautiful
link |
00:59:37.080
formal structure that I found myself spending most of my time doing that as opposed to getting
link |
00:59:42.000
it, parsing a file and extracting a few keywords or whatever the task was trying to do.
link |
00:59:47.520
So what is it about Python?
link |
00:59:49.980
How do you think of productivity in general as you were designing it now, sort of through
link |
00:59:54.520
the decades, last three decades, what do you think it means to be a productive programmer?
link |
01:00:00.120
And how did you try to design it into the language?
link |
01:00:03.560
There are different tasks and as a programmer, it's useful to have different tools available
link |
01:00:10.400
that sort of are suitable for different tasks.
link |
01:00:13.940
So I still write C code, I still write shell code, but I write most of my things in Python.
link |
01:00:25.600
Why do I still use those other languages, because sometimes the task just demands it.
link |
01:00:33.000
And well, I would say most of the time the task actually demands a certain language because
link |
01:00:39.000
the task is not write a program that solves problem X from scratch, but it's more like
link |
01:00:45.600
fix a bug in existing program X or add a small feature to an existing large program.
link |
01:00:56.680
But even if you're not constrained in your choice of language by context like that, there
link |
01:01:10.160
is still the fact that if you write it in a certain language, then you have this balance
link |
01:01:21.360
between how long does it take you to write the code and how long does the code run?
link |
01:01:31.840
And when you're in the phase of exploring solutions, you often spend much more time
link |
01:01:42.760
writing the code than running it because every time you've run it, you see that the output
link |
01:01:50.720
is not quite what you wanted and you spend some more time coding.
link |
01:01:58.480
And a language like Python just makes that iteration much faster because there are fewer
link |
01:02:06.760
details that you have to get right before your program compiles and runs.
link |
01:02:19.480
There are libraries that do all sorts of stuff for you, so you can sort of very quickly take
link |
01:02:26.400
a bunch of existing components, put them together, and get your prototype application running.
link |
01:02:36.320
Just like when I was building electronics, I was using a breadboard most of the time,
link |
01:02:42.860
so I had this sprawl out circuit that if you shook it, it would stop working because it
link |
01:02:51.320
was not put together very well, but it functioned and all I wanted was to see that it worked
link |
01:02:58.800
and then move on to the next schematic or design or add something to it.
link |
01:03:05.000
Once you've sort of figured out, oh, this is the perfect design for my radio or light
link |
01:03:10.500
sensor or whatever, then you can say, okay, how do we design a PCB for this?
link |
01:03:15.800
How do we solder the components in a small space?
link |
01:03:19.920
How do we make it so that it is robust against, say, voltage fluctuations or mechanical disruption?
link |
01:03:32.840
I know nothing about that when it comes to designing electronics, but I know a lot about
link |
01:03:37.320
that when it comes to writing code.
link |
01:03:40.400
So the initial steps are efficient, fast, and there's not much stuff that gets in the
link |
01:03:46.080
way, but you're kind of describing, like Darwin described the evolution of species, right?
link |
01:03:56.680
You're observing of what is true about Python.
link |
01:04:00.520
Now if you take a step back, if the act of creating languages is art and you had three
link |
01:04:07.800
months to do it, initial steps, so you just specified a bunch of goals, sort of things
link |
01:04:15.640
that you observe about Python, perhaps you had those goals, but how do you create the
link |
01:04:19.400
rules, the syntactic structure, the features that result in those?
link |
01:04:25.600
So I have in the beginning and I have follow up questions about through the evolution of
link |
01:04:29.880
Python too, but in the very beginning when you were sitting there creating the lexical
link |
01:04:35.440
analyzer or whatever.
link |
01:04:37.440
Python was still a big part of it because I sort of, I said to myself, I don't want
link |
01:04:47.240
to have to design everything from scratch, I'm going to borrow features from other languages
link |
01:04:53.640
that I like.
link |
01:04:54.640
Oh, interesting.
link |
01:04:55.640
So you basically, exactly, you first observe what you like.
link |
01:04:58.360
Yeah, and so that's why if you're 17 years old and you want to sort of create a programming
link |
01:05:05.240
language, you're not going to be very successful at it because you have no experience with
link |
01:05:11.600
other languages, whereas I was in my, let's say mid 30s, I had written parsers before,
link |
01:05:24.300
so I had worked on the implementation of ABC, I had spent years debating the design of ABC
link |
01:05:30.880
with its authors, with its designers, I had nothing to do with the design, it was designed
link |
01:05:37.520
fully as it ended up being implemented when I joined the team.
link |
01:05:42.080
But so you borrow ideas and concepts and very concrete sort of local rules from different
link |
01:05:51.440
languages like the indentation and certain other syntactic features from ABC, but I chose
link |
01:05:58.920
to borrow string literals and how numbers work from C and various other things.
link |
01:06:07.960
So in then, if you take that further, so yet you've had this funny sounding, but I think
link |
01:06:13.800
surprisingly accurate and at least practical title of benevolent dictator for life for
link |
01:06:21.000
quite, you know, for the last three decades or whatever, or no, not the actual title,
link |
01:06:25.240
but functionally speaking.
link |
01:06:27.940
So you had to make decisions, design decisions.
link |
01:06:34.280
Can you maybe, let's take Python 2, so releasing Python 3 as an example.
link |
01:06:41.960
It's not backward compatible to Python 2 in ways that a lot of people know.
link |
01:06:47.240
So what was that deliberation, discussion, decision like?
link |
01:06:50.640
Yeah.
link |
01:06:51.640
What was the psychology of that experience?
link |
01:06:54.520
Do you regret any aspects of how that experience undergone that?
link |
01:06:58.520
Well, yeah, so it was a group process really.
link |
01:07:03.040
At that point, even though I was BDFL in name and certainly everybody sort of respected
link |
01:07:11.880
my position as the creator and the current sort of owner of the language design, I was
link |
01:07:22.160
looking at everyone else for feedback.
link |
01:07:26.560
Sort of Python 3.0 in some sense was sparked by other people in the community pointing
link |
01:07:35.280
out, oh, well, there are a few issues that sort of bite users over and over.
link |
01:07:46.360
Can we do something about that?
link |
01:07:48.920
And for Python 3, we took a number of those Python words as they were called at the time
link |
01:07:56.360
and we said, can we try to sort of make small changes to the language that address those
link |
01:08:04.800
words?
link |
01:08:06.560
And we had sort of in the past, we had always taken backwards compatibility very seriously.
link |
01:08:15.360
And so many Python words in earlier versions had already been resolved because they could
link |
01:08:20.420
be resolved while maintaining backwards compatibility or sort of using a very gradual path of evolution
link |
01:08:29.740
of the language in a certain area.
link |
01:08:31.960
And so we were stuck with a number of words that were widely recognized as problems, not
link |
01:08:39.760
like roadblocks, but nevertheless sort of things that some people trip over and you know that
link |
01:08:47.680
that's always the same thing that people trip over when they trip.
link |
01:08:52.080
And we could not think of a backwards compatible way of resolving those issues.
link |
01:08:58.480
But it's still an option to not resolve the issues, right?
link |
01:09:01.920
And so yes, for a long time, we had sort of resigned ourselves to, well, okay, the language
link |
01:09:07.920
is not going to be perfect in this way and that way and that way.
link |
01:09:13.400
And we sort of, certain of these, I mean, there are still plenty of things where you
link |
01:09:19.440
can say, well, that particular detail is better in Java or in R or in Visual Basic or whatever.
link |
01:09:32.680
And we're okay with that because, well, we can't easily change it.
link |
01:09:37.960
It's not too bad.
link |
01:09:38.960
We can do a little bit with user education or we can have a static analyzer or warnings
link |
01:09:47.180
in the parse or something.
link |
01:09:49.440
But there were things where we thought, well, these are really problems that are not going
link |
01:09:54.880
away.
link |
01:09:55.880
They are getting worse in the future.
link |
01:10:00.840
We should do something about that.
link |
01:10:03.040
But ultimately there is a decision to be made, right?
link |
01:10:05.640
So was that the toughest decision in the history of Python you had to make as the benevolent
link |
01:10:13.320
dictator for life?
link |
01:10:15.180
Or if not, what are there, maybe even on the smaller scale, what was the decision where
link |
01:10:20.160
you were really torn up about?
link |
01:10:22.040
Well, the toughest decision was probably to resign.
link |
01:10:25.800
All right, let's go there.
link |
01:10:28.120
Hold on a second then.
link |
01:10:29.360
Let me just, because in the interest of time too, because I have a few cool questions for
link |
01:10:33.200
you and let's touch a really important one because it was quite dramatic and beautiful
link |
01:10:38.160
in certain kinds of ways.
link |
01:10:40.400
In July this year, three months ago, you wrote, now that PEP 572 is done, I don't ever want
link |
01:10:47.320
to have to fight so hard for a PEP and find that so many people despise my decisions.
link |
01:10:52.680
I would like to remove myself entirely from the decision process.
link |
01:10:56.240
I'll still be there for a while as an ordinary core developer and I'll still be available
link |
01:11:01.520
to mentor people, possibly more available.
link |
01:11:05.440
But I'm basically giving myself a permanent vacation from being BDFL, benevolent dictator
link |
01:11:11.000
for life.
link |
01:11:12.000
And you all will be on your own.
link |
01:11:14.240
First of all, it's almost Shakespearean.
link |
01:11:19.720
I'm not going to appoint a successor.
link |
01:11:22.300
So what are you all going to do?
link |
01:11:24.640
Create a democracy, anarchy, a dictatorship, a federation?
link |
01:11:29.240
So that was a very dramatic and beautiful set of statements.
link |
01:11:34.560
It's almost, it's open ended nature called the community to create a future for Python.
link |
01:11:40.080
It's just kind of a beautiful aspect to it.
link |
01:11:43.280
So what, and dramatic, you know, what was making that decision like?
link |
01:11:48.320
What was on your heart, on your mind, stepping back now a few months later?
link |
01:11:54.560
I'm glad you liked the writing because it was actually written pretty quickly.
link |
01:12:02.940
It was literally something like after months and months of going around in circles, I had
link |
01:12:14.240
finally approved PEP572, which I had a big hand in its design, although I didn't initiate
link |
01:12:26.240
it originally.
link |
01:12:27.760
I sort of gave it a bunch of nudges in a direction that would be better for the language.
link |
01:12:36.320
So sorry, just to ask, is async IO, that's the one or no?
link |
01:12:40.320
PEP572 was actually a small feature, which is assignment expressions.
link |
01:12:49.320
That had been, there was just a lot of debate where a lot of people claimed that they knew
link |
01:12:58.200
what was Pythonic and what was not Pythonic, and they knew that this was going to destroy
link |
01:13:04.800
the language.
link |
01:13:06.080
This was like a violation of Python's most fundamental design philosophy, and I thought
link |
01:13:11.800
that was all bullshit because I was in favor of it, and I would think I know something
link |
01:13:17.200
about Python's design philosophy.
link |
01:13:19.120
So I was really tired and also stressed of that thing, and literally after sort of announcing
link |
01:13:26.340
I was going to accept it, a certain Wednesday evening I had finally sent the email, it's
link |
01:13:34.560
accepted.
link |
01:13:35.560
I can just go implement it.
link |
01:13:38.920
So I went to bed feeling really relieved, that's behind me.
link |
01:13:44.120
And I wake up Thursday morning, 7 a.m., and I think, well, that was the last one that's
link |
01:13:54.320
going to be such a terrible debate, and that's the last time that I let myself be so stressed
link |
01:14:03.880
out about a pep decision.
link |
01:14:06.520
I should just resign.
link |
01:14:07.920
I've been sort of thinking about retirement for half a decade, I've been joking and sort
link |
01:14:15.520
of mentioning retirement, sort of telling the community at some point in the future
link |
01:14:22.460
I'm going to retire, don't take that FL part of my title too literally.
link |
01:14:29.400
And I thought, okay, this is it.
link |
01:14:32.080
I'm done, I had the day off, I wanted to have a good time with my wife, we were going to
link |
01:14:39.200
a little beach town nearby, and in I think maybe 15, 20 minutes I wrote that thing that
link |
01:14:48.480
you just called Shakespearean.
link |
01:14:51.320
The funny thing is I didn't even realize what a monumental decision it was, because
link |
01:15:01.560
five minutes later I read that link to my message back on Twitter, where people were
link |
01:15:09.200
already discussing on Twitter, Guido resigned as the BDFL.
link |
01:15:15.280
And I had posted it on an internal forum that I thought was only read by core developers,
link |
01:15:22.440
so I thought I would at least have one day before the news would sort of get out.
link |
01:15:28.520
The on your own aspects had also an element of quite, it was quite a powerful element
link |
01:15:36.200
of the uncertainty that lies ahead, but can you also just briefly talk about, for example
link |
01:15:43.080
I play guitar as a hobby for fun, and whenever I play people are super positive, super friendly,
link |
01:15:49.920
they're like, this is awesome, this is great.
link |
01:15:52.680
But sometimes I enter as an outside observer, I enter the programming community and there
link |
01:15:57.520
seems to sometimes be camps on whatever the topic, and the two camps, the two or plus
link |
01:16:05.560
camps, are often pretty harsh at criticizing the opposing camps.
link |
01:16:11.700
As an onlooker, I may be totally wrong on this, but what do you think of this?
link |
01:16:14.880
Yeah, holy wars are sort of a favorite activity in the programming community.
link |
01:16:19.760
And what is the psychology behind that?
link |
01:16:22.120
Is that okay for a healthy community to have?
link |
01:16:25.120
Is that a productive force ultimately for the evolution of a language?
link |
01:16:29.760
Well, if everybody is patting each other on the back and never telling the truth, it would
link |
01:16:39.080
not be a good thing.
link |
01:16:40.840
I think there is a middle ground where sort of being nasty to each other is not okay,
link |
01:16:52.760
but there is a middle ground where there is healthy ongoing criticism and feedback that
link |
01:17:01.760
is very productive.
link |
01:17:04.780
And you mean at every level you see that.
link |
01:17:07.760
I mean, someone proposes to fix a very small issue in a code base, chances are that some
link |
01:17:17.760
reviewer will sort of respond by saying, well, actually, you can do it better the other way.
link |
01:17:27.080
When it comes to deciding on the future of the Python core developer community, we now
link |
01:17:34.360
have, I think, five or six competing proposals for a constitution.
link |
01:17:41.160
So that future, do you have a fear of that future, do you have a hope for that future?
link |
01:17:48.040
I'm very confident about that future.
link |
01:17:51.280
By and large, I think that the debate has been very healthy and productive.
link |
01:17:58.920
And I actually, when I wrote that resignation email, I knew that Python was in a very good
link |
01:18:07.680
spot and that the Python core developer community, the group of 50 or 100 people who sort of
link |
01:18:16.840
write or review most of the code that goes into Python, those people get along very well
link |
01:18:24.720
most of the time.
link |
01:18:27.680
A large number of different areas of expertise are represented, different levels of experience
link |
01:18:40.120
in the Python core dev community, different levels of experience completely outside it
link |
01:18:45.440
in software development in general, large systems, small systems, embedded systems.
link |
01:18:53.040
So I felt okay resigning because I knew that the community can really take care of itself.
link |
01:19:03.880
And out of a grab bag of future feature developments, let me ask if you can comment, maybe on all
link |
01:19:12.360
very quickly, concurrent programming, parallel computing, async IO.
link |
01:19:19.120
These are things that people have expressed hope, complained about, whatever, have discussed
link |
01:19:24.880
on Reddit.
link |
01:19:25.880
Async IO, so the parallelization in general, packaging, I was totally clueless on this.
link |
01:19:32.200
I just used pip to install stuff, but apparently there's pipenv, poetry, there's these dependency
link |
01:19:38.600
packaging systems that manage dependencies and so on.
link |
01:19:41.300
They're emerging and there's a lot of confusion about what's the right thing to use.
link |
01:19:45.520
Then also functional programming, are we going to get more functional programming or not,
link |
01:19:56.360
this kind of idea.
link |
01:19:59.040
And of course the GIL connected to the parallelization, I suppose, the global interpreter lock problem.
link |
01:20:08.280
Can you just comment on whichever you want to comment on?
link |
01:20:12.800
Well, let's take the GIL and parallelization and async IO as one topic.
link |
01:20:25.440
I'm not that hopeful that Python will develop into a sort of high concurrency, high parallelism
link |
01:20:35.820
language.
link |
01:20:37.960
That's sort of the way the language is designed, the way most users use the language, the way
link |
01:20:44.800
the language is implemented, all make that a pretty unlikely future.
link |
01:20:50.280
So you think it might not even need to, really the way people use it, it might not be something
link |
01:20:56.040
that should be of great concern.
link |
01:20:58.160
I think async IO is a special case because it sort of allows overlapping IO and only
link |
01:21:05.620
IO and that is a sort of best practice of supporting very high throughput IO, many connections
link |
01:21:18.160
per second.
link |
01:21:21.680
I'm not worried about that.
link |
01:21:22.780
I think async IO will evolve.
link |
01:21:25.280
There are a couple of competing packages.
link |
01:21:27.440
We have some very smart people who are sort of pushing us to make async IO better.
link |
01:21:36.800
Parallel computing, I think that Python is not the language for that.
link |
01:21:43.800
There are ways to work around it, but you can't expect to write an algorithm in Python
link |
01:21:53.560
and have a compiler automatically parallelize that.
link |
01:21:57.440
What you can do is use a package like NumPy and there are a bunch of other very powerful
link |
01:22:03.520
packages that sort of use all the CPUs available because you tell the package, here's the data,
link |
01:22:12.480
here's the abstract operation to apply over it, go at it, and then we're back in the C++
link |
01:22:19.040
world.
link |
01:22:20.040
Those packages are themselves implemented usually in C++.
link |
01:22:24.600
That's where TensorFlow and all these packages come in, where they parallelize across GPUs,
link |
01:22:28.000
for example, they take care of that for you.
link |
01:22:30.480
In terms of packaging, can you comment on the future of packaging in Python?
link |
01:22:36.600
Packaging has always been my least favorite topic.
link |
01:22:42.640
It's a really tough problem because the OS and the platform want to own packaging, but
link |
01:22:55.600
their packaging solution is not specific to a language.
link |
01:23:01.000
If you take Linux, there are two competing packaging solutions for Linux or for Unix
link |
01:23:07.480
in general, but they all work across all languages.
link |
01:23:15.000
Several languages like Node, JavaScript, Ruby, and Python all have their own packaging solutions
link |
01:23:24.760
that only work within the ecosystem of that language.
link |
01:23:29.480
What should you use?
link |
01:23:31.920
That is a tough problem.
link |
01:23:34.560
My own approach is I use the system packaging system to install Python, and I use the Python
link |
01:23:43.520
packaging system then to install third party Python packages.
link |
01:23:49.280
That's what most people do.
link |
01:23:51.480
Ten years ago, Python packaging was really a terrible situation.
link |
01:23:56.400
Nowadays, pip is the future, there is a separate ecosystem for numerical and scientific Python
link |
01:24:05.360
based on Anaconda.
link |
01:24:08.200
Those two can live together.
link |
01:24:09.760
I don't think there is a need for more than that.
link |
01:24:13.600
That's packaging.
link |
01:24:14.600
Well, at least for me, that's where I've been extremely happy.
link |
01:24:18.720
I didn't even know this was an issue until it was brought up.
link |
01:24:22.320
In the interest of time, let me sort of skip through a million other questions I have.
link |
01:24:27.600
So I watched the five and a half hour oral history that you've done with the Computer
link |
01:24:32.880
History Museum, and the nice thing about it, it gave this, because of the linear progression
link |
01:24:37.600
of the interview, it gave this feeling of a life, you know, a life well lived with interesting
link |
01:24:44.480
things in it, sort of a pretty, I would say a good spend of this little existence we have
link |
01:24:52.160
on Earth.
link |
01:24:53.160
So, outside of your family, looking back, what about this journey are you really proud
link |
01:24:59.840
of?
link |
01:25:00.840
Are there moments that stand out, accomplishments, ideas?
link |
01:25:07.040
Is it the creation of Python itself that stands out as a thing that you look back and say,
link |
01:25:14.040
damn, I did pretty good there?
link |
01:25:16.480
Well, I would say that Python is definitely the best thing I've ever done, and I wouldn't
link |
01:25:25.520
sort of say just the creation of Python, but the way I sort of raised Python, like a baby.
link |
01:25:36.560
I didn't just conceive a child, but I raised a child, and now I'm setting the child free
link |
01:25:42.480
in the world, and I've set up the child to sort of be able to take care of himself, and
link |
01:25:50.200
I'm very proud of that.
link |
01:25:52.640
And as the announcer of Monty Python's Flying Circus used to say, and now for something
link |
01:25:56.740
completely different, do you have a favorite Monty Python moment, or a moment in Hitchhiker's
link |
01:26:02.280
Guide, or any other literature show or movie that cracks you up when you think about it?
link |
01:26:07.720
You can always play me the dead parrot sketch.
link |
01:26:11.320
Oh, that's brilliant.
link |
01:26:13.680
That's my favorite as well.
link |
01:26:14.680
It's pushing up the daisies.
link |
01:26:15.680
Okay, Greta, thank you so much for talking with me today.
link |
01:26:20.680
Lex, this has been a great conversation.