back to indexBrian Kernighan: UNIX, C, AWK, AMPL, and Go Programming | Lex Fridman Podcast #109
link |
The following is a conversation with Brian Kernighan,
link |
a professor of computer science at Princeton University.
link |
He was a key figure in the computer science community
link |
in the early Unix days, alongside Unix creators,
link |
Ken Thompson and Dennis Ritchie.
link |
He coauthored the C programming language with Dennis Ritchie,
link |
the creator of C, and has written a lot of books
link |
on programming, computers, and life,
link |
including The Practice of Programming,
link |
the Go programming language, and his latest,
link |
Unix, A History and a Memoir.
link |
He cocreated AUK, the text processing language
link |
used by Linux folks like myself.
link |
He co designed Ample, an algebraic modeling language
link |
that I personally love and have used a lot in my life
link |
for large scale optimization.
link |
I think I can keep going for a long time
link |
with his creations and accomplishments,
link |
which is funny because given all that,
link |
he's one of the most humble and kind people
link |
I've spoken to on this podcast.
link |
Quick summary of the ads, two new sponsors,
link |
the amazing self cooling 8sleep mattress
link |
and Raycon earbuds.
link |
Please consider supporting the podcast
link |
by going to 8sleep.com slash Lex
link |
and going to buyraycon.com slash Lex.
link |
Click the links, buy the stuff.
link |
It really is the best way to support this podcast
link |
and the journey I'm on.
link |
If you enjoy this thing, subscribe on YouTube,
link |
review it with 5,000 Apple podcasts,
link |
support it on Patreon,
link |
or connect with me on Twitter at Lex Friedman.
link |
As usual, I'll do a few minutes of ads now
link |
and never any ads in the middle
link |
that could break the flow of the conversation.
link |
This show is sponsored by 8sleep
link |
and it's incredible pod pro mattress
link |
that you can check out at 8sleep.com slash Lex
link |
The mattress controls temperature with an app
link |
and can cool down to as low as 55 degrees.
link |
Research shows that temperature has a big impact
link |
on the quality of our sleep.
link |
Anecdotally, it's been a game changer for me.
link |
The pod pro is packed with sensors
link |
that track heart rate, heart rate variability,
link |
and respiratory rate,
link |
showing it all on their app once you wake up.
link |
Plus, if you have a partner,
link |
you can control the temperature of each side of the bed.
link |
I don't happen to have one,
link |
but the 8sleep app reminds me
link |
that I should probably get on that.
link |
So ladies, if a temperature controlled mattress
link |
isn't a good reason to apply,
link |
I don't know what is.
link |
The app's health metrics are amazing,
link |
but the cooling alone is honestly worth the money.
link |
As some of you know, I don't always sleep,
link |
but when I do, I choose the 8sleep pod pro mattress.
link |
Check it out at 8sleep.com slash Lex
link |
This show is also sponsored by Raycon earbuds.
link |
Get them at buyraycon.com slash lex.
link |
They've quickly become my main method
link |
of listening to podcasts, audio books,
link |
and music when I run,
link |
do the pushups and pullups
link |
that I've begun to hate at this point,
link |
or just living life.
link |
In fact, I often listen to brown noise with these
link |
when I'm thinking deeply about something.
link |
It helps me focus the mind.
link |
They're super comfortable, pair easily,
link |
great sound, great bass, six hours of playtime.
link |
In fact, for fun, I have one of the earbuds in now
link |
and I'm listening to Europa by Santana,
link |
probably one of my favorite guitar songs.
link |
It kind of makes me feel like I'm in a music video.
link |
So they told me to say that a bunch of celebrities
link |
use these like Snoop Dogg, Melissa Etheridge, and Cardi B.
link |
I don't even know who Cardi B is,
link |
but her earbud game is on point.
link |
To mention celebrities I actually care about,
link |
I'm sure if Richard Feynman was still with us,
link |
he'd be listening to the Joe Rogan Experience
link |
with Raycon earbuds.
link |
Get them at buyraycon.com slash lex.
link |
It's how they know I sent you
link |
and increases the chance that he'll support
link |
this podcast in the future.
link |
So for all of the sponsors, click all of the links.
link |
It really helps this podcast.
link |
And now, here's my conversation with Brian Kernighan.
link |
Unix started being developed 50 years ago.
link |
It'd be more than 50 years ago.
link |
Can you tell the story like you describe in your new book
link |
of how Unix was created?
link |
Ha, if I can remember that far back,
link |
it was some while ago.
link |
So I think the gist of it is that at Bell Labs,
link |
in 1969, there were a group of people
link |
who had just finished working on the Multics project,
link |
which was itself a follow on to CTSS.
link |
So we can go back sort of an infinite regress in time,
link |
but the CTSS was a very, very, very nice time sharing system.
link |
It was very nice to use.
link |
I actually used it that summer I spent in Cambridge in 1966.
link |
What was the hardware there?
link |
So what's the operating system, what's the hardware there?
link |
What's the CTSS look like?
link |
So CTSS looked like kind of like
link |
a standard time sharing system.
link |
Certainly at the time, it was the only time sharing.
link |
Let's go back to the basics.
link |
What's a time sharing system?
link |
Okay, in the beginning was the word
link |
and the word was the system.
link |
And then there was time sharing systems.
link |
Yeah, if we go back into, let's call it the 1950s
link |
and early 1960s, most computing was done on very big
link |
computers, physically big, although not terribly powerful
link |
by today's standards, that were maintained
link |
in very large rooms and you use things like punch cards
link |
to write your programs on and talk to them.
link |
So you would take a deck of cards,
link |
write your program on it, send it over a counter,
link |
hand it to an operator and some while later
link |
back would come something that said,
link |
oh, you made a mistake and then you'd recycle.
link |
And so it was very, very slow.
link |
So the idea of time sharing was that you take
link |
basically that same computer, but connect to it
link |
with something that looked like an electric typewriter.
link |
They could be a long distance away, it could be close,
link |
but fundamentally what the operating system did
link |
was to give each person who was connected to it
link |
and wanting to do something a small slice of time
link |
to do a particular job.
link |
So I might be editing a file, so I would be typing
link |
and every time I hit a keystroke,
link |
the operating system would wake up and said,
link |
oh, he typed character, let me remember that.
link |
Then it'd go back to doing something else.
link |
So it'd be going around and around a group of people
link |
who were trying to get something done, giving each
link |
a small slice of time and giving them each the illusion
link |
that they pretty much had the whole machine to themselves
link |
and hence time sharing, that is sharing the computing time
link |
resource of the computer among a number of people
link |
who were doing it.
link |
Without the individual people being aware
link |
that there's others in a sense, the illusion,
link |
the feelings that the machine is your own.
link |
Pretty much that was the idea.
link |
Yes, if it were well done and if it were fast enough
link |
and other people weren't doing too much,
link |
you did have the illusion that you had the whole machine
link |
to yourself and it was very much better
link |
than the punch card model.
link |
And so CTSS, the compatible time sharing system
link |
was I think arguably the first of these.
link |
It was done I guess technically in 64 or something like that.
link |
It ran on an IBM 7094, slightly modified
link |
to have twice as much memory as the norm.
link |
It had two banks of 32K words instead of one.
link |
Each word was 36 bits, so call it
link |
about 150 kilobytes times two.
link |
So by today's standards, that's down in the noise.
link |
But at the time, that was a lot of memory
link |
and memory was expensive.
link |
So CTSS was just a wonderful environment to work on.
link |
It was done by the people at MIT,
link |
led by Fernando Corbato, Corby who died just earlier
link |
this year, and a bunch of other folks.
link |
So I spent the summer of 66 working on that,
link |
had a great time, met a lot of really nice people
link |
and indirectly knew of people at Bell Labs
link |
who were also working on a follow on to CTSS
link |
that was called Multics.
link |
So Multics was meant to be the system
link |
that would do everything that CTSS did
link |
but do it better for a larger population.
link |
All the usual stuff.
link |
Now the actual time sharing, the scheduling,
link |
what's the algorithm that performs the scheduling?
link |
What's that look like?
link |
How much magic is there?
link |
What are the metrics?
link |
How does it all work in the beginning?
link |
So the answer is I don't have a clue.
link |
I think the basic idea was nothing more
link |
than who all wants to get something done.
link |
Suppose that things are very quiet
link |
in the middle of the night,
link |
then I get all the time that I want.
link |
Suppose that you and I are contending at high noon
link |
for something like that,
link |
then probably the simplest algorithm is a round robin one
link |
that gives you a bit of time, gives me a bit of time.
link |
And then we could adapt to that.
link |
Like what are you trying to do?
link |
Are you text editing or are you compiling or something?
link |
And then we might adjust the scheduler
link |
according to things like that.
link |
So okay, so Multics was trying to just do some of the,
link |
clean it up a little bit.
link |
Well, it was meant to be much more than that.
link |
So Multics was the multiplexed information
link |
and computing service and it was meant to be
link |
a very large thing that would provide computing utility.
link |
Something that where you could actually think of it
link |
as just a plug in the wall service.
link |
Sort of like cloud computing today.
link |
Same idea, but 50 odd years earlier.
link |
And so what Multics offered
link |
was a richer operating system environment,
link |
a piece of hardware that was better designed
link |
for doing the kind of sharing of resources.
link |
And presumably lots of other things.
link |
Do you think people at that time had the dream
link |
of what cloud computing is starting to become now,
link |
which is computing is everywhere.
link |
That you can just plug in almost,
link |
and you never know how the magic works.
link |
You just kind of plug in, add your little computation
link |
that you need to perform and it does it.
link |
Was that the dream?
link |
I don't know where that was the dream.
link |
I wasn't part of it at that point.
link |
I remember I was an intern for summer.
link |
But my sense is given that it was over 50 years ago,
link |
yeah, they had that idea that it was an information utility.
link |
That it was something where if you had a computing task to do,
link |
you could just go and do it.
link |
Now I'm betting that they didn't have the same view
link |
of computing for the masses, let's call it.
link |
The idea that your grandmother would be shopping on Amazon.
link |
I don't think that was part of it.
link |
But if your grandmother were a programmer,
link |
it might be very easy for her to go and use
link |
this kind of utility.
link |
What was your dream of computers at that time?
link |
What did you see as the future of computers?
link |
Because you have predicted what computers are today.
link |
Oh, short answer, absolutely not.
link |
I'm not sure I had a dream.
link |
It was a dream job in the sense that I really enjoyed
link |
I was surrounded by really, really nice people.
link |
Cambridge is a very fine city to live in in the summer,
link |
less so in the winter when it snows.
link |
But in the summer, it was a delightful time.
link |
And so I really enjoyed all of that stuff.
link |
And I learned things.
link |
And I think the good fortune of being there for summer
link |
led me then to get a summer job at Bell Labs
link |
the following summer.
link |
And that was quite useful for the future.
link |
So Bell Labs is this magical, legendary place.
link |
So first of all, where is Bell Labs?
link |
And can you start talking about that journey
link |
towards Unix at Bell Labs?
link |
Yeah, so Bell Labs is physically scattered around,
link |
at the time, scattered around New Jersey.
link |
The primary location is in a town called Murray Hill,
link |
or a location called Murray Hill is actually
link |
across the boundary between two small towns in New Jersey
link |
called New Providence and Berkeley Heights.
link |
Think of it as about 15, 20 miles straight west
link |
of New York City, and therefore about an hour north
link |
of here in Princeton.
link |
And at that time, it had, make up a number,
link |
three or 4,000 people there, many of whom had PhDs
link |
and mostly doing physical sciences,
link |
chemistry, physics, materials kinds of things,
link |
but very strong math and rapidly growing interest
link |
in computing as people realized you could do things
link |
with computers that you might not have been able
link |
You could replace labs with computers
link |
that had worked on models of what was going on.
link |
So that was the essence of Bell Labs.
link |
And again, I wasn't a permanent employee there.
link |
That was another internship.
link |
I got lucky in internships.
link |
I mean, if you could just linger on it a little bit,
link |
what was the, what was in the air there?
link |
Because some of the, the number of Nobel Prizes,
link |
the number of Turing Awards and just legendary
link |
computer scientists that come from their inventions,
link |
including developments, including Unix,
link |
it's just, it's unbelievable.
link |
So was there something special about that place?
link |
Oh, I think there was very definitely something special.
link |
I mentioned the number of people,
link |
it's a very large number of people, very highly skilled
link |
and working in an environment
link |
where there was always something interesting to work on
link |
because the goal of Bell Labs,
link |
which was a small part of AT&T,
link |
which provided basically the country's phone service.
link |
The goal of AT&T was to provide service for everybody.
link |
And the goal of Bell Labs was to try and make that service
link |
keep getting better, so improving service.
link |
And that meant doing research on a lot of different things,
link |
physical devices, like the transistor
link |
or fiber optical cables or microwave systems,
link |
all of these things the labs worked on.
link |
And it was kind of just the beginning of real boom times
link |
in computing as well.
link |
Because when I was there, I went there first in 66.
link |
So computing was at that point fairly young.
link |
And so people were discovering
link |
that you could do lots of things with computers.
link |
So how was Unix born?
link |
So Multics, in spite of having an enormous number
link |
of really good ideas and lots of good people working on it,
link |
fundamentally didn't live up, at least in the short run,
link |
and I think ultimately really ever,
link |
to its goal of being this information utility.
link |
It was too expensive and certainly what was promised
link |
was delivered much too late.
link |
And so in roughly the beginning of 1969,
link |
Bell Labs pulled out of the project.
link |
The project at that point had included MIT, Bell Labs,
link |
and General Electric, General Electric made computers.
link |
So General Electric was the hardware operation.
link |
So Bell Labs, realizing this wasn't going anywhere
link |
on a timescale they cared about, pulled out of the project.
link |
And this left several people with an acquired taste
link |
for really, really nice computing environments,
link |
but no computing environment.
link |
And so they started thinking about what could you do
link |
if you were going to design a new operating system
link |
that would provide the same kind of comfortable computing
link |
as CTSS had, but also the facilities of something
link |
like Multics sort of brought forward.
link |
And so they did a lot of paper design stuff.
link |
And at the same time, Ken Thompson found
link |
what is characterized as a little used PDP 7,
link |
where he started to do experiments with file systems,
link |
just how do you store information on a computer
link |
in a efficient way, and then this famous story
link |
that his wife went away to California for three weeks,
link |
taking their one year old son, and three weeks,
link |
and he sat down and wrote an operating system,
link |
which ultimately became Unix.
link |
So software productivity was good in those days.
link |
So PDP, what's a PDP 7?
link |
So it's a piece of hardware.
link |
Yeah, it's a piece of hardware.
link |
It was one of early machines made
link |
by Digital Equipment Corporation, DEC,
link |
and it was a mini computer, so called.
link |
It had, I would have to look up the numbers exactly,
link |
but it had a very small amount of memory,
link |
maybe 16K, 16 bit words, or something like that,
link |
relatively slow, probably not super expensive.
link |
Maybe, again, making this up, I'd have to look it up,
link |
$100,000 or something like that.
link |
Which is not super expensive in those days, right?
link |
It was enough that you and I probably
link |
wouldn't be able to buy one,
link |
but a modest group of people could get together.
link |
But in any case, it came out, if I recall, in 1964.
link |
So by 1969, it was getting a little obsolete,
link |
and that's why it was little used.
link |
If you can sort of comment,
link |
what do you think it's like
link |
to write an operating system like that?
link |
So that process that Ken went through in three weeks,
link |
because you were, I mean, you're a part of that process.
link |
You contributed a lot to Unix's early development.
link |
So what do you think it takes to do that first step,
link |
that first kind of, from design to reality on the PDP?
link |
Well, let me correct one thing.
link |
I had nothing to do with it.
link |
So I did not write it.
link |
I have never written operating system code.
link |
And so I don't know.
link |
Now an operating system is simply code.
link |
And this first one wasn't very big,
link |
but it's something that lets you run processes,
link |
lets you execute some kind of code that has been written.
link |
It lets you store information for periods of time
link |
so that it doesn't go away when you turn the power off
link |
or reboot or something like that.
link |
And there's kind of a core set of tools
link |
that are technically not part of an operating system,
link |
but you probably need them.
link |
In this case, Ken wrote an assembler
link |
for the PDP 7 that worked.
link |
He needed a text editor
link |
so that he could actually create text.
link |
He had the file system stuff that he had been working on,
link |
and then the rest of it was just a way
link |
to load things, executable code from the file system
link |
into the memory, give it control,
link |
and then recover control when it was finished
link |
or in some other way quit.
link |
What was the code written in,
link |
primarily the programming language?
link |
Was it in assembly?
link |
Yeah, PDP 7 assembler that Ken created.
link |
These things were assembly language
link |
until probably the, call it 1973 or 74, something like that.
link |
Forgive me if it's a dumb question,
link |
but it feels like a daunting task
link |
to write any kind of complex system in assembly.
link |
It feels like impossible to do any kind
link |
of what we think of as software engineering with assembly,
link |
because to work on a big picture sort of.
link |
I think it's hard.
link |
It's been a long time since I wrote assembly language.
link |
It is absolutely true that in assembly language,
link |
if you make a mistake, nobody tells you.
link |
There are no training wheels whatsoever.
link |
And so stuff doesn't work.
link |
There's no debuggers.
link |
Well, there could be debuggers,
link |
but that's the same problem, right?
link |
How do you actually get something
link |
that will help you debug it?
link |
So part of it is an ability to see the big picture.
link |
Now these systems were not big in the sense
link |
that today's pictures are.
link |
So the big picture was in some sense more manageable.
link |
I mean, then realistically,
link |
there's an enormous variation
link |
in the capabilities of programmers.
link |
And Ken Thompson, who did that first one,
link |
is kind of the singularity, in my experience, of programmers.
link |
With no disrespect to you or even to me,
link |
he's in several leagues removed.
link |
I know there's levels.
link |
It's a fascinating thing that there are unique stars
link |
in particular in the programming space
link |
and at a particular time.
link |
You know, the time matters too,
link |
the timing of when that person comes along.
link |
And a wife does have to leave.
link |
There's this weird timing that happens
link |
and then all of a sudden something beautiful is created.
link |
I mean, how does it make you feel
link |
that there's a system that was created in three weeks
link |
or maybe you can even say on a whim,
link |
but not really, but of course, quickly,
link |
that is now, you could think of most of the computers
link |
in the world run on a Unix like system?
link |
How do you interpret, like,
link |
if you kind of zoom from the alien perspective,
link |
if you were just observing Earth,
link |
and all of a sudden these computers took over the world
link |
and they started from this little initial seed of Unix,
link |
how does that make you feel?
link |
It's quite surprising.
link |
And you asked earlier about prediction.
link |
There's no way you could predict that kind of evolution.
link |
And I don't know whether it was inevitable
link |
or just a whole sequence of blind luck.
link |
I suspect more of the latter.
link |
And so I look at it and think, gee, that's kind of neat.
link |
I think the real question is what does Ken think about that?
link |
Because he's the guy arguably from whom it really came.
link |
You know, tremendous contributions from Dennis Ritchie
link |
and then others around in that Bell Labs environment.
link |
But, you know, if you had to pick a single person,
link |
that would be Ken.
link |
So you've written a new book,
link |
Unix, a history and a memoir.
link |
Are there some memorable human stories,
link |
funny or profound from that time
link |
that just kind of stand out?
link |
Oh, there's a lot of them in his book.
link |
Oh, there's a lot of them in a sense.
link |
And again, it's a question of can you resurrect them
link |
But I think part of it was that Bell Labs at the time
link |
was a very special kind of place to work
link |
because there were a lot of interesting people
link |
and the environment was very, very open and free.
link |
It was a very cooperative environment,
link |
very friendly environment.
link |
And so if you had an interesting problem,
link |
you go and talk to somebody
link |
and they might help you with the solution.
link |
And it was a kind of a fun environment too,
link |
in which people did strange things
link |
and often tweaking the bureaucracy in one way or another.
link |
So rebellious in certain kinds of ways.
link |
In some ways, yeah, absolutely.
link |
I think most people didn't take too kindly
link |
to the bureaucracy and I'm sure the bureaucracy
link |
put up with an enormous amount
link |
that they didn't really want to.
link |
So maybe to linger on it a little bit,
link |
do you have a sense of what the philosophy
link |
that characterizes Unix is, the design?
link |
Not just the initial, but just carry through the years,
link |
just being there, being around it.
link |
What's the fundamental philosophy behind the system?
link |
I think one aspect of fundamental philosophy
link |
was to provide an environment that made it easy to write
link |
or easier, productive to write programs.
link |
So it was meant as a programmer environment.
link |
It wasn't meant specifically as something
link |
to do some other kind of job.
link |
For example, it was used extensively for word processing,
link |
but it wasn't designed as a word processing system.
link |
It was used extensively for lab control,
link |
but it wasn't designed for that.
link |
It was used extensively as a front end
link |
for big other systems, big dumb systems,
link |
but it wasn't designed for that.
link |
It was meant to be an environment
link |
where it was really easy to write programs.
link |
So the programmers could be highly productive.
link |
And part of that was to be a community.
link |
And there's some observation from Dennis Ritchie,
link |
I think at the end of the book,
link |
that says that from his standpoint,
link |
the real goal was to create a community
link |
where people could work as programmers on a system.
link |
And I think in that sense, certainly for many, many years,
link |
it succeeded quite well at that.
link |
And part of that is the technical aspects
link |
of because it made it really easy to write programs,
link |
people did write interesting programs.
link |
Those programs tended to be used by other programmers.
link |
And so it was kind of a virtuous circle
link |
of more and more stuff coming up
link |
that was really good for programmers.
link |
And you were part of that community of programmers.
link |
So what was it like writing programs in that early Unix?
link |
You know, I like to program.
link |
I'm not a terribly good programmer,
link |
but it was a lot of fun to write code.
link |
And in the early days, there was an enormous amount
link |
of what you would today, I suppose,
link |
called low hanging fruit.
link |
People hadn't done things before.
link |
And this was this new environment
link |
and the whole combination of nice tools
link |
and very responsive system and tremendous colleagues
link |
made it possible to write code.
link |
You could have an idea in the morning.
link |
You could do an experiment with it.
link |
You could have something limping along that night
link |
or the next day and people would react to it.
link |
And they would say, oh, that's wonderful,
link |
but you're really screwed up here.
link |
And the feedback loop was then very, very short and tight.
link |
And so a lot of things got developed fairly quickly
link |
that in many cases still exist today.
link |
And I think that was part of what made it fun
link |
because programming itself is fun.
link |
It's puzzle solving in a variety of ways,
link |
but I think it's even more fun when you do something
link |
that somebody else then uses.
link |
Even if they whine about it not working,
link |
the fact that they used it is part of the reward mechanism.
link |
And what was the method of interaction,
link |
the communication, that feedback loop?
link |
I mean, this is before the internet.
link |
Certainly before the internet.
link |
It was mostly physical right there.
link |
Somebody would come into your office and say something.
link |
So these places are all close by,
link |
like offices are nearby, so really lively interaction.
link |
Bell Labs was fundamentally one giant building
link |
and most of the people were involved in this unique stuff.
link |
We're in two or three quarters and there was a room.
link |
Oh, how big was it?
link |
Probably call it 50 feet by 50 feet.
link |
Make up a number of that which had some access
link |
to computers there as well as in offices
link |
and people hung out there and it had a coffee machine.
link |
And so it was mostly very physical.
link |
We did use email, of course.
link |
But it was fundamentally, for a long time,
link |
all on one machine.
link |
So there was no need for internet.
link |
It's fascinating to think about what computing
link |
would be today without Bell Labs.
link |
It seems so many, the people being in the vicinity
link |
of each other, sort of getting that quick feedback,
link |
working together, so many brilliant people.
link |
I don't know where else that could have existed
link |
in the world given how that came together.
link |
Yeah, how does that make you feel
link |
that little element of history?
link |
Well, I think that's very nice,
link |
but in a sense it's survivor bias
link |
and if it hadn't happened at Bell Labs,
link |
there were other places that were doing
link |
really interesting work as well.
link |
Xerox PARC is perhaps the most obvious one.
link |
Xerox PARC contributed an enormous amount
link |
of good material and many of the things
link |
we take for granted today in the same way
link |
came from Xerox PARC experience.
link |
I don't think they capitalized in the long run as much.
link |
Their parent company was perhaps not as lucky
link |
in capitalizing on this, who knows?
link |
But that's certainly another place
link |
where there was a tremendous amount of influence.
link |
There were a lot of good university activities.
link |
MIT was obviously no slouch in this kind of thing
link |
and others as well.
link |
So Unix turned out to be open source
link |
because of the various ways that AT&T operated
link |
and sort of it had to, the focus was on telephones.
link |
I think that's a mischaracterization in a sense.
link |
It absolutely was not open source.
link |
It was very definitely proprietary, licensed,
link |
but it was licensed freely to universities
link |
in source code form for many years.
link |
And because of that, generations of university students
link |
and their faculty people grew up knowing about Unix
link |
and there was enough expertise in the community
link |
that it then became possible for people
link |
to kind of go off in their own direction
link |
and build something that looked Unix like.
link |
The Berkeley version of Unix started with that licensed code
link |
and gradually picked up enough of its own code contributions,
link |
notably from people like Bill Joy,
link |
that eventually it was able to become completely free
link |
Now, there was an enormous amount of legal jockeying
link |
around this in the late, early to late 80s, early 90s,
link |
something like that.
link |
And then, I guess the open source movement
link |
might've started when Richard Stallman started
link |
to think about this in the late 80s.
link |
And by 1991, when Torvalds decided he was going
link |
to do a Unix like operating system,
link |
there was enough expertise in the community
link |
that first he had a target, he could see what to do
link |
because the kind of the Unix system call interface
link |
and the tools and so on were there.
link |
And so he was able to build an operating system
link |
that at this point, when you say Unix,
link |
in many cases, what you're really thinking is Linux.
link |
But it's funny that from my distant perception,
link |
I felt that Unix was open source
link |
without actually knowing it.
link |
But what you're really saying, it was just freely licensed.
link |
It was freely licensed.
link |
So it felt open source in a sense
link |
because universities are not trying to make money,
link |
so it felt open source in a sense
link |
that you can get access if you wanted.
link |
Right, and a very, very, very large number of universities
link |
had the license and they were able to talk
link |
to all the other universities who had the license.
link |
And so technically not open,
link |
technically belonging to AT&T, pragmatically pretty open.
link |
And so there's a ripple effect
link |
that all the faculty and the students then all grew up
link |
and then they went throughout the world
link |
and permeated in that kind of way.
link |
So what kind of features do you think make
link |
for a good operating system?
link |
If you take the lessons of Unix,
link |
you said make it easy for programmers.
link |
That seems to be an important one.
link |
But also Unix turned out to be exceptionally robust
link |
So is that an accident when you focus on the programmer
link |
or is that a natural outcome?
link |
I think part of the reason for efficiency
link |
was that it began on extremely modest hardware,
link |
very, very, very tiny.
link |
And so you couldn't get carried away.
link |
You couldn't do a lot of complicated things
link |
because you just didn't have the resources,
link |
either processor speed or memory.
link |
And so that enforced a certain minimality of mechanisms
link |
and maybe a search for generalizations
link |
so that you would find one mechanism
link |
that served for a lot of different things
link |
rather than having lots of different special cases.
link |
I think the file system in Unix is a good example
link |
of that file system interface in its fundamental form
link |
is extremely straightforward.
link |
And that means that you can write code
link |
very, very effectively for the file system.
link |
And then one of those ideas, one of those generalizations
link |
is that gee, that file system interface works
link |
for all kinds of other things as well.
link |
And so in particular, the idea of reading
link |
and writing to devices is the same as reading
link |
and writing to a disc that has a file system.
link |
And then that gets carried further in other parts
link |
Processes become, in effect, files in a file system.
link |
And the Plan 9 operating system, which came along,
link |
I guess, in the late 80s or something like that,
link |
took a lot of those ideas from the original Unix
link |
and tried to push the generalization even further
link |
so that in Plan 9, a lot of different resources
link |
They all share that interface.
link |
So that would be one example where finding the right model
link |
of how to do something means that an awful lot of things
link |
become simpler, and it means, therefore,
link |
that more people can do useful, interesting things
link |
with them without having to think as hard about it.
link |
So you said you're not a very good programmer.
link |
You're the most modest human being, okay,
link |
but you'll continue saying that.
link |
I understand how this works.
link |
But you do radiate a sort of love for programming.
link |
So let me ask, do you think programming
link |
is more an art or a science?
link |
Is it creativity or kind of rigor?
link |
I think it's some of each.
link |
It's some combination.
link |
Some of the art is figuring out what it is
link |
that you really want to do.
link |
What should that program be?
link |
What would make a good program?
link |
And that's some understanding of what the task is,
link |
what the people who might use this program want.
link |
And I think that's art in many respects.
link |
The science part is trying to figure out how to do it well.
link |
And some of that is real computer sciencey stuff,
link |
like what algorithm should we use at some point?
link |
Mostly in the sense of being careful to use algorithms
link |
that will actually work properly, scale properly,
link |
avoiding quadratic algorithms
link |
when a linear algorithm should be the right thing,
link |
that kind of more formal view of it.
link |
Same thing for data structures.
link |
But also it's, I think, an engineering field as well.
link |
And engineering is not quite the same as science
link |
because engineering, you're working with constraints.
link |
You have to figure out not only what
link |
is a good algorithm for this kind of thing,
link |
but what's the most appropriate algorithm given
link |
the amount of time we have to compute,
link |
the amount of time we have to program,
link |
what's likely to happen in the future with maintenance,
link |
who's going to pick this up in the future, all
link |
of those kind of things that if you're an engineer,
link |
you get to worry about.
link |
Whereas if you think of yourself as a scientist,
link |
well, you can maybe push them over the horizon in a way.
link |
And if you're an artist, what's that?
link |
So just on your own personal level,
link |
what's your process like of writing a program?
link |
Say, a small and large sort of tinkering with stuff.
link |
Do you just start coding right away
link |
and just kind of evolve iteratively with a loose notion?
link |
Or do you plan on a sheet of paper first
link |
and then kind of design in what they teach you
link |
in the kind of software engineering courses
link |
in undergrad or something like that?
link |
What's your process like?
link |
It's certainly much more the informal incremental.
link |
First, I don't write big programs at this point.
link |
It's been a long time since I wrote a program that
link |
was more than I call it a few hundred or more lines,
link |
something like that.
link |
Many of the programs I write are experiments
link |
for either something I'm curious about
link |
or often for something that I want to talk about in a class.
link |
So those necessarily tend to be relatively small.
link |
A lot of the kind of code I write these days
link |
tends to be for sort of exploratory data analysis
link |
where I've got some collection of data
link |
and I want to try and figure out what on earth is going on in it.
link |
And for that, those programs tend to be very small.
link |
Sometimes you're not even programming.
link |
You're just using existing tools like counting things.
link |
Or sometimes you're writing OX scripts
link |
because two or three lines will tell you
link |
something about a piece of data.
link |
And then when it gets bigger, well, then I
link |
will probably write something in Python
link |
because that scales better up to call it a few hundred lines
link |
or something like that.
link |
And it's been a long time since I wrote programs
link |
that were much more than that.
link |
Speaking of data exploration and OX, first, what is OX?
link |
So OX is a scripting language that
link |
was done by myself, Al Aho, and Peter Weinberger.
link |
We did that originally in the late 70s.
link |
It was a language that was meant to make it really easy
link |
to do quick and dirty tasks like counting things
link |
or selecting interesting information from basically
link |
all text files, rearranging it in some way or summarizing it.
link |
It runs a command on each line of a file.
link |
I mean, it's still exceptionally widely used today.
link |
It's so simple and elegant, sort of the way to explore data.
link |
Turns out you can just write a script that
link |
does something seemingly trivial in a single line,
link |
and giving you that slice of the data
link |
somehow reveals something fundamental about the data.
link |
And that seems to work still.
link |
Yeah, it's very good for that kind of thing.
link |
That's sort of what it was meant for.
link |
I think what we didn't appreciate
link |
was that the model was actually quite good for a lot of data
link |
processing kinds of tasks and that it's
link |
kept going as long as it has because at this point,
link |
it's over 40 years old, and it's still, I think, a useful tool.
link |
And well, this is paternal interest, I guess.
link |
But I think in terms of programming languages,
link |
you get the most bang for the buck by learning AUC.
link |
And it doesn't scale the big programs,
link |
but it does pretty darn well on these little things
link |
where you just want to see all the somethings in something.
link |
So yeah, I probably write more AUC than anything else
link |
So what kind of stuff do you love about AUC?
link |
Is there, if you can comment on sort of things
link |
that give you joy when you can, in a simple program,
link |
reveal something about the data.
link |
Is there something that stands out from particular features?
link |
I think it's mostly the selection of default behaviors.
link |
You sort of hinted at it a moment ago.
link |
What AUC does is to read through a set of files,
link |
and then within each file, it writes
link |
through each of the lines.
link |
And then on each of the lines, it has a set of patterns
link |
that it looks for.
link |
That's your AUC program.
link |
And if one of the patterns matches,
link |
there is a corresponding action that you might perform.
link |
And so it's kind of a quadruply nested loop or something
link |
And that's all completely automatic.
link |
You don't have to say anything about it.
link |
You just write the pattern and the action,
link |
and then run the data by it.
link |
And so that paradigm for programming
link |
is a very natural and effective one.
link |
And I think we captured that reasonably well in AUC.
link |
And it does other things for free as well.
link |
It splits the data into fields so that on each line,
link |
there is fields separated by white space or something.
link |
And so it does that for free.
link |
You don't have to say anything about it.
link |
And it collects information as it goes along,
link |
like what line are we on?
link |
How many fields are there on this line?
link |
So lots of things that just make it so that a program which
link |
in another language, let's say Python,
link |
would be five, 10, 20 lines in AUC is one or two lines.
link |
And so because it's one or two lines,
link |
you can do it on the shell.
link |
You don't have to open up another whole thing.
link |
You can just do it right there in the interaction
link |
with the operatives directly.
link |
Is there other shell commands that you love over the years
link |
like you really enjoy using?
link |
Grep's the only one.
link |
Yeah, grep does everything.
link |
So grep is a simpler version of AUC, I would say?
link |
In some sense, yeah, right.
link |
So grep basically searches the input
link |
for particular patterns, regular expressions,
link |
technically, of a certain class.
link |
And it has that same paradigm that AUC does.
link |
It's a pattern action thing.
link |
It reads through all the files and then
link |
all the lines in each file.
link |
But it has a single pattern, which
link |
is the regular expression you're looking for,
link |
and a single action printed if it matches.
link |
So in that sense, it's a much simpler version.
link |
And you could write grep in AUC as a one liner.
link |
And I use grep probably more than anything else
link |
at this point just because it's so convenient and natural.
link |
Why do you think it's such a powerful tool, grep and AUC?
link |
Why do you think operating systems like Windows,
link |
for example, don't have it?
link |
You can, of course, I use, which is amazing now,
link |
there's Windows for Linux.
link |
So which you could basically use all the fun stuff
link |
like AUC and grep inside of Windows.
link |
But Windows naturally, as part of the graphical interface,
link |
the simplicity of grep, searching
link |
through a bunch of files and just popping up naturally.
link |
Why do you think that's unique to the Linux environment?
link |
It's not strictly unique, but it's certainly focused there.
link |
And I think some of it's the weight of history
link |
that Windows came from MS DOS.
link |
MS DOS was a pretty pathetic operating system,
link |
although common on an unboundedly large number
link |
But somewhere in roughly the 90s,
link |
Windows became a graphical system.
link |
And I think Microsoft spent a lot of their energy
link |
on making that graphical interface what it is.
link |
And that's a different model of computing.
link |
It's a model of computing where you point and click
link |
and sort of experiment with menus.
link |
It's a model of computing works rather well for people
link |
who are not programmers and just want to get something done,
link |
whereas teaching something like the command line
link |
to nonprogrammers turns out to sometimes be
link |
an uphill struggle.
link |
And so I think Microsoft probably
link |
was right in what they did.
link |
Now you mentioned Whistle or whatever
link |
it's called, the Winix, Linux.
link |
I wonder what it's pronounced.
link |
WSL is what I've never actually pronounced.
link |
Whistle, I like it.
link |
But there have been things like that for longest.
link |
Cygwin, for example, which is a wonderful collection of take
link |
all your favorite tools from Unix and Linux
link |
and just make them work perfectly on Windows.
link |
And so that's something that's been going on
link |
for at least 20 years, if not longer.
link |
And I use that on my one remaining Windows machine
link |
routinely because if you're doing something that
link |
is batch computing, suitable for command line,
link |
that's the right way to do it.
link |
Because the Windows equivalents are, if nothing else,
link |
not familiar to me.
link |
But I would definitely recommend to people
link |
if they don't use Cygwin to try Whistle.
link |
I've been so excited that I could write scripts quickly
link |
It's changed my life.
link |
OK, what's your perfect programming setup?
link |
What computer, what operating system, what keyboard,
link |
Yeah, perfect is too strong a word.
link |
It's way too strong a word.
link |
What I use by default, I have, at this point,
link |
a 13 inch MacBook Air, which I use
link |
because it's kind of a reasonable balance
link |
of the various things I need.
link |
I can carry it around.
link |
It's got enough computing, horsepower, screen's
link |
big enough, keyboard's OK.
link |
And so I basically do most of my computing on that.
link |
I have a big iMac in my office that I use from time to time
link |
as well, especially when I need a big screen,
link |
but otherwise, it tends not to be used that much.
link |
I use mostly SAM, which is an editor that Rob Pike wrote
link |
long ago at Bell Labs.
link |
Sorry to interrupt.
link |
Does that precede VI?
link |
Does that precede iMac?
link |
It post dates both VI and iMacs.
link |
It is derived from Rob's experience with ED and VI.
link |
That's the original Unix editor.
link |
Dated probably before you were born.
link |
So actually, what's the history of editors?
link |
Can you briefly, because it's such a fact.
link |
I use Emacs, I'm sorry to say.
link |
Sorry to come out with that.
link |
But what's the kind of interplay there?
link |
So in ancient times, call it the first time sharing systems,
link |
going back to what we were talking about.
link |
There was an editor on CTSS that I don't even
link |
remember what it was called.
link |
It might have been edit, where you could type text, program
link |
text, and it would do something, or document text.
link |
You could save the text.
link |
You could edit it.
link |
The usual thing that you would get in an editor.
link |
And Ken Thompson wrote an editor called QED, which
link |
was very, very powerful.
link |
But these were all totally A, command based.
link |
They were not mouse or cursor based,
link |
because it was before mice and even before cursors,
link |
because they were running on terminals that printed on paper.
link |
No CRT type displays, let alone LEDs.
link |
And so then when Unix came along, Ken took QED
link |
and stripped it way, way, way, way down.
link |
And that became an editor that he called ED.
link |
And it was very simple.
link |
But it was a line oriented editor.
link |
And so you could load a file.
link |
And then you could talk about the lines one
link |
through the last line.
link |
And you could print ranges of lines.
link |
You could add text.
link |
You could delete text.
link |
You could change text.
link |
Or you could do a substitute command
link |
that would change things within a line or within groups
link |
So you can work on parts of a file, essentially.
link |
You can work on any part of it, the whole thing or whatever.
link |
But it was entirely command line based.
link |
And it was entirely on paper.
link |
And that meant that you changed it.
link |
And so if you changed a line, you
link |
had to print that line using up another line of paper
link |
to see what the change caused.
link |
So when CRT displays came along, then you
link |
could start to use cursor control.
link |
And you could sort of move where you were on the screen.
link |
Without reprinting every time.
link |
Without reprinting.
link |
And there were a number of editors there.
link |
The one that I was most familiar with and still use
link |
is VI, which was done by Bill Choi.
link |
And so that dates from probably the late 70s, as I guess.
link |
And it took full advantage of the cursor controls.
link |
I suspect that Emacs was roughly at the same time.
link |
I've never internalized Emacs.
link |
So at this point, I stopped using ED, although I still can.
link |
I use VI sometimes, and I use SAM when I can.
link |
And SAM is available on most systems?
link |
You have to download it yourself from, typically,
link |
the Plan 9 operating system distribution.
link |
It's been maintained by people there.
link |
And so I'll get home tonight.
link |
It sounds fascinating.
link |
Although my love is with Lisp and Emacs,
link |
I've went into that hippie world of.
link |
I think it's a lot of things.
link |
Religion, where you're brought up with.
link |
Yeah, that's true.
link |
Most of the actual programming I do is C, C++, and Python.
link |
But my weird sort of, yeah, my religious upbringing is in Lisp.
link |
So can you take on the impossible task
link |
and give a brief history of programming languages
link |
from your perspective?
link |
So I guess you could say programming languages started
link |
probably in, what, the late 40s or something like that.
link |
People used to program computers by basically putting
link |
in zeros and ones.
link |
Using something like switches on a console.
link |
And then, or maybe holes in paper tapes.
link |
Something like that.
link |
So extremely tedious, awful, whatever.
link |
And so I think the first programming languages
link |
were relatively crude assembly languages,
link |
where people would basically write
link |
a program that would convert mnemonics like add ADD
link |
into whatever the bit pattern was
link |
that corresponded to an ADD instruction.
link |
And they would do the clerical work of figuring out
link |
where things were.
link |
So you could put a name on a location in a program,
link |
and the assembler would figure out
link |
where that corresponded to when the thing was all put together
link |
and dropped into memory.
link |
And early on, and this would be the late 40s and very early
link |
50s, there were assemblers written for the various machines
link |
You may have seen in the paper just a couple of days ago,
link |
He did this thing in Manchester called AutoCode, a language
link |
which I knew only by name.
link |
But it sounds like it was a flavor of assembly language,
link |
sort of a little higher in some ways.
link |
And it replaced a language that Alan Turing wrote,
link |
which you put in zeros and ones.
link |
But you put it in backwards order,
link |
because that was a hardware word.
link |
Yeah, yeah, that's right.
link |
So assembly languages, let's call that the early 1950s.
link |
And so every different flavor of computer
link |
has its own assembly language.
link |
So the EDSAC had its, and the Manchester had its,
link |
and the IBM whatever, 790 or 704, or whatever had its,
link |
So everybody had their own assembly language.
link |
And assembly languages have a few commands, additions,
link |
subtraction, then branching of some kind,
link |
if then type of situation.
link |
Right, they have exactly, in their simplest form at least,
link |
one instruction per, or one assembly language instruction
link |
per instruction in the machine's repertoire.
link |
And so you have to know the machine intimately
link |
to be able to write programs in it.
link |
And if you write an assembly language program
link |
for one kind of machine, and then you say,
link |
gee, it's nice, I'd like a different machine, start over.
link |
And so what happened in the late 50s
link |
was people realized you could play this game again,
link |
and you could move up a level in writing or creating languages
link |
that were closer to the way that real people might think
link |
about how to write code.
link |
And there were, I guess, arguably three or four
link |
at that time period.
link |
There was FORTRAN, which came from IBM,
link |
which was formula translation, meant
link |
to make it easy to do scientific and engineering
link |
I didn't know that, formula translation, that's wow.
link |
That's what I stood for.
link |
There was COBOL, which is the Common Business Oriented
link |
Language that Grace Hopper and others worked on,
link |
which was aimed at business kinds of tasks.
link |
There was ALGOL, which was mostly
link |
meant to describe algorithmic computations.
link |
I guess you could argue BASIC was in there somewhere.
link |
I think it's just a little later.
link |
And so all of those moved the level up,
link |
and so they were closer to what you and I might think of
link |
as we were trying to write a program.
link |
And they were focused on different domains, FORTRAN
link |
for formula translation, engineering computations,
link |
let's say COBOL for business, that kind of thing.
link |
And still used today, at least FORTRAN probably.
link |
Oh, yeah, COBOL, too.
link |
But the deal was that once you moved up that level,
link |
then you, let's call it FORTRAN, you
link |
had a language that was not tied to a particular kind
link |
of hardware, because a different compiler would compile
link |
for a different kind of hardware.
link |
And that meant two things.
link |
It meant you only had to write the program once, which
link |
is very important.
link |
And it meant that you could, in fact,
link |
if you were a random engineer, physicist, whatever,
link |
you could write that program yourself.
link |
You didn't have to hire a programmer to do it for you.
link |
It might not be as good as you'd get with a real programmer,
link |
but it was pretty good.
link |
And so it democratized and made much more broadly available
link |
the ability to write code.
link |
So it puts the power of programming
link |
into the hands of people like you.
link |
Yeah, anybody who is willing to invest some time in learning
link |
a programming language and is not then tied
link |
to a particular kind of computer.
link |
And then in the 70s, you get system programming languages,
link |
of which C is the survivor.
link |
And what does system programming language mean?
link |
Programs that, programming languages
link |
that would take on the kinds of things
link |
that were necessary to write so called system programs.
link |
Things like text editors, or assemblers, or compilers,
link |
or operating systems themselves.
link |
Those kinds of things.
link |
They have to be feature rich.
link |
They have to be able to do a lot of stuff.
link |
A lot of memory management, access processes,
link |
and all that kind of stuff.
link |
It's a different flavor of what they're doing.
link |
They're much more in touch with the actual machine,
link |
but in a positive way.
link |
That is, you can talk about memory in a more controlled
link |
You can talk about the different data types
link |
that the machine supports, and more ways
link |
to structure and organize data.
link |
And so the system programming languages,
link |
there was a lot of effort in that in the,
link |
call it the late 60s, early 70s.
link |
C is, I think, the only real survivor of that.
link |
And then what happens after that?
link |
You get things like object oriented programming languages.
link |
Because as you write programs in a language like C,
link |
at some point scale gets to you.
link |
And it's too hard to keep track of the pieces.
link |
And there's no guardrails, or training wheels,
link |
or something like that to prevent you
link |
from doing bad things.
link |
So C++ comes out of that tradition.
link |
And then it took off from there.
link |
I mean, there's also a parallel, slightly parallel track
link |
with a little bit of functional stuff with Lisp and so on.
link |
But I guess from that point, it's
link |
just an explosion of languages.
link |
There's the Java story.
link |
There's the JavaScript.
link |
There's all the stuff that the cool kids these days
link |
are doing with Rust and all that.
link |
You wrote a book, C Programming Language.
link |
And C is probably one of the most important languages
link |
in the history of programming languages,
link |
if you kind of look at impact.
link |
What do you think is the most elegant or powerful part of C?
link |
Why did it survive?
link |
Why did it have such a long lasting impact?
link |
I think it found a sweet spot of expressiveness,
link |
so that you could rewrite things in a pretty natural way,
link |
and efficiency, which was particularly important when
link |
computers were not nearly as powerful as they are today.
link |
You've got to put yourself back 50 years,
link |
almost, in terms of what computers could do.
link |
And that's roughly four or five generations,
link |
decades of Moore's law, right?
link |
So expressiveness and efficiency and, I don't know,
link |
perhaps the environment that it came with as well,
link |
So it meant if you wrote a program,
link |
it could be used on all those computers that ran Unix.
link |
And that was all of those computers,
link |
because they were all written in C.
link |
And that was Unix, the operating system itself,
link |
was portable, as were all the tools.
link |
So it all worked together, again,
link |
in one of these things where things
link |
fit on each other in a positive cycle.
link |
What did it take to write sort of a definitive book,
link |
probably definitive book on all of program,
link |
like it's more definitive to a particular language
link |
than any other book on any other language,
link |
and did two really powerful things,
link |
which is popularized the language,
link |
at least from my perspective, maybe you can correct me.
link |
And second is created a standard of how, you know,
link |
how this language is supposed to be used and applied.
link |
So what did it take?
link |
Did you have those kinds of ambitions in mind
link |
when working on that?
link |
Is this some kind of joke?
link |
No, of course not.
link |
So it's an accident of timing, skill, and just luck?
link |
A lot of it is, clearly.
link |
Now, Dennis and I wrote the book in 1977.
link |
And at that point, Unix was starting to spread.
link |
I don't know how many there were,
link |
but it would be dozens to hundreds of Unix systems.
link |
And C was also available on other kinds of computers
link |
that had nothing to do with Unix.
link |
And so the language had some potential.
link |
And there were no other books on C,
link |
and Bell Labs was really the only source for it.
link |
And Dennis, of course, was authoritative
link |
because it was his language.
link |
And he had written the reference manual,
link |
which is a marvelous example
link |
of how to write a reference manual.
link |
Really, really very, very well done.
link |
So I twisted his arm until he agreed to write a book,
link |
and then we wrote a book.
link |
And the virtue or advantage, at least,
link |
I guess, of going first is that then other people
link |
have to follow you if they're gonna do anything.
link |
And I think it worked well because Dennis
link |
was a superb writer.
link |
I mean, he really, really did.
link |
And the reference manual in that book is his, period.
link |
I had nothing to do with that at all.
link |
So just crystal clear prose and very, very well expressed.
link |
And then he and I, I wrote most of the expository material.
link |
And then he and I sort of did the usual ping ponging
link |
back and forth, refining it.
link |
But I spent a lot of time trying to find examples
link |
that would sort of hang together
link |
and that would tell people what they might need
link |
to know at about the right time
link |
that they should be thinking about needing it.
link |
And I'm not sure it completely succeeded,
link |
but it mostly worked out fairly well.
link |
What do you think is the power of example?
link |
I mean, you're the creator, at least one of the first people
link |
to do the Hello World program, which is like the example.
link |
If aliens discover our civilization hundreds of years
link |
from now, it'll probably be Hello World programs,
link |
just like a half broken robot communicating with them
link |
with the Hello World.
link |
So what, and that's a representative example.
link |
So what do you find powerful about examples?
link |
I think a good example will tell you how to do something
link |
and it will be representative of,
link |
you might not want to do exactly that,
link |
but you will want to do something that's at least
link |
in that same general vein.
link |
And so a lot of the examples in the C book were picked
link |
for these very, very simple, straightforward
link |
text processing problems that were typical of Unix.
link |
I want to read input and write it out again.
link |
There's a copy command.
link |
I want to read input and do something to it
link |
and write it out again.
link |
And so that kind of find things that are representative
link |
of what people want to do and spell those out
link |
so that they can then take those and see the core parts
link |
and modify them to their taste.
link |
And I think that a lot of programming books that,
link |
I don't look at programming books
link |
a tremendous amount these days, but when I do,
link |
a lot of them don't do that.
link |
They don't give you examples that are both realistic
link |
and something you might want to do.
link |
Some of them are pure syntax.
link |
Here's how you add three numbers.
link |
Well, come on, I could figure that out.
link |
Tell me how I would get those three numbers
link |
into the computer and how we would do something useful
link |
with them and then how I put them back out again,
link |
And especially if you follow that example,
link |
there is something magical of doing something
link |
that feels useful.
link |
And I think it's the attempt,
link |
and it's absolutely not perfect,
link |
but the attempt in all cases was to get something
link |
that was going to be either directly useful
link |
or would be very representative of useful things
link |
that a programmer might want to do.
link |
But within that vein of fundamentally text processing,
link |
reading text, doing something, writing text.
link |
So you've also written a book on Go language.
link |
I have to admit, so I worked at Google for a while
link |
and I've never used Go.
link |
Well, you missed something.
link |
Well, I know I missed something for sure.
link |
I mean, so Go and Rust are two languages
link |
that I hear very, spoken very highly of
link |
and I wish I would like to, well, there's a lot of them.
link |
There's Julia, there's all these incredible modern languages.
link |
But if you can comment before,
link |
or maybe comment on what do you find,
link |
where does Go sit in this broad spectrum of languages?
link |
And also, how do you yourself feel
link |
about this wide range of powerful, interesting languages
link |
that you may never even get to try to explore
link |
So I think, so Go first comes from that same
link |
Bell Labs tradition in part, not exclusively,
link |
but two of the three creators, Ken Thompson and Rob Pike.
link |
So literally, the people.
link |
And then with this very, very useful influence
link |
from the European school in particular,
link |
the Claude Speer influence through Robert Griesemer,
link |
who was, I guess, a second generation down student at ETH.
link |
And so that's an interesting combination of things.
link |
And so some ways, Go captures the good parts of C,
link |
it looks sort of like C, it's sometimes characterized as C
link |
for the 21st century.
link |
On the surface, it looks very, very much like C.
link |
But at the same time, it has some interesting
link |
data structuring capabilities.
link |
And then I think the part that I would say
link |
is particularly useful, and again, I'm not a Go expert.
link |
In spite of coauthoring the book,
link |
about 90% of the work was done by Alan Donovan,
link |
my coauthor, who is a Go expert.
link |
But Go provides a very nice model of concurrency.
link |
It's basically the cooperating,
link |
communicating sequential processes that Tony Hoare
link |
set forth, jeez, I don't know, 40 plus years ago.
link |
And Go routines are, to my mind, a very natural way
link |
to talk about parallel computation.
link |
And in the few experiments I've done with them,
link |
they're easy to write, and typically it's gonna work,
link |
and very efficient as well.
link |
So I think that's one place where Go stands out,
link |
that that model of parallel computation
link |
is very, very easy and nice to work with.
link |
Just to comment on that, do you think C foresaw,
link |
or the early Unix days foresaw threads
link |
and massively parallel computation?
link |
I would guess not really.
link |
I mean, maybe it was seen, but not at the level
link |
where it was something you had to do anything about.
link |
For a long time, processors got faster,
link |
and then processors stopped getting faster
link |
because of things like power consumption
link |
and heat generation.
link |
And so what happened instead was that instead
link |
of processors getting faster,
link |
there started to be more of them.
link |
And that's where that parallel thread stuff comes in.
link |
So if you can comment on all the other languages,
link |
is it break your heart that you'll never get to explore them?
link |
How do you feel about the full variety?
link |
It's not break my heart,
link |
but I would love to be able to try more of these languages.
link |
The closest I've come is in a class
link |
that I often teach in the spring here.
link |
It's a programming class, and I often give,
link |
I have one sort of small example that I will write
link |
in as many languages as I possibly can.
link |
I've got it in 20 languages.
link |
At this point, and that's so I do a minimal experiment
link |
with a language just to say, okay,
link |
I have this trivial task, which I understand the task,
link |
and it takes 15 lines in awk,
link |
and not much more in a variety of other languages.
link |
How fast does it run?
link |
And what pain did I go through to learn how to do it?
link |
And that's like anecdotal, right?
link |
It's very, very, very, very, very, very, very,
link |
very, very narrowly focused.
link |
I think data, I like that term.
link |
So yeah, but still, it's a little sample,
link |
because you get to, I think the hardest step
link |
of the programming language is probably the first step,
link |
right, so there you're taking the first step.
link |
Yeah, and so my experience with some languages
link |
is very positive, like Lua,
link |
a scripting language I had never used,
link |
and I took my little program.
link |
The program is a trivial formatter.
link |
It just takes in lines of text of varying lengths,
link |
and it puts them out in lines
link |
that have no more than 60 characters on each line.
link |
So think of it as just kind of the flow of process
link |
in a browser or something.
link |
So it's a very short program.
link |
And in Lua, I downloaded Lua,
link |
and in an hour, I had it working,
link |
never having written Lua in my life,
link |
just going with online documentation.
link |
I did the same thing in Scala,
link |
which you can think of as a flavor of Java, equally trivial.
link |
I did it in Haskell.
link |
It took me several weeks.
link |
But it did run like a turtle.
link |
And I did it in Fortran 90, and it was painful,
link |
but it worked, and I tried it in Rust,
link |
and it took me several days to get it working
link |
because the model of memory management
link |
was just a little unfamiliar to me.
link |
And the problem I had with Rust,
link |
and it's back to what we were just talking about,
link |
I couldn't find good, consistent documentation on Rust.
link |
Now, this was several years ago,
link |
and I'm sure things have stabilized,
link |
but at the time, everything in the Rust world
link |
seemed to be changing rapidly,
link |
and so you would find what looked like a working example,
link |
and it wouldn't work with the version
link |
of the language that I had.
link |
So it took longer than it should have.
link |
Rust is a language I would like to get back to,
link |
but probably won't.
link |
I think one of the issues,
link |
you have to have something you want to do.
link |
If you don't have something that is the right combination,
link |
if I want to do it, and yet I have enough disposable time,
link |
whatever, to make it worth learning a new language
link |
at the same time, it's never gonna happen.
link |
So what do you think about another language of JavaScript?
link |
Well, let me just sort of comment on what I said.
link |
When I was brought up, sort of JavaScript was seen as
link |
probably like the ugliest language possible,
link |
and yet it's quite arguably, quite possibly taking over,
link |
not just the front end and the back end of the internet,
link |
but possibly in the future taking over everything,
link |
because they've now learned to make it very efficient.
link |
And so what do you think about this?
link |
Yeah, well, I think you've captured it in a lot of ways.
link |
When it first came out,
link |
JavaScript was deemed to be fairly irregular
link |
and an ugly language, and certainly in the academy,
link |
if you said you were working on JavaScript,
link |
people would ridicule you.
link |
It was just not fit for academics to work on.
link |
I think a lot of that has evolved.
link |
The language itself has evolved,
link |
and certainly the technology of compiling it
link |
is fantastically better than it was.
link |
And so in that sense,
link |
it's absolutely a viable solution on back ends,
link |
as well as the front ends.
link |
Used well, I think it's a pretty good language.
link |
I've written a modest amount of it,
link |
and I've played with JavaScript translators
link |
and things like that.
link |
I'm not a real expert,
link |
and it's hard to keep up even there
link |
with the new things that come along with it.
link |
So I don't know whether it will ever take over the world.
link |
I think not, but it's certainly an important language,
link |
and worth knowing more about.
link |
There's, maybe to get your comment on something,
link |
which JavaScript, and actually most languages,
link |
sort of Python, such a big part of the experience
link |
of programming with those languages includes libraries,
link |
sort of using, building on top of the code
link |
that other people have built.
link |
I think that's probably different from the experience
link |
that we just talked about from Unix and C days,
link |
when you're building stuff from scratch.
link |
What do you think about this world
link |
of essentially leveraging, building up libraries
link |
on top of each other and leveraging them?
link |
Yeah, no, that's a very perceptive kind of question.
link |
One of the reasons programming was fun in the old days
link |
was that you were really building it all yourself.
link |
The number of libraries you had to deal with
link |
Maybe it was printf, or the standard library,
link |
or something like that, and that is not the case today.
link |
And if you want to do something in,
link |
you mentioned Python and JavaScript,
link |
and those are the two fine examples,
link |
you have to typically download a boatload of other stuff,
link |
and you have no idea what you're getting,
link |
absolutely nothing.
link |
I've been doing some playing with machine learning
link |
over the last couple of days,
link |
and geez, something doesn't work.
link |
Well, you pip install this, okay,
link |
and down comes another one,
link |
okay, and down comes another gazillion megabytes of something
link |
and you have no idea what it was.
link |
And if you're lucky, it works.
link |
And if it doesn't work, you have no recourse.
link |
There's absolutely no way you could figure out
link |
which of these thousand different packages.
link |
And I think it's worse in the NPM environment
link |
I think there's less discipline, less control there.
link |
And there's aspects of not just not understanding
link |
how it works, but there's security issues,
link |
there's robustness issues,
link |
so you don't wanna run a nuclear power plant
link |
using JavaScript, essentially.
link |
So speaking to the variety of languages,
link |
do you think that variety is good,
link |
or do you hope, think that over time,
link |
we should converge towards one, two, three
link |
programming languages?
link |
You mentioned to the Bell Lab days
link |
when people could sort of, the community of it,
link |
and the more languages you have,
link |
the more you separate the communities.
link |
There's the Ruby community,
link |
there's the Python community,
link |
there's C++ community.
link |
Do you hope that they'll unite one day
link |
to just one or two languages?
link |
I certainly don't hope it.
link |
I'm not sure that that's right,
link |
because I honestly don't think there is one language
link |
that will suffice for all the programming needs of the world.
link |
Are there too many at this point?
link |
But I think if you look at the sort of the distribution
link |
of how they are used,
link |
there's something called a dozen languages
link |
that probably account for 95% of all programming
link |
at this point, and that doesn't seem unreasonable.
link |
And then there's another, well, 2,000 languages
link |
that are still in use that nobody uses,
link |
and, or at least don't use in any quantity.
link |
But I think new languages are a good idea in many respects,
link |
because they're often a chance to explore an idea
link |
of how language might help.
link |
I think that's one of the positive things
link |
about functional languages, for example.
link |
They're a particularly good place
link |
where people have explored ideas
link |
that at the time didn't seem feasible,
link |
but ultimately have wound up
link |
as part of mainstream languages as well.
link |
I mean, just go back as early as Recursion Lisp
link |
and then follow forward functions as first class citizens
link |
and pattern based languages,
link |
and gee, I don't know, closures,
link |
and just on and on and on.
link |
Lambda's interesting ideas that showed up first
link |
in, let's call it broadly,
link |
the functional programming community,
link |
and then find their way into mainstream languages.
link |
Yeah, it's a playground for rebels.
link |
Yeah, exactly, and so I think the languages
link |
in the playground themselves are probably not going
link |
to be the mainstream, at least for some while,
link |
but the ideas that come from there are invaluable.
link |
So let's go to something that, when I found out recently,
link |
so I've known that you've done a million things,
link |
but one of the things I wasn't aware of,
link |
that you had a role in Ample,
link |
and before you interrupt me by minimizing your role in it.
link |
Ample is for minimizing functions.
link |
Yeah, minimizing functions, right, exactly.
link |
Can I just say that the elegance and abstraction power
link |
of Ample is incredible,
link |
when I first came to it about 10 years ago or so.
link |
Can you describe what is the Ample language?
link |
Sure, so Ample is a language for mathematical programming,
link |
technical term, think of it as linear programming,
link |
that is setting up systems of linear equations
link |
that are of some sort of system of constraints,
link |
so that you have a bunch of things
link |
that have to be less than this, greater than that,
link |
whatever, and you're trying to find a set of values
link |
for some decision variables that will maximize
link |
or minimize some objective function,
link |
so it's a way of solving a particular kind
link |
of optimization problem,
link |
a very formal sort of optimization problem,
link |
but one that's exceptionally useful.
link |
And it specifies, so there's objective function constraints
link |
and variables that become separate
link |
from the data it operates on.
link |
So that kind of separation allows you to,
link |
put on different hats,
link |
one put the hat of an optimization person
link |
and then put another hat of a data person
link |
and dance back and forth,
link |
and also separate the actual solvers,
link |
the optimization systems that do the solving.
link |
Then you can have other people come to the table
link |
and then build their solvers,
link |
whether it's linear or nonlinear,
link |
convex, nonconvex, that kind of stuff.
link |
to you as, maybe you can comment
link |
how you got into that world
link |
and what is the beautiful or interesting idea to you
link |
from the world of optimization?
link |
So I preface it by saying I'm absolutely not an expert
link |
on this and most of the important work in AMPL
link |
comes from my two partners in crime on that,
link |
Bob Forer, who was a professor
link |
in the Industrial Engineering
link |
and Management Science Department at Northwestern,
link |
and my colleague at Bell Labs, Dave Gay,
link |
who was a numerical analyst and optimization person.
link |
So the deal is linear programming.
link |
Preface this by saying I don't.
link |
Let's stay with linear programming.
link |
Yeah, linear programming is the simplest example of this.
link |
So linear programming, as it's taught in school,
link |
is that you have a big matrix,
link |
which is always called A,
link |
and you say AX is less than or equal to B.
link |
So B is a set of constraints,
link |
X is the decision variables,
link |
and A is how the decision variables are combined
link |
to set up the various constraints.
link |
So A is a matrix and X and B are vectors.
link |
And then there's an objective function,
link |
which is just a sum of a bunch of Xs
link |
and some coefficients on them,
link |
and that's the thing you want to optimize.
link |
The problem is that in the real world,
link |
that matrix A is a very, very, very intricate,
link |
very large and very sparse matrix
link |
where the various components of the model
link |
are distributed among the coefficients
link |
in a way that is totally unobvious to anybody.
link |
And so what you need is some way
link |
to express the original model,
link |
which you and I would write,
link |
you know, we'd write mathematics on the board,
link |
and the sum of this is greater
link |
than the sum of that kind of thing.
link |
So you need a language to write those kinds of constraints.
link |
And Bob Forer, for a long time,
link |
had been interested in modeling languages,
link |
languages that made it possible to do this.
link |
There was a modeling language around called GAMS,
link |
the General Algebraic Modeling System,
link |
but it looked very much like Fortran.
link |
It was kind of clunky.
link |
And so Bob spent a sabbatical year at Bell Labs in 1984,
link |
and he and, there's only the office across from me,
link |
and it's always geography,
link |
and he and Dave Gay and I started talking
link |
about this kind of thing,
link |
and he wanted to design a language that would make it
link |
so that you could take these algebraic specifications,
link |
you know, summation signs over sets,
link |
and that you would write on the board
link |
and convert them into basically this A matrix,
link |
and then pass that off to a solver,
link |
which is an entirely separate thing.
link |
And so we talked about the design of the language.
link |
I don't remember any of the details of this now,
link |
but it's kind of an obvious thing.
link |
You're just writing out mathematical expressions
link |
in a Fortran like, sorry,
link |
an algebraic but textual like language.
link |
And I wrote the first version of this Ample program,
link |
my first C++ program, and.
link |
It's written in C++?
link |
And so I did that fairly quickly.
link |
We wrote, it was, you know, 3,000 lines or something,
link |
so it wasn't very big,
link |
but it sort of showed the feasibility of it
link |
that you could actually do something that was easy
link |
for people to specify models
link |
and convert it into something that a solver could work with.
link |
At the same time, as you say,
link |
the model and the data are separate things.
link |
So one model would then work with all kinds
link |
of different data in the same way
link |
that lots of programs do the same thing,
link |
but with different data.
link |
So one of the really nice things
link |
is the specification of the models,
link |
human, just kind of like, as you say, is human readable.
link |
Like I literally, I remember on stuff I worked,
link |
I would send it to colleagues
link |
that I'm pretty sure never programmed in their life,
link |
just to understand what the optimization problem is.
link |
I think, how hard is it to convert that?
link |
You said there's a first prototype in C++
link |
to convert that into something
link |
that could actually be used by the solver.
link |
because most of the solvers have some mechanism
link |
that lets them import a model in a form.
link |
It might be as simple as the matrix itself
link |
in just some representation,
link |
or if you're doing things that are not linear programming,
link |
then there may be some mechanism
link |
that lets you provide things like functions to be called,
link |
or other constraints on the model.
link |
So all AMPL does is to generate that kind of thing,
link |
and then solver deals with all the hard work,
link |
and then when the solver comes back with numbers,
link |
AMPL converts those back into your original form,
link |
so you know how much of each thing you should be buying,
link |
or making, or shipping, or whatever.
link |
So we did that in 84, and I haven't had a lot to do
link |
with it since, except that we wrote a couple of versions
link |
Which is one of the greatest books ever written.
link |
It's an excellent book.
link |
Bob Farrer wrote most of it,
link |
and so it's really, really well done.
link |
He must have been a dynamite teacher.
link |
And typeset in LaTeX.
link |
No, no, no, are you kidding?
link |
I remember liking the typography, so I don't know.
link |
We did it with DROF.
link |
I don't even know what that is.
link |
I think of DROF as a predecessor
link |
to the tech family of things.
link |
It's a formatter that was done at Bell Labs
link |
in this same period of the very early 70s
link |
that predates tech and things like that
link |
by five to 10 years.
link |
But it was nevertheless, I'm going by memories.
link |
I remember it being beautiful.
link |
Yeah, it was nicely done.
link |
Outside of Unix, C, A, Golang,
link |
all the things we talked about.
link |
All the amazing work you've done.
link |
You've also done work in graph theory.
link |
Let me ask this crazy out there question.
link |
If you had to make a bet,
link |
and I had to force you to make a bet,
link |
do you think P equals NP?
link |
although I'm told that somebody asked Jeff Dean
link |
if that was, under what conditions P would equal NP,
link |
and he said either P is zero or N is one.
link |
Or vice versa, I've forgotten.
link |
This is why Jeff Dean is a lot smarter than I am.
link |
So, but your intuition is, uh.
link |
I have no, I have no intuition,
link |
but I've got a lot of colleagues who've got intuition
link |
and their betting is no.
link |
That's the popular, that's the popular bet.
link |
Okay, so what is computational complexity theory?
link |
And do you think these kinds of complexity classes,
link |
especially as you've taught in this modern world,
link |
are still a useful way to understand
link |
the hardness of problems?
link |
I don't do that stuff.
link |
The last time I touched anything to do with that
link |
was before. Many, many years ago.
link |
Was before it was invented.
link |
Because I, it's literally true.
link |
I did my PhD thesis on graph.
link |
Before Big O notation.
link |
Before, I did this in 1968,
link |
and I worked on graph partitioning,
link |
which is this question.
link |
You've got a graph that is a nodes and edges kind of graph,
link |
and the edges have weights,
link |
and you just want to divide the nodes into two piles
link |
of equal size so that the number of edges
link |
that goes from one side to the other
link |
is as small as possible.
link |
You developed, so that problem is hard.
link |
Well, as it turns out,
link |
I worked with Shen Lin at Bell Labs on this,
link |
and we were never able to come up with anything
link |
that was guaranteed to give the right answer.
link |
We came up with heuristics that worked pretty darn well,
link |
and I peeled off some special cases for my thesis,
link |
but it was just hard.
link |
And that was just about the time that Steve Cook
link |
was showing that there were classes of problems
link |
that appeared to be really hard,
link |
of which graph partitioning was one.
link |
But this, my expertise, such as it was,
link |
totally predates that development.
link |
So the heuristic, which now,
link |
carries the two of yours names
link |
for the traveling salesman problem,
link |
and then for the graph partitioning.
link |
That was, like, how did you,
link |
you weren't even thinking in terms of classes.
link |
You were just trying to find.
link |
There was no such idea.
link |
A heuristic that kinda does the job pretty well.
link |
You were trying to find something that did the job,
link |
and there was nothing that you would call,
link |
let's say, a closed form or algorithmic thing
link |
that would give you a guaranteed right answer.
link |
I mean, compare graph partitioning to max flow min cut,
link |
or something like that.
link |
That's the same problem,
link |
except there's no constraint on the number of nodes
link |
on one side or the other of the cut.
link |
And that means it's an easy problem,
link |
at least as I understand it.
link |
Whereas the constraint that says
link |
the two have to be constrained in size
link |
makes it a hard problem.
link |
Yeah, so Robert Frost says that poem
link |
where you had to choose two paths.
link |
is there another alternate universe
link |
in which you pursued the Don Knuth path
link |
of algorithm design, sort of?
link |
You're infinitely modest,
link |
but so you pursued your kind of love of programming.
link |
I mean, when you look back to those,
link |
I mean, just looking into that world,
link |
does that just seem like a distant world
link |
of theoretical computer science?
link |
Then is it fundamentally different
link |
from the world of programming?
link |
I mean, certainly, in all seriousness,
link |
I just didn't have the talent for it.
link |
When I got here as a grad student to Princeton
link |
and I started to think about research
link |
at the end of my, I don't know,
link |
first year or something like that,
link |
I worked briefly with John Hopcroft,
link |
who is absolutely, you know,
link |
you mentioned during award winner, et cetera,
link |
a great guy, and it became crystal clear
link |
I was not cut out for this stuff, period, okay.
link |
And so I moved into things
link |
where I was more cut out for it,
link |
and that tended to be things like writing programs
link |
and then ultimately writing books.
link |
You said that in Toronto as an undergrad,
link |
you did a senior thesis or a literature survey
link |
on artificial intelligence.
link |
What was the AI landscape, ideas, dreams at that time?
link |
I think that was one of the,
link |
well, you've heard of AI winners.
link |
This is whatever the opposite was,
link |
AI summer or something.
link |
It was one of these things where people thought
link |
that, boy, we could do anything with computers,
link |
that all these hard problems, we could,
link |
computers will solve them.
link |
They will do machine translation.
link |
They will play games like chess.
link |
They will do, you know, prove theorems in geometry.
link |
There are all kinds of examples like that
link |
where people thought, boy,
link |
we could really do those sorts of things.
link |
And, you know, I read The Kool Aid in some sense.
link |
There's a wonderful collection of papers
link |
called Computers and Thought that was published
link |
in about that era and people were very optimistic.
link |
And then of course it turned out that
link |
what people thought was just a few years down the pike
link |
was more than a few years down the pike.
link |
And some parts of that are more or less now
link |
sort of under control.
link |
We finally do play games like Go and chess
link |
and so on better than people do,
link |
but there are others and machine translation
link |
is a lot better than it used to be,
link |
but that's, you know, 50, close to 60 years of progress
link |
and a lot of evolution in hardware
link |
and a tremendous amount more data up on which
link |
you can build systems that actually can learn
link |
from some of that data.
link |
And the infrastructure to support developers
link |
working together, like an open source movement,
link |
the internet, period, is also empowering.
link |
But what lessons do you draw from that,
link |
the opposite of winter, that optimism?
link |
Well, I guess the lesson is that in the short run
link |
it's pretty easy to be too pessimistic
link |
or maybe too optimistic and in the long run
link |
you probably shouldn't be too pessimistic.
link |
I'm not saying that very well.
link |
It reminds me of this remark from Arthur Clarke,
link |
a science fiction author, who says, you know,
link |
when some distinguished but elderly person
link |
says that something is possible, he's probably right.
link |
And if he says it's impossible, he's almost surely wrong.
link |
But you don't know what the time scale is.
link |
The time scale is critical, right.
link |
So what are your thoughts on this new summer of AI
link |
now in the work with machine learning and neural networks?
link |
You've kind of mentioned that you started to try to explore
link |
and look into this world that seems fundamentally different
link |
from the world of heuristics and algorithms like search,
link |
that it's now purely sort of trying to take
link |
huge amounts of data and learn from that data, right,
link |
programs from the data.
link |
Yeah, look, I think it's very interesting.
link |
I am incredibly far from an expert.
link |
Most of what I know I've learned from my students
link |
and they're probably disappointed
link |
in how little I've learned from them.
link |
But I think it has tremendous potential
link |
for certain kinds of things.
link |
I mean, games is one where it obviously has had an effect
link |
on some of the others as well.
link |
I think there's, and this is speaking from
link |
definitely not expertise,
link |
I think there are serious problems
link |
in certain kinds of machine learning at least
link |
because what they're learning from
link |
is the data that we give them.
link |
And if the data we give them has something wrong with it,
link |
then what they learn from it is probably wrong too.
link |
And the obvious thing is some kind of bias in the data.
link |
That the data has stuff in it like, I don't know,
link |
women aren't as good as men at something, okay.
link |
That's just flat wrong.
link |
But if it's in the data because of historical treatment,
link |
then that machine learning stuff will propagate that.
link |
And that is a serious worry.
link |
The positive part of that is what machine learning does
link |
is reveal the bias in the data
link |
and puts a mirror to our own society.
link |
And in so doing helps us remove the bias,
link |
you know, helps us work on ourselves.
link |
Puts a mirror to ourselves.
link |
Yeah, that's an optimistic point of view.
link |
And if it works that way, that would be absolutely great.
link |
And what I don't know is whether it does work that way
link |
or whether the AI mechanisms
link |
or machine learning mechanisms reinforce
link |
and amplify things that have been wrong in the past.
link |
And I don't know, but I think that's a serious thing
link |
that we have to be concerned about.
link |
Let me ask you an out there question, okay.
link |
I know nobody knows, but what do you think it takes
link |
to build a system of human level intelligence?
link |
That's been the dream from the 60s.
link |
We talk about games, about language,
link |
about image recognition, but really the dream
link |
is to create human level or superhuman level intelligence.
link |
What do you think it takes to do that?
link |
I haven't a clue and I don't know, roughly speaking.
link |
I mean, this was Turing.
link |
I was trying to trick you into a hypothesis.
link |
Yeah, I mean, Turing talked about this
link |
in his paper on machine intelligence back in, geez,
link |
I don't know, early 50s or something like that.
link |
And he had the idea of the Turing test.
link |
And I don't know what the Turing test is.
link |
It's a good test of intelligence.
link |
It's an interesting test.
link |
At least it's in some vague sense objective,
link |
whether you can read anything into the conclusions
link |
is a different story.
link |
Do you have worries, concerns, excitement
link |
about the future of artificial intelligence?
link |
So there's a lot of people who are worried
link |
and you can speak broadly
link |
than just artificial intelligence.
link |
It's basically computing taking over the world
link |
Are you excited by this future,
link |
this possibility of computing being everywhere
link |
or are you worried?
link |
It's some combination of those.
link |
I think almost all technologies over the long run
link |
are for good, but there's plenty of examples
link |
where they haven't been good either over a long run
link |
for some people or over a short run.
link |
And computing is one of those.
link |
And AI within it is gonna be one of those as well,
link |
but computing broadly.
link |
I mean, for just a today example is privacy,
link |
that the use of things like social media and so on
link |
means that, and the commercial surveillance
link |
means that there's an enormous amount more known about us
link |
by people, other businesses, government, whatever,
link |
than perhaps one ought to feel comfortable with.
link |
So that's an example.
link |
So that's an example of a possible negative effect
link |
of computing being everywhere.
link |
It's an interesting one
link |
because it could also be a positive, if leveraged correctly.
link |
There's a big if there.
link |
So I have a deep interest in human psychology
link |
and humans seem to be very paranoid about this data thing
link |
that varies depending on age group.
link |
It seems like the younger folks.
link |
So it's exciting to me to see what society looks like
link |
50 years from now, that the concerns about privacy
link |
might be flipped on their head
link |
based purely on human psychology
link |
versus actual concerns or not.
link |
What do you think about Moore's Law?
link |
Well, you said a lot of stuff we've talked,
link |
you talked about programming languages in their design,
link |
in their ideas that come from the constraints
link |
in the systems they operate in.
link |
Do you think Moore's Law,
link |
the exponential improvement of systems
link |
will continue indefinitely?
link |
There's a mix of opinions on that currently,
link |
or do you think there'll be a plateau?
link |
Well, the frivolous answer is no exponential
link |
it can go on forever.
link |
You run out of something.
link |
Just as we said, timescale matters.
link |
So if it goes on long enough, that might be all we need.
link |
Yeah, right, won't matter to us.
link |
So I don't know, we've seen places
link |
where Moore's Law has changed.
link |
For example, mentioned earlier,
link |
processors don't get faster anymore,
link |
but you use that same growth of the ability
link |
to put more things in a given area
link |
to grow them horizontally instead of vertically as it were
link |
so you can get more and more processors
link |
or memory or whatever on the same chip.
link |
Is that gonna run into a limitation?
link |
Presumably, because at some point
link |
you get down to the individual atoms.
link |
And so you gotta find some way around that.
link |
Will we find some way around that?
link |
I don't know, I just said that if I say it won't,
link |
I'll be wrong, so perhaps we will.
link |
So I just talked to Jim Keller and he says,
link |
so he actually describes, he argues
link |
that the Moore's Law will continue for a long, long time
link |
because you mentioned the atom.
link |
We actually have, I think, a thousand fold increase,
link |
still decreased in size, still possible
link |
before we get to the quantum level.
link |
So there's still a lot of possibilities.
link |
He thinks he'll continue indefinitely,
link |
which is an interesting optimistic viewpoint.
link |
But how do you think the programming languages
link |
will change with this increase?
link |
Whether we hit a wall or not,
link |
what do you think, do you think there'll be
link |
a fundamental change in the way
link |
programming languages are designed?
link |
I don't know about that.
link |
I think what will happen is continuation
link |
of what we see in some areas, at least,
link |
which is that more programming will be done
link |
by programs than by people, and that more will be done
link |
by sort of declarative rather than procedural mechanisms
link |
where I'll say, I want this to happen.
link |
You figure out how.
link |
And that is, in many cases, at this point,
link |
domain of specialized languages for narrow domains,
link |
but you can imagine that broadening out.
link |
And so I don't have to say so much, in so much detail,
link |
some collection of software, let's call it languages
link |
or programs or something, will figure out
link |
how to do what I want to do.
link |
Interesting, so increased levels of abstraction.
link |
And one day getting to the human level,
link |
where we can just use natural language.
link |
Could be possible.
link |
So you taught, so teach a course,
link |
Computers in Our World, here at Princeton,
link |
that introduces computing and programming to nonmajors.
link |
What, just from that experience,
link |
what advice do you have for people
link |
who don't know anything about programming
link |
but are kind of curious about this world,
link |
or programming seems to become more and more
link |
of a fundamental skill that people need to be
link |
at least aware of?
link |
Yeah, well, I couldn't recommend a good book.
link |
The book I wrote for the course.
link |
I think this is one of these questions of,
link |
should everybody know how to program?
link |
And I think the answer is probably not,
link |
but I think everybody should at least understand
link |
sort of what it is, so that if you say to somebody,
link |
I'm a programmer, they have a notion of what that might be,
link |
or if you say this is a program,
link |
or this was decided by a computer running a program,
link |
that they have some vague intuitive understanding
link |
and accurate understanding of what that might imply.
link |
So part of what I'm doing in this course,
link |
which is very definitely for nontechnical people,
link |
and a typical person in it is a history or English major,
link |
try and explain how computers work,
link |
how they do their thing, what programming is,
link |
how you write a program,
link |
and how computers talk to each other,
link |
and what do they do when they're talking to each other.
link |
And then I would say nobody, very rarely,
link |
and does anybody in that course go on
link |
to become a real serious programmer,
link |
but at least they've got a somewhat better idea
link |
of what all this stuff is about, not just the programming,
link |
but the technology behind computers and communications.
link |
Do they try and write a program themselves?
link |
Oh yeah, yeah, a very small amount.
link |
I introduced them to how machines work at a level below,
link |
high level languages, so we have kind of a toy machine
link |
that has a very small repertoire, a dozen instructions,
link |
and they write trivial assembly language programs for that.
link |
Wow, that's interesting.
link |
So can you just, if you were to give a flavor
link |
to people of the programming world,
link |
of the competing world,
link |
what are the examples they should go with?
link |
So a little bit of assembly to get a sense
link |
at the lowest level of what the program is really doing.
link |
Yeah, I mean, in some sense,
link |
there's no such thing as the lowest level
link |
because you can keep going down,
link |
but that's the place where I drew the line.
link |
So the idea that computers have a fairly small repertoire
link |
of very simple instructions that they can do,
link |
like add and subtract and branch and so on,
link |
as you mentioned earlier,
link |
and that you can write code at that level
link |
and it will get things done,
link |
and then you have the levels of abstraction
link |
that we get with higher level languages,
link |
like Fortran or C or whatever,
link |
and that makes it easier to write the code
link |
and less dependent on particular architectures.
link |
And then we talk about a lot of the different kinds
link |
of programs that they use all the time
link |
that they don't probably realize are programs,
link |
like they're running Mac OS on their computers
link |
or maybe Windows, and they're downloading apps
link |
on their phones, and all of those things are programs
link |
that are just what we just talked about,
link |
except at a grand scale.
link |
And it's easy to forget that they're actual programs
link |
that people program.
link |
There's engineers that wrote those things.
link |
And so in a way, I'm expecting them
link |
to make an enormous conceptual leap
link |
from their five or 10 line toy assembly language thing
link |
that adds two or three numbers to something
link |
that is a browser on their phone or whatever,
link |
but it's really the same thing.
link |
So if you look in broad strokes at history,
link |
what do you think the world,
link |
how do you think the world changed because of computers?
link |
It's hard to sometimes see the big picture
link |
when you're in it, but I guess I'm asking
link |
if there's something you've noticed over the years
link |
that, like you were mentioning,
link |
the students are more distracted looking at their,
link |
now there's a device to look at.
link |
I think computing has changed a tremendous amount,
link |
obviously, but I think one aspect of that
link |
is the way that people interact with each other,
link |
both locally and far away.
link |
And when I was the age of those kids,
link |
making a phone call to somewhere was a big deal
link |
because it costs serious money.
link |
And this was in the 60s, right?
link |
And today people don't make phone calls,
link |
they send texts or something like that.
link |
So there's an up and down in what people do.
link |
People think nothing of having correspondence,
link |
regular meetings, video, whatever,
link |
with friends or family or whatever
link |
in any other part of the world,
link |
and they don't think about that at all.
link |
And so that's just the communication aspect of it.
link |
Do you think that brings us closer together
link |
or does it make us,
link |
does it take us away from the closeness
link |
of human to human contact?
link |
I think it depends a lot on all kinds of things.
link |
So I trade mail with my brother and sister in Canada
link |
much more often than I used to talk to them on the phone.
link |
So probably every two or three days,
link |
I get something or send something to them.
link |
Whereas 20 years ago,
link |
I probably wouldn't have talked to them
link |
on the phone nearly as much.
link |
So in that sense, that's brought my brother and sister
link |
and I closer together.
link |
That's a good thing.
link |
I watch the kids on campus
link |
and they're mostly walking around with their heads down,
link |
fooling with their phones
link |
to the point where I have to duck them.
link |
I don't know that that has brought them closer together
link |
There's sociological research that says people are,
link |
in fact, not as close together as they used to be.
link |
I don't know where that's really true,
link |
but I can see potential downsides
link |
and kids where you think,
link |
come on, wake up and smell the coffee or whatever.
link |
But if you look at, again, nobody can predict the future,
link |
but are you excited?
link |
Kind of touched this a little bit with AI,
link |
but are you excited by the future in the next 10, 20 years
link |
that computing will bring?
link |
You were there when there was no computers really.
link |
And now computers are everywhere all over the world
link |
and Africa and Asia and just every person,
link |
almost every person in the world has a device.
link |
So are you hopeful, optimistic about that future?
link |
It's mixed, if the truth be told.
link |
I mean, I think there are some things about that
link |
I think there's the potential for people
link |
to improve their lives all over the place
link |
and that's obviously good.
link |
And at the same time, at least in the short run,
link |
you can see lots and lots of bad
link |
as people become more tribalistic or parochial
link |
in their interests and it's an enormous amount
link |
more us than them and people are using computers
link |
in all kinds of ways to mislead or misrepresent
link |
or flat out lie about what's going on
link |
and that is affecting politics locally
link |
and I think everywhere in the world.
link |
Yeah, the long term effect on political systems
link |
and so on is who knows.
link |
The people now have a voice which is a powerful thing.
link |
People who are oppressed have a voice
link |
but also everybody has a voice
link |
and the chaos that emerges from that
link |
is fascinating to watch.
link |
Yeah, yeah, it's kind of scary.
link |
If you can go back and relive a moment in your life,
link |
one that made you truly happy outside of family
link |
or was profoundly transformative,
link |
is there a moment or moments that jump out at you
link |
I don't think specific moments.
link |
I think there were lots and lots and lots of good times
link |
at Bell Labs where you would build something
link |
Did you say it worked?
link |
So the moment it worked.
link |
Yeah, and somebody used it and they said,
link |
Those kinds of things happened quite often
link |
in that sort of golden era in the 70s when Unix was young
link |
and there was all this low hanging fruit
link |
and interesting things to work on
link |
and a group of people who kind of,
link |
we were all together in this and if you did something,
link |
they would try it out for you.
link |
And I think that was in some sense,
link |
a really, really good time.
link |
And AUK was, was AUK an example of that?
link |
That when you built it and people used it?
link |
And now millions of people use it.
link |
And all your stupid mistakes are right there
link |
for them to look at, right?
link |
Yeah, it's terrifying, vulnerable
link |
but it's beautiful because it does have a positive impact
link |
on so, so many people.
link |
So I think there's no better way to end it.
link |
Brian, thank you so much for talking to us, it was an honor.
link |
Okay, my pleasure.
link |
Thank you for listening to this conversation
link |
with Brian Kernighan and thank you to our sponsors,
link |
8 Sleep Mattress and Raycon Earbuds.
link |
Please consider supporting this podcast
link |
by going to 8sleep.com slash Lex and to buyraycon.com
link |
slash Lex, click the links, buy the stuff.
link |
These both are amazing products.
link |
It really is the best way to support this podcast
link |
and the journey I'm on.
link |
It's how they know I sent you and increases the chance
link |
that they'll actually support this podcast in the future.
link |
If you enjoy this thing, subscribe on YouTube,
link |
review it with 5 Stars and Apple Podcast,
link |
support it on Patreon or connect with me on Twitter
link |
at Lex Friedman, spelled somehow miraculously
link |
without the letter E, just F R I D M A N
link |
because when we immigrated to this country,
link |
we were not so good at spelling.
link |
And now let me leave you with some words
link |
from Brian Kernighan, don't comment bad code, rewrite it.
link |
Thank you for listening and hope to see you next time.