back to indexBjarne Stroustrup: C++ | Lex Fridman Podcast #48
link |
The following is a conversation with Bjarne Stroustrup.
link |
He is the creator of C++, a programming language that, after 40 years, is still one of the most
link |
popular and powerful languages in the world. Its focus on fast, stable, robust code underlies many
link |
of the biggest systems in the world that we have come to rely on as a society. If you're watching
link |
this on YouTube, for example, many of the critical back end components of YouTube are written in C++.
link |
The same goes for Google, Facebook, Amazon, Twitter, most Microsoft applications, Adobe applications,
link |
most database systems, and most physical systems that operate in the real world, like cars, robots,
link |
rockets that launch us into space and one day will land us on Mars.
link |
C++ also happens to be the language that I use more than any other in my life. I've written
link |
several hundred thousand lines of C++ source code. Of course, lines of source code don't mean much,
link |
but they do give hints of my personal journey through the world of software.
link |
I've enjoyed watching the development of C++ as a programming language,
link |
leading up to the big update in the standard in 2011 and those that followed in 14, 17,
link |
and toward the new C++20 standard hopefully coming out next year.
link |
TITLE This is the Artificial Intelligence podcast.
link |
If you enjoy it, subscribe on YouTube, give it five stars on iTunes, support it on Patreon,
link |
or simply connect with me on Twitter at Lex Friedman, spelled F R I D M A N.
link |
And now, here's my conversation with Björn Stroustrup.
link |
What was the first program you've ever written? Do you remember?
link |
BJÖRN It was my second year in university, first year of computer science, and it was an Alco 60.
link |
I calculated the shape of a super ellipse and then connected points on the perimeter,
link |
creating star patterns. It was with a wet ink on a paper printer.
link |
TITLE And that was in college, university?
link |
BJÖRN Yeah, yeah. I learned to program the second year in university.
link |
TITLE And what was the first programming language,
link |
if I may ask it this way, that you fell in love with?
link |
BJÖRN I think Alco 60. And after that, I remember
link |
Snowball. I remember Fortran, didn't fall in love with that. I remember Pascal,
link |
didn't fall in love with that. It all got in the way of me. And then I discovered Assembler,
link |
and that was much more fun. And from there, I went to Micro Code.
link |
TITLE So you were drawn to the, you found the low level stuff beautiful.
link |
BJÖRN I went through a lot of languages, and then I spent significant time in Assembler and
link |
Micro Code. That was sort of the first really profitable things that paid for my masters,
link |
actually. And then I discovered Simula, which was absolutely great.
link |
BJÖRN Simula was the extension of Alco 60,
link |
done primarily for simulation. But basically, they invented object oriented programming at
link |
inheritance and runtime polymorphism while they were doing it. And that was the language that
link |
taught me that you could have the sort of the problems of a program grow with the size of the
link |
program rather than with the square of the size of the program. That is, you can actually modularize
link |
very nicely. And that was a surprise to me. It was also a surprise to me that a stricter type
link |
system than Pascal's was helpful, whereas Pascal's type system got in my way all the time. So you
link |
need a strong type system to organize your code well, but it has to be extensible and flexible.
link |
TITLE Let's get into the details a little bit. If you remember, what kind of type system did
link |
Pascal have? What type system, typing system did Alco 60 have?
link |
BJÖRN Basically, Pascal was sort of the simplest language that Niklaus Wirth could
link |
define that served the needs of Niklaus Wirth at the time. And it has a sort of a highly moral
link |
tone to it. That is, if you can say it in Pascal, it's good. And if you can't, it's not so good.
link |
Whereas Simula allowed you basically to build your own type system. So instead of trying to fit
link |
yourself into Niklaus Wirth's world, Christen Nygaard's language and Johan Dahl's language
link |
allowed you to build your own. So it's sort of close to the original idea of you build a domain
link |
specific language. As a matter of fact, what you build is a set of types and relations among types
link |
that allows you to express something that's suitable for an application.
link |
TITLE So when you say types,
link |
stuff you're saying has echoes of object oriented programming.
link |
BJÖRN Yes, they invented it. Every language
link |
that uses the word class for type is a descendant of Simula, directly or indirectly. Christen Nygaard
link |
and Ole Johan Dahl were mathematicians and they didn't think in terms of types, but they understood
link |
sets and classes of elements. And so they called their types classes. And basically in C++,
link |
as in Simula, classes are user defined type.
link |
TITLE So can you try the impossible task and give
link |
a brief history of programming languages from your perspective? So we started with Algol 60,
link |
Simula, Pascal, but that's just the 60s and 70s. BJÖRN I can try. The most sort of interesting and
link |
major improvement of programming languages was Fortran, the first Fortran. Because before that,
link |
all code was written for a specific machine and each specific machine had a language,
link |
a simple language or a cross simpler or some extension of that idea. But you're writing for
link |
a specific machine in the language of that machine. And Bacchus and his team at IBM built a language
link |
that would allow you to write what you really wanted. That is, you could write it in a language
link |
that was natural for people. Now, these people happen to be engineers and physicists. So the
link |
language that came out was somewhat unusual for the rest of the world. But basically they said
link |
formula translation because they wanted to have the mathematical formulas translated into the
link |
machine. And as a side effect, they got portability because now they're writing in the terms that the
link |
humans used and the way humans thought. And then they had a program that translated it into the
link |
machine's needs. And that was new and that was great. And it's something to remember. We want to
link |
raise the language to the human level, but we don't want to lose the efficiency.
link |
And that was the first step towards the human. That was the first step. And of course,
link |
they were a very particular kind of humans. Business people were different, so they got
link |
cobalt instead, et cetera, et cetera. And Simula came out. No, let's not go to Simula yet. Let's
link |
go to Algol. Fortran didn't have, at the time, the notions of not a precise notion of type,
link |
not a precise notion of scope, not a set of translation phases that was what we have today,
link |
lexical syntax, semantics. It was sort of a bit of a model in the early days, but
link |
hey, they've just done the biggest breakthrough in the history of programming, right?
link |
So you can't criticize them for not having gotten all the technical details right.
link |
So we got Algol. That was very pretty. And most people in commerce and science considered it
link |
useless because it was not flexible enough, and it wasn't efficient enough, and et cetera,
link |
et cetera. But that was a breakthrough from a technical point of view.
link |
Then Simula came along to make that idea more flexible, and you could define your own types.
link |
And that's where I got very interested. Christen Nygård was the main idea man behind Simula.
link |
That was late 60s.
link |
This was late 60s. Well, I was a visiting professor in Aarhus, and so I learned object
link |
oriented programming by sitting around and, well, in theory, discussing with Christen Nygård. But
link |
Christen, once you get started and in full flow, it's very hard to get a word in edgeways.
link |
Where you just listen.
link |
So it was great. I learned it from there.
link |
Not to romanticize the notion, but it seems like a big leap to think about
link |
object oriented programming. It's really a leap of abstraction.
link |
And was that as big and beautiful of a leap as it seems from now in retrospect,
link |
or was it an obvious one at the time?
link |
It was not obvious, and many people have tried to do something like that,
link |
and most people didn't come up with something as wonderful as Simula.
link |
Lots of people got their PhDs and made their careers out of forgetting about Simula or never
link |
knowing it. For me, the key idea was basically I could get my own types. And that's the idea that
link |
goes further into C++, where I can get better types and more flexible types and more efficient
link |
types. But it's still the fundamental idea. When I want to write a program, I want to write it with
link |
my types that is appropriate to my problem and under the constraints that I'm under with hardware,
link |
software, environment, et cetera. And that's the key idea.
link |
People picked up on the class hierarchies and the virtual functions and the inheritance,
link |
and that was only part of it. It was an interesting and major part and still a major
link |
part and a lot of graphic stuff, but it was not the most fundamental. It was when you wanted to
link |
relate one type to another. You don't want them all to be independent. The classical example is
link |
that you don't actually want to write a city simulation with vehicles where you say, well,
link |
if it's a bicycle, write the code for turning a bicycle to the left. If it's a normal car,
link |
turn right the normal car way. If it's a fire engine, turn right the fire engine way. You get
link |
these big case statements and bunches of if statements and such. Instead, you tell the base
link |
class that that's the vehicle saying, turn left the way you want to. And this is actually a real
link |
example. They used it to simulate and optimize the emergency services for somewhere in Norway back in
link |
the 60s. So this was one of the early examples for why you needed inheritance and you needed
link |
a runtime polymorphism because you wanted to handle this set of vehicles in a manageable way.
link |
You can't just rewrite your code each time a new kind of vehicle comes along.
link |
Yeah, that's a beautiful, powerful idea. And of course it stretches through your work
link |
with C++ as we'll talk about. But I think you've structured it nicely. What other breakthroughs
link |
came along in the history of programming languages if we were to tell the history in that way?
link |
Obviously, I'm better at telling the part of the history that is the path I'm on as opposed to all
link |
the paths. Yeah, you skipped the hippie John McCarthy and Lisp, one of my favorite languages.
link |
But Lisp is not one of my favorite languages. It's obviously important. It's obviously interesting.
link |
Lots of people write code in it and then they rewrite it into C or C++ when they want to go
link |
to production. It's in the world I'm at, which are constrained by performance, reliability,
link |
issues, deployability, cost of hardware. I don't like things to be too dynamic.
link |
It is really hard to write a piece of code that's perfectly flexible that you can also deploy on a
link |
small computer and that you can also put in, say, a telephone switch in Bogota. What's the chance?
link |
If you get an error and you find yourself in the debugger that the telephone switch in Bogota on
link |
late Sunday night has a programmer around, their chance is zero. A lot of the things I think most
link |
about can't afford that flexibility. I'm quite aware that maybe 70%, 80% of all code are not
link |
under the kind of constraints I'm interested in. But somebody has to do the job I'm doing
link |
because you have to get from these high level flexible languages to the hardware.
link |
The stuff that lasts for 10, 20, 30 years is robust, operates under very constrained conditions.
link |
Yes, absolutely. That's right. And it's fascinating and beautiful in its own way.
link |
C++ is one of my favorite languages, and so is Lisp. So I can embody two for different reasons
link |
as a programmer. I understand why Lisp is popular, and I can see the beauty of the ideas
link |
and similarly with Smalltalk. It's just not as relevant in my world. And by the way, I distinguish
link |
between those and the functional languages where I go to things like ML and Haskell. Different kind
link |
of languages, they have a different kind of beauty and they're very interesting. And I think
link |
that's interesting. And I actually try to learn from all the languages I encounter to see what is
link |
there that would make working on the kind of problems I'm interested in with the kind of
link |
constraints that I'm interested in, what can actually be done better? Because we can surely
link |
do better than we do today. You've said that it's good for any professional programmer to know at
link |
least five languages as speaking about a variety of languages that you've taken inspiration from,
link |
and you've listed yours as being, at least at the time, C++, obviously, Java, Python, Ruby,
link |
script. Can you first of all, update that list, modify it? You don't have to be constrained
link |
to just five, but can you describe what you picked up also from each of these languages?
link |
How do you see them as inspirations for you when you're working with C++?
link |
This is a very hard question to answer. So about languages, you should know
link |
languages. I reckon I knew about 25 or thereabouts when I did C++. It was easier in those days
link |
because the languages were smaller, and you didn't have to learn a whole programming environment and
link |
such to do it. You could learn the language quite easily. And it's good to learn so many languages.
link |
I imagine, just like with natural language for communication, there's different
link |
paradigms that emerge in all of them, that there's commonalities and so on.
link |
So I picked five out of a hat. You picked five out of a hat.
link |
Obviously. The important thing that the number is not one.
link |
It's like, I don't like, I mean, if you're a monoglot, you are likely to think that your
link |
own culture is the only one superior to everybody else's. A good learning of a foreign language and
link |
a foreign culture is important. It helps you think and be a better person. With programming
link |
languages, you become a better programmer, better designer with the second language.
link |
Now, once you've got two, the weight of five is not that long. It's the second one that's
link |
most important. And then when I had to pick five, I sort of thinking what kinds of languages are
link |
there? Well, there's a really low level stuff. It's good. It's actually good to know machine code.
link |
Even today. The C++ optimizers write better machine code than I do.
link |
Yes. But I don't think I could appreciate them if I actually didn't understand machine code and
link |
machine architecture. At least in my position, I have to understand a bit of it because you mess
link |
up the cache and you're off in performance by a factor of 100. It shouldn't be that if you are
link |
interested in either performance or the size of the computer you have to deploy. So I would
link |
go as a simpler. I used to mention C, but these days going low level is not actually what gives
link |
you the performance. It is to express your ideas so cleanly that you can think about it and the
link |
optimizer can understand what you're up to. My favorite way of optimizing these days is to throw
link |
away out the clever bits and see if it still runs fast. And sometimes it runs faster. So I need the
link |
abstraction mechanisms or something like C++ to write compact high performance code. There was a
link |
beautiful keynote by Jason Turner at the CppCon a couple of years ago where he decided he was going
link |
to program Pong on Motorola 6800, I think it was. And he says, well, this is relevant because it
link |
looks like a microcontroller. It has specialized hardware. It has not very much memory and it's
link |
relatively slow. And so he shows in real time how he writes Pong starting with fairly straightforward
link |
low level stuff, improving his abstractions and what he's doing. He's writing C++ and it translates
link |
into 86 assembler, which you can do with Clang and you can see it in real time. It's
link |
the compiler explorer, which you can use on the web. And then he wrote a little program
link |
that translated 86 assembler into Motorola assembler. And so he types and you can see this
link |
thing in real time. Wow. You can see it in real time. And even if you can't read the assembly code,
link |
you can just see it. His code gets better. The code, the assembler gets smaller.
link |
He increases the abstraction level, uses C++ 11 as it were better.
link |
This code gets cleaner. It gets easier maintainable. The code shrinks and it keeps shrinking. And
link |
I could not in any reasonable amount of time write that assembler as good as the compiler
link |
generated from really quite nice modern C++. And I'll go as far as to say the thing that looked
link |
like C was significantly uglier and smaller and larger when it became machine code.
link |
So the abstractions that can be optimized are important.
link |
I would love to see that kind of visualization in larger code bases.
link |
Yeah. That might be beautiful.
link |
But you can't show a larger code base in a one hour talk and have it fit on screen.
link |
Right. So that's C and C++.
link |
So my two languages would be machine code and C++. And then I think you can learn a
link |
lot from the functional languages. So PIC has gloy ML. I don't care which. I think actually
link |
you learn the same lessons of expressing especially mathematical notions really clearly
link |
and having a type system that's really strict. And then you should probably have a language for sort
link |
of quickly churning out something. You could pick JavaScript. You could pick Python. You could pick
link |
Ruby. What do you make of JavaScript in general? So you're talking in the platonic sense about
link |
languages, about what they're good at, what their philosophy of design is. But there's also a large
link |
user base behind each of these languages and they use it in the way sometimes maybe it wasn't
link |
really designed for. That's right. JavaScript is used way beyond probably what it was designed for.
link |
Let me say it this way. When you build a tool, you do not know how it's going to be used.
link |
You try to improve the tool by looking at how it's being used and when people cut their fingers
link |
off and try and stop that from happening. But really you have no control over how something
link |
is used. So I'm very happy and proud of some of the things C++ is being used at and some of the
link |
things I wish people wouldn't do. Bitcoin mining being my favorite example uses as much energy as
link |
Switzerland and mostly serves criminals. But back to the languages, I actually think that having
link |
JavaScript run in the browser was an enabling thing for a lot of things. Yes, you could have
link |
done it better, but people were trying to do it better and they were using more principles,
link |
language designs, but they just couldn't do it right. And the nonprofessional programmers that
link |
write lots of that code just couldn't understand them. So it did an amazing job for what it was.
link |
It's not the prettiest language and I don't think it ever will be the prettiest language, but
link |
let's not be bigots here. So what was the origin story of C++?
link |
Yeah, you basically gave a few perspectives of your inspiration of object oriented programming.
link |
That's you had a connection with C and performance efficiency was an important
link |
thing you were drawn to. Efficiency and reliability. Reliability. You have to get both.
link |
What's reliability? I really want my telephone calls to get through and I want the quality
link |
of what I am talking, coming out at the other end. The other end might be in London or wherever.
link |
And you don't want the system to be crashing. If you're doing a bank, you mustn't crash. It might
link |
be your bank account that is in trouble. There's different constraints like in games, it doesn't
link |
matter too much if there's a crash, nobody dies and nobody gets ruined.
link |
But I am interested in the combination of performance, partly because of sort of speed
link |
of things being done, part of being able to do things that is necessary to have reliability
link |
of larger systems. If you spend all your time interpreting a simple function call,
link |
a simple function call, you are not going to have enough time to do proper signal processing to get
link |
the telephone calls to sound right. Either that or you have to have ten times as many computers
link |
and you can't afford your phone anymore. It's a ridiculous idea in the modern world because
link |
we have solved all of those problems. I mean, they keep popping up in different ways because
link |
we tackle bigger and bigger problems. So efficiency remains always an important aspect.
link |
But you have to think about efficiency, not just as speed, but as an enabler to
link |
important things. And one of the things it enables is reliability, is dependability.
link |
When I press the pedal, the brake pedal of a car, it is not actually connected directly
link |
to anything but a computer. That computer better work.
link |
Let's talk about reliability just a little bit. So modern cars have ECUs, have millions of lines
link |
of code today. So this is certainly especially true of autonomous vehicles where some of the
link |
aspects of the control or driver assistance systems that steer the car, that keep it in the
link |
lane and so on. So how do you think, you know, I talked to regulators, people in government
link |
who are very nervous about testing the safety of these systems of software. Ultimately software
link |
that makes decisions that could lead to fatalities. So how do we test software systems like these?
link |
First of all, safety, like performance and like security is the system's property.
link |
People tend to look at one part of a system at a time and saying something like, this is secure.
link |
That's all right. I don't need to do that. Yeah, that piece of code is secure. I'll buy
link |
your operator. If you want to have reliability, if you want to have performance, if you want to
link |
have security, you have to look at the whole system. I did not expect you to say that,
link |
but that's very true. Yes, I'm dealing with one part of the system and I want my part to be really
link |
good, but I know it's not the whole system. Furthermore, if making an individual part perfect,
link |
may actually not be the best way of getting the highest degree of reliability and performance and
link |
such. There's people that say C++ is not type safe. You can break it. Sure. I can break anything
link |
that runs on a computer. I may not go through your type system. If I wanted to break into your
link |
computer, I'll probably try SQL injection. And it's very true. If you think about
link |
safety or even reliability at the system level, especially when a human being is involved,
link |
it starts becoming hopeless pretty quickly in terms of proving that something is
link |
safe to a certain level. Yeah. Because there's so many variables. It's so complex. Well, let's get
link |
back to something we can talk about and actually talk about it. Yeah.
link |
Talk about and actually make some progress on. Yes. We can look at C++ programs and we can
link |
try and make sure they crash this often. The way you do that is largely by simplification.
link |
The first step is to simplify the code, have less code, have code that are less likely to go wrong.
link |
It's not by runtime testing everything. It is not by big test frameworks that you are using.
link |
Yes, we do that also. But the first step is actually to make sure that when you want to
link |
express something, you can express it directly in code rather than going through endless loops
link |
and convolutions in your head before it gets down the code. The way you are thinking about
link |
a problem is not in the code. There is a missing piece that's just in your head. And the code,
link |
you can see what it does, but it cannot see what you thought about it unless you have expressed
link |
things directly. When you express things directly, you can maintain it. It's easier to find errors.
link |
It's easier to make modifications. It's actually easier to test it. And lo and behold, it runs
link |
faster. And therefore, you can use a smaller number of computers, which means there's less
link |
hardware that could possibly break. So I think the key here is simplification.
link |
But it has to be, to use the Einstein quote, as simple as possible and no simpler.
link |
There are other areas with under constraints where you can be simpler than you can be in C++.
link |
But in the domain I'm dealing with, that's the simplification I'm after.
link |
So how do you inspire or ensure that the Einstein level of simplification is reached?
link |
So can you do code review? Can you look at code? If I gave you the code for the Ford F150
link |
and said, here, is this a mess or is this okay? Is it possible to tell? Is it possible to regulate?
link |
An experienced developer can look at code and see if it smells. Mixed metaphors deliberately.
link |
Yes. The point is that it is hard to generate something that is really obviously clean and
link |
can be appreciated. But you can usually recognize when you haven't reached that point.
link |
And so I've never looked at the F150 code, so I wouldn't know. But I know what I ought to be
link |
looking for. I'll be looking for some tricks that correlate with bugs and elsewhere. And I have tried
link |
to formulate rules for what good code looks like. And the current version of that is called the C++
link |
core guidelines. One thing people should remember is there's what you can do in a language and what
link |
you should do. In a language, you have lots of things that is necessary in some context,
link |
but not in others. There's things that exist just because there's 30 year old code out there and
link |
you can't get rid of it. But you can't have rules that says when you create it, try and follow these
link |
rules. This does not create good programs by themselves, but it limits the damage from mistakes.
link |
It limits the possibilities of mistakes. And basically, we are trying to say, what is it that
link |
a good programmer does? At the fairly simple level of where you use the language and how you use it.
link |
Now, I can put all the rules for chiseling in marble. It doesn't mean that somebody who follows
link |
all of those rules can do a masterpiece by Michelangelo. That is, there's something else
link |
to write a good program. Just is there something else to create an important work of art? That is,
link |
there's some kind of inspiration, understanding, gift. But we can approach the sort of technical,
link |
the craftsmanship level of it. The famous painters, the famous sculptures was among other things,
link |
superb craftsmen. They could express their ideas using their tools very well. And so these days,
link |
I think what I'm doing, what a lot of people are doing, we are still trying to figure out how it is
link |
to use our tools very well. For a really good piece of code, you need a spark of inspiration,
link |
and you can't, I think, regulate that. You cannot say that I'll take a picture only,
link |
I'll buy your picture only if you're at least Van Gogh. There are other things you can regulate,
link |
but not the inspiration. I think that's quite beautifully put. It is true that there is as an
link |
experienced programmer, when you see code that's inspired, that's like Michelangelo, you know it
link |
when you see it. And the opposite of that is code that is messy, code that smells, you know,
link |
when you see it. And I'm not sure you can describe it in words, except vaguely through guidelines and
link |
so on. Yes, it's easier to recognize ugly than to recognize beauty in code. And for the reason is
link |
that sometimes beauty comes from something that's innovative and unusual. And you have to sometimes
link |
think reasonably hard to appreciate that. On the other hand, the messes have things that are
link |
in common. And you can have static checkers and dynamic checkers that find
link |
a large number of the most common mistakes. You can catch a lot of sloppiness mechanically. I'm
link |
a great fan of static analysis in particular, because you can check for not just the language
link |
rules, but for the usage of language rules. And I think we will see much more static analysis
link |
in the coming decade. Can you describe what static analysis is? You represent a piece of code
link |
so that you can write a program that goes over that representation and look for things that are
link |
are right and not right. So, for instance, you can analyze a program to see if
link |
resources are leaked. That's one of my favorite problems. It's not actually all that hard and
link |
modern C++, but you can do it. If you are writing in the C level, you have to have a malloc and a
link |
free. And they have to match. If you have them in a single function, you can usually do it very
link |
easily. If there's a malloc here, there should be a free there. On the other hand, in between can be
link |
showing complete code and then it becomes impossible. If you pass that pointer to the
link |
memory out of a function and then want to make sure that the free is done somewhere else,
link |
now it gets really difficult. And so for static analysis, you can run through a program and you
link |
can try and figure out if there's any leaks. And what you will probably find is that you will find
link |
some leaks and you'll find quite a few places where your analysis can't be complete. It might
link |
depend on runtime. It might depend on the cleverness of your analyzer and it might take a
link |
long time. Some of these programs run for a long time. But if you combine such analysis
link |
with a set of rules that says how people could use it, you can actually see why the rules are
link |
violated. And that stops you from getting into the impossible complexities. You don't want to
link |
solve the halting problem. So static analysis is looking at the code without running the code.
link |
Yes. And thereby it's almost not a production code, but it's almost like an education tool
link |
of how the language should be used. It guides you like it at its best, right? It would
link |
guide you in how you write future code as well. And you learn together.
link |
Yes. So basically you need a set of rules for how you use the language. Then you need a static
link |
analysis that catches your mistakes when you violate the rules or when your code ends up
link |
doing things that it shouldn't, despite the rules, because there is the language rules.
link |
We can go further. And again, it's back to my idea that I'd much rather find errors before
link |
I start running the code. If nothing else, once the code runs, if it catches an error at run times,
link |
I have to have an error handler. And one of the hardest things to write in code is error handling
link |
code, because you know something went wrong. Do you know really exactly what went wrong?
link |
Usually not. How can you recover when you don't know what the problem was? You can't be 100% sure
link |
what the problem was in many, many cases. And this is part of it. So yes, we need good languages,
link |
we need good type systems, we need rules for how to use them, we need static analysis. And the
link |
ultimate for static analysis is of course program proof, but that still doesn't scale to the kind
link |
of systems we deploy. Then we start needing testing and the rest of the stuff.
link |
So C++ is an object oriented programming language that creates, especially with its newer versions,
link |
as we'll talk about, higher and higher levels of abstraction. So how do you design?
link |
Let's even go back to the origin of C++. How do you design something with so much abstraction
link |
that's still efficient and is still something that you can manage, do static analysis on,
link |
you can have constraints on, they can be reliable, all those things we've talked about.
link |
To me, there's a slight tension between high level abstraction and efficiency.
link |
That's a good question. I could probably have a year's course just trying to answer it.
link |
Yes, there's a tension between efficiency and abstraction, but you also get the interesting
link |
situation that you get the best efficiency out of the best abstraction. And my main tool
link |
for efficiency for performance actually is abstraction. So let's go back to how C++ was
link |
got there. You said it was object oriented programming language. I actually never said that.
link |
It's always quoted, but I never did. I said C++ supports object oriented programming and other
link |
techniques. And that's important because I think that the best solution to most complex,
link |
interesting problems require ideas and techniques from things that has been called object oriented
link |
data abstraction, functional, traditional C style code, all of the above. And so when I was designing
link |
C++, I soon realized I couldn't just add features. If you just add what looks pretty or what people
link |
ask for or what you think is good, one by one, you're not going to get a coherent whole. What
link |
you need is a set of guidelines that that guides your decisions. Should this feature be in or should
link |
this feature be out? How should a feature be modified before it can go in and such?
link |
And in the book I wrote about that, the design evolution of C++, there's a whole bunch of rules
link |
like that. Most of them are not language technical. They're things like don't violate static type
link |
system because I like static type system for the obvious reason that I like things to be reliable
link |
on reasonable amounts of hardware. But one of these rules is the zero overhead principle.
link |
The what kind of principle?
link |
The zero overhead principle. It basically says that if you have an abstraction,
link |
it should not cost anything compared to write the equivalent code at a lower level.
link |
So if I have, say, a matrix multiply, it should be written in such a way that you could not drop to
link |
the C level of abstraction and use arrays and pointers and such and run faster.
link |
And so people have written such matrix multiplications, and they've actually gotten
link |
code that ran faster than Fortran because once you had the right abstraction, you can eliminate
link |
temporaries and you can do loop fusion and other good stuff like that. That's quite hard to do by
link |
hand and in a lower level language. And there's some really nice examples of that.
link |
And the key here is that that matrix multiplication, the matrix abstraction,
link |
allows you to write code that's simple and easy. You can do that in any language.
link |
But with C++, it has the features so that you can also have this thing run faster than if you hand
link |
coded it. Now, people have given that lecture many times, I and others, and a very common
link |
question after the talk where you have demonstrated that you can outperform Fortran for
link |
dense matrix multiplication, people come up and says, yeah, but that was C++.
link |
If I rewrote your code in C, how much faster would it run? The answer is much slower.
link |
This happened the first time actually back in the 80s with a friend of mine called Doug McElroy,
link |
who demonstrated exactly this effect. And so the principle is you should give programmers the tools
link |
so that the abstractions can follow the zero void principle. Furthermore, when you put in a language
link |
feature in C++ or a standard library feature, you try to meet this. It doesn't mean it's absolutely
link |
optimal, but it means if you hand code it with the usual facilities in the language in C++ in C,
link |
you should not be able to better it. Usually you can do better if you use embedded assembler for
link |
machine code for some of the details to utilize part of a computer that the compiler doesn't know
link |
about. But you should get to that point before you beat to the abstraction. So that's a beautiful
link |
ideal to reach for. And we meet it quite often. Quite often. So where's the magic of that coming
link |
from? There's some of it is the compilation process. So the implementation of C++, some of it
link |
is the design of the feature itself, the guidelines. So I think it's important that you
link |
think about the guidelines. So I've recently and often talked to Chris Latner, so Clang.
link |
What, just out of curiosity, is your relationship in general with the different implementations of
link |
C++ as you think about you and committee and other people in C++, think about the design of
link |
features or design of previous features. In trying to reach the ideal of zero overhead,
link |
does the magic come from the design, the guidelines, or from the implementations?
link |
And not all. You go for programming technique,
link |
programming language features, and implementation techniques. You need all three.
link |
And how can you think about all three at the same time?
link |
It takes some experience, takes some practice, and sometimes you get it wrong. But after a while,
link |
you sort of get it right. I don't write compilers anymore. But Brian Kernighan pointed out that one
link |
of the reasons C++ succeeded was some of the craftsmanship I put into the early compilers.
link |
And of course, I did the language assign. Of course, I wrote a fair amount of code using
link |
this kind of stuff. And I think most of the successes involve progress in all three areas
link |
together. A small group of people can do that. Two, three people can work together to do something
link |
like that. It's ideal if it's one person that has all the skills necessary. But nobody has all the
link |
skills necessary in all the fields where C++ is used. So if you want to approach my ideal in, say,
link |
concurrent programming, you need to know about algorithms from current programming. You need to
link |
know the trigger of lock free programming. You need to know something about compiler techniques.
link |
And then you have to know some of the application areas where this is, like some forms of graphics
link |
or some forms of what we call web server kind of stuff. And that's very hard to get into a single
link |
head. But small groups can do it too. So is there differences in your view, not saying which is
link |
better or so on, but differences in the different implementations of C++? Why are there several
link |
sort of maybe naive questions for me? GCC, clang, so on? This is a very reasonable question. When
link |
I designed C++, most languages had multiple implementations. Because if you run on an IBM,
link |
if you run on a Sun, if you run on a Motorola, there was just many, many companies and they each
link |
have their own compilation structure and their own compilers. It was just fairly common that
link |
there was many of them. And I wrote C Front assuming that other people would write compilers
link |
with C++ if successful. And furthermore, I wanted to utilize all the backend infrastructures that
link |
were available. I soon realized that my users were using 25 different linkers. I couldn't write my
link |
own linker. Yes, I could, but I couldn't write 25 linkers and also get any work done on the language.
link |
And so it came from a world where there was many linkers, many optimizers, many
link |
compiler front ends, not to start, but many operating systems. The whole world was not an
link |
86 and a Linux box or something, whatever is the standard today. In the old days, they set a VAX.
link |
So basically, I assumed there would be lots of compilers. It was not a decision that there should
link |
be many compilers. It was just a fact. That's the way the world is. And yes, many compilers
link |
emerged. And today, there's at least four front ends, Clang, GCC, Microsoft, and EDG,
link |
it is design group. They supply a lot of the independent organizations and the embedded
link |
systems industry. And there's lots and lots of backends. We have to think about how many dozen
link |
backends there are. Because different machines have different things, especially in the embedded
link |
world, the machines are very different, the architectures are very different. And so having
link |
a single implementation was never an option. Now, I also happen to dislike monocultures.
link |
They are dangerous. Because whoever owns the monoculture can go stale. And there's no
link |
competition. And there's no incentive to innovate. There's a lot of incentive to put barriers in the
link |
way of change. Because hey, we own the world. And it's a very comfortable world for us. And who are
link |
you to mess with that? So I really am very happy that there's four front ends for C++. Clang's
link |
great. But GCC was great. But then it got somewhat stale. Clang came along. And GCC is much better
link |
now. Microsoft is much better now. So at least a low number of front ends puts a lot of pressure on
link |
standards compliance and also on performance and error messages and compile time speed,
link |
all this good stuff that we want.
link |
Do you think, crazy question, there might come along, do you hope there might come along
link |
implementation of C++ written, given all its history, written from scratch?
link |
So written today from scratch?
link |
Well, Clang and the LLVM is more or less written from scratch.
link |
But there's been C++ 11, 14, 17, 20. You know, there's been a lot of
link |
I think sooner or later somebody's going to try again. There has been attempts to write
link |
new C++ compilers and some of them has been used and some of them has been absorbed into
link |
others and such. Yeah, it'll happen.
link |
So what are the key features of C++? And let's use that as a way to sort of talk about
link |
the evolution of C++, the new features. So at the highest level, what are the features
link |
that were there in the beginning? What features got added?
link |
Let's first get a principle or an aim in place. C++ is for people who want to use hardware
link |
really well and then manage the complexity of doing that through abstraction.
link |
And so the first facility you have is a way of manipulating the machines at a fairly low
link |
level. That looks very much like C. It has loops, it has variables, it has pointers like
link |
machine addresses, it can access memory directly, it can allocate stuff in the absolute minimum
link |
of space needed on the machine. There's a machine facing part of C++ which is roughly
link |
equivalent to C. I said C++ could beat C and it can. It doesn't mean I dislike C. If I
link |
disliked C, I wouldn't have built on it. Furthermore, after Dennis Ritchie, I'm probably the major
link |
contributor to modern C. I had lunch with Dennis most days for 16 years and we never
link |
had a harsh word between us. So these C versus C++ fights are for people who don't quite
link |
understand what's going on. Then the other part is the abstraction. The key is the class.
link |
There, the key is the class which is a user defined type. My idea for the class is that
link |
you should be able to build a type that's just like the building types in the way you
link |
use them, in the way you declare them, in the way you get the memory and you can do
link |
just as well. So in C++ there's an int as in C. You should be able to build an abstraction,
link |
a class which we can call capital int that you can use exactly like an integer and run
link |
just as fast as an integer. There's the idea right there. And of course you probably don't
link |
want to use the int itself but it has happened. People have wanted integers that were range
link |
checked so that you couldn't overflow and such, especially for very safety critical
link |
applications like the fuel injection for a marine diesel engine for the largest ships.
link |
This is a real example by the way. This has been done. They built themselves an integer
link |
that was just like integer except that couldn't overflow. If there was an overflow you went
link |
into the error handling. And then you built more interesting types. You can build a matrix
link |
which you need to do graphics or you could build a gnome for a video game.
link |
And all these are classes and they appear just like the built in types.
link |
In terms of efficiency and so on. So what else is there?
link |
So I don't know, for people who are not familiar with object oriented programming there's inheritance.
link |
There's a hierarchy of classes. You can just like you said create a generic vehicle that can turn
link |
So what people found was that you don't actually know. How do I say this? A lot of types are
link |
related. That is the vehicles, all vehicles are related. Bicycles, cars, fire engines, tanks. They
link |
have some things in common and some things that differ. And you would like to have the common
link |
things common and having the differences specific. And when you didn't want to know about
link |
the differences, just turn left. You don't have to worry about it. That's how you get the traditional
link |
object oriented programming coming out of Simula adopted by Smalltalk and C++ and all the other
link |
languages. The other kind of obvious similarity between types comes when you have something like
link |
a vector. Fortran gave us the vector as called array of doubles. But the minute you have a
link |
vector of doubles, you want a vector of double precision doubles and for short doubles for
link |
graphics. And why should you not have a vector of integers while you're added or a vector of
link |
vectors and a vector of vectors of chess pieces? Now you have a board, right? So this is you
link |
express the commonality as the idea of a vector and the variations come through parameterization.
link |
And so here we get the two fundamental ways of abstracting or of having similarities of
link |
types in C++. There's the inheritance and there's a parameterization. There's the object oriented
link |
programming and there's the generic programming. With the templates for the generic programming.
link |
Yep. So you've presented it very nicely, but now you have to make all that happen and make it
link |
efficient. So generic programming with templates, there's all kinds of magic going on, especially
link |
recently that you can help catch up on. But it feels to me like you can do way more than what
link |
you just said with templates. You can start doing this kind of metaprogramming, this kind of...
link |
You can do metaprogramming also. I didn't go there in that explanation. We're trying to be
link |
very basic, but go back on to the implementation. If you couldn't implement this efficiently,
link |
if you couldn't use it so that it became efficient, it has no place in C++ because
link |
it will violate the zero overhead principle. So when I had to get object oriented programming
link |
inheritance, I took the idea of virtual functions from Simula. Virtual functions is a Simula term,
link |
class is a Simula term. If you ever use those words, say thanks to Christen Nygaard and Olli
link |
Høndahl. And I did the simplest implementation I knew of, which was basically a jump table.
link |
So you get the virtual function table, the function goes in, does an indirection through
link |
a table and get the right function. That's how you pick the right thing there.
link |
And I thought that was trivial. It's close to optimal and it was obvious. It turned out the
link |
Simula had a more complicated way of doing it and therefore was slower. And it turns out that most
link |
languages have something that's a little bit more complicated, sometimes more flexible,
link |
but you pay for it. And one of the strengths of C++ was that you could actually do this object
link |
oriented stuff and your overhead compared to ordinary functions, there's no indirection. It's
link |
sort of in 5, 10, 25% just the call. It's down there. It's not two. And that means you can
link |
afford to use it. Furthermore, in C++, you have the distinction between a virtual function and
link |
a nonvirtual function. If you don't want any overhead, if you don't need the indirection that
link |
gives you the flexibility in object oriented programming, just don't ask for it. So the idea
link |
is that you only use virtual functions if you actually need the flexibility. So it's not zero
link |
overhead, but it's zero overhead compared to any other way of achieving the flexibility.
link |
Now, auto parameterization. Basically, the compiler looks at the template, say the vector,
link |
and it looks at the parameter, and then combines the two and generates a piece of code that is
link |
exactly as if you've written a vector of that specific type. So that's the minimal overhead.
link |
If you have many template parameters, you can actually combine code that the compiler couldn't
link |
usually see at the same time and therefore get code that is faster than if you had handwritten
link |
the stuff, unless you are very, very clever. So the thing is, parameterized code, the compiler
link |
fills stuff in during the compilation process, not during runtime. That's right. And furthermore,
link |
it gives all the information it's gotten, which is the template, the parameter, and the context
link |
of use. It combines the three and generates good code. But it can generate, now, it's a little
link |
outside of what I'm even comfortable thinking about, but it can generate a lot of code. Yes.
link |
And how do you, I remember being both amazed at the power of that idea, and
link |
how ugly the debugging looked? Yes. Debugging can be truly horrid.
link |
Come back to this, because I have a solution. Anyway, the debugging was ugly.
link |
The code generated by C++ has always been ugly, because there's these inherent optimizations.
link |
A modern C++ compiler has front end, middle end, and back end.
link |
Even C Front, back in 83, had front end and back end optimizations. I actually took the code,
link |
generated an internal representation, munched that representation to generate good code.
link |
So people say, it's not a compiler, it generates C. The reason it generated C was I wanted to use
link |
C's code generator, and I wanted to use C's code generator to generate good code.
link |
C was I wanted to use C's code generators that was really good at back end optimizations.
link |
But I needed front end optimizations, and therefore, the C I generated was optimized C.
link |
The way a really good handcrafted optimizer human could generate it, and it was not meant
link |
for humans. It was the output of a program, and it's much worse today. And with templates,
link |
it gets much worse still. So it's hard to combine simple debugging with the optimal code,
link |
because the idea is to drag in information from different parts of the code to generate good code,
link |
machine code. And that's not readable. So what people often do for debugging
link |
is they turn the optimizer off. And so you get code that when something in your source code
link |
looks like a function call, it is a function call. When the optimizer is turned on, it may disappear,
link |
the function call, it may inline. And so one of the things you can do is you can actually get code
link |
that is smaller than the function call, because you eliminate the function preamble and return.
link |
And there's just the operation there. One of the key things when I did
link |
templates was I wanted to make sure that if you have, say, a sort algorithm, and you give it a
link |
sorting criteria, if that sorting criteria is simply comparing things with less than,
link |
the code generated should be the less than, not an indirect function call to a comparison
link |
object, which is what it is in the source code. But we really want down to the single instruction.
link |
But anyway, turn off the optimizer, and you can debug. The first level of debugging can be done,
link |
and I always do without the optimization on, because then I can see what's going on.
link |
And then there's this idea of concepts that puts some, now I've never even,
link |
I don't know if it was ever available in any form, but it puts some constraints
link |
on the stuff you can parameterize, essentially.
link |
Let me try and explain this. So yes, it wasn't there 10 years ago. We have had versions of it
link |
that actually work for the last four or five years. It was a design by Gabi Dos Reis, Drew
link |
Sutton and me. We were professors and postdocs in Texas at the time. And the implementation by
link |
Andrew Sutton has been available for that time. And it is part of C++20. And there's a standard
link |
library that uses it. So this is becoming really very real. It's available in Clang and GCC. GCC
link |
for a couple of years, and I believe Microsoft is soon going to do it. We expect all of C++20
link |
to be available in all the major compilers in 20. But this kind of stuff is available now.
link |
I'm just saying that because otherwise people might think I was talking about science fiction.
link |
And so what I'm going to say is concrete. You can run it today.
link |
And there's production uses of it. So the basic idea is that when you have a generic component,
link |
like a sort function, the sort function will require at least two parameters. One is the
link |
data structure with a given type and a comparison criteria. And these things are related, but
link |
obviously you can't compare things if you don't know what the type of things you compare.
link |
And so you want to be able to say, I'm going to sort something and it is to be sortable.
link |
What does it mean to be sortable? You look it up in the standard. It has to have a
link |
it has to be a sequence with a beginning and an end. There has to be random access to that sequence.
link |
And there has to be the element types has to be comparable by default.
link |
Which means less than operator can operate on.
link |
Less than logical operator can operate.
link |
Basically what concepts are, they're compile time predicates. They're predicates you can ask,
link |
are you a sequence? Yes, I have a beginning and end. Are you a random access sequence? Yes,
link |
I have a subscripting and plus. Is your element type something that has a less than? Yes,
link |
I have a less than it's and so basically that's the system. And so instead of saying,
link |
I will take a parameter of any type, it'll say, I'll take something that's sortable.
link |
And it's well defined. And so we say, okay, you can sort with less than, I don't want less than,
link |
I want greater than or something I invent. So you have two parameters, the sortable thing and the
link |
comparison criteria. And the comparison criteria will say, well, I can, you can write it saying it
link |
should operate on the element type. And then you can say, well, I can sort with less than,
link |
and it has the comparison operations. So that's just simply the fundamental thing. It's compile
link |
time predicates. Do you have the properties I need? So it specifies the requirements of the code
link |
on the parameters that it gets. It's very similar to types actually. But operating in the space of
link |
concepts. Concepts. The word concept was used by Alex Stefanov, who is sort of the father of generic
link |
programming in the context of C++. There's other places that use that word, but the way we call
link |
it generic programming is Alex's. And he called them concepts because he said they are the sort
link |
of the fundamental concepts of an area. So they should be called concepts. And we've had
link |
concepts all the time. If you look at the KNR book about C, C has arithmetic types and it has
link |
integral types. It says so in the book. And then it lists what they are and they have certain
link |
properties. The difference today is that we can actually write a concept that will ask a type,
link |
are you an integral type? Do you have the properties necessary to be an integral type?
link |
Do you have plus, minus, divide and such? So maybe the story of concepts, because I thought
link |
it might be part of C++11. C O X or whatever it was at the time. What was the, why didn't it,
link |
what, like what we'll, we'll talk a little bit about this fascinating process of standards,
link |
because I think it's really interesting for people. It's interesting for me,
link |
but why did it take so long? What shapes did the idea of concepts take?
link |
What were the challenges? Back in 87 or thereabouts. 1987?
link |
Well, 1987 or thereabouts when I was designing templates, obviously I wanted to express the
link |
notion of what is required by a template of its arguments. And so I looked at this and basically
link |
for templates, I wanted three properties. I wanted to be very flexible. It had to be able to express
link |
things I couldn't imagine because I know I can't imagine everything. And I've been suffering from
link |
languages that try to constrain you to only do what the designer thought good. Didn't want to
link |
do that. Secondly, it had to run faster, as fast or faster than handwritten code. So basically,
link |
if I have a vector of T and I take a vector of char, it should run as fast as you built a vector
link |
of char yourself without parameterization. And thirdly, I wanted to be able to express
link |
the constraints of the arguments, have proper type checking of the interfaces.
link |
And neither I nor anybody else at the time knew how to get all three. And I thought for C++,
link |
I must have the two first. Otherwise, it's not C++. And it bothered me for another couple of
link |
decades that I couldn't solve the third one. I mean, I was the one that put function argument
link |
type checking into C. I know the value of good interfaces. I didn't invent that idea. It's very
link |
common, but I did it. And I wanted to do the same for templates, of course, and I couldn't.
link |
So it bothered me. Then we tried again, 2002, 2003. Gaby DesRays and I started analyzing the
link |
problem, explained possible solutions. It was not a complete design. A group in University of Indiana,
link |
an old friend of mine, they started a project at Indiana and we thought we could get
link |
a good system of concepts in another two or three years that would have made C++ 11 to C++
link |
06 or 07. Well, it turns out that I think we got a lot of the fundamental ideas wrong. They were
link |
too conventional. They didn't quite fit C++ in my opinion. Didn't serve implicit conversions very
link |
well. It didn't serve mixed type arithmetic, mixed type computations very well. A lot of
link |
stuff came out of the functional community and that community didn't deal with multiple types
link |
in the same way as C++ does, had more constraints on what you could express and didn't have the
link |
draconian performance requirements. And basically we tried. We tried very hard. We had some
link |
successes, but it just in the end wasn't, didn't compile fast enough, was too hard to use and
link |
didn't run fast enough unless you had optimizers that was beyond the state of the art. They still
link |
are. So we had to do something else. Basically it was the idea that a set of parameters has
link |
defined a set of operations and you go through an interaction table just like for virtual functions
link |
and then you try to optimize the interaction away to get performance. And we just couldn't
link |
do all of that. But get back to the standardization. We are standardizing C++ under ISO rules,
link |
which are very open process. People come in, there's no requirements for education or experience.
link |
So you started to develop C++ and there's a whole, when was the first standard established? What is
link |
that like? The ISO standard, is there a committee that you're referring to? There's a group of
link |
people. What was that like? How often do you meet? What's the discussion?
link |
I'll try and explain that. So sometime in early 1989, two people, one from IBM, one from HP,
link |
turned up in my office and told me I would like to standardize C++. This was a new idea to me and
link |
when I pointed out that it wasn't finished yet and it wasn't ready for formal standardization
link |
and such. And they say, no, Bjarne, you haven't gotten it. You really want to do this.
link |
Our organizations depend on C++. We cannot depend on something that's owned by another
link |
corporation that might be a competitor. Of course we could rely on you, but you might get run over
link |
by a boss. We really need to get this out in the open. It has to be standardized under formal rules
link |
and we are going to standardize it under ISO rules and you really want to be part of it because
link |
basically otherwise we'll do it ourselves. And we know you can do it better. So through a combination
link |
of arm twisting and flattery, it got started. So in late 89, there was a meeting in DC at the,
link |
actually no, it was not ISO then, it was ANSI, the American National Standard doing.
link |
We met there. We were lectured on the rules of how to do an ANSI standard. There was about 25 of us
link |
there, which apparently was a new record for that kind of meeting. And some of the old C guys that
link |
has been standardized in C was there. So we got some expertise in. So the way this works is that
link |
it's an open process. Anybody can sign up if they pay the minimal fee, which is about a thousand
link |
dollars, less than a little bit more now. And I think it's $1,280. It's not going to kill you.
link |
And we have three meetings a year. This is fairly standard. We tried two meetings a year for a
link |
couple of years that didn't work too well. So three one week meetings a year and you meet
link |
and you have technical discussions, and then you bring proposals forward for votes. The votes are
link |
done one person per, one vote per organization. So you can't have say IBM come in with 10 people
link |
and dominate things that's not allowed. And these are organizations that extensively UC
link |
plus plus. Yes. Or individuals or individuals. I mean, it's a bunch of people in the room
link |
deciding the design of a language based on which a lot of the world's systems run.
link |
Right. Well, I think most people would agree it's better than if I decided it
link |
or better than if a single organization like AG&T decides it. I don't know if everyone agrees to
link |
that, by the way. Bureaucracies have their critics too. Yes. Look, standardization is not pleasant.
link |
It's horrifying. It's like democracy. Exactly. As Churchill says, democracy is the worst way,
link |
except for the others. Right. And it's, I would say the same with formal standardization.
link |
But anyway, so we meet and we have these votes and that determines what the standard is.
link |
A couple of years later, we extended this so it became worldwide. We have standard organizations
link |
that are active in currently 15 to 20 countries and another 15 to 20 are sort of looking and voting
link |
based on the rest of the work on it. And we meet three times a year. Next week I'll be in Cologne,
link |
Germany, spending a week doing standardization and we'll vote out the committee draft of C++20,
link |
which goes to the national standards committees for comments and requests for changes and
link |
improvements. Then we do that and there's a second set of votes where hopefully everybody
link |
votes in favor. This has happened several times. The first time we finished, we started in the
link |
first technical meeting was in 1990. The last was in 98. We voted it out. That was the standard
link |
that people used until 11 or a little bit past 11. And it was an international standard. All the
link |
countries voted in favor. It took longer with 11. I'll mention why, but all the nations voted in
link |
favor. And we work on the basis of consensus. That is, we do not want something that passes 6040
link |
because then we're going to get dialects and opponents and people complain too much. They
link |
all complain too much, but basically it has no real effect. The standards has been obeyed. They
link |
have been working to make it easier to use many compilers, many computers and all of that kind of
link |
stuff. It was traditional with ISO standards to take 10 years. We did the first one in eight,
link |
brilliant. And we thought we were going to do the next one in six because now we are good at it.
link |
Right. It took 13. Yeah. It was named OX. It was named OX. Hoping that you would at least get it
link |
within the single, within the odds, the single digits. I thought we would get, I thought we'd
link |
get six, seven or eight. The confidence of youth. That's right. Well, the point is that this was
link |
sort of like a second system effect. That is, we now knew how to do it. And so we're going to do
link |
it much better. And we've got more ambitious and it took longer. Furthermore, there is this tendency
link |
because it's a 10 year cycle or it doesn't matter. Just before you're about to ship,
link |
somebody has a bright idea. And so we really, really must get that in. We did that successfully
link |
with the STL. We got the standard library that gives us all the STL stuff. That basically,
link |
I think it saved C++. It was beautiful. And then people tried it with other things
link |
and it didn't work so well. They got things in, but it wasn't as dramatic and it took longer and
link |
longer and longer. So after C++ 11, which was a huge improvement and what, basically what most
link |
people are using today, we decided never again. And so how do you avoid those slips? And the
link |
answer is that you ship more often. So that if you have a slip on a 10 year cycle, by the time
link |
you know it's a slip, there's 11 years till you get it. Now with a three year cycle, there is
link |
about three or four years till you get it. Like the delay between feature freeze and shipping. So
link |
you always get one or two years more. And so we shipped 14 on time, we shipped 17 on time,
link |
and we ship, we will ship 20 on time. It'll happen. And furthermore, this gives a predictability
link |
that allows the implementers, the compiler implementers, the library implementers,
link |
they have a target and they deliver on it. 11 took two years before most compilers were good
link |
enough. 14, most compilers were actually getting pretty good in 14. 17, everybody shipped in 17.
link |
We are going to have at least almost everybody ship almost everything in 20. And I know this
link |
and I know this because they're shipping in 19. Predictability is good. Delivery on time is good.
link |
And so yeah. That's great. That's how it works.
link |
There's a lot of features that came in in C++ 11. There's a lot of features at the birth of C++
link |
that were amazing and ideas with concepts in 2020. What to you is the most,
link |
just to you personally, beautiful or just you sit back and think, wow, that's just nice and clean
link |
feature of C++? I have written two papers for the History of Programming Languages Conference,
link |
which basically asked me such questions. And I'm writing a third one, which I will deliver
link |
at the History of Programming Languages Conference in London next year. So I've been thinking about
link |
that. And there is one clear answer. Constructors and destructors. The way a constructor can
link |
establish the environment for the use of a type for an object and the destructor that cleans up
link |
any messes at the end of it. That is key to C++. That's why we don't have to use garbage
link |
collection. That's how we can get predictable performance. That's how you can get the minimal
link |
overhead in many, many cases, and have really clean types. It's the idea of constructor destructor
link |
pairs. Sometimes it comes out under the name RAII. Resource acquisition is initialization,
link |
which is the idea that you grab resources in the constructor and release them in destructor.
link |
It's also the best example of why I shouldn't be in advertising. I get the best idea and I call it
link |
resource acquisition is initialization. Not the greatest naming I've ever heard.
link |
Not the greatest naming I've ever heard. So it's types, abstraction of types.
link |
You said, I want to create my own types. So types is an essential part of C++ and making them
link |
efficient is the key part. And to you, this is almost getting philosophical, but the construction
link |
and the destruction, the creation of an instance of a type and the freeing of resources from that
link |
instance of a type is what defines the object. It's almost like birth and death is what defines
link |
human life. That's right. By the way, philosophy is important. You can't do good language design
link |
without philosophy because what you are determining is what people can express and how.
link |
This is very important. By the way, constructors destructors came into C++ in 79 in about the
link |
second week of my work with what was then called C of the classes. It is a fundamental idea.
link |
Next comes the fact that you need to control copying because once you control, as you said,
link |
birth and death, you have to control taking copies, which is another way of creating an object.
link |
And finally, you have to be able to move things around so you get the move operations. And that's
link |
the set of key operations you can define on a C++ type. And so to you, those things are just
link |
just a beautiful part of C++ that is at the core of it all. Yes. You mentioned that you hope there
link |
will be one unified set of guidelines in the future for how to construct a programming language.
link |
So perhaps not one programming language, but a unification of how we build programming languages,
link |
if you remember such statements. I have some trouble remembering it, but I know the origin
link |
of that idea. So maybe you can talk about sort of C++ has been improving. There's been a lot
link |
of programming language. Do you, where does the arc of history taking us? Do you hope that there
link |
is a unification about the languages with which we communicate in the digital space?
link |
Well, I think that languages should be designed not by clobbering language features together and
link |
and doing slightly different versions of somebody else's ideas, but through the creation of a set of
link |
principles, rules of thumbs, whatever you call them. I made them for C++. And we're trying to
link |
teach people in the standards committee about these rules, because a lot of people come in
link |
and says, I've got a great idea. Let's put it in the language. And then you have to ask, why does
link |
it fit in the language? Why does it fit in this language? It may fit in another language and not
link |
here, or it may fit here and not the other language. So you have to work from a set of
link |
principles and you have to develop that set of principles. And one example that I sometimes
link |
remember is I was sitting down with some of the designers of Common Lisp and we were talking about
link |
languages and language features. And obviously we didn't agree about anything because, well,
link |
Lisp is not C++ and vice versa. It's too many parentheses. But suddenly we started making
link |
progress. I said, I had this problem and I developed it according to these ideas. And
link |
they said, why? We had that problem, different problem, and we developed it with the same kind
link |
of principles. And so we worked through large chunks of C++ and large chunks of Common Lisp
link |
and figured out we actually had similar sets of principles of how to do it. But the constraints
link |
on our designs were very different and the aims for the usage was very different. But there was
link |
commonality in the way you reason about language features and the fundamental principles you are
link |
trying to do. So do you think that's possible? So there, just like there is perhaps a unified
link |
theory of physics, of the fundamental forces of physics, that I'm sure there is commonalities
link |
among the languages, but there's also people involved that help drive the development of these
link |
languages. Do you have a hope or an optimism that there will be a unification? If you think about
link |
physics and Einstein towards a simplified language, do you think that's possible?
link |
Let's remember sort of modern physics, I think, started with Galileo in the 1300s. So they've had
link |
700 years to get going. Modern computing started in about 49. We've got, what is it, 70 years. They
link |
have 10 times. Furthermore, they are not as bothered with people using physics the way
link |
we are worried about programming is done by humans. So each have problems and constraints
link |
the others have, but we are very immature compared to physics. So I would look at sort of the
link |
philosophical level and look for fundamental principles. Like you don't leak resources,
link |
you shouldn't. You don't take errors at runtime that you don't need to. You don't violate some
link |
kind of type system. There's many kinds of type systems, but when you have one, you don't break it,
link |
etc., etc. There will be quite a few, and it will not be the same for all languages. But I think
link |
if we step back at some kind of philosophical level, we would be able to agree on sets of
link |
principles that applied to sets of problem areas. And within an area of use, like in C++'s case,
link |
what used to be called systems programming, the area between the hardware and the fluffier parts
link |
of the system, you might very well see a convergence. So these days you see Rust having
link |
adopted RAII and sometimes accuse me for having borrowed it 20 years before they discovered it.
link |
But we're seeing some kind of convergence here instead of relying on garbage collection all the
link |
time. The garbage collection languages are doing things like the dispose patterns and such that
link |
imitate some of the construction destruction stuff. And they're trying not to use the garbage
link |
collection all the time and things like that. So there's a conversion. But I think we have to step
link |
back to the philosophical level, agree on principles, and then we'll see some conversions,
link |
convergences. And it will be application domain specific.
link |
So a crazy question, but I work a lot with machine learning, with deep learning. I'm not sure if you
link |
touch that world that much, but you could think of programming as a thing that takes some input.
link |
A programming is the task of creating a program and a program takes some input and produces some
link |
output. So machine learning systems train on data in order to be able to take an input and produce
link |
output. But they're messy, fuzzy things, much like we as children grow up. We take some input,
link |
we make some output, but we're noisy. We mess up a lot. We're definitely not reliable. Biological
link |
system are a giant mess. So there's a sense in which machine learning is a kind of way of
link |
programming, but just fuzzy. It's very, very, very different than C++. Because C++ is just like you
link |
said, it's extremely reliable, it's efficient, you can measure it, you can test it in a bunch of
link |
different ways. With biological systems or machine learning systems, you can't say much except sort
link |
of empirically saying that 99.8% of the time, it seems to work. What do you think about this fuzzy
link |
kind of programming? Do you even see it as programming? Is it totally another kind of world?
link |
I think it's a different kind of world. And it is fuzzy. And in my domain, I don't like fuzziness.
link |
That is, people say things like they want everybody to be able to program. But I don't
link |
want everybody to program my airplane controls or the car controls. I want that to be done by
link |
engineers. I want that to be done with people that are specifically educated and trained for doing
link |
building things. And it is not for everybody. Similarly, a language like C++ is not for
link |
everybody. It is generated to be a sharp and effective tool for professionals, basically,
link |
and definitely for people who aim at some kind of precision. You don't have people doing
link |
calculations without understanding math. Counting on your fingers is not going to cut it if you want
link |
to fly to the moon. And so there are areas where an 84% accuracy rate, 16% false positive rate,
link |
is perfectly acceptable and where people will probably get no more than 70. You said 98%. What
link |
I have seen is more like 84. And by really a lot of blood, sweat, and tears, you can get up to 92.5.
link |
So this is fine if it is, say, prescreening stuff before the human look at it. It is not good enough
link |
for life threatening situations. And so there's lots of areas where the fuzziness is perfectly
link |
acceptable and good and better than humans, cheaper than humans, cheaper than humans.
link |
But it's not the kind of engineering stuff I'm mostly interested in. I worry a bit about
link |
machine learning in the context of cars. You know much more about this than I do.
link |
But I'm sort of an amateur here. I've read some of the papers, but I've not ever done it. And the
link |
idea that scares me the most is the one I have heard, and I don't know how common it is, that
link |
you have this AI system, machine learning, all of these trained neural nets. And when there's
link |
something that's too complicated, they ask the human for help. But the human is reading a book or
link |
asleep, and he has 30 seconds or three seconds to figure out what the problem was that the AI
link |
system couldn't handle and do the right thing. This is scary. I mean, how do you do the cutting
link |
work between the machine and the human? It's very, very difficult. And for the designer of
link |
one of the most reliable, efficient, and powerful programming languages, C++, I can understand why
link |
that world is actually unappealing. It is for most engineers. To me, it's extremely
link |
appealing because we don't know how to get that interaction right. But I think it's possible. But
link |
it's very, very hard. It is. And I was stating a problem, not a solution. That is impossible.
link |
I mean, I would much rather never rely on the human. If you're driving a nuclear reactor,
link |
if you're or an autonomous vehicle, it's much better to design systems written in C++ than
link |
never ask human for help. Let's just get one fact in. Yeah. All of this AI stuff is on top of C++.
link |
So that's one reason I have to keep a weather eye out on what's going on in that field. But
link |
I will never become an expert in that area. But it's a good example of how you separate
link |
different areas of applications and you have to have different tools, different principles. And
link |
then they interact. No major system today is written in one language. And there are good
link |
reasons for that. When you look back at your life work, what is a moment? What is a
link |
event creation that you're really proud of? They say, damn, I did pretty good there.
link |
Is it as obvious as the creation of C++? It's obvious. I've spent a lot of time with C++. And
link |
there's a combination of a few good ideas, a lot of hard work, and a bit of work that I've done.
link |
And I've tried to get away from it a few times, but I get dragged in again, partly because I'm
link |
most effective in this area and partly because what I do has much more impact if I do it in
link |
the context of C++. I have four and a half million people that pick it up tomorrow if I
link |
get something right. If I did it in another field, I would have to start learning, then I have to
link |
build it and then we'll see if anybody wants to use it. One of the things that has kept me going
link |
for all of these years is one, the good things that people do with it and the interesting things
link |
they do with it. And also, I get to see a lot of interesting stuff and talk to a lot of interesting
link |
people. I mean, if it has just been statements on paper on a screen, I don't think I could have kept
link |
going. But I get to see the telescopes up on Mauna Kea and I actually went and see how Ford built
link |
cars and I got to JPL and see how they do the Mars rovers. There's so much cool stuff going on. And
link |
most of the cool stuff is done by pretty nice people and sometimes in very nice places.
link |
Cambridge, Sophia, Silicon Valley. There's more to it than just code. But code is central.
link |
On top of the code are the people in very nice places. Well, I think I speak for millions of
link |
people, Yaron, in saying thank you for creating this language that so many systems are built on
link |
top of that make a better world. So thank you and thank you for talking today. Yeah, thanks.
link |
And we'll make it even better. Good.