back to indexBjarne Stroustrup: C++ | Lex Fridman Podcast #48
link |
The following is a conversation with Bjarn Strelstrom. He's a creator of C++, a programming
link |
language that after 40 years is still one of the most popular and powerful languages in the world.
link |
Its focus on fast, stable, robust code underlies many of the biggest systems in the world
link |
that we have come to rely on as a society. If you're watching this on YouTube, for example,
link |
many of the critical backend components of YouTube are written in C++. Same goes for Google,
link |
Facebook, Amazon, Twitter, most Microsoft applications, Adobe applications, most database
link |
systems and most physical systems that operate in the real world like cars, robots, rockets that
link |
launch us into space and one day will land us on Mars. C++ also happens to be the language
link |
that I use more than any other in my life. I've written several hundred thousand lines of C++
link |
source code. Of course, lines of source code don't mean much, but they do give hints of my
link |
personal journey through the world of software. I've enjoyed watching the development of C++
link |
as a programming language leading up to the big update in the standard in 2011 and those that
link |
followed in 14, 17 and toward the new C++20 standard hopefully coming out next year.
link |
This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, give it
link |
five stars on iTunes, support it on Patreon, or simply connect with me on Twitter at Lex
link |
Friedman, spelled F R I D M A N. And now here's my conversation with Bjorn Straussstroke.
link |
What was the first program you've ever written? Do you remember?
link |
It was my second year in university, first year of computer science and it was an Algor 60.
link |
I calculated the shape of a super ellipse and then connected points on the perimeter,
link |
creating star patterns. It was with a wedding on a paper printer.
link |
And that was in college, university? Yeah. I learned to program the second year in university.
link |
And what was the first programming language, if I may ask it this way, that you fell in love with?
link |
I think Algor 60 and after that I remember I remember Snowball. I remember Fortran didn't
link |
fall in love with that. I remember Pascal didn't fall in love with that. It already got in the
link |
way of me. And then I discovered Assembler and that was much more fun. And from there I went to
link |
microcode. So you were drawn to the, you found the low level stuff beautiful?
link |
I went through a lot of languages and then I spent significant time in Assembler and microcode.
link |
That was sort of the first really profitable things I paid for my masters actually.
link |
And then I discovered Simula, which was absolutely great.
link |
Simula? Simula was the extension of Algor 60 done primarily for simulation, but basically
link |
they invented object oriented programming at inheritance and runtime polymorphism while
link |
they were doing it. And that was a language that taught me that you could have the sort of the
link |
problems of a program grow with the size of the program rather than with the square of the size
link |
of the program. That is, you can actually modularize very nicely. And that was a surprise to me.
link |
It was also a surprise to me that a stricter type system than Pascal's was helpful, whereas
link |
Pascal's type system got in my way all the time. So you need a strong type system to organize your
link |
code well, but it has to be extensible and flexible. Let's get into the details a little bit. What
link |
if you remember what kind of type system did Pascal have? What type system typing system did Algor
link |
60 have? Basically, Pascal was sort of the simplest language that Nicholas could define
link |
that served the needs of Nicholas. We had at the time and it has a sort of a highly
link |
moral tone to it. That is, if you can say it in Pascal, it's good. And if you can't, it's not so good.
link |
Whereas, Simula, allowed you basically to build your own type system. So instead of trying to fit
link |
yourself into Nicholas Vietz's world, Christen Nugol's language and Oliwann Dahl's language
link |
allowed you to build your own. So it's sort of close to the original idea of you build a domain
link |
specific language. As a matter of fact, what you build is a set of types and relations among types
link |
that allows you to express something that's suitable for an application.
link |
So when you say types, stuff you're saying has echoes of object during a programming.
link |
Yes, they invented it. Every language that uses the word class for type is a descent of Simula,
link |
directly or indirectly. Christen Nugol and Oliwann Dahl were mathematicians and they didn't think in
link |
terms of types, but they understood sets and classes of elements and so they called their
link |
types classes. And basically in C++, as in Simula, classes are user defined type.
link |
So can you try the impossible task and give a brief history of programming languages from
link |
your perspective? So we started with Algal 60, Simula, Pascal, but that's just the 60s and 70s.
link |
I can try. The most sort of interesting and major improvement of programming languages was
link |
Fortran, the first Fortran. Because before that, all code was written for a specific machine and
link |
each specific machine had a language, a simply language or a core Simula or some extension
link |
of that idea, but you are writing for a specific machine in the language of that machine.
link |
And Barker and his team at IBM built a language that would allow you to write what you really
link |
wanted. That is, you could write it in a language that was natural for people. Now these people
link |
happened to be engineers and physicists, so the language that came out was somewhat unusual for
link |
the rest of the world. But basically they said formula translation because they wanted to have
link |
the mathematical formulas translated into the machine. And as a side effect, they got portability
link |
because now they are writing in the terms that the humans used and the way humans thought.
link |
And then they had a program that translated it into the machine's needs. And that was new and
link |
that was great. And it's something to remember. We want to raise the language to the human level,
link |
but we don't want to lose the efficiency. And that was the first step towards the human?
link |
That was the first step. And of course, there were very particular kinds of humans,
link |
business people who were different, so they got cobalt instead and et cetera, et cetera.
link |
And Simula came out. No, let's not go to Simula yet. Let's go to Algor. Fortran didn't have,
link |
at the time, the notions of not a precise notion of type, not a precise notion of scope,
link |
not a set of translation phases that was what we have today, lexical, syntax, semantics. It was
link |
sort of a bit of a model in the early days, but hey, they had just done the biggest breakthrough
link |
in the history of programming. So you can't criticize them for not having gotten all the
link |
technical details right. So we got Algor. That was very pretty. And most people in commerce
link |
and science considered it useless because it was not flexible enough and it wasn't efficient enough
link |
and et cetera, et cetera. But that was the breakthrough from the technical point of view.
link |
And then Simula came along to make that idea more flexible and you could define your own types. And
link |
that's where I got very interested. Preston Nygo, who's the main idea man behind Simula.
link |
That was late 60s.
link |
This was late 60s. Well, I was a visiting professor in Aarhus. And so I learned object
link |
oriented programming by sitting around and well, in theory, discussing with
link |
Kaisen Nygo. But Kaisen, once you get started and in full flow, it's very hard to get a word in
link |
edge ways with you just listen. So it was great. I learned it from there.
link |
Not to romanticize the notion, but it seems like a big leap to think about object oriented programming.
link |
It's really a leap of abstraction. Yes.
link |
And was that as big and beautiful of a leap as it seems from now in retrospect,
link |
or was it an obvious one at the time?
link |
It was not obvious. And many people have tried to do something like that. And most people didn't
link |
come up with something as wonderful as Simula. Lots of people got their PhDs and made their
link |
careers out of forgetting about Simula or never knowing it. For me, the key idea was basically
link |
I could get my own types. And that's the idea that goes further into C++, where I can get
link |
better types and more flexible types and more efficient types. But it's still the
link |
fundamental idea when I want to write a program, I want to write it with my types.
link |
That is appropriate to my problem and under the constraints that I'm under with hardware,
link |
software, environment, etc. And that's the key idea. People picked up on the class hierarchies
link |
and the virtual functions and the inheritance. And that was only part of it. It was an
link |
interesting and major part and still a major part and a lot of graphic stuff.
link |
But it was not the most fundamental. It was when you wanted to relate one type to another,
link |
you don't want them all to be independent. The classical example is that you don't
link |
actually want to write a city simulation with vehicles where you say, well, if it's a bicycle,
link |
write the code for turning a bicycle to the left, if it's a normal car, turn right the normal car
link |
way, if it's a fire engine, turn right the fire engine, you get these big case statements and
link |
bunches of if statements and such. Instead, you tell the base class that that's the vehicle and say,
link |
turn left the way you want to. And this is actually a real example. They used it to simulate
link |
and optimize the emergency services for somewhere in Norway back in the 60s.
link |
Wow. So this was one of the early examples for why you needed inheritance and you needed
link |
runtime polymorphism because you wanted to handle this set of vehicles in a manageable way.
link |
You can't just rewrite your code each time a new kind of vehicle comes along.
link |
Yeah, that's a beautiful, powerful idea. And of course, it is stretched through your work
link |
with C++ as we'll talk about. But I think you structured nicely what other breakthroughs came
link |
along in the history of programming languages. If we were to tell the history in that way.
link |
Obviously, I'm better telling the part of the history that that is the part that I'm on as
link |
opposed to all the parts. Yeah, you skipped the hippie John McCarthy and Lisb, one of my favorite
link |
languages. But Lisb is not one of my favorite languages. It's obviously important. It's obviously
link |
interesting. Lots of people write code in it. And then they rewrite it into CSE++ when they want to
link |
go to production. It's in the world I'm at, which are constrained by performance, reliability,
link |
issues, deployability, cost of hardware. I don't like things to be too dynamic.
link |
It is really hard to write a piece of code that's perfectly flexible
link |
that you can also deploy on a small computer. And that you can also put in, say, a telephone switch
link |
in Bogota. What's the chance if you get an error and you find yourself in the debugger
link |
that the telephone switch in Bogota on late Sunday night has a programmer around?
link |
Right. The chance is zero. And so a lot of things I think most about
link |
most about can't afford that flexibility. I'm quite aware that maybe 70, 80% of all code
link |
are not under the kind of constraints I'm interested in. But somebody has to do the job I'm doing
link |
because you have to get from these high level flexible languages to the hardware.
link |
The stuff that lasts for 10, 20, 30 years is robust, operates under very constrained conditions.
link |
Yes, absolutely. That's right. And it's fascinating and beautiful in its own way.
link |
It's C++ is one of my favorite languages and so is Lisp. So I can embody it too
link |
for different reasons as a programmer. I understand why Lisp is popular and I can see
link |
the beauty of the ideas and similarly with Smalltalk. It's just not as relevant in my world.
link |
And by the way, I distinguish between those in the functional languages
link |
where I go to things like ML and Haskell. Different kind of languages. They have a
link |
different kind of beauty and they're very interesting. And I actually try to learn from
link |
all the languages I encounter to see what is there that would make
link |
working on the kind of problems I'm interested in with the kind of constraints
link |
that I'm interested in. What can actually be done better because we can surely do better than we do
link |
today. You've said that it's good for any professional programmer to know at least
link |
five languages. Speaking about a variety of languages that you've taken inspiration from
link |
and you've listed yours as being at least at the time C++ obviously, Java, Python, Ruby and JavaScript.
link |
Can you first of all update that list? Modify it. You don't have to be constrained
link |
to just five. But can you describe what you picked up also from each of these languages?
link |
How you see them as inspirations for even when you're working with C++?
link |
This is a very hard question to answer. So about languages, you should know languages.
link |
I reckon I knew about 25 or thereabouts when I did C++. It was easier in those days because
link |
the languages were smaller and you didn't have to learn a whole programming environment and such
link |
to do it. You could learn the language quite easily and it's good to learn so many languages.
link |
I imagine just like with natural language for communication, there's different paradigms that
link |
emerge in all of them. That there's commonalities and so on.
link |
So I picked five out of a hat. Obviously. The important thing that the number is not one.
link |
That's right. I don't like, I mean, if you're a monoglot, you are likely to think that your
link |
own culture is the only one's period for everybody else's. A good learning of a foreign language
link |
and a foreign culture is important. It helps you think and be a better person. With programming
link |
languages, you become a better programmer, a better designer with the second language.
link |
Now, once you've got two, the way to five is not that long. It's the second one that's most
link |
important. And then when I had to pick five, I sort of thinking what kinds of languages are there.
link |
Well, there's a really low level stuff. It's good. It's actually good to know machine code.
link |
Even still, sorry. Even today. Even today. The C++ optimizes right better machine code than I do.
link |
Yes. But I don't think I could appreciate them if I actually didn't understand machine code and
link |
machine architecture. At least in my position, I have to understand a bit of it. Because
link |
you mess up the cash and you're off in performance by a factor of 100. Right? It shouldn't be that
link |
if you are interested in either performance or the size of the computer you have to deploy.
link |
Yeah. So I would go, that's a simpler. I used to mention C, but these days going low level
link |
is not actually what gives you the performance. It is to express your ideas so cleanly that you
link |
can think about it and the optimizer can understand what you're up to. My favorite way of optimizing
link |
these days is to throw out the clever bits and see if it still runs fast. And sometimes it runs
link |
faster. So I need the abstraction mechanisms or something like C++ to write compact high
link |
performance code. There was a beautiful keynote by Jason Turner at the CPP Con a couple of years ago
link |
where he decided he was going to program Pong on Motorola 6800, I think it was. And he says,
link |
well this is relevant because it looks like a microcontroller. It has specialized hardware,
link |
it has not very much memory and it's relatively slow. And so he shows in real time how he writes
link |
Pong starting with fairly straightforward low level stuff, improving his abstractions
link |
and what he's doing, he's writing C++ and it translates into 86 assembler which you can do
link |
with Clang and you can see it in real time. It's the compiler explorer which you can use on the web
link |
and then he wrote a little program that translated 86 assembler into Motorola assembler. And so he
link |
types and you can see this thing in real time. You can see it in real time and even if you can't
link |
read the assembly code, you can just see it, his code gets better, the code, the assembler gets
link |
smaller, he increases the abstraction level, uses C++ 11 as it were better, this code gets cleaner,
link |
it gets easier to maintain what the code shrinks and it keeps shrinking. And I could not in any
link |
reasonable amount of time write that assembler as good as the compiler generated from really
link |
quite nice modern C++. And I'll go as far as to say that the thing that looked like C was
link |
significantly uglier and smaller when it became and larger when it became machine code.
link |
So the abstractions that can be optimized are important. I would love to see that kind of
link |
visualization in larger code bases. That might be beautiful. But you can't show a larger code base
link |
in a one hour talk and have it fit on screen. Right. So that's C and C++. So my two languages
link |
would be machine code and C++. And then I think you can learn a lot from the functional languages.
link |
So PIG has Goliom L. I don't care which. I think actually you learn the same lessons
link |
of expressing especially mathematical notions really clearly and having a type system that's
link |
really strict. And then you should probably have a language for sort of quickly churning out
link |
something. You could pick JavaScript, you could pick Python, you could pick Ruby.
link |
What do you make of JavaScript in general? So you're talking in the platonic sense about
link |
languages, about what they're good at, what their philosophy of design is. But there's also a large
link |
user base behind each of these languages and they use it in the way sometimes maybe it wasn't
link |
really designed for. That's right. JavaScript is used way beyond probably what it was designed for.
link |
Let me say it this way. When you build a tool, you do not know how it's going to be used.
link |
You try to improve the tool by looking at how it's being used and when people cut their fingers off
link |
and try and stop that from happening. But really you have no control over how something is used.
link |
So I'm very happy and proud of some of the things C++ is being used at and some of the things I wish
link |
people wouldn't do. Bitcoin mining being my favorite example uses as much energy as Switzerland
link |
and mostly serves criminals. But back to the languages, I actually think that having JavaScript
link |
run in the browser was an enabling thing for a lot of things. Yes, you could have done it better,
link |
but people were trying to do it better and they were using more principles, language designs,
link |
but they just couldn't do it right. And the nonprofessional programmers that write lots of
link |
that code just couldn't understand them. So it did an amazing job for what it was. It's not
link |
the prettiest language and I don't think it ever will be the prettiest language, but that's not
link |
the biggest here. So what was the origin story of C++? You basically gave a few perspectives of
link |
your inspiration of object oriented programming. You had a connection with C in performance,
link |
efficiency was an important thing you were drawn to. Efficiency and reliability. You have to get
link |
both. What's reliability? I really want my telephone calls to get through and I want the
link |
quality of what I am talking coming out at the other end. The other end might be in London or
link |
wherever. And you don't want the system to be crashing. If you're doing a bank,
link |
you mustn't crash. It might be your bank account that is in trouble. There's different constraints
link |
like in games, it doesn't matter too much if there's a crash, nobody dies and nobody gets ruined.
link |
But I'm interested in the combination of performance partly because of sort of speed
link |
of things being done, part of being able to do things that is necessary to have reliable
link |
reliability of larger systems. If you spend all your time interpreting a simple function call,
link |
you are not going to have enough time to do proper signal processing to get the telephone
link |
calls to sound right. Either that or you have to have 10 times as many computers and you can't
link |
afford your phone anymore. It's a ridiculous idea in the modern world because we've solved all of
link |
those problems. I mean they keep popping up in different ways because we tackle bigger and
link |
bigger problems so efficiency remains always an important aspect. But you have to think about
link |
efficiency not just as speed but as an enabler to important things. And one of the things it
link |
enables is reliability, is dependability. When I press the brake pedal of a car, it is not actually
link |
connected directly to anything but a computer. That computer better work.
link |
Let's talk about reliability just a little bit. So modern cars have ECUs, have millions of lines
link |
of code today. So this is certainly especially true of autonomous vehicles where some of the
link |
aspect of the control or driver assistance systems that steer the car, keeping the lanes on.
link |
So how do you think, you know, I talked to regulators, people in the government who are
link |
very nervous about testing the safety of these systems of software, ultimately software that
link |
makes decisions that could lead to fatalities. So how do we test software systems like these?
link |
First of all, safety, like performance and like security is a systems property. People tend to
link |
look at one part of a system at a time and saying something like, this is secure. That's all right.
link |
I don't need to do that. Yeah, that piece of code is secure. I'll buy your operator.
link |
If you want to have reliability, if you want to have performance, if you want to have security,
link |
you have to look at the whole system. I did not expect you to say that, but that's very true.
link |
Yes. I'm dealing with one part of the system and I want my part to be really good,
link |
but I know it's not the whole system. Furthermore, making an individual part perfect
link |
may actually not be the best way of getting the highest degree of reliability and performance and
link |
such. Those people say C++ type safe, not type safe, you can break it. Sure. I can break anything
link |
that runs on a computer. I may not go through your type system. If I wanted to break into your
link |
computer, I'll probably try SQL injection. It's very true. If you think about safety or even
link |
reliability at a system level, especially when a human being is involved, it starts becoming
link |
hopeless pretty quickly in terms of proving that something is safe to a certain level,
link |
because there are so many variables. It's so complex.
link |
Well, let's get back to something we can talk about and actually make some progress on.
link |
We can look at C++ programs and we can try and make sure they crash less often.
link |
The way you do that is largely by simplification. The first step is to simplify the code, have less
link |
code, have code that are less likely to go wrong. It's not by runtime testing everything. It is not
link |
by big test frameworks that you're using. Yes, we do that also, but the first step is actually to
link |
make sure that when you want to express something, you can express it directly in code rather than
link |
going through endless loops and convolutions in your head before it gets down the code.
link |
The way you're thinking about a problem is not in the code. There is a missing piece that's just
link |
in your head and the code, you can see what it does, but it cannot see what you thought about it
link |
unless you have expressed things directly. When you express things directly, you can maintain it.
link |
It's easier to find errors. It's easier to make modifications. It's actually easier to test it
link |
and, lo and behold, it runs faster. Therefore, you can use a smaller number of computers,
link |
which means there's less hardware that could possibly break. I think the key here is simplification,
link |
but it has to be to use the Einstein quote as simple as possible and no simpler.
link |
Not simpler. There are other areas with under constraints where you can be simpler than you
link |
can be in C++, but in the domain I'm dealing with, that's the simplification I'm after.
link |
How do you inspire or ensure that the Einstein level simplification is reached?
link |
Can you do code review? Can you look at code? If I gave you the code for the Ford F150 and said,
link |
here, is this a mess or is this okay? Is it possible to tell? Is it possible to regulate?
link |
An experienced developer can look at code and see if it smells. I mixed made it for us deliberately.
link |
Yes. The point is that it is hard to generate something that is really obviously clean and
link |
can be appreciated, but you can usually recognize when you haven't reached that point.
link |
If I have never looked at the F150 code, so I wouldn't know, but I know what I would be looking
link |
for. I would be looking for some tricks that correlate with bugs and elsewhere. I have tried
link |
to formulate rules for what good code looks like. The current version of that is called the C++
link |
core guidelines. One thing people should remember is there's what you can do in a language and what
link |
you should do. In a language, you have lots of things that is necessary in some context,
link |
but not in others. There are things that exist just because there's 30 year old code out there
link |
and you can't get rid of it, but you can't have rules that says when you create it, try and follow
link |
these rules. This does not create good programs by themselves, but it limits the damage for
link |
mistakes. It limits the possibilities of mistakes. Basically, we are trying to say what is it
link |
that a good programmer does at the fairly simple level of where you use the language and how you
link |
use it. Now, I can put all the rules for chiseling and marble. It doesn't mean that somebody who
link |
follows all of those rules can do a masterpiece by Michelangelo. That is, there's something else
link |
to write a good program. Is there something else to create important work of art? There's
link |
some kind of inspiration, understanding, gift, but we can approach the technical, the craftsmanship
link |
level of it. The famous painters, the famous sculptures was, among other things, superb
link |
craftsmen. They could express their ideas using their tools very well. These days, I think what
link |
I'm doing, what a lot of people are doing, we're still trying to figure out how it is to use our
link |
tools very well. For a really good piece of code, you need a spark of inspiration and you
link |
can't, I think, regulate that. You cannot say that I'll buy your picture only if you're,
link |
at least, Van Gogh. There are other things you can regulate, but not the inspiration.
link |
I think that's quite beautifully put. It is true that there is, as an experienced programmer,
link |
when you see code that's inspired, that's like Michelangelo, you know it when you see it.
link |
The opposite of that is code that is messy, code that smells, you know when you see it.
link |
I'm not sure you can describe it in words except vaguely through guidelines and so on.
link |
Yes. It's easier to recognize ugly than to recognize beauty in code. The reason is that
link |
sometimes beauty comes from something that's innovative and unusual and you have to
link |
sometimes think reasonably hard to appreciate that. On the other hand, the messes have things
link |
in common and you can have static checkers and dynamic checkers that finds a large number of the
link |
most common mistakes. You can catch a lot of sloppiness mechanically. I'm a great fan of
link |
static analysis in particular because you can check for not just the language rules,
link |
but for the usage of language rules. I think we will see much more static analysis in the
link |
coming decade. Can you describe what static analysis is? You represent a piece of code
link |
so that you can write a program that goes over that representation and look for things that are
link |
right and not right. For instance, you can analyze a program to see if resources are leaked.
link |
That's one of my favorite problems. It's not actually all that hard in modern C++,
link |
but you can do it. If you are writing in the C level, you have to have a malloc and a free
link |
function. They have to match. If you have them in a single function, you can usually do it very
link |
easily. If there's a malloc here, there should be a free there. On the other hand, in between
link |
can be showing complete code and then it becomes impossible. If you pass that pointer to the memory
link |
out of a function and then want to make sure that the free is done somewhere else,
link |
now it gets really difficult. For static analysis, you can run through a program
link |
and you can try and figure out if there's any leaks. What you will probably find is that you
link |
will find some leaks and you'll find quite a few places where your analysis can't be complete. It
link |
might depend on runtime. It might depend on the cleverness of your analyzer and it might take
link |
a long time. Some of these programs run for a long time, but if you combine such analysis
link |
with a set of rules, such as how people could use it, you can actually see why the rules are
link |
violated and that stops you from getting into the impossible complexities. You don't want to solve
link |
the holding problem. The static analysis is looking at the code without running the code.
link |
Not in production code, but it's almost like an education tool of how the language should be used.
link |
It guides you at its best. It would guide you in how you write future code as well and you learn
link |
together. Basically, you need a set of rules for how you use the language. Then you need
link |
a static analysis that catches your mistakes when you violate the rules or when your code ends up
link |
doing things that it shouldn't, despite the rules, because those are the language rules. We can go
link |
further. Again, it's back to my idea that I would much rather find errors before I start
link |
running the code. If nothing else, once the code runs, if it catches an error at runtime,
link |
I have to have an error handler. One of the hardest things to write in code is error handling code,
link |
because you know something went wrong. Do you know really exactly what went wrong?
link |
Usually not. How can you recover when you don't know what the problem was? You can't be 100%
link |
sure what the problem was in many, many cases. This is part of it. Yes, we need good languages,
link |
with good type systems. We need rules for how to use them. We need static analysis. The ultimate
link |
for static analysis is course program proof, but that still doesn't scale to the kind of systems
link |
we deploy. Then we start needing testing and the rest of the stuff. C++ is an object oriented
link |
programming language that creates, especially with newer versions, as we'll talk about,
link |
higher and higher levels of abstraction. Let's even go back to the origin C++. How do you design
link |
something with so much abstraction that's still efficient and is still something that you can
link |
manage, do static analysis on, you can have constraints on, they can be,
link |
relive all those things we've talked about. To me, there's a slight tension between
link |
high level abstraction and efficiency. That's a good question. I could probably have a year's
link |
course just trying to answer it. Yes, there's a tension between efficiency and abstraction,
link |
but you also get the interesting situation that you get the best efficiency out of the best
link |
abstraction. My main tool for efficiency, for performance, actually is abstraction.
link |
Let's go back to how C++ got there. You said it was object oriented programming language. I
link |
actually never said that. It's always quoted, but I never did. I said C++ supports object oriented
link |
programming and other techniques. That's important because I think that the best solution to most
link |
complex interesting problems require ideas and techniques from things that have been called
link |
object oriented, data abstraction, functional, traditional C style code, all of the above.
link |
When I was designing C++, I soon realized I couldn't just add features.
link |
If you just add what looks pretty or what people ask for or what you think is good,
link |
one by one, you're not going to get a coherent whole. What you need is a set of guidelines that
link |
guides your decisions. Should this feature be in or should this feature be out? How should
link |
a feature be modified before it can go in and such? In the book I wrote about that,
link |
that's signed an evolution of C++. There's a whole bunch of rules like that. Most of them are not
link |
language technical. There are things like don't violate static type system because I like static
link |
type system for the obvious reason that I like things to be reliable on reasonable amounts of
link |
hardware. One of these rules is the zero overhead principle. It basically says that
link |
if you have an abstraction, it should not cost anything compared to write the equivalent code
link |
at a lower level. If I have, say, a matrix multiply, it should be written in such a way that you
link |
could not drop to the C level of abstraction and use arrays and pointers and such and run faster.
link |
People have written such matrix multiplications. They've actually gotten code that ran faster than
link |
Fortran because once you had the right abstraction, you can eliminate temporaries and you can do
link |
loop fusion and other good stuff like that. That's quite hard to do by hand and in a lower
link |
level language. There's some really nice examples of that. The key here is that that matrix
link |
multiplication, the matrix abstraction, allows you to write code that's simple and easy. You can
link |
do that in any language. With C++, it has the features so that you can also have this thing
link |
run faster than if you hand coded it. People have given that lecture many times, I and others,
link |
and a very common question after the talk where you have demonstrated that you're going
link |
to perform Fortran for dense matrix multiplication, people come up and say, yeah,
link |
but that was C++. If I rewrote your code and see how much faster would it run, the answer is much
link |
slower. This happened the first time actually back in the ages with a friend of mine called
link |
Doug McElroy who demonstrated exactly this effect. The principle is you should give programmers the
link |
tools so that the abstractions can follow the zero word principle. Furthermore, when you put in a
link |
language feature on C++ or a standard library feature, you try to meet this. It doesn't mean
link |
it's absolutely optimal, but it means if you hand code it with the usual facilities in the language
link |
in C++ in C, you should not be able to better it. Usually you can do better if you use embedded
link |
assembler for machine code for some of the details to utilize part of a computer that
link |
the compiler doesn't know about, but you should get to that point before you beat the abstraction.
link |
That's a beautiful idea to reach for. And we meet it quite often.
link |
Quite often. Where's the magic of that coming from? Some of it is the compilation process,
link |
so the implementation is C++. Some of it is the design of the feature itself, the guidelines.
link |
I've recently often talked to Chris Ladner, so Clang. Just out of curiosity, is your
link |
relationship in general with the different implementations in C++, as you think about
link |
you and committee and other people in C++, think about the design of new features or design of
link |
previous features in trying to reach the ideal of zero overhead? Does the magic come from the
link |
design, the guidelines, or from the implementations? And not all. You go for programming technique,
link |
program language features, and implementation techniques. You need all three.
link |
And how can you think about all three at the same time?
link |
It takes some experience, takes some practice, and sometimes you get it wrong. But after a while,
link |
you sort of get it right. I don't write compilers anymore. But Brian Kernighan pointed out that
link |
one of the reasons C++ succeeded was some of the craftsmanship I put into the early compilers.
link |
And of course, I did the languages sign. Of course, I wrote a fair amount of code
link |
using this kind of stuff. And I think most of the successes involves progress in all three areas
link |
together. A small group of people can do that. Two, three people can work together to do something
link |
like that. It's ideal if it's one person that has all the skills necessary. But nobody has all
link |
the skills necessary in all the fields where C++ is used. So if you want to approach my ideal in,
link |
say, concurrent programming, you need to know about algorithms on concurrent programming. You
link |
need to know the trigger of lock free programming. You need to know something about the compiler
link |
techniques. And then you have to know some of the application areas where this is,
link |
like some forms of graphics or some forms of what we call a web serving kind of stuff.
link |
And that's very hard to get into a single head. But small groups can do it too.
link |
So is there differences in your view? Not saying which is better or so on, but difference in
link |
the different implementations of C++? Why are there several sort of maybe naive questions for me?
link |
This is a very reasonable question. When I designed C++,
link |
most languages have multiple implementations. Because if you run on an IBM, if you run on a
link |
Sun, if you run on a Motorola, there was just many, many companies and they each have their
link |
own compilation structure, the old compilers. It was just fairly common that there was many of them.
link |
And I wrote Cfront assuming that other people would write compilers with C++ if they were
link |
successful. And furthermore, I wanted to utilize all the back end infrastructures that were available.
link |
I soon realized that my users were using 25 different linkers. I couldn't write my own linker.
link |
Yes, I could, but I couldn't write 25 linkers and also get any work done on the language.
link |
And so it came from a world where there was many linkers, many optimizers, many compiler front ends,
link |
not to start, but many operating systems. The whole world was not an 86 and Linux box or something,
link |
whatever is the standard today. In the old days, they said a Vax. So basically, I assumed there
link |
would be lots of compilers. It was not a decision that there should be many compilers. It was just
link |
the fact that's the way the world is. And yes, many compilers emerged. And today, there's
link |
at least four front ends, Clang, GCC, Microsoft, and EDG. It is the sign group. They supply a lot
link |
of the independent organizations and the embedded systems industry. And there's lots and lots of
link |
back ends. We have to think about how many dozen back ends there are because different machines
link |
have different things, especially in the embedded world, the machines are very different. The
link |
architectures are very different. And so having a single implementation was never an option.
link |
Now, I also happen to dislike monocultures. Monocultures. They are dangerous because whoever
link |
owns the monoculture can go stale and there's no competition and there's no incentive to innovate.
link |
There's a lot of incentive to put barriers in the way of change because, hey, we own the world and
link |
it's a very comfortable world for us. And who are you to mess with that? So I really am very happy
link |
that there's four front ends for C++. Clang's great, but GCC was great. But then it got somewhat
link |
stale. Clang came along and GCC is much better now. Competition is good. Microsoft is much better
link |
now. So at least a low number of front ends puts a lot of pressure on
link |
standards compliance and also on performance and error messages and compile time speed,
link |
all of this good stuff that we want. Do you think, crazy question, there might come along,
link |
do you hope there might come along implementation of C++ written given all of its history written
link |
from scratch? So written today from scratch? Well, Clang and the LLVM is more or less written
link |
from scratch. But there's been C++ 11, 14, 17, 20. Sooner or later somebody is going to try again.
link |
There has been attempts to write new C++ compilers and some of them has been used and some of them
link |
has been absorbed into others and such. Yeah, it'll happen. So what are the key features of C++?
link |
And let's use that as a way to sort of talk about the evolution of C++, the new feature.
link |
So at the highest level, what are the features that were there in the beginning and what features
link |
got added? Let's first get a principle or an aim in place. C++ is for people who want to use
link |
hardware really well and then manage the complexity of doing that through abstraction.
link |
And so the first facility you have is a way of manipulating the machines at a fairly low level.
link |
That looks very much like C. It has loops, it has variables, it has pointers like machine
link |
addresses, it can access memory directly, it can allocate stuff in the absolute minimum of space
link |
needed on the machine. There's a machine facing part of C++, which is roughly equivalent to C.
link |
I said C++ could beat C and it can. It doesn't mean I dislike C. If I disliked C, I wouldn't have
link |
built on it. Furthermore, after Dennis Ritchie, I'm probably the major contributor to modern C.
link |
And well, I had lunch with Dennis most days for 16 years and we never had a harsh word between us.
link |
So these C versus C++ fights are for people who don't quite understand what's going on.
link |
Then the other part is the abstraction. And there, the key is the class, which is a user defined type.
link |
And my ideal for the class is that you should be able to build a type that's just like the
link |
built in types in the way you use them, in the way you declare them, in the way you get the memory
link |
and you can do just as well. So in C++ there's an int. As in C, you should be able to build
link |
an abstraction, a class, which we can call capital int, that you can use exactly like an integer
link |
and run just as fast as an integer. There's the idea right there. And of course, you probably
link |
don't want to use the int itself, but it has happened. People have wanted integers that were
link |
range checked so that you couldn't overflow and such, especially for very safety critical
link |
applications, like the fuel injection for a marine diesel engine for the largest ships.
link |
This is a real example, by the way. This has been done. They built themselves an integer
link |
that was just like integer, except that it couldn't overflow. If there was an overflow,
link |
you went into the error handling. And then you built more interesting types. You can build
link |
a matrix, which you need to do graphics, or you could build a gnome for a video game.
link |
And all of these are classes and they appear just like the built in types,
link |
in terms of efficiency and so on. So what else is there? And flexibility.
link |
I don't know. For people who are not familiar with object joining programming,
link |
there's inheritance. There's a hierarchy of classes. You can, just like you said,
link |
create a generic vehicle that can turn left. So what people found was that you don't actually
link |
know. How do I say this? A lot of types are related. That is, the vehicles, all vehicles are
link |
related. Bicycles, cars, fire engines, tanks, they have some things in common and some things
link |
that differ. And you would like to have the common things common and having the differences
link |
specific. And when you didn't want to know about the differences, like just turn left,
link |
you don't have to worry about it. That's how you get the traditional object oriented
link |
programming coming out of Simula adopted by Smalltalk and C++ and all the other languages.
link |
The other kind of obvious similarity between types comes when you have something like a vector.
link |
Fortune gave us the vector as called array of doubles. But the minute you have a vector of
link |
doubles, you want a vector of double precision doubles and for short doubles for graphics.
link |
And why should you have not have a vector of integers while you're at it? Or that vector of
link |
vectors and a vector of vectors of chess pieces. Now we have a board, right? So this is you express
link |
the commonality as the idea of a vector and the variations come through parameterization.
link |
And so here we get the two fundamental ways of abstracting of having similarities of
link |
types in C++. There's the inheritance and there's a parameterization. There's the
link |
object oriented programming and there's the generic programming with the templates for
link |
the generic programming. So you've presented it very nicely. But now you have to make all that
link |
happen and make it efficient. So generic programming with templates, there's all kinds of magic going
link |
on, especially recently, that you can help catch up on. But it feels to me like you can do way more
link |
than what you just said with templates. You can start doing this kind of metaprogramming.
link |
You can do metaprogramming also. I didn't go there in that explanation. We're trying to be
link |
very basics. But go back on to the implementation. If you couldn't implement this efficiently,
link |
if you couldn't use it so that it became efficient, it has no place in C++ because it
link |
will violate the zero overhead principle. So when I had to get object oriented programming
link |
inheritance, I took the idea of virtual functions from Simula. Virtual functions is a Simula term.
link |
Class is a Simula term. If you ever use those words, say thanks to Christian Nugor and Oli Handal.
link |
And I get the simplest implementation I knew of, which was basically a jump table. So you get the
link |
virtual function table, the function goes in, do it, does an interaction through a table and get
link |
the right function. That's how you pick the right thing there. And I thought that was trivial. It's
link |
close to optimal. And it was obvious. It turned out the Simula had a more complicated way of doing
link |
it and therefore was slower. And it turns out that most languages have something that's a little
link |
bit more complicated, sometimes more flexible, but you pay for it. And one of the strengths of
link |
C++ was that you could actually do this object oriented stuff and your overhead compared to
link |
ordinary functions. There's no indirection. It's sort of in 5, 10, 25 percent. Just the
link |
core. It's down there. It's not two. And that means you can afford to use it. Furthermore,
link |
in C++, you have the distinction between a virtual function and a non virtual function.
link |
If you don't want any overhead, if you don't need the interaction that gives you the flexibility
link |
in object oriented programming, just don't ask for it. So the idea is that you only use
link |
virtual functions if you actually need the flexibility. So it's not zero overhead,
link |
but it's zero overhead compared to any other way of achieving the flexibility.
link |
Now, auto parameterization. Basically, the compiler looks at
link |
at the template, say the vector, and it looks at the parameter and then combines the two and
link |
generates a piece of code that is exactly as if you're ridden a vector of that specific type.
link |
Yes. So that's the that's the minimal overhead. If you have many template parameters, you can
link |
actually combine code that the compiler couldn't usually see at the same time and therefore get
link |
code that is faster than if you had handwritten the stuff, unless you were very, very clever.
link |
So the thing is parameterized code, the compiler fills stuff in during the compilation process,
link |
not during runtime. That's right. And furthermore, it gives all the information it's gotten,
link |
which is the template, the parameter, and the context of use. It combines the three and generates
link |
good code. But it can generate. Now, it's a little outside of what I'm even comfortable
link |
thinking about, but it can generate a lot of code. Yes. And how do you remember being both
link |
amazed at the power of that idea? And how ugly the debugging looked? Yes, debugging can be truly
link |
horrid. Come back to this because I have a solution. Anyway, the debugging was ugly.
link |
The code generated by C++ has always been ugly, because there's these inherent optimizations.
link |
A modern C++ compiler has front end, middle end, and back end optimizations. Even C front,
link |
back in 83, had front end and back end optimizations. I actually took the code,
link |
generated an internal representation, munched that representation to generate good code.
link |
So people say it's not a compiler, it generates C. The reason it generated C was I wanted to use
link |
a C code generator that was really good at back end optimizations. But I needed front end
link |
optimizations. And therefore, the C I generated was optimized C. The way a really good handcrafted
link |
optimizer human could generate it, and it was not meant for humans. It was the output of a program,
link |
and it's much worse today. And with templates, it gets much worse still. So it's hard to combine
link |
simple debugging with the optimal code, because the idea is to drag in information from different
link |
parts of the code to generate good code, machine code. And that's not readable. So what people
link |
often do for debugging is they turn the optimizer off. And so you get code that when something in
link |
your source code looks like a function core, it is a function core. When the optimizer is turned on,
link |
it may disappear. The function core, it may inline. And so one of the things you can do
link |
is you can actually get code that is smaller than the function core, because you eliminate the
link |
function preamble and return. And there's just the operation there. One of the key things when I did
link |
templates was I wanted to make sure that if you have, say, a sort algorithm, and you give it a
link |
sorting criteria, if that sorting criteria is simply comparing things with less than,
link |
the code generated should be the less than, not a indirect function core to a comparison
link |
object, which is what it is in the source code. But we really want down to the single instruction.
link |
But anyway, turn off the optimizer and you can debug. The first level of debugging
link |
can be done and I always do without the optimization on, because then I can see what's going on.
link |
And then there's this idea of concepts that put some, now I've never even,
link |
I don't know if it was ever available in any form, but it puts some constraints on the stuff you
link |
can parameterize, essentially. Let me try and explain. So yes, it wasn't there 10 years ago.
link |
We have had versions of it that actually work for the last four or five years.
link |
It was a design by Gabbidas Rays, Drew Sodden and me. We were professors in postdocs in Texas at
link |
the time. And the implementation by Andrew Sodden has been available for that time. And it is
link |
part of C++20. And there's a standard library that uses it. So this is becoming really very real.
link |
It's available in Klangen and GCC, GCC for a couple of years. And I believe Microsoft is soon
link |
going to do it. We expect all of C++20 to be available. So in all the major compilers in 20.
link |
But this kind of stuff is available now. I'm just saying that because otherwise people might think
link |
I was talking about science fiction. And so what I'm going to say is it's concrete. You can run it
link |
today. And there's production uses of it. So the basic idea is that when you have
link |
a generic component, like a sort function, the sort function will require at least two parameters.
link |
One, a data structure with a given type and a comparison criteria. And these things are related.
link |
But obviously, you can't compare things if you don't know what the type of things you compare.
link |
Yeah. And so you want to be able to say, I'm going to sort something and it is to be sortable.
link |
What does it mean to be sortable? You look it up in the standard. It has to be a sequence with a
link |
beginning and an end. There has to be random access to that sequence. And the element types
link |
has to be comparable. Which means less than operator can operate on. Yes, less than logical
link |
operator can operate. Basically, what concepts are their compile time predicates, their predicates
link |
you can ask, are you a sequence? Yes, I have a beginning and end. Are you a random access sequence?
link |
Yes, I have a subscripting and plus. Is your element type something that has a less than?
link |
Yes, I have a less than. And so basically, that's the system. And so instead of saying,
link |
I will take a parameter of any type, it'll say I'll take something that's sortable.
link |
And it's well defined. And so we say, okay, you can sort with less than. I don't want less than.
link |
I want greater than or something I invent. So you have two parameters, the sortable thing and the
link |
comparison criteria. And the comparison criteria will say, well, you can write in saying it should
link |
operate on the element type. And it has the comparison operations. So that's just simply
link |
the fundamental thing. It's compile time predicates. Do you have the properties I need? So it specifies
link |
the requirements of the code on the parameters that gets. It's very similar to types, actually.
link |
But operating in the space of concepts. The word concept was used by Alex Stefanov,
link |
who is sort of the father of generic programming in the context of C++.
link |
You know, there's other places that use that word, but the way we call generic programming is
link |
Alex's. And he called them concepts, because he said they're the sort of the fundamental
link |
concepts of an area. So they should be called concepts. And we've had concepts all the time.
link |
If you look at the K&R book about C, C has arithmetic types, and it has
link |
integral types. It says so in the book. And then it lists what they are, and they have
link |
certain properties. The difference today is that we can actually write a concept that will ask a
link |
type, are you an integral type? Do you have the properties necessary to be an integral type?
link |
Do you have plus, minus, divide, and such? So maybe the story of concepts, because I thought
link |
it might be part of C++11, C0X, whatever it was at the time. We'll talk a little bit about
link |
this fascinating process of standards, because I think it's really interesting for people.
link |
It's interesting for me. But why did it take so long? What shapes the idea of concepts take?
link |
What were the challenges? Back in 1987 or thereabouts. 1987?
link |
Well, 1987 or thereabouts. When I was designing templates, obviously, I wanted to express the
link |
notion of what is required by a template of its arguments. And so I looked at this. And basically
link |
for templates, I wanted three properties. I wanted to be very flexible. It had to be able to express
link |
things I couldn't imagine, because I know I can't imagine everything, and I've been suffering from
link |
languages that try to constrain you to only do what the designer thought good. I didn't want to do that.
link |
Secondly, it had to run faster, as fast or faster than handwritten code. So basically,
link |
if I have a vector of t and I take a vector of char, it should run as fast as you build
link |
a vector of char yourself without parameterization. And thirdly, I wanted to be able to express
link |
the constraints of the arguments, have proper type checking of the interfaces,
link |
and neither I nor anybody else at the time knew how to get all three. And I thought for C++,
link |
I must have the two first. Otherwise, it's not C++. And it bothered me for another couple of decades
link |
that I couldn't solve the third one. I mean, I was the one that put function argument type checking
link |
into C. I know the value of good interfaces. I didn't invent that idea. It's very common.
link |
But I did it. And I wanted to do the same for templates, of course, and I could. So it bothered
link |
me. Then we tried again, 2002, 2003. Gaby just raised and I started analyzing the problem,
link |
explained possible solutions. It was not a complete design. A group in University of Indiana,
link |
an old friend of mine, they started a project at Indiana. And we thought we could get
link |
a good system of concepts in another two or three years. That would have made C++ 11 to C++
link |
0607. Well, it turned out that I think we got a lot of the fundamental ideas wrong.
link |
They were too conventional. They didn't quite fit C++, in my opinion. Didn't serve implicit
link |
conversions very well. It didn't serve mixed type arithmetic, mixed type computations very well.
link |
Well, a lot of stuff came out of the functional community. That community
link |
didn't deal with multiple types in the same way as C++ does. Had more constraints on what you could
link |
express and didn't have the draconian performance requirements. Basically, we tried. We tried very
link |
hard. We had some successes, but it just in the end wasn't. Didn't compile fast enough, was too
link |
hard to use and didn't run fast enough unless you had optimizers that was beyond the state of the
link |
art. They still are. So we had to do something else. Basically, it was the idea that a set of
link |
parameters defines a set of operations and you go through an interaction table just like for
link |
virtual functions. Then you try to optimize the interaction away to get performance. We just couldn't
link |
do all of that. But get back to the standardization. We are standardizing C++ under ISO rules,
link |
which is a very open process. People come in. There's no requirements for education or experience.
link |
So you've started to develop C++. When was the first standard established? What is that like?
link |
The ISO standard? Is there a committee that you're referring to? There's a group of people. What's
link |
that like? How often do you meet? What's the discussion? I'll try and explain that. So sometime
link |
in early 1989, two people, one from IBM, one from HP turned up in my office and told me I would like
link |
to standardize C++. This was a new idea to me. I pointed out that it wasn't finished yet and it
link |
wasn't ready for formal standardization and such. They said, no, Bjarne, you haven't gotten it.
link |
You really want to do this. Our organizations depend on C++. We cannot depend on something
link |
that's owned by another corporation that might be a competitor. Of course, we could rely on you,
link |
but you might get run over by a boss. We really need to get this out in the open. It has to be
link |
standardized under formal rules. We are going to standardize it under ISO rules. You really want
link |
to be part of it because, basically, otherwise, we'll do it ourselves. We know you can do it better.
link |
Through a combination of arm twisting and flattery, it got started. In late
link |
in late 89, there was a meeting in DC at the, actually, no, it was not ISO, then it was ANSI,
link |
the American National Standard we're doing. We met there. We were lectured on the rules of
link |
how to do an ANSI standard. There was about 25 of us there, which apparently was a new record for
link |
that kind of meeting. Some of the old C guys that has been standardizing C was there, so we
link |
got some expertise in. The way this works is that it's an open process. Anybody can sign up if they
link |
pay the minimum fee, which is about $1,000. They're less than, it's a little bit more now. I think
link |
it's $1,280. It's not going to kill you. We have three meetings a year. This is fairly standard.
link |
We tried two meetings a year for a couple of years. That didn't work too well. Three one
link |
week meetings a year, and you meet and you have technical meetings, technical discussions,
link |
and then you bring proposals forward for votes. The votes are done one person per,
link |
one vote per organization. You can't have, say, IBM come in with 10 people and dominate things
link |
that's not allowed. These are organizations that extensively use C++? Yes.
link |
All individuals. It's a bunch of people in the room deciding the design of a language based
link |
on which a lot of the world's systems run. That's right. Well, I think most people would agree it's
link |
better than if I decided it, or better than if a single organization like H&C decides it.
link |
I don't know if everyone agrees to that, by the way. Bureaucracies have their critics too.
link |
Yes, look, standardization is not pleasant. It's horrifying. It's like democracy.
link |
But exactly. As Churchill says, democracy is the worst way except for the others.
link |
And it's, I would say, the same with formal standardization. But anyway, so we meet and we
link |
have these votes and that determines what the standard is. A couple of years later, we extended
link |
this so it became worldwide. We have standard organizations that are active in currently 15
link |
to 20 countries. And another 15 to 20 are sort of looking and voting based on the rest of the work
link |
on it. And we meet three times a year. Next week, I'll be in Cologne, Germany, spending a week
link |
doing standardization. And then we will vote out the committee draft of C++20, which goes to the
link |
national standards committees for comments and requests for changes and improvements.
link |
Then we do that. And there's a second set of votes where hopefully everybody votes in favor.
link |
This has happened several times. The first time we finished, we started in the first technical
link |
meeting was in 1990. The last was in 98. We voted it out. That was the standard that people used
link |
till 11 or a little bit past 11. And it was an international standard. All the countries voted
link |
in favor. It took longer with 11. I'll mention why, but all the nations voted in favor. And we
link |
work on the basis of consensus. That is, we do not want something that passes 6040,
link |
because then we're getting getting a dialects and opponents and people complain too much.
link |
They all complain too much. But basically, it has no real effect. The standards has been obeyed.
link |
They have been working to make it easier to use many compilers, many computers,
link |
and all of that kind of stuff. And so the first, it was traditional with ISO standards to take
link |
10 years. We did the first one in eight, brilliant. And we thought we were going to do the next one
link |
in six, because now we're good at it. It took 13. Yeah, it was named OX. It was named OX.
link |
Hoping that you would at least get it within the single, within the arts, the single digits.
link |
I thought we would get, I thought we would get the six, seven or eight. The confidence of youth.
link |
Yeah, that's right. Well, the point is that this was sort of like a second system effect.
link |
That is, we now knew how to do it. And so we're going to do it much better. And we've got more
link |
ambitious and it took longer. Furthermore, there is this tendency because it's a 10 year cycle,
link |
or it doesn't matter. Just before you're about to ship, somebody has a bright idea.
link |
And so we really, really must get that in. We did that successfully with the STL.
link |
We got the standard line that gives us all the STL stuff that that I basically,
link |
I think it saved C++. It was beautiful. And then people tried it with other things.
link |
And it didn't work so well. They got things in, but it wasn't as dramatic and it took longer and
link |
longer and longer. So after C++ 11, which was a huge improvement and what basically what most
link |
people are using today, we decided never again. And so how do you avoid those slips? And the
link |
answer is that you ship more often so that if you have a slip on a 10 year cycle, by the time
link |
you know it's a slip, there's 11 years till you get it. Now with a three year cycle,
link |
there is about three, four years till you get it. Like the delay between feature freeze and
link |
shipping. So you always get one or two years more. And so we shipped 14 on time. We shipped
link |
17 on time. And we shipped, we will ship 20 on time. It'll happen. And furthermore,
link |
this allows, this gives a predictability that allows the implementers, the compiler implementers,
link |
the library implementers, to, they have a target and they deliver on it. 11 took two years before
link |
most compilers were good enough. 14, most compilers were actually getting pretty good in 14.
link |
17, everybody shipped in 17. We are going to have at least almost everybody ship almost
link |
everything in 20. And I know this because they're shipping in 19. Predictability is good. Delivery
link |
on time is good. And so yeah. That's great. So that's how it works. There's a lot of features
link |
that came in in C++11. There's a lot of features at the birth of C++ that were amazing and ideas
link |
with concepts in 2020. What to you is the most, just to you personally, beautiful or
link |
just do you sit back and think, wow, that's just nice and clean feature of C++?
link |
I have written two papers for the history of programming languages conference, which basically
link |
asked me such questions. And I'm writing a third one, which I will deliver at the history of
link |
programming languages conference in London next year. So I've been thinking about that and there
link |
is one clear answer. Constructors and destructors. The way a constructor can establish the environment
link |
for the use of a type for an object and the destructor that cleans up any messes at the end
link |
of it. That is key to C++. That's why we don't have to use garbage collection. That's how we can
link |
get predictable performance. That's how you can get the minimal overhead in many, many cases
link |
and have really clean types. It's the idea of constructor, destructor pairs. Sometimes it comes
link |
out under the name RIAII. Resource acquisition is initialization, which is the idea that you
link |
grab resources and the constructor and release them and destructor. It's also the best example
link |
of why I shouldn't be in advertising. I get the best idea and I call it resource acquisition is
link |
initialization. Not the greatest naming I've ever heard. So it's types, abstraction of types.
link |
You said, I want to create my own types. Types is an essential part of C++ and making them
link |
efficient is the key part. To you, this is almost getting philosophical, but the construction
link |
and the destruction, the creation of an instance of a type and the freeing of resources from that
link |
instance of a type is what defines the object. It's almost like birth and death is what defines
link |
human life. Yeah, that's right. By the way, philosophy is important. You can't do good
link |
language design without philosophy because what you are determining is what people can express
link |
and how. This is very important. By the way, constructors, destructors came into C++ in 1979
link |
in about the second week of my work with what was then called C++. It is a fundamental idea.
link |
Next comes the fact that you need to control copying because once you control, as you said,
link |
birth and death, you have to control taking copies, which is another way of creating an object.
link |
Finally, you have to be able to move things around so you get the move operations. That's
link |
a set of key operations you can define on a C++ type. To you, those things are just a beautiful
link |
part of C++ that is at the core of it all. Yes. You mentioned that you hope there will be one
link |
unified set of guidelines in the future for how to construct the programming language.
link |
So perhaps not one programming language, but a unification of how we build programming languages
link |
if you remember such statements. I have some trouble remembering it, but I know the origin
link |
of that idea. Maybe you can talk about, C++ has been improving. There's been a lot of programming
link |
language. Where does the archer history taking us? Do you hope that there is a unification about
link |
the languages with which we communicate in the digital space?
link |
I think that languages should be designed not by clobbering language features together and
link |
doing slightly different versions of somebody else's ideas, but through the creation of a set
link |
of principles, rules of thumbs, whatever you call them. I made them for C++ and we're trying to
link |
teach people in the Standards Committee about these rules because a lot of people come in and say,
link |
I've got a great idea. Let's put it in the language. Then you have to ask, why does it fit in the
link |
language? Why does it fit in this language? It may fit in another language and not here,
link |
or it may fit here and not the other language. So you have to work from a set of principles and
link |
you have to develop that set of principles. One example that I sometimes remember is I was sitting
link |
down with some of the designers of Common Lisp and we were talking about languages and language
link |
features and obviously we didn't agree about anything because well, Lisp is not C++ and vice
link |
versa. Too many parentheses. But suddenly we started making progress. I said, I had this problem
link |
and I developed it according to these ideas and they said, why? We had that problem, different
link |
problem and we developed it with the same kind of principles. And so we worked through large chunks
link |
of C++ and large chunks of Common Lisp and figured out we actually had similar sets
link |
of principles of how to do it. But the constraints on our designs were very different
link |
and the aims for the usage was very different. But there was commonality in the way you reason
link |
about language features and the fundamental principles you were trying to do.
link |
So do you think that's possible? So they're just like there is perhaps a unified theory of physics,
link |
of the fundamental forces of physics, that I'm sure there is commonalities among the language
link |
but there's also people involved that help drive the development of these languages. Do you have a
link |
hope or an optimism that there will be a unification if you think about physics in Einstein towards
link |
a simplified language? Do you think that's possible? Let's remember sort of modern physics,
link |
I think started with Galileo in the 1300s. So they've had 700 years to get going.
link |
Modern computing started in about 49. We've got, what is it, 70 years. They have 10 times.
link |
And furthermore, they are not as bothered with people using physics the way we are worried about
link |
programming. It's done by humans. So each have problems and constraints the others have but
link |
we are very immature compared to physics. So I would look at sort of the philosophical level
link |
and look for fundamental principles like you don't leak resources, you shouldn't.
link |
You don't take errors at runtime that you don't need to. You don't violate some kind of type
link |
system. There's many kinds of type systems, but when you have one, you don't break it, etc. etc.
link |
There will be quite a few and it will not be the same for all languages. But I think if we step
link |
back at some kind of philosophical level, we would be able to agree on sets of principles
link |
that applied to sets of problem areas. And within an area of use, like in C++'s case,
link |
what used to be called systems programming, the area between the hardware and the fluffier
link |
the fluffier parts of the system, you might very well see a convergence. So these days you see
link |
Rust having adopted RAII and some time accuses me for having borrowed it 20 years before they
link |
discovered it. But we're seeing some kind of convergence here instead of relying on garbage
link |
collection all the time. The garbage collection languages are doing things like the dispose
link |
patterns and such that imitate some of the construction, destruction stuff. And they're
link |
trying not to use the garbage collection all the time and things like that. So there's a
link |
conversion. But I think we have to step back to the philosophical level, agree on principles,
link |
and then we'll see some conversions, convergences, and it will be application domain specific.
link |
So a crazy question, but I work a lot with machine learning with deep learning. I'm not
link |
sure if you touch that world much. But you could think of programming as a thing that takes some
link |
input. Programming is the task of creating a program, and the program takes some input and
link |
produces some output. So machine learning systems train on data in order to be able
link |
to take an input and produce output. But they're messy, fuzzy things, much like
link |
we as children grow up, we take some input, we make some output, but we're noisy, we mess up a
link |
lot, we're definitely not reliable biological system or a giant mess. So there's a sense in
link |
which machine learning is a kind of way of programming, but just fuzzy. It's very, very,
link |
very different than C++. Because C++ is a, like it's just like you said, it's extremely reliable,
link |
it's efficient, it's, you know, you can, you can measure, you can test in a bunch of different
link |
ways with biological systems or machine learning systems. You can't say much, except sort of
link |
empirically saying that 99.8% of the time, it seems to work. What do you think about this fuzzy
link |
kind of programming? Do you even see it as programming? Is it solely and totally another
link |
kind of world? I think it's a different kind of world. And it is fuzzy. And in my domain,
link |
I don't like fuzziness. That is, people say things like they want everybody to be able to program.
link |
But I don't want everybody to program my airplane controls or the car controls.
link |
I want that to be done by engineers. I want that to be done with people that are specifically
link |
educated and trained for doing building things. And it is not for everybody. Similarly, a language
link |
like C++ is not for everybody. It is generated to be a sharp and effective tool for professionals,
link |
basically, and definitely for people who, who, who aim at some kind of precision. You don't
link |
have people doing calculations without understanding math, right? Counting on your fingers is not going
link |
to cut it if you want to fly to the moon. And so there are areas where an 84% accuracy rate,
link |
16% false positive rate is perfectly acceptable and where people will probably get no more than
link |
70. You said 98%. I, the, what I've seen is more like 84. And by, by really a lot of blood,
link |
sweat and tears, you can get up to the 92 and a half. So this is fine if it is, say,
link |
pre screening stuff before the human look at it. It is not good enough for, for life
link |
threatening situations. And so there's lots of areas where, where the fuzziness is, is perfectly
link |
acceptable and good and better than humans, cheaper than humans. But it's not the kind of
link |
engineering stuff I'm mostly interested in. I worry a bit about machine learning in the context
link |
of cars. You know, much more about this than I do. I worry too. But I'm, I'm, I'm sort of an
link |
amateur here. I've read some of the papers, but I've not ever done it. And the, the, the idea
link |
that scares me the most is the one I have heard, and I don't know how common it is, that
link |
you have this AI system, machine learning, all of these trained neural nets. And when there's
link |
something that's too complicated, they ask the human for help. But the human is reading a book
link |
or sleep. And he has 30 seconds or three seconds to figure out what the problem was that the AI
link |
system couldn't handle and do the right thing. This is scary. I mean, how do you do the cut over
link |
between the machine and the human? It's very, very difficult. And for the designer of one of the most
link |
reliable, efficient and powerful programming languages, C plus plus, I can understand why
link |
that world is actually unappealing. It is for most engineers. To me, it's extremely appealing,
link |
because we don't know how to get that interaction right. But I think it's possible,
link |
but it's very, very hard. It is. And now it's stating a problem, not a solution.
link |
That is impossible. I mean, I would much rather never rely on a human. If you're driving a nuclear
link |
reactor, if you're or an autonomous vehicle, it's much better to design systems written in C plus
link |
plus that never ask human for help. Let, let, let's just get one fact in. Yeah. All of this AI
link |
stuff is on top of C plus plus. So, so that's one reason I have to keep a weather eye out on
link |
what's going on in that field, but I will never become an expert in that area. But it's a good
link |
example of how you separate different areas of applications and you have to have different tools,
link |
different principles. And then they interact. No major system today is written in one language.
link |
And there are good reasons for that. When you look back at your life work,
link |
what is a, what is a moment? What is a event creation that you're really proud of?
link |
They say, damn, I did pretty good there. Is it as obvious as the creation of C plus plus?
link |
It's obvious. I've spent a lot of time with C plus plus and there's a combination of a few good
link |
ideas, a lot of hard work and a bit of luck. And I've tried to get away from it a few times,
link |
but I get dragged in again, partly because I'm most effective in this area and partly because
link |
what I do has much more impact. If I do it in the context of C plus plus, I have four and a half
link |
million people that pick it up tomorrow. If I get something right, if I did it in another field,
link |
I would have to start learning, then I have to build it and then we'll see if anybody wants to use it.
link |
One of the things that has kept me going for all of these years is one, the good things that people
link |
do with it and the interesting things they do with it. And also, I get to see a lot of
link |
interesting stuff and talk to a lot of interesting people. I mean, if it has just been statements
link |
on paper or on a screen, I don't think I could have kept going. But I get to see the telescopes
link |
up on Mauna Kea and I actually went and see how Ford built cars and I got to JPL and see how they
link |
do the Mars rovers. There's so much cool stuff going on and most of the cool stuff is done by
link |
pretty nice people and sometimes in very nice places, Cambridge, Sofia and Tbilisi, Silicon Valley.
link |
Yeah, there's more to it than just code. But code is central.
link |
On top of the code are the people in very nice places. Well, I think I speak for millions of
link |
people. We are in saying thank you for creating this language that so many systems are built on
link |
top of that make a better world. So thank you and thank you for talking today.
link |
Yeah, thanks and we'll make it even better. Good.