back to indexDmitri Dolgov: Waymo and the Future of Self-Driving Cars | Lex Fridman Podcast #147
link |
The following is a conversation with Dimitri Dolgov, the CTO of Waymo, which
link |
is an autonomous driving company that started as Google self driving car
link |
project in 2009 and became Waymo in 2016.
link |
Dimitri was there all along.
link |
Waymo is currently leading in the fully autonomous vehicle space and that they
link |
actually have an at scale deployment of publicly accessible autonomous vehicles
link |
driving passengers around with no safety driver, with nobody in the driver's seat.
link |
This to me is an incredible accomplishment of engineering on one of
link |
the most difficult and exciting artificial intelligence challenges of
link |
Quick mention of a sponsor, followed by some thoughts related to the episode.
link |
Thank you to Triolabs, a company that helps businesses apply machine
link |
learning to solve real world problems.
link |
Blinkist, an app I use for reading through summaries of books, better
link |
help, online therapy with a licensed professional, and Cash App, the app
link |
I use to send money to friends.
link |
Please check out the sponsors in the description to get a discount
link |
at the support this podcast.
link |
As a side note, let me say that autonomous and semi autonomous driving
link |
was the focus of my work at MIT and as a problem space that I find
link |
fascinating and full of open questions from both robotics and a human
link |
psychology perspective.
link |
There's quite a bit that I could say here about my experiences in academia
link |
on this topic that revealed to me, let's say the less admirable size of human
link |
beings, but I choose to focus on the positive, on solutions.
link |
I'm brilliant engineers like Dimitri and the team at Waymo, who work
link |
tirelessly to innovate and to build amazing technology that will define
link |
Because of Dimitri and others like him, I'm excited for this future.
link |
And who knows, perhaps I too will help contribute something of value to it.
link |
If you enjoy this thing, subscribe on YouTube, review it with five stars
link |
and up a podcast, follow on Spotify, support on Patreon, or connect with
link |
me on Twitter at Lex Friedman.
link |
And now here's my conversation with Dimitri Dolgov.
link |
When did you first fall in love with MIT?
link |
When did you first fall in love with robotics or even computer
link |
science more in general?
link |
Computer science first at a fairly young age, then robotics happened much later.
link |
I think my first interesting introduction to computers was in the late 80s when
link |
we got our first computer, I think it was an IBM, I think IBM AT.
link |
Those things that had like a turbo button in the front, the radio
link |
precedent, you know, make, make the thing goes faster.
link |
Did that already have floppy disks?
link |
Like the, the 5.4 inch ones.
link |
I think there was a bigger inch.
link |
When something then five inches and three inches.
link |
Yeah, I think that was the five.
link |
I don't, I maybe that was before that was the giant plates and it didn't get that.
link |
But it was definitely not the, not the three inch ones.
link |
Anyway, so that, that, you know, we got that computer, I spent the first few
link |
months just playing video games as you would expect, I got bored of that.
link |
So I started messing around and trying to figure out how to, you know, make
link |
the thing do other stuff, got into exploring programming and a couple of
link |
years later, it got to a point where, I actually wrote a game, a lot of games
link |
and a game developer, a Japanese game developer actually offered to buy it
link |
for me for a few hundred bucks.
link |
But you know, for, for a kid in Russia, that's a big deal.
link |
That's a big deal.
link |
I did not take the deal.
link |
I, I instead, yes, that was not the most acute financial move that I made in my
link |
life, you know, looking back at it now, I, I instead put it, well, you know, I had
link |
a reason I put it online, it was, what'd you call it back in the days?
link |
It was a freeware thing, right?
link |
It was not open source, but you could upload the binaries, you would put the
link |
game online and the idea was that, you know, people like it and then they, you
link |
know, contribute on the send you a little donations, right?
link |
So I did my quick math of like, you know, of course, you know, thousands and
link |
millions of people are going to play my game, send me a couple of bucks a piece,
link |
you know, should definitely do that.
link |
As I said, not, not the best.
link |
You're already playing with business models at that young age.
link |
Remember what language it was?
link |
What programming, it was a Pascal, which what Pascal, Pascal, and that
link |
a graphical component, so it's not text based.
link |
It was, uh, like, uh, I think there are 300, 320 by 200, uh, whatever it was.
link |
I think that kind of the earlier, that's the resolution, right?
link |
And I actually think the reason why this company wanted to buy it is not like the
link |
fancy graphics or the implementation.
link |
That was maybe the idea, uh, of my actual game, the idea of the game.
link |
Well, one of the things I, it's so funny.
link |
I'm used to play this game called golden X and the simplicity of the graphics and
link |
something about the simplicity of the music, like it's still haunts me.
link |
I don't know if that's a childhood thing.
link |
I don't know if that's the same thing for call of duty these days for young kids,
link |
but I still think that the simple one of the games are simple.
link |
That simple purity makes for like allows your imagination to take over and
link |
thereby creating a more magical experience.
link |
Like now with better and better graphics, it feels like your
link |
imagination doesn't get to, uh, create worlds, which is kind of interesting.
link |
Um, it could be just an old man on a porch, like way waving at kids
link |
these days that have no respect.
link |
But I still think that graphics almost get in the way of the experience.
link |
Yeah, I don't know if the imagination is closed.
link |
I don't yet, but that that's more about games that op like that's more
link |
like Tetris world where they optimally masterfully, like create a fun, short
link |
term dopamine experience versus I'm more referring to like role playing
link |
games where there's like a story you can live in it for months or years.
link |
Um, like, uh, there's an elder scroll series, which is probably my favorite
link |
set of games that was a magical experience.
link |
And that the graphics are terrible.
link |
The characters were all randomly generated, but they're, I don't know.
link |
That's it pulls you in.
link |
It's like an interactive version of an elder scrolls Tolkien world.
link |
And you get to live in it.
link |
It's one of the things that suck about being an adult is there's no, you have
link |
to live in the real world as opposed to the elder scrolls world, you know, whatever
link |
brings you joy, right?
link |
Minecraft is a great example.
link |
You create, like it's not the fancy graphics, but it's the creation of your own worlds.
link |
Yeah, that one is crazy.
link |
You know, one of the pitches for being a parent that people tell me is that you
link |
can like use the excuse of parenting to, to go back into the video game world.
link |
And like, like that's like, you know, father, son, father, daughter time, but
link |
really you just get to play video games with your kids.
link |
So anyway, at that time, did you have any ridiculously ambitious dreams of where as
link |
a creator, you might go as an engineer?
link |
Did you, what, what did you think of yourself as, as an engineer, as a tinker,
link |
or did you want to be like an astronaut or something like that?
link |
You know, I'm tempted to make something up about, you know, robots, uh, engineering
link |
or, you know, mysteries of the universe, but that's not the actual memory that
link |
pops into my mind when you, when you asked me about childhood dreams.
link |
So I'll actually share the, the, the real thing, uh, when I was maybe four or five
link |
years old, I, you know, as we all do, I thought about, you know, what I wanted
link |
to do when I grow up and I had this dream of being a traffic control cop.
link |
Uh, you know, they don't have those today's I think, but you know, back in
link |
the eighties and in Russia, uh, you probably are familiar with that Lex.
link |
They had these, uh, you know, police officers that would stand in the middle
link |
of intersection all day and they would have their like stripe back, black and
link |
white batons that they would use to control the flow of traffic and, you
link |
know, for whatever reasons, I was strangely infatuated with this whole
link |
process and like that, that was my dream.
link |
Uh, that's what I wanted to do when I grew up and, you know, my parents, uh,
link |
both physics profs, by the way, I think were, you know, a little concerned, uh,
link |
with that level of ambition coming from their child.
link |
Uh, uh, you know, that age.
link |
Well, that it's an interesting, I don't know if you can relate,
link |
but I very much love that idea.
link |
I have a OCD nature that I think lends itself very close to the engineering
link |
mindset, which is you want to kind of optimize, you know, solve a problem by
link |
create, creating an automated solution, like a, like a set of rules, that set
link |
of rules you can follow and then thereby make it ultra efficient.
link |
I don't know if that's, it was of that nature.
link |
I certainly have that.
link |
There's like fact, like SimCity and factory building games, all those
link |
kinds of things kind of speak to that engineering mindset, or
link |
did you just like the uniform?
link |
I think it was more of the latter.
link |
I think it was the uniform and the, you know, the, the stripe baton that
link |
made cars go in the right directions.
link |
But I guess, you know, I, it is, I did end up, uh, I guess, uh,
link |
you know, working on the transportation industry one way or another uniform.
link |
No, but that's right.
link |
Maybe, maybe, maybe it was my, you know, deep inner infatuation with the,
link |
you know, traffic control batons that led to this career.
link |
What, uh, when did you, when was the leap from programming to robotics?
link |
That happened later.
link |
That was after grad school, uh, after, and I actually, the most self driving
link |
cars was I think my first real hands on introduction to robotics.
link |
But I never really had that much hands on experience in school and training.
link |
I, you know, worked on applied math and physics.
link |
Then in college, I did more half, uh, abstract computer science.
link |
And it was after grad school that I really got involved in robotics, which
link |
was actually self driving cars.
link |
And, you know, that was a big flip.
link |
What, uh, what grad school?
link |
So I went to grad school in Michigan, and then I did a postdoc at Stanford,
link |
uh, which is, that was the postdoc where I got to play with self driving cars.
link |
So we'll return there.
link |
Let's go back to, uh, to Moscow.
link |
So, uh, you know, for episode 100, I talked to my dad and also I
link |
grew up with my dad, I guess.
link |
Uh, so I had to put up with them for many years and, uh, he, he went to the
link |
FISTIEG or MIPT, it's weird to say in English, cause I've heard all this
link |
in Russian, Moscow Institute of Physics and Technology.
link |
And to me, that was like, I met some super interesting, as a child, I met
link |
some super interesting characters.
link |
It felt to me like the greatest university in the world, the most elite
link |
university in the world, and just the people that I met that came out of there
link |
were like, not only brilliant, but also special humans.
link |
It seems like that place really tested the soul, uh, both like in terms
link |
of technically and like spiritually.
link |
So that could be just the romanticization of that place.
link |
I'm not sure, but so maybe you can speak to it, but is it correct to
link |
say that you spent some time at FISTIEG?
link |
Yeah, that's right.
link |
Uh, I got my bachelor's and master's in physics and math there.
link |
And it's actually interesting cause my, my dad, and actually both my parents,
link |
uh, went there and I think all the stories that I heard, uh, like, just
link |
like you, Alex, uh, growing up about the place and, you know, how interesting
link |
and special and magical it was, I think that was a significant, maybe the
link |
main reason, uh, I wanted to go there, uh, for college, uh, enough so that
link |
I actually went back to Russia from the U S I graduated high school in the U S.
link |
Um, and you went back there.
link |
I went back there.
link |
Yeah, that's exactly the reaction most of my peers in college had.
link |
But, you know, perhaps a little bit stronger that like, you know, point
link |
me out as this crazy kid, were your parents supportive of that?
link |
My games, your previous question, they, uh, they supported me and, you know,
link |
letting me kind of pursue my passions and the things that I was interested in.
link |
That's a bold move.
link |
What was it like there?
link |
It was interesting, you know, definitely fairly hardcore on the fundamentals
link |
of, you know, math and physics and, uh, you know, lots of good memories,
link |
uh, from, you know, from those times.
link |
How'd you get into autonomous vehicles?
link |
I had the great fortune, uh, and great honor to join Stanford's
link |
DARPA urban challenge team.
link |
And, uh, 2006 there, this was a third in the sequence of the DARPA challenges.
link |
There were two grand challenges prior to that.
link |
And then in 2007, they held the DARPA urban challenge.
link |
So, you know, I was doing my, my postdoc I had, I joined the team and, uh, worked
link |
on motion planning, uh, for, you know, that, that competition.
link |
So for people who might not know, I know from, from certain autonomous vehicles is
link |
a funny world in a certain circle of people, everybody knows everything.
link |
And then the certain circle, uh, nobody knows anything in terms of general public.
link |
So it's interesting.
link |
It's, it's a good question of what to talk about, but I do think that the urban
link |
challenge is worth revisiting. It's a fun little challenge.
link |
One that, first of all, like sparked so much, so many incredible minds to focus
link |
on one of the hardest problems of our time in artificial intelligence.
link |
So that's, that's a success from a perspective of a single little challenge.
link |
But can you talk about like, what did the challenge involve?
link |
So were there pedestrians, were there other cars, what was the goal?
link |
Uh, who was on the team?
link |
How long did it take any fun, fun sort of specs?
link |
So the way the challenge was constructed and just a little bit of backgrounding,
link |
as I mentioned, this was the third, uh, competition in that series.
link |
The first year we're at the grand challenge called the grand challenge.
link |
The goal there was to just drive in a completely static environment.
link |
You know, you had to drive in a desert, uh, that was very successful.
link |
So then DARPA followed with what they called the urban challenge, where the
link |
goal was to have, you know, build vehicles that could operate in more dynamic
link |
environments and, you know, share them with other vehicles.
link |
There were no pedestrians there, but what DARPA did is they took over
link |
an abandoned air force base.
link |
Uh, and it was kind of like a little fake city that they built out there.
link |
And they had a bunch of, uh, robots, uh, you know, cars, uh, that were
link |
autonomous, uh, in there all at the same time.
link |
Uh, mixed in with other vehicles driven by professional, uh, drivers and each
link |
car, uh, had a mission and so there's a crude map that they received, uh,
link |
beginning and they had a mission and go here and then there and over here.
link |
Um, and they kind of all were sharing this environment at the same time.
link |
They had to interact with each other.
link |
They had to interact with the human drivers.
link |
There's this very first, very rudimentary, um, version of, uh,
link |
self driving car that, you know, could operate, uh, and, uh, in a, in an
link |
environment, you know, shared with other dynamic actors that, as you said,
link |
you know, really, you know, many ways, you know, kickstarted this whole industry.
link |
So who was on the team and how'd you do?
link |
Uh, I came in second.
link |
Uh, perhaps that was my contribution to the team.
link |
I think the Stanford team came in first in the DARPA challenge.
link |
Uh, but then I joined the team and, you know, you were the one with the
link |
bug in the code, I mean, do you have sort of memories of some
link |
particularly challenging things or, you know, one of the cool things,
link |
it's not, you know, this isn't a product, this isn't the thing that, uh, you know,
link |
it there's, you have a little bit more freedom to experiment so you can take
link |
risks and there's, uh, so you can make mistakes.
link |
Uh, so is there interesting mistakes?
link |
Is there interesting challenges that stand out to you as some, like, taught
link |
you, um, a good technical lesson or a good philosophical lesson from that time?
link |
Uh, you know, definitely, definitely a very memorable time, not really
link |
challenged, but like one of the most vivid memories that I have from the time.
link |
And I think that was actually one of the days that really got me hooked, uh, on
link |
this whole field was, uh, the first time I got to run my software and I got to
link |
software on the car and, uh, I was working on a part of our planning algorithm,
link |
uh, that had to navigate in parking lots.
link |
So it was something that, you know, called free space emotion planning.
link |
So the very first version of that, uh, was, you know, we tried on the car, it
link |
was on Stanford's campus, uh, in the middle of the night and you had this
link |
little course constructed with cones, uh, in the middle of a parking lot.
link |
So we're there in like 3 am, you know, by the time we got the code to, you
link |
know, uh, uh, you know, compile and turn over, uh, and, you know, it drove, I
link |
could actually did something quite reasonable and, you know, it was of
link |
course very buggy at the time and had all kinds of problems, but it was pretty
link |
I remember going back and, you know, later at night and trying to fall
link |
asleep and just, you know, being unable to fall asleep for the rest of the
link |
night, uh, just my mind was blown.
link |
Just like, and that, that, that's what I've been doing ever since for more
link |
than a decade, uh, in terms of challenges and, uh, you know, interesting
link |
memories, like on the day of the competition, uh, it was pretty nerve
link |
Uh, I remember standing there with Mike Montemarillo, who was, uh, the
link |
software lead and wrote most of the code.
link |
I think I did one little part of the planner, Mike, you know, incredibly
link |
that, you know, pretty much the rest of it, uh, with, with, you know, a bunch
link |
of other incredible people, but I remember standing on the day of the
link |
competition, uh, you know, watching the car, you know, with Mike and cars
link |
are completely empty, right?
link |
They're all there lined up in the beginning of the race and then, you
link |
know, DARPA sends them, you know, on their mission one by one.
link |
So then leave and Mike, you just, they had these sirens, they all had
link |
their different silence silence, right?
link |
Each siren had its own personality, if you will.
link |
So, you know, off they go and you don't see them.
link |
You just kind of, and then every once in a while they come a little bit
link |
closer to where the audience is and you can kind of hear, you know, the
link |
sound of your car and then, you know, it seems to be moving along.
link |
So that, you know, gives you hope.
link |
And then, you know, it goes away and you can't hear it for too long.
link |
You start getting anxious, right?
link |
So it's a little bit like, you know, sending your kids to college and like,
link |
you know, kind of you invested in them.
link |
You hope you, you, you, you, you, you, you build it properly, but like,
link |
it's still, uh, anxiety inducing.
link |
Uh, so that was, uh, an incredibly, uh, fun, uh, few days in terms of, you
link |
know, bugs, as you mentioned, you know, one that that was my bug that caused
link |
us the loss of the first place, uh, is still a debate that, you know,
link |
occasionally have with people on the CMU team, CMU came first, I should
link |
mention, uh, that you haven't heard of them, but yeah, it's something, you
link |
know, it's a small school, but it's, it's, it's, you know, really a glitch
link |
that, you know, they happen to succeed at something robotics related.
link |
Very scenic though.
link |
So most people go there for the scenery.
link |
Um, yeah, it's a beautiful campus.
link |
I'm like, unlike Stanford.
link |
So for people, yeah, that's true.
link |
Unlike Stanford, for people who don't know, CMU is one of the great robotics
link |
and sort of artificial intelligence universities in the world, CMU, Carnegie
link |
Mellon university, okay, sorry, go ahead.
link |
So in the part that I contributed to, which was navigating parking lots and
link |
the way that part of the mission work is, uh, you in a parking lot, you
link |
would get from DARPA an outline of the map.
link |
You basically get this, you know, giant polygon that defined the
link |
perimeter of the parking lot, uh, and there would be an entrance and, you
link |
know, so maybe multiple entrances or access to it, and then you would get a
link |
goal, uh, within that open space, uh, X, Y, you know, heading where the car had
link |
to park and had no information about the optical, so obstacles that the car might
link |
So it had to navigate a kind of completely free space, uh, from the
link |
entrance to the parking lot into that parking space.
link |
And then, uh, once parked there, it had to, uh, exit the parking lot, you know,
link |
while of course, I'm counting and reasoning about all the obstacles that
link |
it encounters in real time.
link |
So, uh, Our interpretation, or at least my interpretation of the rules was that
link |
you had to reverse out of the parking spot.
link |
And that's what our cars did.
link |
Even if there's no obstacle in front, that's not what CMU's car did.
link |
And it just kind of drove right through.
link |
So there's still a debate.
link |
And of course, you know, as you stop and then reverse out and go out the
link |
different way that costs you some time.
link |
And so there's still a debate whether, you know, it was my poor implementation
link |
that cost us extra time or whether it was, you know, CMU, uh, violating an
link |
important rule of the competition.
link |
And, you know, I have my own, uh, opinion here in terms of other bugs.
link |
And like, uh, I, I have to apologize to Mike Montemarila, uh, for sharing this
link |
on air, but it is actually, uh, one of the more memorable ones.
link |
Uh, and it's something that's kind of become a bit of, uh, a metaphor and
link |
a label in the industry, uh, since then, I think, you know, at least in some
link |
circles, it's called the victory circle or victory lap.
link |
Um, and, uh, uh, our cars did that.
link |
So in one of the missions in the urban challenge, in one of the courses, uh,
link |
there was this big oval, right by the start and finish of the race.
link |
So the ARPA had a lot of the missions would finish kind of in that same location.
link |
Uh, and it was pretty cool because you could see the cars come by, you know,
link |
kind of finished that part leg of the trip, that leg of the mission, and then,
link |
you know, go on and finish the rest of it.
link |
Uh, and other vehicles would, you know, come hit their waypoint, uh, and, you
link |
know, exit the oval and off they would go.
link |
Our car on the hand, which hit the checkpoint, and then it would do an extra
link |
lap around the oval and only then, you know, uh, leave and go on its merry way.
link |
So over the course of the full day, it accumulated, uh, uh,
link |
some extra time and the problem was that we had a bug where it wouldn't, you know,
link |
start reasoning about the next waypoint and plan a route to get to that next
link |
point until it hit a previous one.
link |
And in that particular case, by the time you hit the, that, that one, it was too
link |
late for us to consider the next one and kind of make a lane change.
link |
So at every time we would do like an extra lap.
link |
So, you know, and that's the Stanford victory lap.
link |
Oh, that's there's, I feel like there's something philosophically
link |
profound in there somehow, but, uh, I mean, ultimately everybody is
link |
a winner in that kind of competition.
link |
And it led to sort of famously to the creation of, um, Google self driving
link |
car project and now Waymo.
link |
So can we, uh, give an overview of how is Waymo born?
link |
How's the Google self driving car project born?
link |
What's the, what is the mission?
link |
What is it is the engineering kind of, uh, set of milestones that
link |
it seeks to accomplish, there's a lot of questions in there.
link |
Uh, yeah, uh, I don't know, kind of the DARPA urban challenge and the DARPA
link |
and previous DARPA grand challenges, uh, kind of led, I think to a very large
link |
degree to that next step and then, you know, Larry and Sergey, um, uh, Larry
link |
Page and Sergey Brin, uh, uh, Google founders course, uh, I saw that
link |
competition and believed in the technology.
link |
So, you know, the Google self driving car project was born, you know, at that time.
link |
And we started in 2009, it was a pretty small group of us, about a dozen people,
link |
uh, who came together, uh, to, to work on this project at Google.
link |
At that time we saw an incredible early result in the DARPA urban challenge.
link |
I think we're all incredibly excited, uh, about where we got to and we believed
link |
in the future of the technology, but we still had a very, you know,
link |
very, you know, rudimentary understanding of the problem space.
link |
So the first goal of this project in 2009 was to really better
link |
understand what we're up against.
link |
Uh, and, you know, with that goal in mind, when we started the project, we created a
link |
few milestones for ourselves, uh, that.
link |
Maximized learnings.
link |
Well, the two milestones were, you know, uh, one was to drive a hundred thousand
link |
miles in autonomous mode, which was at that time, you know, orders of magnitude
link |
that, uh, more than anybody has ever done.
link |
And the second milestone was to drive 10 routes, uh, each one was a hundred miles
link |
long, uh, and there were specifically chosen to become extra spicy and extra
link |
complicated and sample the full complexity of the, that, that, uh, domain.
link |
Um, uh, and you had to drive each one from beginning to end with no intervention,
link |
no human intervention.
link |
So you would get to the beginning of the course, uh, you would press the button
link |
that would engage in autonomy and you had to go for a hundred miles, you know,
link |
beginning to end, uh, with no interventions.
link |
Um, and it sampled again, the full complexity of driving conditions.
link |
Some, uh, were on freeways.
link |
We had one route that went all through all the freeways and all
link |
the bridges in the Bay area.
link |
You know, we had, uh, some that went around Lake Tahoe and kind of mountains,
link |
We had some that drove through dense urban, um, environments like in downtown
link |
Palo Alto and through San Francisco.
link |
So it was incredibly, uh, interesting, uh, to work on.
link |
And it, uh, it took us just under two years, uh, about a year and a half,
link |
a little bit more to finish both of these milestones.
link |
And in that process, uh, you know, it was an incredible amount of fun,
link |
probably the most fun I had in my professional career.
link |
And you're just learning so much.
link |
You are, you know, the goal here is to learn and prototype.
link |
You're not yet starting to build a production system, right?
link |
So you just, you were, you know, this is when you're kind of working 24 seven
link |
and you're hacking things together.
link |
And you also don't know how hard this is.
link |
I mean, that's the point.
link |
Like, so, I mean, that's an ambitious, if I put myself in that mindset, even
link |
still, that's a really ambitious set of goals.
link |
Like just those two picking, picking 10 different, difficult, spicy challenges.
link |
And then having zero interventions.
link |
So like not saying gradually we're going to like, you know, over a period of 10
link |
years, we're going to have a bunch of routes and gradually reduce the number
link |
of interventions, you know, that literally says like, by as soon as
link |
possible, we want to have zero and on hard roads.
link |
So like, to me, if I was facing that, it's unclear that whether that takes
link |
two years or whether that takes 20 years.
link |
I mean, it took us under two.
link |
I guess that that speaks to a really big difference between doing something
link |
once and having a prototype where you are going after, you know, learning
link |
about the problem versus how you go about engineering a product that, you
link |
know, where you look at, you know, you do properly do evaluation, you look
link |
at metrics, you drive down and you're confident that you can do that.
link |
And I guess that's the, you know, why it took a dozen people, you know, 16
link |
months or a little bit more than that back in 2009 and 2010 with the
link |
technology of, you know, the more than a decade ago that amount of time to
link |
achieve that milestone of, you know, 10 routes, a hundred miles each and no
link |
interventions, and, you know, it took us a little bit longer to get to, you
link |
know, a full driverless product that customers use.
link |
That's another really important moment.
link |
Is there some memories of technical lessons or just one, like, what did you
link |
learn about the problem of driving from that experience?
link |
I mean, we can, we can now talk about like what you learned from modern day
link |
Waymo, but I feel like you may have learned some profound things in those
link |
early days, even more so because it feels like what Waymo is now is to trying
link |
to, you know, how to do scale, how to make sure you create a product, how to
link |
make sure it's like safety and all those things, which is all fascinating
link |
challenges, but like you were facing the more fundamental philosophical
link |
problem of driving in those early days.
link |
Like what the hell is driving as an autonomous, or maybe I'm again
link |
romanticizing it, but is it, is there, is there some valuable lessons you
link |
picked up over there at those two years?
link |
The most important one is probably that we believe that it's doable and we've
link |
gotten far enough into the problem that, you know, we had a, I think only a
link |
glimpse of the true complexity of the, that the domain, you know, it's a
link |
little bit like, you know, climbing a mountain where you kind of, you know,
link |
see the next peak and you think that's kind of the summit, but then you get
link |
to that and you kind of see that, that this is just the start of the journey.
link |
But we've tried, we've sampled enough of the problem space and we've made
link |
enough rapid success, even, you know, with technology of 2009, 2010, that
link |
it gave us confidence to then, you know, pursue this as a real product.
link |
So the next step, you mentioned the milestones that you had in the, in those
link |
two years, what are the next milestones that then led to the creation of Waymo
link |
Yeah, we had a, it was a really interesting journey and, you know, Waymo
link |
came a little bit later, then, you know, we completed those milestones in 2010.
link |
That was the pivot when we decided to focus on actually building a product
link |
using this technology.
link |
The initial couple of years after that, we were focused on a freeway, you
link |
know, what you would call a driver assist, maybe, you know, an L3 driver
link |
Then around 2013, we've learned enough about the space and thought more deeply
link |
about, you know, the product that we wanted to build, that we pivoted, we
link |
pivoted towards this vision of building a driver and deploying it fully driverless
link |
vehicles without a person.
link |
And that that's the path that we've been on since then.
link |
And very, it was exactly the right decision for us.
link |
So there was a moment where you're also considered like, what is the right
link |
What is the right role of automation in the, in the task of driving?
link |
There's still, it wasn't from the early days, obviously you want to go fully
link |
From the early days, it was not.
link |
I think it was in 20, around 2013, maybe that we've, that became very clear and
link |
we made that pivot and also became very clear and that it's either the way you
link |
go building a driver assist system is, you know, fundamentally different from
link |
how you go building a fully driverless vehicle.
link |
So, you know, we've pivoted towards the ladder and that's what we've been
link |
working on ever since.
link |
And so that was around 2013, then there's sequence of really meaningful for us
link |
really important defining milestones since then.
link |
And in 2015, we had our first, actually the world's first fully driverless
link |
trade on public roads.
link |
It was in a custom built vehicle that we had.
link |
I must've seen those.
link |
We called them the Firefly, that, you know, funny looking marshmallow looking
link |
And we put a passenger, his name was Steve Mann, you know, great friend of
link |
our project from the early days, the man happens to be blind.
link |
So we put them in that vehicle.
link |
The car had no steering wheel, no pedals.
link |
It was an uncontrolled environment.
link |
You know, no, you know, lead or chase cars, no police escorts.
link |
And, you know, we did that trip a few times in Austin, Texas.
link |
So that was a really big milestone.
link |
But that was in Austin.
link |
And, you know, we only, but at that time we're only, it took a tremendous
link |
amount of engineering.
link |
It took a tremendous amount of validation to get to that point.
link |
But, you know, we only did it a few times.
link |
It was a fixed route.
link |
It was not kind of a controlled environment, but it was a fixed route.
link |
And we only did a few times.
link |
Then in 2016, end of 2016, beginning of 2017 is when we founded Waymo, the
link |
That's when we kind of, that was the next phase of the project where I
link |
wanted, we believed in kind of the commercial vision of this technology.
link |
And it made sense to create an independent entity, you know, within
link |
that alphabet umbrella to pursue this product at scale.
link |
Beyond that in 2017, later in 2017 was another really huge step for us.
link |
Really big milestone where we started, I think it was October of 2017 where
link |
when we started regular driverless operations on public roads, that first
link |
day of operations, we drove in one day.
link |
And that first day, a hundred miles and driverless fashion.
link |
And then we've now the most, the most important thing about that milestone
link |
was not that, you know, a hundred miles in one day, but that it was the
link |
start of kind of regular ongoing driverless operations.
link |
And when you say driverless, it means no driver.
link |
That's exactly right.
link |
So on that first day, we actually hit a mix and in some, we didn't want
link |
to like, you know, be on YouTube and Twitter that same day.
link |
So in, in many of the rides we had somebody in the driver's seat, but
link |
they could not disengage like the car, not disengage, but actually on that
link |
first day, some of the miles were driven and just completely empty driver's seat.
link |
And this is the key distinction that I think people don't realize it's, you
link |
know, that oftentimes when you talk about autonomous vehicles, you're, there's
link |
often a driver in the seat that's ready to to take over what's called a safety
link |
driver and then Waymo is really one of the only companies at least that I'm
link |
aware of, or at least as like boldly and carefully and all, and all of that is
link |
actually has cases.
link |
And now we'll talk about more and more where there's literally no driver.
link |
So that's another, the interesting case of where the driver's not supposed
link |
to disengage, that's like a nice middle ground, they're still there, but
link |
they're not supposed to disengage, but really there's the case when there's
link |
no, okay, there's something magical about there being nobody in the driver's seat.
link |
Like, just like to me, you mentioned the first time you wrote some code for free
link |
space navigation of the parking lot, that was like a magical moment to me, just
link |
sort of as an observer of robots, the first magical moment is seeing an
link |
autonomous vehicle turn, like make a left turn, like apply sufficient torque to
link |
the steering wheel to where it, like, there's a lot of rotation and for some
link |
reason, and there's nobody in the driver's seat, for some reason that
link |
communicates that here's a being with power that makes a decision.
link |
There's something about like the steering wheel, cause we perhaps romanticize
link |
the notion of the steering wheel, it's so essential to our conception, our 20th
link |
century conception of a car and it turning the steering wheel with nobody
link |
in driver's seat, that to me, I think maybe to others, it's really powerful.
link |
Like this thing is in control and then there's this leap of trust that you give.
link |
Like I'm going to put my life in the hands of this thing that's in control.
link |
So in that sense, when there's no, but no driver in the driver's seat, that's a
link |
magical moment for robots.
link |
So I'm, I've gotten a chance to last year to take a ride in a, in a
link |
way more vehicle and that, that was the magical moment. There's like nobody in
link |
the driver's seat. It's, it's like the little details. You would think it
link |
doesn't matter whether there's a driver or not, but like if there's no driver
link |
and the steering wheel is turning on its own, I don't know. That's magical.
link |
It's absolutely magical. I, I have taken many of these rides and like completely
link |
empty car, no human in the car pulls up, you know, you call it on your cell phone.
link |
It pulls up, you get in, it takes you on its way. There's nobody in the car, but
link |
you, right? That's something called, you know, fully driverless, you know, our
link |
writer only mode of operation. Yeah. It, it is magical. It is, you know,
link |
transformative. This is what we hear from our writers. It kind of really
link |
changes your experience. And not like that, that really is what unlocks the
link |
real potential of this technology. But, you know, coming back to our journey,
link |
you know, that was 2017 when we started, you know, truly driverless operations.
link |
Then in 2018, we've launched our public commercial service that we called
link |
Waymo One in Phoenix. In 2019, we started offering truly driverless writer
link |
only rides to our early rider population of users. And then, you know, 2020 has
link |
also been a pretty interesting year. One of the first ones, less about
link |
technology, but more about the maturing and the growth of Waymo as a company.
link |
We raised our first round of external financing this year, you know, we were
link |
part of Alphabet. So obviously we have access to, you know, significant resources
link |
but as kind of on the journey of Waymo maturing as a company, it made sense
link |
for us to, you know, partially go externally in this round. So, you know,
link |
we're raised about $3.2 billion from that round. We've also started putting
link |
our fifth generation of our driver, our hardware, that is on the new vehicle,
link |
but it's also a qualitatively different set of self driving hardware.
link |
That is now on the JLR pace. So that was a very important step for us.
link |
Hardware specs, fifth generation. I think it'd be fun to maybe, I apologize if
link |
I'm interrupting, but maybe talk about maybe the generations with a focus on
link |
what we're talking about on the fifth generation in terms of hardware specs,
link |
like what's on this car.
link |
Sure. So we separated out, you know, the actual car that we are driving from
link |
the self driving hardware we put on it. Right now we have, so this is, as I
link |
mentioned, the fifth generation, you know, we've gone through, we started,
link |
you know, building our own hardware, you know, many, many years ago. And
link |
that, you know, Firefly vehicle also had the hardware suite that was mostly
link |
designed, engineered, and built in house. Lighters are one of the more important
link |
components that we design and build from the ground up. So on the fifth
link |
generation of our drivers of our self driving hardware that we're switching
link |
to right now, we have, as with previous generations, in terms of sensing,
link |
we have lighters, cameras, and radars, and we have a pretty beefy computer
link |
that processes all that information and makes decisions in real time on
link |
board the car. So in all of the, and it's really a qualitative jump forward
link |
in terms of the capabilities and the various parameters and the specs of
link |
the hardware compared to what we had before and compared to what you can
link |
kind of get off the shelf in the market today.
link |
Meaning from fifth to fourth or from fifth to first?
link |
Definitely from first to fifth, but also from the fourth.
link |
That was the world's dumbest question.
link |
Definitely from fourth to fifth, as well as the last step is a big step forward.
link |
So everything's in house. So like LIDAR is built in house and cameras are
link |
You know, it's different. We work with partners and there's some components
link |
that we get from our manufacturing and supply chain partners. What exactly
link |
is in house is a bit different. We do a lot of custom design on all of
link |
our sensing modalities, lighters, radars, cameras, you know, exactly.
link |
There's lighters are almost exclusively in house and some of the
link |
technologies that we have, some of the fundamental technologies there
link |
are completely unique to Waymo. That is also largely true about radars
link |
and cameras. It's a little bit more of a mix in terms of what we do
link |
ourselves versus what we get from partners.
link |
Is there something super sexy about the computer that you can mention
link |
that's not top secret? Like for people who enjoy computers for, I
link |
mean, there's a lot of machine learning involved, but there's a lot
link |
of just basic compute. You have to probably do a lot of signal
link |
processing on all the different sensors. You have to integrate everything
link |
has to be in real time. There's probably some kind of redundancy
link |
type of situation. Is there something interesting you can say about
link |
the computer for the people who love hardware? It does have all of
link |
the characteristics, all the properties that you just mentioned.
link |
Redundancy, very beefy compute for general processing, as well as
link |
inference and ML models. It is some of the more sensitive stuff that
link |
I don't want to get into for IP reasons, but it can be shared a
link |
little bit in terms of the specs of the sensors that we have on the
link |
car. We actually shared some videos of what our
link |
lighters see in the world. We have 29 cameras. We have five lighters.
link |
We have six radars on these vehicles, and you can get a feel for
link |
the amount of data that they're producing. That all has to be
link |
processed in real time to do perception, to do complex
link |
reasoning. That kind of gives you some idea of how beefy those computers
link |
are, but I don't want to get into specifics of exactly how we build
link |
them. Okay, well, let me try some more questions that you can get
link |
into the specifics of, like GPU wise. Is that something you can get
link |
into? I know that Google works with GPUs and so on. I mean, for
link |
machine learning folks, it's kind of interesting. Or is there no...
link |
How do I ask it? I've been talking to people in the government about
link |
UFOs and they don't answer any questions. So this is how I feel
link |
right now asking about GPUs. But is there something interesting that
link |
you could reveal? Or is it just... Or leave it up to our
link |
imagination, some of the compute. Is there any, I guess, is there any
link |
fun trickery? Like I talked to Chris Latner for a second time and he
link |
was a key person about GPUs, and there's a lot of fun stuff going
link |
on in Google in terms of hardware that optimizes for machine
link |
learning. Is there something you can reveal in terms of how much,
link |
you mentioned customization, how much customization there is for
link |
hardware for machine learning purposes? I'm going to be like that
link |
government person who bought UFOs. I guess I will say that it's
link |
really... Compute is really important. We have very data hungry
link |
and compute hungry ML models all over our stack. And this is where
link |
both being part of Alphabet, as well as designing our own sensors
link |
and the entire hardware suite together, where on one hand you
link |
get access to really rich raw sensor data that you can pipe
link |
from your sensors into your compute platform and build like
link |
build the whole pipe from sensor raw sensor data to the big
link |
compute as then have the massive compute to process all that
link |
data. And this is where we're finding that having a lot of
link |
control of that hardware part of the stack is really
link |
advantageous. One of the fascinating magical places to me
link |
again, might not be able to speak to the details, but it is
link |
the other compute, which is like, we're just talking about a
link |
single car, but the driving experience is a source of a lot
link |
of fascinating data. And you have a huge amount of data
link |
coming in on the car and the infrastructure of storing some
link |
of that data to then train or to analyze or so on. That's a
link |
fascinating piece of it that I understand a single car. I
link |
don't understand how you pull it all together in a nice way.
link |
Is that something that you could speak to in terms of the
link |
challenges of seeing the network of cars and then
link |
bringing the data back and analyzing things that like edge
link |
cases of driving, be able to learn on them to improve the
link |
system to see where things went wrong, where things went right
link |
and analyze all that kind of stuff. Is there something
link |
interesting there from an engineering perspective?
link |
Oh, there's an incredible amount of really interesting
link |
work that's happening there, both in the real time operation
link |
of the fleet of cars and the information that they exchange
link |
with each other in real time to make better decisions as well
link |
as on the kind of the off board component where you have to
link |
deal with massive amounts of data for training your ML
link |
models, evaluating the ML models for simulating the entire
link |
system and for evaluating your entire system. And this is
link |
where being part of Alphabet has once again been tremendously
link |
advantageous because we consume an incredible amount of
link |
compute for ML infrastructure. We build a lot of custom
link |
frameworks to get good at data mining, finding the
link |
interesting edge cases for training and for evaluation of
link |
the system for both training and evaluating some components
link |
and your sub parts of the system and various ML models,
link |
as well as the evaluating the entire system and simulation.
link |
Okay. That first piece that you mentioned that cars
link |
communicating to each other, essentially, I mean, through
link |
perhaps through a centralized point, but what that's
link |
fascinating too, how much does that help you? Like if you
link |
imagine, you know, right now the number of way more vehicles
link |
is whatever X. I don't know if you can talk to what that
link |
number is, but it's not in the hundreds of millions yet. And
link |
imagine if the whole world is way more vehicles, like that
link |
changes potentially the power of connectivity. Like the more
link |
cars you have, I guess, actually, if you look at
link |
Phoenix, cause there's enough vehicles, there's enough, when
link |
there's like some level of density, you can start to
link |
probably do some really interesting stuff with the fact
link |
that cars can negotiate, can be, can communicate with each
link |
other and thereby make decisions. Is there something
link |
interesting there that you can talk to about like, how does
link |
that help with the driving problem from, as compared to
link |
just a single car solving the driving problem by itself?
link |
Yeah, it's a spectrum. I first and say that, you know, it's,
link |
it helps and it helps in various ways, but it's not required
link |
right now with the way we build our system, like each cars can
link |
operate independently. They can operate with no connectivity.
link |
So I think it is important that, you know, you have a fully
link |
autonomous, fully capable driver that, you know, computerized
link |
driver that each car has. Then, you know, they do share
link |
information and they share information in real time. It
link |
really, really helps. So the way we do this today is, you know,
link |
whenever one car encounters something interesting in the
link |
world, whether it might be an accident or a new construction
link |
zone, that information immediately gets, you know,
link |
uploaded over the air and it's propagated to the rest of the
link |
fleet. So, and that's kind of how we think about maps as
link |
priors in terms of the knowledge of our drivers, of our fleet of
link |
drivers that is distributed across the fleet and it's
link |
updated in real time. So that's one use case. And
link |
you know, you can imagine as the, you know, the density of
link |
these vehicles go up, that they can exchange more information
link |
in terms of what they're planning to do and start
link |
influencing how they interact with each other, as well as,
link |
you know, potentially sharing some observations, right, to
link |
help with, you know, if you have enough density of these
link |
vehicles where, you know, one car might be seeing something
link |
that another is relevant to another car that is very
link |
dynamic. You know, it's not part of kind of your updating
link |
your static prior of the map of the world, but it's more of a
link |
dynamic information that could be relevant to the decisions
link |
that another car is making real time. So you can see them
link |
exchanging that information and you can build on that. But
link |
again, I see that as an advantage, but it's not a
link |
requirement. So what about the human in the loop? So when I
link |
got a chance to drive with a ride in a Waymo, you know,
link |
there's customer service. So like there is somebody that's
link |
able to dynamically like tune in and help you out. What role
link |
does the human play in that picture? That's a fascinating
link |
like, you know, the idea of teleoperation, be able to
link |
remotely control a vehicle. So here, what we're talking
link |
about is like, like frictionless, like a human being
link |
able to in a in a frictionless way, sort of help you out. I
link |
don't know if they're able to actually control the vehicle.
link |
Is that something you could talk to? Yes. Okay. To be clear,
link |
we don't do teleporation. I kind of believe in
link |
teleporation for various reasons. That's not what we
link |
have in our cars. We do, as you mentioned, have, you know,
link |
version of, you know, customer support. You know, we call it
link |
life health. In fact, we find it that it's very important for
link |
our ride experience, especially if it's your first trip, you've
link |
never been in a fully driverless ride or only way more
link |
vehicle you get in, there's nobody there. And so you can
link |
imagine having all kinds of, you know, questions in your head,
link |
like how this thing works. So we've put a lot of thought into
link |
kind of guiding our, our writers or customers through that
link |
experience, especially for the first time they get some
link |
information on the phone. If the fully driverless vehicle is
link |
used to service their trip, when you get into the car, we
link |
have an in car, you know, screen and audio that kind of guides
link |
them and explains what to expect. They also have a button
link |
that they can push that will connect them to, you know, a
link |
real life human being that they can talk to, right, about this
link |
whole process. So that's one aspect of it. There is, you
link |
know, I should mention that there is another function that
link |
humans provide to our cars, but it's not teleoperation. You can
link |
think of it a little bit more like, you know, fleet
link |
assistance, kind of like, you know, traffic control that you
link |
have, where our cars, again, they're responsible on their own
link |
for making all of the decisions, all of the driving decisions
link |
that don't require connectivity. They, you know,
link |
anything that is safety or latency critical is done, you
link |
know, purely autonomously by onboard, our onboard system.
link |
But there are situations where, you know, if connectivity is
link |
available, when a car encounters a particularly challenging
link |
situation, you can imagine like a super hairy scene of an
link |
accident, the cars will do their best, they will recognize that
link |
it's an off nominal situation, they will do their best to come
link |
up with the right interpretation, the best course
link |
of action in that scenario. But if connectivity is available,
link |
they can ask for confirmation from, you know, human
link |
assistant to kind of confirm those actions and perhaps
link |
provide a little bit of kind of contextual information and
link |
guidance. So October 8th was when you're talking about the
link |
was Waymo launched the fully self, the public version of
link |
its fully driverless, that's the right term, I think, service
link |
in Phoenix. Is that October 8th? That's right. It was the
link |
introduction of fully driverless, right, our only
link |
vehicles into our public Waymo One service. Okay, so that's
link |
that's amazing. So it's like anybody can get into Waymo in
link |
Phoenix. So we previously had early people in our early
link |
rider program, taking fully driverless rides in Phoenix.
link |
And just this a little while ago, we opened on October 8th,
link |
we opened that mode of operation to the public. So I
link |
can download the app and go on a ride. There's a lot more
link |
demand right now for that service. And then we have
link |
capacity. So we're kind of managing that. But that's
link |
exactly the way to describe it. Yeah, that's interesting. So
link |
there's more demand than you can handle. Like what has been
link |
reception so far? I mean, okay, so this is a product,
link |
right? That's a whole nother discussion of like how
link |
compelling of a product it is. Great. But it's also like one
link |
of the most kind of transformational technologies of
link |
the 21st century. So it's also like a tourist attraction.
link |
Like it's fun to, you know, to be a part of it. So it'd be
link |
interesting to see like, what do people say? What do people,
link |
what have been the feedback so far? You know, still early
link |
days, but so far, the feedback has been incredible, incredibly
link |
positive. They, you know, we asked them for feedback during
link |
the ride, we asked them for feedback after the ride as part
link |
of their trip. We asked them some questions, we asked them
link |
to rate the performance of our driver. Most by far, you know,
link |
most of our drivers give us five stars in our app, which is
link |
absolutely great to see. And you know, that's and we're
link |
they're also giving us feedback on you know, things we can
link |
improve. And you know, that's that's one of the main reasons
link |
we're doing this as Phoenix and you know, over the last couple
link |
of years, and every day today, we are just learning a
link |
tremendous amount of new stuff from our users. There's there's
link |
no substitute for actually doing the real thing, actually
link |
having a fully driverless product out there in the field
link |
with, you know, users that are actually paying us money to
link |
get from point A to point B. So this is a legitimate like,
link |
there's a paid service. That's right. And the idea is you use
link |
the app to go from point A to point B. And then what what are
link |
the A's? What are the what's the freedom of the of the starting
link |
and ending places? It's an area of geography where that
link |
service is enabled. It's a decent size of geography of
link |
territory. It's actually larger than the size of San Francisco.
link |
And you know, within that, you have full freedom of, you know,
link |
selecting where you want to go. You know, of course, there's
link |
some and you on your app, you get a map, you tell the car
link |
where you want to be picked up, where you want the car to pull
link |
over and pick you up. And then you tell it where you want to
link |
be dropped off. All right. And of course, there are some
link |
exclusions, right? You want to be you know, you were in terms
link |
of where the car is allowed to pull over, right? So that you
link |
can do. But you know, besides that, it's amazing. It's not
link |
like a fixed just would be very I guess. I don't know. Maybe
link |
that's what's the question behind your question. But it's
link |
not a, you know, preset set of yes, I guess. So within the
link |
geographic constraints with that within that area anywhere
link |
else, it can be you can be picked up and dropped off
link |
anywhere. That's right. And you know, people use them on like
link |
all kinds of trips. They we have and we have an incredible
link |
spectrum of riders. We I think the youngest actually have car
link |
seats them and we have, you know, people taking their kids
link |
and rides. I think the youngest riders we had on cars are, you
link |
know, one or two years old, you know, and the full spectrum of
link |
use cases people you can take them to, you know, schools to,
link |
you know, go grocery shopping, to restaurants, to bars, you
link |
know, run errands, you know, go shopping, etc, etc. You can go
link |
to your office, right? Like the full spectrum of use cases,
link |
and people are going to use them in their daily lives to get
link |
around. And we see all kinds of really interesting use cases
link |
and that that that's providing us incredibly valuable
link |
experience that we then, you know, use to improve our
link |
product. So as somebody who's been on done a few long rants
link |
with Joe Rogan and others about the toxicity of the internet
link |
and the comments and the negativity in the comments, I'm
link |
fascinated by feedback. I believe that most people are
link |
good and kind and intelligent and can provide, like, even in
link |
disagreement, really fascinating ideas. So on a product
link |
side, it's fascinating to me, like, how do you get the richest
link |
possible user feedback, like, to improve? What's, what are the
link |
channels that you use to measure? Because, like, you're
link |
no longer, that's one of the magical things about autonomous
link |
vehicles is it's not like it's frictionless interaction with
link |
the human. So like, you don't get to, you know, it's just
link |
giving a ride. So like, how do you get feedback from people
link |
to in order to improve?
link |
Yeah, great question, various mechanisms. So as part of the
link |
normal flow, we ask people for feedback, they as the car is
link |
driving around, we have on the phone and in the car, and we
link |
have a touchscreen in the car, you can actually click some
link |
buttons and provide real time feedback on how the car is
link |
doing, and how the car is handling a particular situation,
link |
you know, both positive and negative. So that's one
link |
channel, we have, as we discussed, customer support or
link |
life help, where, you know, if a customer wants to, has a
link |
question, or he has some sort of concern, they can talk to a
link |
person in real time. So that that is another mechanism that
link |
gives us feedback. At the end of a trip, you know, we also ask
link |
them how things went, they give us comments, and you know, star
link |
rating. And you know, if it's, we also, you know, ask them to
link |
explain what you know, one, well, and you know, what could
link |
be improved. And we have our writers providing very rich
link |
feedback, they're a lot, a large fraction is very passionate,
link |
very excited about this technology. So we get really
link |
good feedback. We also run UXR studies, right, you know,
link |
specific and that are kind of more, you know, go more in
link |
depth. And we will run both kind of lateral and longitudinal
link |
studies, where we have deeper engagement with our customers,
link |
you know, we have our user experience research team,
link |
tracking over time, that's things about longitudinal is
link |
cool. That's that's exactly right. And you know, that's
link |
another really valuable feedback, source of feedback.
link |
And we're just covering a tremendous amount, right?
link |
People go grocery shopping, and they like want to load, you
link |
know, 20 bags of groceries in our cars and like that, that's
link |
one workflow that you maybe don't think about, you know,
link |
getting just right when you're building the driverless
link |
product. I have people like, you know, who bike as part of
link |
their trip. So they, you know, bike somewhere, then they get
link |
on our cars, they take apart their bike, they load into our
link |
vehicle, then go and that's, you know, how they, you know,
link |
where we want to pull over and how that, you know, get in and
link |
get out process works, provides very useful feedback in terms
link |
of what makes a good pickup and drop off location, we get
link |
really valuable feedback. And in fact, we had to do some really
link |
interesting work with high definition maps, and thinking
link |
about walking directions. And if you imagine you're in a store,
link |
right in some giant space, and then you know, you want to be
link |
picked up somewhere, like if you just drop a pin at a current
link |
location, which is maybe in the middle of a shopping mall, like
link |
what's the best location for the car to come pick you up? And
link |
you can have simple heuristics where you're just going to take
link |
your you know, you clean in distance and find the nearest
link |
spot where the car can pull over that's closest to you. But
link |
oftentimes, that's not the most convenient one. You know, I have
link |
many anecdotes where that heuristic breaks in horrible
link |
ways. One example that I often mentioned is somebody wanted to
link |
be, you know, dropped off in Phoenix. And you know, we got
link |
car picked location that was close, the closest to there,
link |
you know, where the pin was dropped on the map in terms of,
link |
you know, latitude and longitude. But it happened to be
link |
on the other side of a parking lot that had this row of
link |
cacti. And the poor person had to like walk all around the
link |
parking lot to get to where they wanted to be in 110 degree
link |
heat. So that, you know, that was about so then, you know, we
link |
took all take all of these, all that feedback from our users
link |
and incorporate it into our system and improve it. Yeah, I
link |
feel like that's like requires AGI to solve the problem of
link |
like, when you're, which is a very common case, when you're in
link |
a big space of some kind, like apartment building, it doesn't
link |
matter, it's some large space. And then you call the, like a
link |
Waymo from there, right? Like, whatever, it doesn't matter,
link |
ride share vehicle. And like, where's the pin supposed to
link |
drop? I feel like that's, you don't think, I think that
link |
requires AGI. I'm gonna, in order to solve. Okay, the
link |
alternative, which I think the Google search engine is taught
link |
is like, there's something really valuable about the
link |
perhaps slightly dumb answer, but a really powerful one,
link |
which is like, what was done in the past by others? Like, what
link |
was the choice made by others? That seems to be like in terms
link |
of Google search, when you have like billions of searches, you
link |
could, you could see which, like when they recommend what you
link |
might possibly mean, they suggest based on not some machine
link |
learning thing, which they also do, but like, on what was
link |
successful for others in the past and finding a thing that
link |
they were happy with. Is that integrated at all? Waymo, like
link |
what, what pickups worked for others? It is. I think you're
link |
exactly right. So there's a real, it's an interesting
link |
problem. Naive solutions have interesting failure modes. So
link |
there's definitely lots of things that can be done to
link |
improve. And both learning from, you know, what works, but
link |
doesn't work in actual heal from getting richer data and
link |
getting more information about the environment and richer
link |
maps. But you're absolutely right, that there's something
link |
like there's some properties of solutions that in terms of the
link |
effect that they have on users so much, much, much better than
link |
others, right? And predictability and
link |
understandability is important. So you can have maybe
link |
something that is not quite as optimal, but is very natural
link |
and predictable to the user and kind of works the same way all
link |
the time. And that matters, that matters a lot for the user
link |
experience. And but you know, to get to the basics, the pretty
link |
fundamental property is that the car actually arrives where you
link |
told it to, right? Like, you can always, you know, change it,
link |
see it on the map, and you can move it around if you don't
link |
like it. And but like, that property that the car actually
link |
shows up reliably is critical, which, you know, where compared
link |
to some of the human driven analogs, I think, you know, you
link |
can have more predictability. It's actually the fact, if I
link |
have a little bit of a detour here, I think the fact that
link |
it's, you know, your phone and the cars, two computers talking
link |
to each other, can lead to some really interesting things we
link |
can do in terms of the user interfaces, both in terms of
link |
function, like the car actually shows up exactly where you told
link |
it, you want it to be, but also some, you know, really
link |
interesting things on the user interface, like as the car is
link |
driving, as you call it, and it's on the way to come pick
link |
you up. And of course, you get the position of the car and the
link |
route on the map. But and they actually follow that route, of
link |
course. But it can also share some really interesting
link |
information about what it's doing. So, you know, our cars, as
link |
they are coming to pick you up, if it's come, if a car is
link |
coming up to a stop sign, it will actually show you that
link |
like, it's there sitting, because it's at a stop sign or
link |
a traffic light will show you that it's got, you know,
link |
sitting at a red light. So, you know, they're like little
link |
things, right? But I find those little touches really
link |
interesting, really magical. And it's just, you know, little
link |
things like that, that you can do to kind of delight your
link |
users. You know, this makes me think of, there's some products
link |
that I just love. Like, there's a there's a company called
link |
Rev, Rev.com, where I like for this podcast, for example, I
link |
can drag and drop a video. And then they do all the
link |
captioning. It's humans doing the captioning, but they
link |
connect, they automate everything of connecting you to
link |
the humans, and they do the captioning and transcription.
link |
It's all effortless. And it like, I remember when I first
link |
started using them, I was like, life's good. Like, because it
link |
was so painful to figure that out earlier. The same thing
link |
with something called iZotope RX, this company I use for
link |
cleaning up audio, like the sound cleanup they do. It's
link |
like drag and drop, and it just cleans everything up very
link |
nicely. Another experience like that I had with Amazon
link |
OneClick purchase, first time. I mean, other places do that
link |
now, but just the effortlessness of purchasing,
link |
making it frictionless. It kind of communicates to me, like,
link |
I'm a fan of design. I'm a fan of products that you can just
link |
create a really pleasant experience. The simplicity of
link |
it, the elegance just makes you fall in love with it. So on
link |
the, do you think about this kind of stuff? I mean, it's
link |
exactly what we've been talking about. It's like the little
link |
details that somehow make you fall in love with the product.
link |
Is that, we went from like urban challenge days, where
link |
love was not part of the conversation, probably. And to
link |
this point where there's a, where there's human beings and
link |
you want them to fall in love with the experience. Is that
link |
something you're trying to optimize for? Try to think
link |
about, like, how do you create an experience that people love?
link |
Absolutely. I think that's the vision is removing any friction
link |
or complexity from getting our users, our writers to where
link |
they want to go. Making that as simple as possible. And then,
link |
you know, beyond that, just transportation, making things
link |
and goods get to their destination as seamlessly as
link |
possible. I talked about a drag and drop experience where I
link |
kind of express your intent and then it just magically happens.
link |
And for our writers, that's what we're trying to get to is
link |
you download an app and you click and car shows up. It's
link |
the same car. It's very predictable. It's a safe and
link |
high quality experience. And then it gets you in a very
link |
reliable, very convenient, frictionless way to where you
link |
want to be. And along the journey, I think we also want to
link |
do little things to delight our users. Like the ride sharing
link |
companies, because they don't control the experience, I
link |
think they can't make people fall in love necessarily with
link |
the experience. Or maybe they, they haven't put in the effort,
link |
but I think if I were to speak to the ride sharing experience
link |
I currently have, it's just very, it's just very
link |
convenient, but there's a lot of room for like falling in love
link |
with it. Like we can speak to sort of car companies, car
link |
companies do this. Well, you can fall in love with a car,
link |
right? And be like a loyal car person, like whatever. Like I
link |
like badass hot rods, I guess, 69 Corvette. And at this point,
link |
you know, you can't really, cars are so, owning a car is so
link |
20th century, man. But is there something about the Waymo
link |
experience where you hope that people will fall in love with
link |
it? Is that part of it? Or is it part of, is it just about
link |
making a convenient ride, not ride sharing, I don't know what
link |
the right term is, but just a convenient A to B autonomous
link |
transport or like, do you want them to fall in love with
link |
Waymo? To maybe elaborate a little bit. I mean, almost like
link |
from a business perspective, I'm curious, like how, do you
link |
want to be in the background invisible or do you want to be
link |
like a source of joy that's in very much in the foreground? I
link |
want to provide the best, most enjoyable transportation
link |
solution. And that means building it, building our
link |
product and building our service in a way that people do.
link |
Kind of use in a very seamless, frictionless way in their
link |
day to day lives. And I think that does mean, you know, in
link |
some way falling in love in that product, right, just kind of
link |
becomes part of your routine. It comes down my mind to safety,
link |
predictability of the experience, and privacy aspects
link |
of it, right? Our cars, you get the same car, you get very
link |
predictable behavior. And you get a lot of different
link |
things. And that is important. And if you're going to use it
link |
in your daily life, privacy, and when you're in a car, you
link |
can do other things. You're spending a bunch, just another
link |
space where you're spending a significant part of your life.
link |
And so not having to share it with other people who you don't
link |
want to share it with, I think is a very nice property. Maybe
link |
you want to take a phone call or do something else in the
link |
vehicle. And, you know, safety on the quality of the driving,
link |
as well as the physical safety of not having to share that
link |
ride is important to a lot of people. What about the idea
link |
that when there's somebody like a human driving, and they do
link |
a rolling stop on a stop sign, like sometimes like, you know,
link |
you get an Uber or Lyft or whatever, like human driver,
link |
and, you know, they can be a little bit aggressive as
link |
drivers. It feels like there's not all aggression is bad. Now
link |
that may be a wrong, again, 20th century conception of
link |
driving. Maybe it's possible to create a driving experience.
link |
Like if you're in the back, busy doing something, maybe
link |
aggression is not a good thing. It's a very different kind of
link |
experience perhaps. But it feels like in order to navigate
link |
this world, you need to, how do I phrase this? You need to kind
link |
of bend the rules a little bit, or at least test the rules. I
link |
don't know what language politicians use to discuss this,
link |
but whatever language they use, you like flirt with the rules.
link |
I don't know. But like you sort of have a bit of an aggressive
link |
way of driving that asserts your presence in this world,
link |
thereby making other vehicles and people respect your
link |
presence and thereby allowing you to sort of navigate
link |
through intersections in a timely fashion. I don't know if
link |
any of that made sense, but like, how does that fit into the
link |
experience of driving autonomously? Is that?
link |
It's a lot of thoughts. This is you're hitting on a very
link |
important point of a number of behavioral components and, you
link |
know, parameters that make your driving feel assertive and
link |
natural and comfortable and predictable. Our cars will
link |
follow rules, right? They will do the safest thing possible in
link |
all situations. Let me be clear on that. But if you think of
link |
really, really good drivers, just think about
link |
professional lemon drivers, right? They will follow the
link |
rules. They're very, very smooth, and yet they're very
link |
efficient. But they're assertive. They're comfortable
link |
for the people in the vehicle. They're predictable for the
link |
other people outside the vehicle that they share the
link |
environment with. And that's the kind of driver that we want
link |
to build. And you think if maybe there's a sport analogy
link |
there, right? You can do in very many sports, the true
link |
professionals are very efficient in their movements,
link |
right? They don't do like, you know, hectic flailing, right?
link |
They're, you know, smooth and precise, right? And they get
link |
the best results. So that's the kind of driver that we want to
link |
build. In terms of, you know, aggressiveness. Yeah, you can
link |
like, you know, roll through the stop signs. You can do crazy
link |
lane changes. It typically doesn't get you to your
link |
destination faster. Typically not the safest or most
link |
predictable, very most comfortable thing to do. But
link |
there is a way to do both. And that's what we're
link |
doing. We're trying to build the driver that is safe,
link |
comfortable, smooth, and predictable. Yeah, that's a
link |
really interesting distinction. I think in the early days of
link |
autonomous vehicles, the vehicles felt cautious as
link |
opposed to efficient. And I'm still probably, but when I
link |
rode in the Waymo, I mean, there was, it was, it was quite
link |
assertive. It moved pretty quickly. Like, yeah, then he's
link |
one of the surprising feelings was that it actually, it went
link |
fast. And it didn't feel like, awkwardly cautious than
link |
autonomous vehicle. Like, like, so I've also programmed
link |
autonomous vehicles and everything I've ever built was
link |
felt awkwardly, either overly aggressive. Okay, especially
link |
when it was my code, or like, awkwardly cautious is the way
link |
I would put it. And Waymo's vehicle felt like, assertive
link |
and I think efficient is like the right terminology here.
link |
It wasn't, and I also like the professional limo driver,
link |
because we often think like, you know, an Uber driver or a
link |
bus driver or a taxi. This is the funny thing is people
link |
think they track taxi drivers are professionals. They, I
link |
mean, it's, it's like, that that's like saying, I'm a
link |
professional walker, just because I've been walking all
link |
my life. I think there's an art to it, right? And if you take
link |
it seriously as an art form, then there's a certain way that
link |
mastery looks like. It's interesting to think about what
link |
does mastery look like in driving? And perhaps what we
link |
associate with like aggressiveness is unnecessary,
link |
like, it's not part of the experience of driving. It's
link |
like, unnecessary fluff, that efficiency, you can be,
link |
you can create a good driving experience within the rules.
link |
That's, I mean, you're the first person to tell me this.
link |
So it's, it's kind of interesting. I need to think
link |
about this, but that's exactly what it felt like with Waymo.
link |
I kind of had this intuition. Maybe it's the Russian thing.
link |
I don't know that you have to break the rules in life to get
link |
anywhere, but maybe, maybe it's possible that that's not the
link |
case in driving. I have to think about that, but it
link |
certainly felt that way on the streets of Phoenix when I was
link |
there in Waymo, that, that, that that was a very pleasant
link |
experience and it wasn't frustrating in that like, come
link |
on, move already kind of feeling. It wasn't, that wasn't
link |
there. Yeah. I mean, that's what, that's what we're going
link |
after. I don't think you have to pick one. I think truly good
link |
driving. It gives you both efficiency, a certainness, but
link |
also comfort and predictability and safety. And, you know, it's,
link |
that's what fundamental improvements in the core
link |
capabilities truly unlock. And you can kind of think of it as,
link |
you know, a precision and recall trade off. You have certain
link |
capabilities of your model. And then it's very easy when, you
link |
know, you have some curve of precision and recall, you can
link |
move things around and can choose your operating point and
link |
your training of precision versus recall, false positives
link |
versus false negatives. Right. But then, and you know, you can
link |
tune things on that curve and be kind of more cautious or more
link |
aggressive, but then aggressive is bad or, you know, cautious is
link |
bad, but true capabilities come from actually moving the whole
link |
curve up. And then you are kind of on a very different plane of
link |
those trade offs. And that, that's what we're trying to do
link |
here is to move the whole curve up. Before I forget, let's talk
link |
about trucks a little bit. So I also got a chance to check out
link |
some of the Waymo trucks. I'm not sure if we want to go too
link |
much into that space, but it's a fascinating one. So maybe we
link |
can mention at least briefly, you know, Waymo is also now
link |
doing autonomous trucking and how different like
link |
philosophically and technically is that whole space of
link |
problems. It's one of our two big products and you know,
link |
commercial applications of our driver, right? Right. Hailing
link |
and deliveries. You know, we have Waymo One and Waymo Via
link |
moving people and moving goods. You know, trucking is an
link |
example of moving goods. We've been working on trucking since
link |
2017. It is a very interesting space. And your question of
link |
how different is it? It has this really nice property that
link |
the first order challenges, like the science, the hard
link |
engineering, whether it's, you know, hardware or, you know,
link |
onboard software or off board software, all of the, you know,
link |
systems that you build for, you know, training your ML models
link |
for, you know, evaluating your time system. Like those
link |
fundamentals carry over. Like the true challenges of, you
link |
know, driving perception, semantic understanding,
link |
prediction, decision making, planning, evaluation, the
link |
simulator, ML infrastructure, those carry over. Like the data
link |
and the application and kind of the domains might be
link |
different, but the most difficult problems, all of that
link |
carries over between the domains. So that's very nice.
link |
So that's how we approach it. We're kind of build investing
link |
in the core, the technical core. And then there's
link |
specialization of that core technology to different
link |
product lines, to different commercial applications. So on
link |
just to tease it apart a little bit on trucks. So starting with
link |
the hardware, the configuration of the sensors is different.
link |
They're different physically, geometrically, you know, different
link |
vehicles. So for example, we have two of our main laser on
link |
the trucks on both sides so that we have, you know, not have the
link |
blind spots. Whereas on the JLR eye pace, we have, you know, one
link |
of it sitting at the very top, but the actual sensors are
link |
almost the same. Now we're largely the same. So all of the
link |
investment that over the years we've put into building our
link |
custom lighters, custom radars, pulling the whole system
link |
together, that carries over very nicely. Then, you know, on the
link |
perception side, the like the fundamental challenges of
link |
seeing, understanding the world, whether it's, you know, object
link |
detection, classification, you know, tracking, semantic
link |
understanding, all that carries over. You know, yes, there's
link |
some specialization when you're driving on freeways, you know,
link |
range becomes more important. The domain is a little bit
link |
different. But again, the fundamentals carry over very,
link |
very nicely. Same, and you guess you get into prediction or
link |
decision making, right, the fundamentals of what it takes to
link |
predict what other people are going to do to find the long
link |
tail to improve your system in that long tail of behavior
link |
prediction and response that carries over right and so on and
link |
so on. So I mean, that's pretty exciting. By the way, does
link |
Waymo via include using the smaller vehicles for
link |
transportation of goods? That's an interesting distinction. So
link |
I would say there's three interesting modes of operation.
link |
So one is moving humans, one is moving goods, and one is like
link |
moving nothing, zero occupancy, meaning like you're going to
link |
the destination, your empty vehicle. I mean, it's the third
link |
is the less of it. If that's the entirety of it, it's the less,
link |
you know, exciting from the commercial perspective.
link |
Well, I mean, in terms of like, if you think about what's
link |
inside a vehicle as it's moving, because it does, you
link |
know, some significant fraction of the vehicle's movement has
link |
to be empty. I mean, it's kind of fascinating. Maybe just on
link |
that small point, is there different control and like
link |
policies that are applied for zero occupancy vehicle? So
link |
vehicle with nothing in it, or is it just move as if there is
link |
a person inside? What was with some subtle differences?
link |
As a first order approximation, there are no differences. And
link |
if you think about, you know, safety and comfort and quality
link |
of driving, only part of it has to do with the people or the
link |
goods inside of the vehicle. But you don't want to be, you
link |
know, you want to drive smoothly, as we discussed, not
link |
for the purely for the benefit of whatever you have inside the
link |
car, right? It's also for the benefit of the people outside
link |
kind of fitting naturally and predictably into that whole
link |
environment, right? So, you know, yes, there are some
link |
second order things you can do, you can change your route, and
link |
you optimize maybe kind of your fleet, things at the fleet
link |
scale. And you would take into account whether some of your
link |
you know, some of your cars are actually, you know, serving a
link |
useful trip, whether with people or with goods, whereas, you
link |
know, other cars are, you know, driving completely empty to that
link |
next valuable trip that they're going to provide. But that those
link |
are mostly second order effects. Okay, cool. So Phoenix
link |
is, is an incredible place. And what you've announced in
link |
Phoenix is, it's kind of amazing. But, you know, that's
link |
just like one city. How do you take over the world? I mean,
link |
I'm asking for a friend. One step at a time.
link |
Is that a cartoon pinky in the brain? Yeah. Okay. But, you
link |
know, gradually is a true answer. So I think the heart of
link |
your question is, can you ask a better question than I asked?
link |
You're asking a great question. Answer that one. I'm just
link |
gonna, you know, phrase it in the terms that I want to
link |
answer. Exactly right. Brilliant. Please. You know,
link |
where are we today? And, you know, what happens next? And
link |
what does it take to go beyond Phoenix? And what does it
link |
take to get this technology to more places and more people
link |
around the world, right? So our next big area of focus is
link |
exactly that. Larger scale commercialization and just,
link |
you know, scaling up. If I think about, you know, the
link |
main, and, you know, Phoenix gives us that platform and
link |
gives us that foundation of upon which we can build. And
link |
it's, there are few really challenging aspects of this
link |
whole problem that you have to pull together in order to build
link |
the technology in order to deploy it into the field to go
link |
from a driverless car to a fleet of cars that are providing a
link |
service, and then all the way to commercialization. So, and
link |
then, you know, this is what we have in Phoenix. We've taken
link |
the technology from a proof point to an actual deployment
link |
and have taken our driver from, you know, one car to a fleet
link |
that can provide a service. Beyond that, if I think about
link |
what it will take to scale up and, you know, deploy in, you
link |
know, more places with more customers, I tend to think about
link |
three main dimensions, three main axes of scale. One is the
link |
core technology, you know, the hardware and software core
link |
capabilities of our driver. The second dimension is
link |
evaluation and deployment. And the third one is the, you know,
link |
product, commercial, and operational excellence. So you
link |
can talk a bit about where we are along, you know, each one of
link |
those three dimensions about where we are today and, you
link |
know, what has, what will happen next. On, you know, the core
link |
technology, you know, the hardware and software, you
link |
know, together comprise a driver, we, you know, obviously
link |
have that foundation that is providing fully driverless
link |
trips to our customers as we speak, in fact. And we've
link |
learned a tremendous amount from that. So now what we're
link |
doing is we are incorporating all those lessons into some
link |
pretty fundamental improvements in our core technology, both on
link |
the hardware side and on the software side to build a more
link |
general, more robust solution that then will enable us to
link |
massively scale beyond Phoenix. So on the hardware side, all of
link |
those lessons are now incorporated into this fifth
link |
generation hardware platform that is, you know, being
link |
deployed right now. And that's the platform, the fourth
link |
generation, the thing that we have right now driving in
link |
Phoenix, it's good enough to operate fully driverlessly,
link |
you know, night and day, you know, various speeds and
link |
various conditions, but the fifth generation is the platform
link |
upon which we want to go to massive scale. We, in turn,
link |
we've really made qualitative improvements in terms of the
link |
capability of the system, the simplicity of the architecture,
link |
the reliability of the redundancy. It is designed to be
link |
manufacturable at very large scale and, you know, provides
link |
the right unit economics. So that's the next big step for us
link |
on the hardware side. That's already there for scale,
link |
the version five. That's right. And is that coincidence or
link |
should we look into a conspiracy theory that it's the
link |
same version as the pixel phone? Is that what's the
link |
hardware? They neither confirm nor deny. All right, cool. So,
link |
sorry. So that's the, okay, that's that axis. What else?
link |
So similarly, you know, hardware is a very discreet
link |
jump, but, you know, similar to how we're making that change
link |
from the fourth generation hardware to the fifth, we're
link |
making similar improvements on the software side to make it
link |
more, you know, robust and more general and allow us to kind of
link |
quickly scale beyond Phoenix. So that's the first dimension of
link |
core technology. The second dimension is evaluation and
link |
deployment. How do you measure your system? How do you
link |
evaluate it? How do you build a release and deployment process
link |
where, you know, with confidence, you can, you know,
link |
regularly release new versions of your driver into a fleet?
link |
How do you get good at it so that it is not, you know, a
link |
huge tax on your researchers and engineers that, you know, so
link |
you can, how do you build all these, you know, processes, the
link |
frameworks, the simulation, the evaluation, the data science,
link |
the validation, so that, you know, people can focus on
link |
improving the system and kind of the releases just go out the
link |
door and get deployed across the fleet. So we've gotten really
link |
good at that in Phoenix. That's been a tremendously difficult
link |
problem, but that's what we have in Phoenix right now that gives
link |
us that foundation. And now we're working on kind of
link |
incorporating all the lessons that we've learned to make it
link |
more efficient, to go to new places, you know, and scale up
link |
and just kind of, you know, stamp things out. So that's that
link |
second dimension of evaluation and deployment. And the third
link |
dimension is product, commercial, and operational
link |
excellence, right? And again, Phoenix there is providing an
link |
incredibly valuable platform. You know, that's why we're doing
link |
things end to end in Phoenix. We're learning, as you know, we
link |
discussed a little earlier today, tremendous amount of
link |
really valuable lessons from our users getting really
link |
incredible feedback. And we'll continue to iterate on that and
link |
incorporate all those lessons into making our product, you
link |
know, even better and more convenient for our users.
link |
So you're converting this whole process in Phoenix into
link |
something that could be copy and pasted elsewhere. So like,
link |
perhaps you didn't think of it that way when you were doing
link |
the experimentation in Phoenix, but so how long did you
link |
basically, and you can correct me, but you've, I mean, it's
link |
still early days, but you've taken the full journey in
link |
Phoenix, right? As you were saying of like what it takes to
link |
basically automate. I mean, it's not the entirety of Phoenix,
link |
right? But I imagine it can encompass the entirety of
link |
Phoenix. That's some near term date, but that's not even
link |
perhaps important. Like as long as it's a large enough
link |
geographic area. So what, how copy pastable is that process
link |
currently and how like, you know, like when you copy and
link |
paste in Google docs, I think now in, or in word, you can
link |
like apply source formatting or apply destination formatting.
link |
So how, when you copy and paste the Phoenix into like, say
link |
Boston, how do you apply the destination formatting? Like
link |
how much of the core of the entire process of bringing an
link |
actual public transportation, autonomous transportation
link |
service to a city is there in Phoenix that you understand
link |
enough to copy and paste into Boston or wherever? So we're
link |
not quite there yet. We're not at a point where we're kind of
link |
massively copy and pasting all over the place. But Phoenix,
link |
what we did in Phoenix, and we very intentionally have chosen
link |
Phoenix as our first full deployment area, you know,
link |
exactly for that reason to kind of tease the problem apart,
link |
look at each dimension and focus on the fundamentals of
link |
complexity and de risking those dimensions, and then bringing
link |
the entire thing together to get all the way and force
link |
ourselves to learn all those hard lessons on technology,
link |
hardware and software, on the evaluation deployment, on
link |
operating a service, operating a business using actually
link |
serving our customers all the way so that we're fully
link |
informed about the most difficult, most important
link |
challenges to get us to that next step of massive copy and
link |
pasting as you said. And that's what we're doing right now.
link |
We're incorporating all those things that we learned into
link |
that next system that then will allow us to kind of copy and
link |
paste all over the place and to massively scale to, you know,
link |
more users and more locations. I mean, you know, just talk a
link |
little bit about, you know, what does that mean along those
link |
different dimensions? So on the hardware side, for example,
link |
again, it's that switch from the fourth to the fifth
link |
generation. And the fifth generation is designed to kind
link |
of have that property. Can you say what other cities you're
link |
thinking about? Like, I'm thinking about, sorry, we're in
link |
San Francisco now. I thought I want to move to San Francisco,
link |
but I'm thinking about moving to Austin. I don't know why
link |
people are not being very nice about San Francisco currently,
link |
but maybe it's a small, maybe it's in vogue right now.
link |
But Austin seems, I visited there and it was, I was in a
link |
Walmart. It's funny, these moments like turn your life.
link |
There's this very nice woman with kind eyes, just like stopped
link |
and said, he looks so handsome in that tie, honey, to me. This
link |
has never happened to me in my life, but just the sweetness of
link |
this woman is something I've never experienced, certainly on
link |
the streets of Boston, but even in San Francisco where people
link |
wouldn't, that's just not how they speak or think. I don't
link |
know. There's a warmth to Austin that love. And since
link |
Waymo does have a little bit of a history there, is that a
link |
possibility? Is this your version of asking the question
link |
of like, you know, Dimitri, I know you can't share your
link |
commercial and deployment roadmap, but I'm thinking about
link |
moving to San Francisco, Austin, like, you know, blink twice if
link |
you think I should move to it. That's true. That's true. You
link |
got me. You know, we've been testing all over the place. I
link |
think we've been testing more than 25 cities. We drive
link |
in San Francisco. We drive in, you know, Michigan for snow.
link |
We are doing significant amount of testing in the Bay Area,
link |
including San Francisco, which is not like, because we're
link |
talking about the very different thing, which is like a
link |
full on large geographic area, public service. You can't share
link |
and you, okay. What about Moscow? When is that happening?
link |
Take on Yandex. I'm not paying attention to those folks.
link |
They're doing, you know, there's a lot of fun. I mean,
link |
maybe as a way of a question, you didn't speak to sort of like
link |
policy or like, is there tricky things with government and so
link |
on? Like, is there other friction that you've
link |
encountered except sort of technological friction of
link |
solving this very difficult problem? Is there other stuff
link |
that you have to overcome when deploying a public service in
link |
a city? That's interesting. It's very important. So we
link |
put significant effort in creating those partnerships and
link |
you know, those relationships with governments at all levels,
link |
local governments, municipalities, state level,
link |
federal level. We've been engaged in very deep
link |
conversations from the earliest days of our projects.
link |
Whenever at all of these levels, whenever we go
link |
to test or operate in a new area, we always lead
link |
with a conversation with the local officials.
link |
But the result of that investment is that no,
link |
it's not challenges we have to overcome, but it is very
link |
important that we continue to have this conversation.
link |
Oh, yeah. I love politicians too. Okay, so Mr. Elon Musk said that
link |
LiDAR is a crutch. What are your thoughts?
link |
I wouldn't characterize it exactly that way. I know I think LiDAR is
link |
very important. It is a key sensor that we use just like
link |
other modalities, right? As we discussed, our cars use cameras, LiDAR
link |
and radars. They are all very important. They are
link |
at the kind of the physical level. They are very different. They have very
link |
different, you know, physical characteristics.
link |
Cameras are passive. LiDARs and radars are active.
link |
Use different wavelengths. So that means they complement each other
link |
very nicely and together combined, they can be used to
link |
build a much safer and much more capable system.
link |
So, you know, to me it's more of a question,
link |
you know, why the heck would you handicap yourself and not use one
link |
or more of those sensing modalities when they, you know, undoubtedly just make your
link |
system more capable and safer. Now,
link |
it, you know, what might make sense for one product or
link |
one business might not make sense for another one.
link |
So if you're talking about driver assist technologies, you make certain design
link |
decisions and you make certain trade offs and make different ones if you are
link |
building a driver that you deploy in fully driverless
link |
vehicles. And, you know, in LiDAR specifically, when this question comes up,
link |
I, you know, typically the criticisms that I hear or, you know, the
link |
counterpoints is that cost and aesthetics.
link |
And I don't find either of those, honestly, very compelling.
link |
So on the cost side, there's nothing fundamentally prohibitive
link |
about, you know, the cost of LiDARs. You know, radars used to be very expensive
link |
before people started, you know, before people made certain advances in
link |
technology and, you know, started to manufacture them at massive scale and
link |
deploy them in vehicles, right? You know, similar with LiDARs. And this is
link |
where the LiDARs that we have on our cars, especially the fifth generation,
link |
you know, we've been able to make some pretty qualitative discontinuous
link |
jumps in terms of the fundamental technology that allow us to
link |
manufacture those things at very significant scale and at a fraction
link |
of the cost of both our previous generation
link |
as well as a fraction of the cost of, you know, what might be available
link |
on the market, you know, off the shelf right now. And, you know, that improvement
link |
will continue. So I think, you know, cost is not a
link |
real issue. Second one is, you know, aesthetics.
link |
You know, I don't think that's, you know, a real issue either.
link |
Beauty is in the eye of the beholder. Yeah. You can make LiDAR sexy again.
link |
I think you're exactly right. I think it is sexy. Like, honestly, I think form
link |
all of function. Well, okay. You know, I was actually, somebody brought this up to
link |
me. I mean, all forms of LiDAR, even
link |
like the ones that are like big, you can make
link |
look, I mean, you can make look beautiful.
link |
There's no sense in which you can't integrate it into design.
link |
Like, there's all kinds of awesome designs. I don't think
link |
small and humble is beautiful. It could be
link |
like, you know, brutalism or like, it could be
link |
like harsh corners. I mean, like I said, like hot rods. Like, I don't like, I don't
link |
necessarily like, like, oh man, I'm going to start so much
link |
controversy with this. I don't like Porsches. Okay.
link |
The Porsche 911, like everyone says it's the most beautiful.
link |
No, no. It's like, it's like a baby car. It doesn't make any sense.
link |
But everyone, it's beauty is in the eye of the beholder. You're already looking at
link |
me like, what is this kid talking about? I'm happy to talk about. You're digging your
link |
own hole. The form and function and my take on the
link |
beauty of the hardware that we put on our vehicles,
link |
you know, I will not comment on your Porsche monologue.
link |
Okay. All right. So, but aesthetics, fine. But there's an underlying, like,
link |
philosophical question behind the kind of lighter question is
link |
like, how much of the problem can be solved
link |
with computer vision, with machine learning?
link |
So I think without sort of disagreements and so on,
link |
it's nice to put it on the spectrum because Waymo is doing a lot of machine
link |
learning as well. It's interesting to think how much of
link |
driving, if we look at five years, 10 years, 50 years down the road,
link |
what can be learned in almost more and more and more
link |
end to end way. If we look at what Tesla is doing
link |
with, as a machine learning problem, they're doing a multitask learning
link |
thing where it's just, they break up driving into a bunch of learning tasks
link |
and they have one single neural network and they're just collecting huge amounts
link |
of data that's training that. I've recently hung out with George
link |
Hotz. I don't know if you know George.
link |
I love him so much. He's just an entertaining human being.
link |
We were off mic talking about Hunter S. Thompson. He's the Hunter S. Thompson
link |
of autonomous driving. Okay. So he, I didn't realize this with comma
link |
AI, but they're like really trying to end to end.
link |
They're the machine, like looking at the machine learning problem, they're
link |
really not doing multitask learning, but it's
link |
computing the drivable area as a machine learning task
link |
and hoping that like down the line, this level two system, this driver
link |
assistance will eventually lead to
link |
allowing you to have a fully autonomous vehicle. Okay. There's an underlying
link |
deep philosophical question there, technical question
link |
of how much of driving can be learned. So LiDAR is an effective tool today
link |
for actually deploying a successful service in Phoenix, right? That's safe,
link |
that's reliable, et cetera, et cetera. But the question,
link |
and I'm not saying you can't do machine learning on LiDAR, but the question is
link |
that like how much of driving can be learned eventually.
link |
Can we do fully autonomous? That's learned.
link |
Yeah. You know, learning is all over the place
link |
and plays a key role in every part of our system.
link |
As you said, I would, you know, decouple the sensing modalities
link |
from the, you know, ML and the software parts of it.
link |
LiDAR, radar, cameras, like it's all machine learning.
link |
All of the object detection classification, of course, like that's
link |
what, you know, these modern deep nets and count nets are very
link |
good at. You feed them raw data, massive amounts of raw data,
link |
and that's actually what our custom build LiDARs and radars are really good
link |
at. And radars, they don't just give you point
link |
estimates of, you know, objects in space, they give you raw,
link |
like, physical observations. And then you take all of that raw information,
link |
you know, there's colors of the pixels, whether it's, you know, LiDARs returns
link |
and some auxiliary information. It's not just distance,
link |
right? And, you know, angle and distance is much richer information that you get
link |
from those returns, plus really rich information from the
link |
radars. You fuse it all together and you feed it into those massive
link |
ML models that then, you know, lead to the best results in terms of, you
link |
know, object detection, classification, state estimation.
link |
So there's a side to interop, but there is a fusion. I mean, that's something
link |
that people didn't do for a very long time,
link |
which is like at the sensor fusion level, I guess,
link |
like early on fusing the information together, whether
link |
so that the the sensory information that the vehicle receives from the different
link |
modalities or even from different cameras is
link |
combined before it is fed into the machine learning models.
link |
Yeah, so I think this is one of the trends you're seeing more of that you
link |
mentioned end to end. There's different interpretation of end to end. There is
link |
kind of the purest interpretation of I'm going to
link |
like have one model that goes from raw sensor data to like,
link |
you know, steering torque and, you know, gas breaks. That, you know,
link |
that's too much. I don't think that's the right way to do it.
link |
There's more, you know, smaller versions of end to end
link |
where you're kind of doing more end to end learning or core training or
link |
depropagation of kind of signals back and forth across
link |
the different stages of your system. There's, you know, really good ways it
link |
gets into some fairly complex design choices where on one
link |
hand you want modularity and decomposability,
link |
decomposability of your system. But on the other hand,
link |
you don't want to create interfaces that are too narrow or too brittle
link |
to engineered where you're giving up on the generality of the solution or you're
link |
unable to properly propagate signal, you know, reach signal forward and losses
link |
and, you know, back so you can optimize the whole system jointly.
link |
So I would decouple and I guess what you're seeing in terms of the fusion
link |
of the sensing data from different modalities as well as kind of fusion
link |
at in the temporal level going more from, you know, frame by frame
link |
where, you know, you would have one net that would do frame by frame detection
link |
and camera and then, you know, something that does frame by frame and
link |
lighter and then radar and then you fuse it, you know, in a weaker engineered way
link |
later. Like the field over the last, you know,
link |
decade has been evolving in more kind of joint fusion, more end to end models that
link |
are, you know, solving some of these tasks, you know, jointly and there's
link |
tremendous power in that and, you know, that's the
link |
progression that kind of our technology, our stack has been on as well.
link |
Now to your, you know, that so I would decouple the kind of sensing and how
link |
that information is fused from the role of ML and the entire stack.
link |
And, you know, I guess it's, there's trade offs and, you know, modularity and
link |
how do you inject inductive bias into your system?
link |
All right, this is, there's tremendous power
link |
in being able to do that. So, you know, we have, there's no
link |
part of our system that is not heavily, that does not heavily, you know, leverage
link |
data driven development or state of the art ML.
link |
But there's mapping, there's a simulator, there's perception, you know, object
link |
level, you know, perception, whether it's
link |
semantic understanding, prediction, decision making, you know, so forth and
link |
It's, you know, of course, object detection and classification, like you're
link |
finding pedestrians and cars and cyclists and, you know, cones and signs
link |
and vegetation and being very good at estimating
link |
kind of detection, classification, and state estimation. There's just stable
link |
stakes, like that's step zero of this whole stack. You can be
link |
incredibly good at that, whether you use cameras or light as a
link |
radar, but that's just, you know, that's stable stakes, that's just step zero.
link |
Beyond that, you get into the really interesting challenges of semantic
link |
understanding at the perception level, you get into scene level reasoning, you
link |
get into very deep problems that have to do with prediction and joint
link |
prediction and interaction, so the interaction
link |
between all the actors in the environment, pedestrians, cyclists, other
link |
cars, and you get into decision making, right? So, how do you build a lot of
link |
systems? So, we leverage ML very heavily in all of
link |
these components. I do believe that the best results you
link |
achieve by kind of using a hybrid approach and
link |
having different types of ML, having
link |
different models with different degrees of inductive bias
link |
that you can have, and combining kind of model,
link |
you know, free approaches with some model based approaches and some
link |
rule based, physics based systems. So, you know, one example I can give
link |
you is traffic lights. There's a problem of the detection of
link |
traffic light state, and obviously that's a great problem for, you know, computer
link |
vision confidence, or, you know, that's their bread and
link |
butter, right? That's how you build that. But then the
link |
interpretation of, you know, of a traffic light, that you're
link |
gonna need to learn that, right? You don't need to build some,
link |
you know, complex ML model that, you know, infers
link |
with some, you know, precision and recall that red means stop.
link |
Like, it's a very clear engineered signal
link |
with very clear semantics, right? So you want to induce that bias, like how you
link |
induce that bias, and that whether, you know, it's a
link |
constraint or a cost, you know, function in your stack, but like
link |
it is important to be able to inject that, like, clear semantic
link |
signal into your stack. And, you know, that's what we do.
link |
And, but then the question of, like, and that's when you
link |
apply it to yourself, when you are making decisions whether you want to stop
link |
for a red light, you know, or not.
link |
But if you think about how other people treat traffic lights,
link |
we're back to the ML version of that. You know they're supposed to stop
link |
for a red light, but that doesn't mean they will.
link |
So then you're back in the, like, very heavy
link |
ML domain where you're picking up on, like, very subtle cues about,
link |
you know, they have to do with the behavior of objects, pedestrians, cyclists,
link |
cars, and the whole, you know, entire configuration of the scene
link |
that allow you to make accurate predictions on whether they will, in
link |
fact, stop or run a red light. So it sounds like already for Waymo,
link |
like, machine learning is a huge part of the stack.
link |
So it's a huge part of, like, not just, so obviously the first, the level
link |
zero, or whatever you said, which is, like,
link |
just the object detection of things that, you know, with no other machine learning
link |
can do, but also starting to do prediction behavior and so on to
link |
model the, what other, what the other parties in the
link |
scene, entities in the scene are going to do.
link |
So machine learning is more and more playing a role in that
link |
as well. Of course. Oh, absolutely. I think we've been
link |
going back to the, you know, earliest days, like, you know, DARPA,
link |
the DARPA Grand Challenge, our team was leveraging, you know, machine
link |
learning. It was, like, pre, you know, ImageNet, and it was a very
link |
different type of ML, but, and I think actually it was before
link |
my time, but the Stanford team during the Grand Challenge had a very
link |
interesting machine learned system that would, you know, use
link |
LiDAR and camera. We've been driving in the
link |
desert, and it, we had built the model where it would kind of
link |
extend the range of free space reasoning. We get a
link |
clear signal from LiDAR, and then it had a model that said, hey, like,
link |
this stuff on camera kind of sort of looks like this stuff in LiDAR, and I
link |
know this stuff that I'm seeing in LiDAR, I'm very confident it's free space,
link |
so let me extend that free space zone into the camera range that would allow
link |
the vehicle to drive faster. And then we've been building on top of
link |
that and kind of staying and pushing the state of the art in ML,
link |
in all kinds of different ML over the years. And in fact,
link |
from the early days, I think, you know, 2010 is probably the year
link |
where Google, maybe 2011 probably, got pretty heavily involved in
link |
machine learning, kind of deep nuts, and at that time it was probably the only
link |
company that was very heavily investing in kind of state of the art ML and
link |
self driving cars. And they go hand in hand.
link |
And we've been on that journey ever since. We're doing, pushing
link |
a lot of these areas in terms of research at Waymo, and we
link |
collaborate very heavily with the researchers in
link |
Alphabet, and all kinds of ML, supervised ML,
link |
unsupervised ML, published some
link |
interesting research papers in the space,
link |
especially recently. It's just a super active learning as well.
link |
Yeah, so super, super active. Of course, there's, you know, kind of the more
link |
mature stuff, like, you know, ConvNets for, you know, object detection.
link |
But there's some really interesting, really active work that's happening
link |
in kind of more, you know, in bigger models and, you know,
link |
models that have more structure to them,
link |
you know, not just, you know, large bitmaps and reason about temporal sequences.
link |
And some of the interesting breakthroughs that you've, you know, we've seen
link |
in language models, right? You know, transformers,
link |
you know, GPT3 inference. There's some really interesting applications of some
link |
of the core breakthroughs to those problems
link |
of, you know, behavior prediction, as well as, you know, decision making and
link |
planning, right? You can think about it, kind of the the behavior,
link |
how, you know, the path, the trajectories, the how people drive.
link |
They have kind of a share, a lot of the fundamental structure,
link |
you know, this problem. There's, you know, sequential,
link |
you know, nature. There's a lot of structure in this representation.
link |
There is a strong locality, kind of like in sentences, you know, words that follow
link |
each other. They're strongly connected, but there's
link |
also kind of larger context that doesn't have that locality, and you also see that
link |
in driving, right? What, you know, is happening in the scene
link |
as a whole has very strong implications on,
link |
you know, the kind of the next step in that sequence where
link |
whether you're, you know, predicting what other people are going to do, whether
link |
you're making your own decisions, or whether in the simulator you're
link |
building generative models of, you know, humans walking, cyclists
link |
riding, and other cars driving. That's all really fascinating, like how
link |
it's fascinating to think that transformer models and all this,
link |
all the breakthroughs in language and NLP that might be applicable to like
link |
driving at the higher level, at the behavioral level, that's kind of
link |
fascinating. Let me ask about pesky little creatures
link |
called pedestrians and cyclists. They seem, so humans are a problem. If we
link |
can get rid of them, I would. But unfortunately, they're all sort of
link |
a source of joy and love and beauty, so let's keep them around.
link |
They're also our customers. For your perspective, yes, yes,
link |
for sure. They're a source of money, very good.
link |
But I don't even know where I was going. Oh yes,
link |
pedestrians and cyclists, you know,
link |
they're a fascinating injection into the system of
link |
uncertainty of like a game theoretic dance of what to do. And also
link |
they have perceptions of their own, and they can tweet
link |
about your product, so you don't want to run them over
link |
from that perspective. I mean, I don't know, I'm joking a lot, but
link |
I think in seriousness, like, you know, pedestrians are a complicated
link |
computer vision problem, a complicated behavioral problem. Is there something
link |
interesting you could say about what you've learned
link |
from a machine learning perspective, from also an autonomous vehicle,
link |
and a product perspective about just interacting with the humans in this
link |
world? Yeah, just to state on record, we care
link |
deeply about the safety of pedestrians, you know, even the ones that don't have
link |
Twitter accounts. Thank you. All right, cool.
link |
Not me. But yes, I'm glad, I'm glad somebody does.
link |
Okay. But you know, in all seriousness, safety
link |
of vulnerable road users, pedestrians or cyclists, is one of our
link |
highest priorities. We do a tremendous amount of testing
link |
and validation, and put a very significant emphasis
link |
on, you know, the capabilities of our systems that have to do with safety
link |
around those unprotected vulnerable road users.
link |
You know, cars, just, you know, discussed earlier in Phoenix, we have completely
link |
empty cars, completely driverless cars, you know, driving in this very large area,
link |
and you know, some people use them to, you know, go to school, so they'll drive
link |
through school zones, right? So, kids are kind of the very special
link |
class of those vulnerable user road users, right? You want to be,
link |
you know, super, super safe, and super, super cautious around those. So, we take
link |
it very, very, very seriously. And you know, what does it take to
link |
be good at it? You know,
link |
an incredible amount of performance across your whole stack. You know,
link |
starts with hardware, and again, you want to use all
link |
sensing modalities available to you. Imagine driving on a residential road
link |
at night, and kind of making a turn, and you don't have, you know, headlights
link |
covering some part of the space, and like, you know, a kid might
link |
run out. And you know, lighters are amazing at that. They
link |
see just as well in complete darkness as they do during the day, right? So, just
link |
again, it gives you that extra,
link |
you know, margin in terms of, you know, capability, and performance, and safety,
link |
and quality. And in fact, we oftentimes, in these
link |
kinds of situations, we have our system detect something,
link |
in some cases even earlier than our trained operators in the car might do,
link |
right? Especially, you know, in conditions like, you know, very dark nights.
link |
So, starts with sensing, then, you know, perception
link |
has to be incredibly good. And you have to be very, very good
link |
at kind of detecting pedestrians in all kinds of situations, and all kinds
link |
of environments, including, you know, people in weird poses,
link |
people kind of running around, and you know, being partially occluded.
link |
So, you know, that's step number one, right?
link |
Then, you have to have in very high accuracy,
link |
and very low latency, in terms of your reactions
link |
to, you know, what, you know, these actors might do, right? And we've put a
link |
tremendous amount of engineering, and tremendous amount of validation, in to
link |
make sure our system performs properly. And, you know, oftentimes, it
link |
does require a very strong reaction to do the safe thing. And, you know, we
link |
actually see a lot of cases like that. That's the long tail of really rare,
link |
you know, really, you know, crazy events that contribute to the safety
link |
around pedestrians. Like, one example that comes to mind, that we actually
link |
happened in Phoenix, where we were driving
link |
along, and I think it was a 45 mile per hour road, so you have pretty high speed
link |
traffic, and there was a sidewalk next to it, and
link |
there was a cyclist on the sidewalk. And as we were in the right lane,
link |
right next to the side, so it was a multi lane road, so as we got close
link |
to the cyclist on the sidewalk, it was a woman, you know, she tripped and fell.
link |
Just, you know, fell right into the path of our vehicle, right?
link |
And our, you know, car, you know, this was actually with a
link |
test driver, our test drivers, did exactly the right thing.
link |
They kind of reacted, and came to stop. It requires both very strong steering,
link |
and, you know, strong application of the brake. And then we simulated what our
link |
system would have done in that situation, and it did, you know,
link |
exactly the same thing. And that speaks to, you know, all of
link |
those components of really good state estimation and
link |
tracking. And, like, imagine, you know, a person
link |
on a bike, and they're falling over, and they're doing that right in front of you,
link |
right? So you have to be really, like, things are changing. The appearance of
link |
that whole thing is changing, right? And a person goes one way, they're falling on
link |
the road, they're, you know, being flat on the ground in front of
link |
you. You know, the bike goes flying the other direction.
link |
Like, the two objects that used to be one, they're now, you know,
link |
are splitting apart, and the car has to, like, detect all of that.
link |
Like, milliseconds matter, and it doesn't, you know, it's not good enough to just
link |
brake. You have to, like, steer and brake, and there's traffic around you.
link |
So, like, it all has to come together, and it was really great
link |
to see in this case, and other cases like that, that we're actually seeing in the
link |
wild, that our system is, you know, performing
link |
exactly the way that we would have liked, and is able to,
link |
you know, avoid collisions like this.
link |
That's such an exciting space for robotics.
link |
Like, in that split second to make decisions of life and death.
link |
I don't know. The stakes are high, in a sense, but it's also beautiful
link |
that for somebody who loves artificial intelligence, the possibility that an AI
link |
system might be able to save a human life.
link |
That's kind of exciting as a problem, like, to wake up.
link |
It's terrifying, probably, for an engineer to wake up,
link |
and to think about, but it's also exciting because it's, like,
link |
it's in your hands. Let me try to ask a question that's often brought up about
link |
autonomous vehicles, and it might be fun to see if you have
link |
anything interesting to say, which is about the trolley problem.
link |
So, a trolley problem is an interesting philosophical construct
link |
that highlights, and there's many others like it,
link |
of the difficult ethical decisions that we humans have before us in this
link |
complicated world. So, specifically is the choice
link |
between if you are forced to choose to kill
link |
a group X of people versus a group Y of people, like
link |
one person. If you did nothing, you would kill one person, but if
link |
you would kill five people, and if you decide to swerve out of the way, you
link |
would only kill one person. Do you do nothing, or you choose to do
link |
something? You can construct all kinds of, sort of,
link |
ethical experiments of this kind that, I think, at least on a positive note,
link |
inspire you to think about, like, introspect
link |
what are the physics of our morality, and there's usually not
link |
good answers there. I think people love it because it's just an exciting
link |
thing to think about. I think people who build autonomous
link |
vehicles usually roll their eyes, because this is not,
link |
this one as constructed, this, like, literally never comes up
link |
in reality. You never have to choose between killing
link |
one or, like, one of two groups of people,
link |
but I wonder if you can speak to, is there some something interesting
link |
to you as an engineer of autonomous vehicles that's within the trolley
link |
problem, or maybe more generally, are there
link |
difficult ethical decisions that you find
link |
that an algorithm must make? On the specific version of the trolley problem,
link |
which one would you do, if you're driving? The question itself
link |
is a profound question, because we humans ourselves
link |
cannot answer, and that's the very point. I would kill both.
link |
Yeah, humans, I think you're exactly right in that, you know, humans are not
link |
particularly good. I think they're kind of phrased as, like, what would a computer do,
link |
but, like, humans, you know, are not very good, and actually oftentimes
link |
I think that, you know, freezing and kind of not doing anything, because,
link |
like, you've taken a few extra milliseconds to just process, and then
link |
you end up, like, doing the worst of the possible outcomes, right? So,
link |
I do think that, as you've pointed out, it can be
link |
a bit of a distraction, and it can be a bit of a kind of red herring. I think
link |
it's an interesting, you know, discussion
link |
in the realm of philosophy, right? But in terms of
link |
what, you know, how that affects the actual
link |
engineering and deployment of self driving vehicles,
link |
it's not how you go about building a system, right? We've talked
link |
about how you engineer a system, how you, you know, go about evaluating
link |
the different components and, you know, the safety of the entire thing.
link |
How do you kind of inject the, you know, various
link |
model based, safety based arguments, and, like, yes, you reason at parts of the
link |
system, you know, you reason about the
link |
probability of a collision, the severity of that collision, right?
link |
And that is incorporated, and there's, you know, you have to properly reason
link |
about the uncertainty that flows through the system, right? So,
link |
you know, those, you know, factors definitely play a role in how
link |
the cars then behave, but they tend to be more
link |
of, like, the emergent behavior. And what you see, like, you're absolutely right
link |
that these, you know, clear theoretical problems that they, you
link |
know, you don't encounter that in the system, and really kind of being
link |
back to our previous discussion of, like, what, you know, what, you
link |
know, which one do you choose? Well, you know, oftentimes, like,
link |
you made a mistake earlier. Like, you shouldn't be in that situation
link |
in the first place, right? And in reality, the system comes up.
link |
If you build a very good, safe, and capable driver,
link |
you have enough, you know, clues in the environment that you
link |
drive defensively, so you don't put yourself in that situation, right? And
link |
again, you know, it has, you know, this, if you go back to that analogy of, you
link |
know, precision and recoil, like, okay, you can make a, you know, very hard trade
link |
off, but like, neither answer is really good.
link |
But what instead you focus on is kind of moving
link |
the whole curve up, and then you focus on building the right capability on the
link |
right defensive driving, so that, you know, you don't put yourself in the
link |
situation like this. I don't know if you have a good answer
link |
for this, but people love it when I ask this question
link |
about books. Are there books in your life that you've enjoyed,
link |
philosophical, fiction, technical, that had a big impact on you as an engineer or
link |
as a human being? You know, everything from science fiction
link |
to a favorite textbook. Is there three books that stand out that
link |
you can think of? Three books. So I would, you know, that
link |
impacted me, I would say,
link |
and this one is, you probably know it well,
link |
but not generally well known, I think, in the U.S., or kind of
link |
internationally, The Master and Margarita. It's one of, actually, my
link |
favorite books. It is, you know, by
link |
Russian, it's a novel by Russian author Mikhail Bulgakov, and it's just, it's a
link |
great book. It's one of those books that you can, like,
link |
reread your entire life, and it's very accessible. You can read it as a kid,
link |
and, like, it's, you know, the plot is interesting. It's, you know, the
link |
devil, you know, visiting the Soviet Union,
link |
and, you know, but it, like, you read it, reread it
link |
at different stages of your life, and you enjoy it for
link |
different, very different reasons, and you keep finding, like, deeper and deeper
link |
meaning, and, you know, kind of affected, you know,
link |
had a, definitely had an, like, imprint on me, you know, mostly from the,
link |
probably kind of the cultural, stylistic aspect. Like, it makes you think one of
link |
those books that, you know, is good and makes you think, but also has,
link |
like, this really, you know, silly, quirky, dark sense of, you know,
link |
humor. It captures the Russian soul more than
link |
many, perhaps, many other books. On that, like, slight note,
link |
just out of curiosity, one of the saddest things is I've read that book
link |
in English. Did you, by chance, read it in English or in Russian?
link |
In Russian, only in Russian, and I actually, that is a question I had,
link |
kind of posed to myself every once in a while, like, I wonder how well it
link |
translates, if it translates at all, and there's the
link |
language aspect of it, and then there's the cultural aspect, so
link |
I, actually, I'm not sure if, you know, either of those would
link |
work well in English. Now, I forget their names, but, so, when the COVID lifts a
link |
little bit, I'm traveling to Paris for several reasons. One is just, I've
link |
never been to Paris, I want to go to Paris, but
link |
there's the most famous translators of Dostoevsky, Tolstoy, of most of
link |
Russian literature live there. There's a couple, they're famous,
link |
a man and a woman, and I'm going to, sort of, have a series of conversations with
link |
them, and in preparation for that, I'm starting
link |
to read Dostoevsky in Russian, so I'm really embarrassed to say that I read
link |
this, everything I've read in Russian literature of, like,
link |
serious depth has been in English, even though
link |
I can also read, I mean, obviously, in Russian, but
link |
for some reason, it seemed,
link |
in the optimization of life, it seemed the improper decision to do, to read in
link |
Russian, like, you know, like, I don't need to,
link |
I need to think in English, not in Russian, but now I'm changing my mind on
link |
that, and so, the question of how well I translate, it's a
link |
really fun to method one, like, even with Dostoevsky.
link |
So, from what I understand, Dostoevsky translates easier,
link |
others don't as much. Obviously, the poetry doesn't translate as well,
link |
I'm also the music big fan of Vladimir Vosotsky,
link |
he doesn't obviously translate well, people have tried,
link |
but mastermind, I don't know, I don't know about that one, I just know in
link |
English, you know, as fun as hell in English, so, so, but
link |
it's a curious question, and I want to study it rigorously from both the
link |
machine learning aspect, and also because I want to do a
link |
couple of interviews in Russia, that
link |
I'm still unsure of how to properly conduct an interview
link |
across a language barrier, it's a fascinating question
link |
that ultimately communicates to an American audience. There's a few
link |
Russian people that I think are truly special human beings,
link |
and I feel, like, I sometimes encounter this with some
link |
incredible scientists, and maybe you encounter this
link |
as well at some point in your life, that it feels like because of the language
link |
barrier, their ideas are lost to history. It's a sad thing, I think about, like,
link |
Chinese scientists, or even authors that, like,
link |
that we don't, in an English speaking world, don't get to appreciate
link |
some, like, the depth of the culture because it's lost in translation,
link |
and I feel like I would love to show that to the world,
link |
like, I'm just some idiot, but because I have this,
link |
like, at least some semblance of skill in speaking Russian,
link |
I feel like, and I know how to record stuff on a video camera,
link |
I feel like I want to catch, like, Grigori Perlman, who's a mathematician, I'm not
link |
sure if you're familiar with him, I want to talk to him, like, he's a
link |
fascinating mind, and to bring him to a wider audience in English speaking
link |
will be fascinating, but that requires to be rigorous about this question
link |
of how well Bulgakov translates. I mean, I know it's a silly
link |
concept, but it's a fundamental one, because how do you translate, and
link |
that's the thing that Google Translate is also facing
link |
as a more machine learning problem, but I wonder as a more
link |
bigger problem for AI, how do we capture the magic
link |
that's there in the language? I think that's a really interesting,
link |
really challenging problem. If you do read it, Master and Margarita
link |
in English, sorry, in Russian, I'd be curious
link |
to get your opinion, and I think part of it is language, but part of it's just,
link |
you know, centuries of culture, that, you know, the cultures are
link |
different, so it's hard to connect that.
link |
Okay, so that was my first one, right? You had two more. The second one I
link |
would probably pick is the science fiction by the
link |
Strogatsky brothers. You know, it's up there with, you know,
link |
Isaac Asimov and, you know, Ray Bradbury and, you know, company. The
link |
Strogatsky brothers kind of appealed more to me. I think it made more of an
link |
impression on me growing up. I apologize if I'm
link |
showing my complete ignorance. I'm so weak on sci fi. What did
link |
they write? Oh, Roadside Picnic,
link |
Heart to Be a God,
link |
Beetle in an Ant Hill, Monday Starts on Saturday. Like, it's
link |
not just science fiction. It also has very interesting, you know,
link |
interpersonal and societal questions, and some of the
link |
language is just completely hilarious.
link |
That's the one. Oh, interesting. Monday Starts on Saturday. So,
link |
I need to read. Okay, oh boy. You put that in the category of science fiction?
link |
That one is, I mean, this was more of a silly,
link |
you know, humorous work. I mean, there is kind of...
link |
It's profound too, right? Science fiction, right? It's about, you know, this
link |
research institute, and it has deep parallels to
link |
serious research, but the setting, of course,
link |
is that they're working on, you know, magic, right? And there's a
link |
lot of stuff. And that's their style, right?
link |
And, you know, other books are very different, right? You know,
link |
Heart to Be a God, right? It's about kind of this higher society being injected
link |
into this primitive world, and how they operate there,
link |
and some of the very deep ethical questions there,
link |
right? And, like, they've got this full spectrum. Some is, you know, more about
link |
kind of more adventure style. But, like, I enjoy all of
link |
their books. There's just, you know, probably a couple.
link |
Actually, one I think that they consider their most important work.
link |
I think it's The Snail on a Hill. I'm not exactly sure how it
link |
translates. I tried reading a couple times. I still don't get it.
link |
But everything else I fully enjoyed. And, like, for one of my birthdays as a kid, I
link |
got, like, their entire collection, like, occupied a giant shelf in my room, and
link |
then, like, over the holidays, I just, like,
link |
you know, my parents couldn't drag me out of the room, and I read the whole thing
link |
cover to cover. And I really enjoyed it.
link |
And that's one more. For the third one, you know, maybe a little bit
link |
darker, but, you know, comes to mind is Orwell's
link |
1984. And, you know, you asked what made an
link |
impression on me and the books that people should read. That one, I think,
link |
falls in the category of both. You know, definitely it's one of those
link |
books that you read, and you just kind of, you know, put it
link |
down and you stare in space for a while. You know, that kind of work. I think
link |
there's, you know, lessons there. People should
link |
not ignore. And, you know, nowadays, with, like,
link |
everything that's happening in the world, I,
link |
like, can't help it, but, you know, have my mind jump to some,
link |
you know, parallels with what Orwell described. And, like, there's this whole,
link |
you know, concept of double think and ignoring logic and, you know, holding
link |
completely contradictory opinions in your mind and not have that not bother
link |
you and, you know, sticking to the party line
link |
at all costs. Like, you know, there's something there.
link |
If anything, 2020 has taught me, and I'm a huge fan of Animal Farm, which is a
link |
kind of friendly, as a friend of 1984 by Orwell.
link |
It's kind of another thought experiment of how our society
link |
may go in directions that we wouldn't like it to go.
link |
But if anything that's been kind of heartbreaking to an
link |
optimist about 2020 is that
link |
that society is kind of fragile. Like, we have this,
link |
this is a special little experiment we have going on.
link |
And not, it's not unbreakable. Like, we should be careful to, like, preserve
link |
whatever the special thing we have going on. I mean, I think 1984
link |
and these books, The Brave New World, they're
link |
helpful in thinking, like, stuff can go wrong
link |
in nonobvious ways. And it's, like, it's up to us to preserve it.
link |
And it's, like, it's a responsibility. It's been weighing heavy on me because, like,
link |
for some reason, like, more than my mom follows me on Twitter and I
link |
feel like I have, like, now somehow a
link |
do this world. And it dawned on me that, like,
link |
me and millions of others are, like, the little ants
link |
that maintain this little colony, right? So we have a responsibility not to
link |
be, I don't know what the right analogy is, but
link |
I'll put a flamethrower to the place. We want to
link |
not do that. And there's interesting complicated ways of doing that as 1984
link |
shows. It could be through bureaucracy. It could
link |
be through incompetence. It could be through misinformation.
link |
It could be through division and toxicity.
link |
I'm a huge believer in, like, that love will be
link |
the, somehow, the solution. So, love and robots. Love and robots, yeah.
link |
I think you're exactly right. Unfortunately, I think it's less of a
link |
flamethrower type of thing. It's more of a,
link |
in many cases, it's going to be more of a slow boil. And that's the
link |
danger. Let me ask, it's a fun thing to make
link |
a world class roboticist, engineer, and leader uncomfortable with a
link |
ridiculous question about life. What is the meaning of life,
link |
Dimitri, from a robotics and a human perspective?
link |
You only have a couple minutes, or one minute to answer, so.
link |
I don't know if that makes it more difficult or easier, actually.
link |
You know, they're very tempted to quote one of the
link |
stories by Isaac Asimov, actually. Actually, titled,
link |
appropriately titled, The Last Question. It's a short story where, you know, the
link |
plot is that, you know, humans build this supercomputer,
link |
you know, this AI intelligence, and, you know, once it
link |
gets powerful enough, they pose this question to it, you know,
link |
how can the entropy in the universe be reduced, right? So the computer replies,
link |
as of yet, insufficient information to give a meaningful answer,
link |
right? And then, you know, thousands of years go by, and they keep posing the
link |
same question, and the computer, you know, gets more and more powerful, and keeps
link |
giving the same answer, you know, as of yet, insufficient
link |
information to give a meaningful answer, or something along those lines,
link |
right? And then, you know, it keeps, you know, happening, and
link |
happening, you fast forward, like, millions of years into the future, and,
link |
you know, billions of years, and, like, at some point, it's just the only entity in
link |
the universe, it's, like, absorbed all humanity,
link |
and all knowledge in the universe, and it, like, keeps posing the same question
link |
to itself, and, you know, finally, it gets to the
link |
point where it is able to answer that question, but, of course, at that point,
link |
you know, there's, you know, the heat death of the universe has occurred, and
link |
that's the only entity, and there's nobody else to provide that
link |
answer to, so the only thing it can do is to,
link |
you know, answer it by demonstration, so, like, you know, it recreates the big bang,
link |
right, and resets the clock, right?
link |
But, like, you know, I can try to give kind of a
link |
different version of the answer, you know, maybe
link |
not on the behalf of all humanity, I think that that might be a little
link |
presumptuous for me to speak about the meaning of life on the behalf of all
link |
humans, but at least, you know, personally,
link |
it changes, right? I think if you think about kind of what
link |
gives, you know, you and your life meaning and purpose, and kind of
link |
what drives you, it seems to
link |
change over time, right, and that lifespan
link |
of, you know, kind of your existence, you know, when
link |
just when you just enter this world, right, it's all about kind of new
link |
experiences, right? You get, like, new smells, new sounds, new emotions, right,
link |
and, like, that's what's driving you, right? You're experiencing
link |
new amazing things, right, and that's magical, right? That's pretty
link |
pretty awesome, right? That gives you kind of meaning.
link |
Then, you know, you get a little bit older, you start more intentionally
link |
learning about things, right? I guess, actually, before you start intentionally
link |
learning, it's probably fun. Fun is a thing that gives you kind of
link |
meaning and purpose and purpose and the thing you optimize for, right?
link |
And, like, fun is good. Then you get, you know, start learning, and I guess that
link |
this joy of comprehension
link |
and discovery is another thing that, you know, gives you
link |
meaning and purpose and drives you, right? Then, you know, you
link |
learn enough stuff and you want to give some of it back, right? And so
link |
impact and contributions back to, you know, technology or society,
link |
you know, people, you know, local or more globally
link |
becomes a new thing that, you know, drives a lot of kind of your behavior
link |
and is something that gives you purpose and
link |
that you derive, you know, positive feedback from, right?
link |
You know, then you go and so on and so forth. You go through various stages of
link |
life. If you have kids,
link |
like, that definitely changes your perspective on things. You know, I have
link |
three that definitely flips some bits in your
link |
head in terms of, you know, what you care about and what you
link |
optimize for and, you know, what matters, what doesn't matter, right?
link |
So, you know, and so on and so forth, right? And I,
link |
it seems to me that, you know, it's all of those things and as
link |
kind of you go through life, you know,
link |
you want these to be additive, right? New experiences,
link |
fun, learning, impact. Like, you want to, you know, be accumulating.
link |
I don't want to, you know, stop having fun or, you know, experiencing new things and
link |
I think it's important that, you know, it just kind of becomes
link |
additive as opposed to a replacement or subtraction.
link |
But, you know, those fewest problems as far as I got, but, you know, ask me in a
link |
few years, I might have one or two more to add to the list.
link |
And before you know it, time is up, just like it is for this conversation,
link |
but hopefully it was a fun ride. It was a huge honor to meet you.
link |
As you know, I've been a fan of yours and a fan of Google Self Driving Car and
link |
Waymo for a long time. I can't wait. I mean, it's one of the
link |
most exciting, if we look back in the 21st century, I
link |
truly believe it'll be one of the most exciting things we
link |
descendants of apes have created on this earth. So,
link |
I'm a huge fan and I can't wait to see what you do
link |
next. Thanks so much for talking to me. Thanks, thanks for having me and it's a
link |
also a huge fan doing work, honestly, and I really
link |
enjoyed it. Thank you. Thanks for listening to this
link |
conversation with Dmitry Dolgov and thank you to our sponsors,
link |
Triolabs, a company that helps businesses apply machine learning to
link |
solve real world problems, Blinkist, an app I use for reading
link |
through summaries of books, BetterHelp, online therapy with a licensed
link |
professional, and CashApp, the app I use to send money to
link |
friends. Please check out these sponsors in the
link |
description to get a discount and to support this podcast. If you
link |
enjoy this thing, subscribe on YouTube, review it with Five Stars
link |
and Upper Podcast, follow on Spotify, support on Patreon,
link |
or connect with me on Twitter at Lex Friedman. And now,
link |
let me leave you with some words from Isaac Asimov.
link |
Science can amuse and fascinate us all, but it is engineering
link |
that changes the world. Thank you for listening and hope to see you