back to indexGeorge Hotz: Comma.ai, OpenPilot, and Autonomous Vehicles | Lex Fridman Podcast #31
link |
The following is a conversation with George Hotz.
link |
He's the founder of Comma AI,
link |
a machine learning based vehicle automation company.
link |
He is most certainly an outspoken personality
link |
in the field of AI and technology in general.
link |
He first gained recognition for being the first person
link |
to carry on lock and iPhone.
link |
And since then, he's done quite a few interesting things
link |
at the intersection of hardware and software.
link |
This is the artificial intelligence podcast.
link |
If you enjoy it, subscribe on YouTube,
link |
give it five stars on iTunes, support it on Patreon,
link |
or simply connect with me on Twitter.
link |
Alex Friedman, spelled F R I D M A N.
link |
And I'd like to give a special thank you to Jennifer
link |
from Canada for her support of the podcast on Patreon.
link |
Merci beaucoup, Jennifer.
link |
She's been a friend and an engineering colleague
link |
for many years since I was in grad school.
link |
Your support means a lot and inspires me
link |
to keep this series going.
link |
And now here's my conversation with George Hotz.
link |
Do you think we're living in a simulation?
link |
Yes, but it may be unfalsifiable.
link |
What do you mean by unfalsifiable?
link |
So if the simulation is designed in such a way
link |
that they did like a formal proof
link |
to show that no information can get in and out.
link |
And if their hardware is designed for the anything
link |
in the simulation to always keep the hardware in spec,
link |
it may be impossible to prove
link |
whether we're in a simulation or not.
link |
So they've designed it such that it's a closed system,
link |
you can't get outside the system.
link |
Well, maybe it's one of three worlds.
link |
We're either in a simulation which can be exploited,
link |
we're in a simulation which not only can't be exploited,
link |
but like the same thing's true about VMs.
link |
A really well designed VM,
link |
you can't even detect if you're in a VM or not.
link |
So we're, yeah, so the simulation is running
link |
on a virtual machine.
link |
But now in reality, all VMs have ways to detect.
link |
I mean, is it, you've done quite a bit of hacking yourself.
link |
So you should know that really any complicated system
link |
will have ways in and out.
link |
So this isn't necessarily true going forward.
link |
I spent my time away from comma,
link |
I learned a cock, it's a dependently typed,
link |
like it's a language for writing math proofs.
link |
And if you write code that compiles in a language like that,
link |
it is correct by definition.
link |
The types check it's correctance.
link |
So it's possible that the simulation
link |
is written in a language like this, in which case, yeah.
link |
Yeah, but that can't be sufficiently expressive
link |
of language like that.
link |
Okay, well, so, all right, so.
link |
The simulation doesn't have to be tearing complete
link |
if it has a scheduled end date.
link |
Looks like it does actually with entropy.
link |
I mean, I don't think that a simulation
link |
that results in something as complicated as the universe
link |
would have a formal proof of correctness, right?
link |
It's possible, of course.
link |
We have no idea how good their tooling is.
link |
And we have no idea how complicated
link |
the universe computer really is.
link |
It may be quite simple.
link |
It's just very large, right?
link |
It's very, it's definitely very large.
link |
But the fundamental rules might be super simple.
link |
Yeah, Conway's gonna like kinda stop.
link |
Right, so if you could hack,
link |
so imagine the simulation that is hackable,
link |
if you could hack it,
link |
what would you change about the universe?
link |
Like how would you approach hacking a simulation?
link |
The reason I gave that talk?
link |
By the way, I'm not familiar with the talk you gave.
link |
I just read that you talked about escaping the simulation
link |
or something like that.
link |
So maybe you can tell me a little bit
link |
about the theme and the message there too.
link |
It wasn't a very practical talk
link |
about how to actually escape a simulation.
link |
It was more about a way of restructuring
link |
an us versus them narrative.
link |
If we continue on the path we're going with technology,
link |
I think we're in big trouble,
link |
like as a species and not just as a species,
link |
but even as me as an individual member of the species.
link |
So if we could change rhetoric to be more like,
link |
like to think about that we're in a simulation
link |
and how we could get out,
link |
already we'd be on the right path.
link |
What you actually do once you do that,
link |
well, I assume I would have acquired way more intelligence
link |
in the process of doing that, so I'll just ask that.
link |
So the thinking upwards,
link |
what kind of ideas,
link |
what kind of breakthrough ideas do you think thinking
link |
in that way could inspire?
link |
And why did you say upwards?
link |
Are you thinking sort of exploration in all forms?
link |
The space narrative that held for the modernist generation
link |
doesn't hold as well for the postmodern generation.
link |
What's the space narrative?
link |
Are we talking about the same space?
link |
The three dimensional space?
link |
No, no, space, like going up space,
link |
like building like Elon Musk,
link |
like we're going to build rockets,
link |
we're going to go to Mars,
link |
we're going to colonize the universe.
link |
And the narrative you're referring,
link |
I was born in the Soviet Union,
link |
you're referring to the race to space?
link |
The race to space, yeah.
link |
Yes, explore, okay.
link |
That was a great modernist narrative.
link |
It doesn't seem to hold the same weight in today's culture.
link |
I'm hoping for good postmodern narratives that replace it.
link |
So let's think, so you work a lot with AI.
link |
So AI is one formulation of that narrative.
link |
There could be also,
link |
I don't know how much you do in VR and AR.
link |
That's another, I know less about it,
link |
but every time I play with it and our research,
link |
it's fascinating, that virtual world.
link |
Are you interested in the virtual world?
link |
I would like to move to virtual reality.
link |
In terms of your work?
link |
No, I would like to physically move there.
link |
The apartment I can rent in the cloud
link |
is way better than the apartment I can rent in the real world.
link |
Well, it's all relative, isn't it?
link |
Because others will have very nice apartments too,
link |
so you'll be inferior in the virtual world as well.
link |
But that's not how I view the world, right?
link |
I don't view the world.
link |
I mean, that's a very like, almost zero summish way
link |
to view the world.
link |
Say like, my great apartment isn't great
link |
because my neighbor has one too.
link |
No, my great apartment is great
link |
because like, look at this dishwasher, man.
link |
You just touch the dish and it's washed, right?
link |
And that is great in and of itself
link |
if I had the only apartment
link |
or if everybody had the apartment.
link |
So you have fundamental gratitude.
link |
The world first learned of Geohot, George Hots
link |
in August 2007, maybe before then,
link |
but certainly in August 2007
link |
when you were the first person to unlock,
link |
carry on lock an iPhone.
link |
How did you get into hacking?
link |
What was the first system you discovered
link |
vulnerabilities for and broke into?
link |
So that was really kind of the first thing.
link |
I had a book in 2006 called Gray Hat Hacking.
link |
And I guess I realized that
link |
if you acquired these sort of powers
link |
you could control the world.
link |
But I didn't really know that much
link |
about computers back then.
link |
I started with electronics.
link |
The first iPhone hack was physical.
link |
You had to open it up and pull an address line high.
link |
And it was because I didn't really know
link |
about software exploitation.
link |
I learned that all in the next few years
link |
and I got very good at it.
link |
But back then I knew about like
link |
how memory chips are connected to processors and stuff.
link |
But you knew about software and programming.
link |
Oh really, so your view of the world
link |
and computers was physical, was hardware.
link |
Actually, if you read the code that I released with that
link |
in August 2007, it's atrocious.
link |
What language was it?
link |
And in a broken sort of state machine, ask C.
link |
I didn't know how to program.
link |
So how did you learn to program?
link |
What was your journey?
link |
I mean, we'll talk about it.
link |
You've live streamed some of your programming.
link |
This chaotic, beautiful mess.
link |
How did you arrive at that?
link |
Years and years of practice.
link |
I interned at Google after,
link |
the summer after the iPhone unlock.
link |
And I did a contract for them
link |
where I built a hardware for Street View
link |
and I wrote a software library to interact with it.
link |
And it was terrible code.
link |
And for the first time I got feedback
link |
from people who I respected saying,
link |
no, like, don't write code like this.
link |
Now, of course, just getting that feedback is not enough.
link |
The way that I really got good was,
link |
I wanted to write this thing that could emulate
link |
and then visualize like arm binaries
link |
because I wanted to hack the iPhone better.
link |
And I didn't like that I couldn't see what the,
link |
I couldn't single step through the processor
link |
because I had no debugger on there,
link |
especially for the low level things like the boot ROM
link |
and the boot loader.
link |
So I tried to build this tool to do it.
link |
And I built the tool once and it was terrible.
link |
I built the tool second times, it was terrible.
link |
I built the tool third time.
link |
This by the time I was at Facebook, it was kind of okay.
link |
And then I built the tool fourth time
link |
when I was a Google intern again in 2014.
link |
And that was the first time I was like,
link |
this is finally usable.
link |
How do you pronounce this, Kira?
link |
So it's essentially the most efficient way
link |
to visualize the change of state of the computer
link |
as the program is running.
link |
That's what you mean by debugger.
link |
Yeah, it's a timeless debugger.
link |
So you can rewind just as easily as going forward.
link |
Think about, if you're using GDB,
link |
you have to put a watch on a variable.
link |
If you want to see if that variable changes.
link |
In Kira, you can just click on that variable.
link |
And then it shows every single time
link |
when that variable was changed or accessed.
link |
Think about it like get for your computer's, the run lock.
link |
So there's like a deep log of the state of the computer
link |
as the program runs and you can rewind.
link |
Why isn't that, maybe it is, maybe you can educate me.
link |
Why isn't that kind of debugging used more often?
link |
Because the tooling's bad.
link |
One, if you're trying to debug Chrome,
link |
Chrome is a 200 megabyte binary
link |
that runs slowly on desktops.
link |
So that's gonna be really hard to use for that.
link |
But it's really good to use for like CTFs
link |
and for boot ROMs and for small parts of code.
link |
So it's hard if you're trying to debug like massive systems.
link |
What's a CTF and what's a boot ROM?
link |
A boot ROM is the first code that executes
link |
the minute you give power to your iPhone.
link |
And CTF were these competitions that I played.
link |
I was gonna ask you about that.
link |
What are those, those look at,
link |
I watched a couple of videos on YouTube.
link |
Those look fascinating.
link |
What have you learned about maybe at the high level
link |
in the vulnerability of systems from these competitions?
link |
I feel like in the heyday of CTFs,
link |
you had all of the best security people in the world
link |
challenging each other and coming up
link |
with new toy exploitable things over here.
link |
And then everybody, okay, who can break it?
link |
And when you break it, you get like,
link |
there's like a file in the server called flag.
link |
And then there's a program running,
link |
listening on a socket that's vulnerable.
link |
So you write an exploit, you get a shell,
link |
and then you cat flag, and then you type the flag
link |
into like a web based scoreboard and you get points.
link |
So the goal is essentially to find an exploit in the system
link |
that allows you to run shell,
link |
to run arbitrary code on that system.
link |
That's one of the categories.
link |
That's like the Poneable category.
link |
It's like, you know, you Pone the program.
link |
You know, first of all, I apologize, I'm gonna say,
link |
it's because I'm Russian,
link |
but maybe you can help educate me.
link |
Some video game like misspelled
link |
to own way back in the day.
link |
Yeah, and it's just,
link |
I wonder if there's a definition
link |
and I'll have to go to Urban Dictionary for it.
link |
Yeah, it'd be interesting to see what it says.
link |
Okay, so what was the heyday of CTL, by the way,
link |
but was it, what decade are we talking about?
link |
I think like, I mean, maybe I'm biased
link |
because it's the era that I played,
link |
but like 2011 to 2015,
link |
because the modern CTF scene
link |
is similar to the modern competitive programming scene.
link |
You have people who like do drills.
link |
You have people who practice.
link |
And then once you've done that,
link |
you've turned it less into a game of generic computer skill
link |
and more into a game of, okay, you memorize,
link |
you drill on these five categories.
link |
And then before that, it wasn't,
link |
it didn't have like as much attention as it had.
link |
I don't know, they were like,
link |
I won $30,000 once in Korea
link |
for one of these competitions.
link |
Yeah, they were, they were, that was...
link |
So that means, I mean, money is money,
link |
but that means there was probably good people there.
link |
Are the challenges human constructed
link |
or are they grounded in some real flaws in real systems?
link |
Usually they're human constructed,
link |
but they're usually inspired by real flaws.
link |
What kind of systems are imagined
link |
is really focused on mobile?
link |
Like what has vulnerabilities these days?
link |
Is it primarily mobile systems like Android?
link |
Oh, everything does.
link |
The price has kind of gone up
link |
because less and less people can find them.
link |
And what's happened in security is now,
link |
if you want to like jailbreak an iPhone,
link |
you don't need one exploit anymore, you need nine.
link |
Nine change together?
link |
What would you mean?
link |
Okay, so it's really, what's the benefit?
link |
Speaking higher level philosophically about hacking.
link |
I mean, it sounds from everything I've seen about you,
link |
you just love the challenge and you don't want to do anything.
link |
You don't want to bring that exploit out into the world
link |
and do any actual, let it run wild.
link |
You just want to solve it
link |
and then you go on to the next thing.
link |
Oh yeah, I mean, doing criminal stuff's not really worth it.
link |
And I'll actually use the same argument
link |
for why I don't do defense for why I don't do crime.
link |
If you want to defend a system,
link |
say the system has 10 holes, right?
link |
If you find nine of those holes as a defender,
link |
you still lose because the attacker gets in
link |
through the last one.
link |
If you're an attacker,
link |
you only have to find one out of the 10.
link |
But if you're a criminal,
link |
if you log on with a VPN nine out of the 10 times,
link |
but one time you forget, you're done.
link |
Because you're caught, okay.
link |
Because you only have to mess up once
link |
to be caught as a criminal.
link |
That's why I'm not a criminal.
link |
cause I was having a discussion with somebody
link |
just at a high level about nuclear weapons,
link |
actually why we're having blown ourselves up yet.
link |
And my feeling is all the smart people in the world,
link |
if you look at the distribution of smart people,
link |
smart people are generally good.
link |
And then the Southern person,
link |
I was talking to Sean Carroll, the physicist,
link |
and he was saying no good and bad people
link |
are evenly distributed amongst everybody.
link |
My sense was good hackers are in general good people
link |
and they don't want to mess with the world.
link |
What's your sense?
link |
I'm not even sure about that.
link |
Like, I have a nice life.
link |
Crime wouldn't get me anything.
link |
But if you're good and you have these skills,
link |
you probably have a nice life too, right?
link |
Right, you can use the father things.
link |
But is there an ethical,
link |
is there a little voice in your head that says,
link |
well, yeah, if you could hack something
link |
to where you could hurt people
link |
and you could earn a lot of money doing it though,
link |
not hurt physically perhaps,
link |
but disrupt their life in some kind of way.
link |
Isn't there a little voice that says,
link |
One, I don't really care about money.
link |
So like the money wouldn't be an incentive.
link |
The thrill might be an incentive.
link |
But when I was 19, I read crime and punishment.
link |
That was another great one
link |
that talked me out of ever really doing crime.
link |
Cause it's like, that's gonna be me.
link |
I'd get away with it, but it would just run through my head.
link |
Even if I got away with it, you know?
link |
And then you do crime for long enough,
link |
you'll never get away with it.
link |
That's right, in the end.
link |
That's a good reason to be good.
link |
I wouldn't say I'm good, I would just say I'm not bad.
link |
You're a talented programmer and a hacker
link |
in a good positive sense of the word.
link |
You've played around, found vulnerabilities
link |
in various systems.
link |
What have you learned broadly
link |
about the design of systems and so on
link |
from that whole process?
link |
You learn to not take things
link |
for what people say they are,
link |
but you look at things for what they actually are.
link |
I understand that's what you tell me it is,
link |
but what does it do?
link |
And you have nice visualization tools
link |
to really know what it's really doing.
link |
Oh, I wish I'm a better programmer now than I was in 2014.
link |
I said, Kira, that was the first tool
link |
that I wrote that was usable.
link |
I wouldn't say the code was great.
link |
I still wouldn't say my code is great.
link |
So how was your evolution as a programmer?
link |
You started with C,
link |
what point did you pick up Python?
link |
Because you're pretty big in Python now.
link |
Now, yeah, in college,
link |
I went to Carnegie Mellon when I was 22.
link |
I went back, I'm like,
link |
I'm gonna take all your hardest CS courses
link |
and we'll see how I do, right?
link |
Like, did I miss anything
link |
by not having a real undergraduate education?
link |
Took operating systems, compilers, AI,
link |
and they're like a freshman Weeder math course.
link |
And some of those classes you mentioned,
link |
pretty tough, actually.
link |
At least when the 2012,
link |
circa 2012 operating systems and compilers
link |
were two of the best classes I've ever taken in my life.
link |
Because you write an operating system
link |
and you write a compiler.
link |
I wrote my operating system in C
link |
and I wrote my compiler in Haskell,
link |
but somehow I picked up Python that semester as well.
link |
I started using it for the CTFs, actually.
link |
That's when I really started to get into CTFs
link |
and CTFs, you're all to race against the clock.
link |
So I can't write things and see.
link |
Oh, there's a clock component.
link |
So you really want to use the programming language
link |
just so you can be fastest.
link |
Pwn as many of these challenges as you can.
link |
You got like 100 points of challenge,
link |
whatever team gets the most.
link |
You were both at Facebook and Google for a brief stint.
link |
With Project Zero, actually, at Google for five months
link |
where you develop Kira.
link |
What was Project Zero about in general?
link |
Just curious about the security efforts in these companies.
link |
Well, Project Zero started the same time I went there.
link |
What year is it there?
link |
So that was right at the beginning of Project Zero.
link |
It's Google's offensive security team.
link |
I'll try to give the best public facing explanation
link |
So the idea is basically,
link |
these vulnerabilities exist in the world.
link |
Nation states have them.
link |
Some high powered bad actors have them.
link |
Sometimes people will find these vulnerabilities
link |
and submit them in bug bounties to the companies.
link |
But a lot of the companies don't only care.
link |
They don't even fix the bug.
link |
It doesn't hurt for there to be a vulnerability.
link |
So Project Zero is like, we're going to do it different.
link |
We're going to announce a vulnerability
link |
and we're going to give them 90 days to fix it.
link |
And then whether they fix it or not,
link |
we're going to drop the Zero Day.
link |
We're going to drop the weapon on the textbook.
link |
I love that deadlines.
link |
Oh, that's so cool.
link |
Give them real deadlines.
link |
And I think it's done a lot for moving the industry forward.
link |
I watched your coding sessions on the streamed online.
link |
You code things up, the basic projects, usually from scratch.
link |
I would say, sort of as a programmer myself,
link |
just watching you, that you type really fast
link |
and your brain works in both brilliant and chaotic ways.
link |
I don't know if that's always true,
link |
but certainly for the live streams.
link |
So it's interesting to me because I'm much slower
link |
and systematic and careful.
link |
And you just move probably in order of magnitude faster.
link |
So I'm curious, is there a method to your madness?
link |
Or is it just who you are?
link |
There's pros and cons.
link |
There's pros and cons to my programming style.
link |
And I'm aware of them.
link |
If you ask me to get something up and working quickly
link |
with an API that's kind of undocumented,
link |
I will do this super fast because I will throw things
link |
at it until it works.
link |
If you ask me to take a vector and rotate it 90 degrees
link |
and then flip it over the X, Y plane,
link |
I'll spam program for two hours and won't get it.
link |
Oh, because it's something that you
link |
could do with a sheet of paper or think through design
link |
and then just you really just throw stuff at the wall
link |
and you get so good at it that it usually works.
link |
I should become better at the other kind as well.
link |
Sometimes I will do things methodically.
link |
It's nowhere near as entertaining on the Twitch streams.
link |
I do exaggerate it a bit on the Twitch streams as well.
link |
The Twitch streams, I mean, what do you want to see a game
link |
or you want to see actions permit, right?
link |
I'll show you APM for programming too.
link |
Yeah, I'd recommend people go to it.
link |
I think I watched probably several hours that you put,
link |
like I've actually left you programming in the background
link |
while I was programming because you made me,
link |
it was like watching a really good gamer.
link |
It's like energizes you because you're like moving so fast
link |
and so it's awesome, it's inspiring.
link |
It made me jealous that like,
link |
because my own programming is inadequate
link |
in terms of speed, so I was like.
link |
So I'm twice as frantic on the live streams
link |
as I am when I code without, oh.
link |
It's super entertaining.
link |
So I wasn't even paying attention to what you were coding,
link |
which is great, it's just watching you switch windows
link |
and Vim, I guess is the most way.
link |
Yeah, does Vim on screen?
link |
I've developed a workload Facebook and stuck with it.
link |
How do you learn new programming tools,
link |
ideas, techniques these days?
link |
What's your like methodology for learning new things?
link |
So I wrote for comma,
link |
the distributed file systems out in the world
link |
are extremely complex.
link |
Like if you want to install something like like like Ceph,
link |
Ceph is I think the like open infrastructure
link |
distributed file system or there's like newer ones
link |
like seaweed FS, but these are all like 10,000
link |
plus line projects.
link |
I think some of them are even 100,000 line
link |
and just configuring them as a nightmare.
link |
So I wrote, I wrote one, it's 200 lines
link |
and it uses like engine X of the line servers
link |
and has this little master server that I wrote and go.
link |
And the way I go, this, if I would say
link |
that I'm proud per line of any code I wrote,
link |
maybe there's some exploits that I think are beautiful
link |
and then this, this is 200 lines
link |
and just the way that I thought about it,
link |
I think was very good and the reason it's very good
link |
is because that was the fourth version of it that I wrote
link |
and I had three versions that I threw away.
link |
You mentioned, did you say go?
link |
I wrote a go, yeah.
link |
Is that a functional language?
link |
I forget what go is.
link |
Go is Google's language.
link |
It's not functional.
link |
It's some, it's like, in a way it's C++, but easier.
link |
It's strongly typed.
link |
It has a nice ecosystem around it.
link |
When I first looked at it, I was like,
link |
this is like Python, but it takes twice as long
link |
Now that I've open pilot is migrating to C,
link |
but it still has large Python components,
link |
I now understand why Python doesn't work
link |
for large code bases and why you want something like go.
link |
So why, why doesn't Python work for,
link |
so even most, speaking for myself at least,
link |
like we do a lot of stuff, basically demo level work
link |
with autonomous vehicles and most of the work is Python.
link |
Why doesn't Python work for large code bases?
link |
Because, well, lack of type checking is a big one.
link |
So errors creep in.
link |
Yeah, and like you don't know,
link |
the compiler can tell you like nothing, right?
link |
So everything is either, you know,
link |
like syntax errors, fine,
link |
but if you misspell a variable in Python,
link |
the compiler won't catch that.
link |
There's like linters that can catch it some of the time.
link |
This is really the biggest downside.
link |
And then we'll Python slow, but that's not related to it.
link |
Well, maybe it's kind of related to it, so it's lack of.
link |
So what's in your toolbox these days?
link |
Is it Python or what else?
link |
I need to move to something else.
link |
My adventure into dependently typed languages,
link |
I love these languages.
link |
They just have like syntax from the 80s.
link |
What do you think about JavaScript?
link |
ES6, like the modern type script?
link |
JavaScript is, the whole ecosystem
link |
is unbelievably confusing.
link |
NPM updates a package from 022 to 025
link |
and that breaks your Babel linter,
link |
which translates your ES5 into ES6, which doesn't run on.
link |
So why do I have to compile my JavaScript again, huh?
link |
It may be the future though.
link |
You think about, I mean,
link |
I've embraced JavaScript recently
link |
just because just like I've continually embraced PHP,
link |
it seems that these worst possible languages live on
link |
for the longest, like cockroaches never die.
link |
Yeah, well, it's in the browser and it's fast.
link |
It's in the browser and compute might stay become,
link |
you know, the browser,
link |
it's unclear what the role of the browser is
link |
in terms of distributed computation in the future.
link |
JavaScript is definitely here to stay.
link |
It's interesting if autonomous vehicles
link |
will run on JavaScript one day.
link |
I mean, you have to consider these possibilities.
link |
Well, all our debug tools are JavaScript.
link |
We actually just open source them.
link |
We have a tool explorer, which you can annotate
link |
your disengagements and we have tool Kibana,
link |
which lets you analyze the can traffic from the car.
link |
So basically any time you're visualizing something
link |
about the log using JavaScript.
link |
Well, the web is the best UI toolkit by far.
link |
So, and then, you know what?
link |
You're coding in JavaScript.
link |
We have a React guy.
link |
Let's get into it.
link |
So let's talk autonomous vehicles.
link |
You found a comma AI.
link |
Let's, at a high level,
link |
how did you get into the world of vehicle automation?
link |
Can you also just, for people who don't know,
link |
tell the story of comma AI?
link |
So I was working at this AI startup
link |
and a friend approached me and he's like,
link |
dude, I don't know where this is going,
link |
but the coolest applied AI problem today
link |
is self driving cars.
link |
I'm like, well, absolutely.
link |
You wanna meet with Elon Musk
link |
and he's looking for somebody to build a vision system
link |
This is when they were still on AP one.
link |
They were still using Mobileye.
link |
Elon back then was looking for a replacement.
link |
And he brought me in and we talked about a contract
link |
where I would deliver something
link |
that meets Mobileye level performance.
link |
I would get paid $12 million if I could deliver it tomorrow
link |
and I would lose $1 million for every month I didn't deliver.
link |
So I was like, okay, this is a great deal.
link |
This is a super exciting challenge.
link |
It takes me 10 months, I get $2 million, it's good.
link |
Maybe I can finish up in five.
link |
Maybe I don't finish it at all and I get paid nothing
link |
and I'll work for 12 months for free.
link |
So maybe just take a pause on that.
link |
I'm also curious about this
link |
because I've been working in robotics for a long time.
link |
And I'm curious to see a person like you just step in
link |
and sort of somewhat naive, but brilliant, right?
link |
So that's the best place to be
link |
because you basically full steam take on a problem.
link |
How confident, from that time,
link |
because you know a lot more now,
link |
at that time, how hard do you think it is
link |
to solve all of autonomous driving?
link |
I remember I suggested to Elon in the meeting
link |
on putting a GPU behind each camera
link |
to keep the compute local.
link |
This is an incredibly stupid idea.
link |
I leave the meeting 10 minutes later and I'm like,
link |
I could have spent a little bit of time
link |
thinking about this problem before I went in.
link |
Why is this a stupid idea?
link |
Oh, just send all your cameras to one big GPU.
link |
You're much better off doing that.
link |
Oh, sorry, you said behind every camera.
link |
I was like, oh, I'll put the first few layers
link |
of my comms there.
link |
Like why did I say that?
link |
It's possible, but it's a bad idea.
link |
It's not obviously a bad idea.
link |
Pretty obviously bad.
link |
But whether it's actually a bad idea or not,
link |
I left that meeting with Elon, like beating myself up.
link |
I'm like, why did I say something stupid?
link |
Yeah, you haven't, like you haven't at least
link |
like thought through every aspect fully.
link |
He's very sharp too.
link |
Like usually in life, I get away with saying stupid things
link |
and then kind of course,
link |
right away he called me out about it.
link |
And like, usually in life,
link |
I get away with saying stupid things.
link |
And then like people will, you know,
link |
a lot of times people don't even notice.
link |
And I'll like correct it and bring the conversation back.
link |
But with Elon, it was like, nope, like, okay.
link |
Well, that's not at all why the contract fell through.
link |
I was much more prepared the second time I met him.
link |
But in general, how hard did you think it,
link |
like 12 months is a tough timeline?
link |
Oh, I just thought I'd clone Mobileye IQ three.
link |
I didn't think I'd solve level five self driving
link |
So the goal there was to do lane keeping,
link |
good lane keeping.
link |
I saw my friend showed me the outputs from Mobileye.
link |
And the outputs from Mobileye was just basically two lanes
link |
and a position of a lead car.
link |
I'm like, I can gather a data set
link |
and train this net in weeks.
link |
Well, first time I tried the implementation of Mobileye
link |
in a Tesla, I was really surprised how good it is.
link |
It's quite incredibly good.
link |
Cause I thought it's just cause I've done
link |
a lot of computer vision.
link |
I thought it'd be a lot harder to create a system
link |
that that's stable.
link |
So I was personally surprised.
link |
Just, you know, have to admit it.
link |
Cause I was kind of skeptical before trying it.
link |
Cause I thought it would go in and out a lot more.
link |
It would get disengaged a lot more.
link |
And it's pretty robust.
link |
So what, how, how, how hard is the problem
link |
when you, when you tackled it?
link |
So I think AP one was great. Like Elon talked
link |
about disengagements on the 405 down in LA
link |
with like the lane marks were kind of faded
link |
and the Mobileye system would drop out.
link |
Like I had something up and working
link |
that I would say was like the same quality in three months.
link |
Same quality, but how do you know?
link |
You say stuff like that confidently, but you can't,
link |
and I love it, but the question is you can't,
link |
you're kind of going by feel cause you just,
link |
You're going by feel, absolutely, absolutely.
link |
Like, like I would take, I hadn't,
link |
I borrowed my friend's Tesla.
link |
I would take AP one out for a drive.
link |
And then I would take my system out for a drive.
link |
And seems reasonably like the same.
link |
So the 405, how hard is it to create something
link |
that could actually be a product that's deployed?
link |
I mean, I've read an article where Elon, this respond,
link |
it said something about you saying that
link |
to build autopilot is more complicated
link |
than a single George Hodds level job.
link |
How hard is that job to create something
link |
that would work across the globally?
link |
Why don't the global is the challenge,
link |
but Elon followed that up by saying
link |
it's going to take two years and a company of 10 people.
link |
And here I am four years later with a company of 12 people.
link |
And I think we still have another two to go.
link |
So yeah, so what do you think,
link |
what do you think about how Tesla's progressing
link |
with autopilot of V2, V3?
link |
I think we've kept pace with them pretty well.
link |
I think navigating autopilot is terrible.
link |
We had some demo features internally of the same stuff
link |
and we would test it and I'm like,
link |
I'm not shipping this even as like open source software
link |
What do you think is terrible?
link |
Consumer Reports does a great job of describing it.
link |
Like when it makes a lane change,
link |
it does it worse than a human.
link |
You shouldn't ship things like autopilot, open pilot,
link |
they lane keep better than a human.
link |
If you turn it on for a stretch of highway,
link |
like an hour long, it's never going to touch a lane line.
link |
Human will touch probably a lane line twice.
link |
You just inspired me.
link |
I don't know if you're grounded in data on that.
link |
I read your paper.
link |
Okay, but no, but that's interesting.
link |
I wonder actually how often we touch lane lines
link |
a little bit because it is.
link |
I could answer that question pretty easily
link |
with the common data side.
link |
Yeah, I'm curious.
link |
I've never answered it.
link |
I just too was like my personal.
link |
It feels right, but that's interesting
link |
because every time you touch a lane,
link |
that's a source of a little bit of stress
link |
and kind of lane keeping is removing that stress.
link |
That's ultimately the biggest value add
link |
honestly is just removing the stress
link |
of having to stay in lane.
link |
And I think I don't think people fully realize
link |
first of all that that's a big value add,
link |
but also that that's all it is.
link |
And that not only I find it a huge value add.
link |
I drove down when we moved to San Diego,
link |
I drove down in an enterprise rental car
link |
So I missed having the system so much.
link |
It's so much more tiring to drive
link |
It's, it is that lane centering.
link |
That's the key feature.
link |
And in a way it's the only feature
link |
that actually adds value to people's lives
link |
in autonomous vehicles today.
link |
Waymo does not add value to people's lives.
link |
It's a more expensive, slower Uber.
link |
Maybe someday it'll be this big cliff where it adds value,
link |
but I don't usually.
link |
I haven't talked to, this is good.
link |
Cause I haven't, I have intuitively,
link |
but I think we're making it explicit now.
link |
I actually believe that really good lane keeping
link |
is a reason to buy a car.
link |
Will be a reason to buy a car.
link |
It is a huge value add.
link |
I've never, until we just started talking about it,
link |
haven't really quite realized it,
link |
that I've felt with Elon's chase of level four
link |
is not the correct chase.
link |
It was on, cause you should just say Tesla has the best
link |
as if from a Tesla perspective say,
link |
Tesla has the best lane keeping.
link |
Kama AI should say Kama AI is the best lane keeping.
link |
You have to do the longitudinal as well.
link |
You can't just lane keep.
link |
You have to do ACC,
link |
but ACC is much more forgiving than lane keep,
link |
especially on the highway.
link |
By the way, are you Kama AI's camera only, correct?
link |
No, we use the radar.
link |
We, from the car, you're able to get to, okay.
link |
We can do it camera only now.
link |
It's gotten to the point,
link |
but we leave the radar there as like a,
link |
Okay, so let's maybe talk through
link |
some of the system specs on the hardware.
link |
What's the hardware side of what you're providing?
link |
What's the capabilities on the software side
link |
with OpenPilot and so on?
link |
So OpenPilot as the box that we sell that it runs on,
link |
it's a phone in a plastic case.
link |
It's nothing special.
link |
We sell it without the software.
link |
So you're like, you know, you buy the phone,
link |
It'll be easy set up,
link |
but it's sold with no software.
link |
OpenPilot right now is about to be 0.6.
link |
When it gets to 1.0,
link |
I think we'll be ready for a consumer product.
link |
We're not gonna add any new features.
link |
We're just gonna make the lane keeping really, really good.
link |
So what do we have right now?
link |
It's a Snapdragon 820.
link |
It's a Sony IMX 298 forward facing camera,
link |
driver monitoring camera.
link |
It's just a selfie cam on the phone.
link |
And a can transceiver,
link |
maybe it's a little thing called pandas.
link |
And they talk over USB to the phone
link |
and then they have three can buses
link |
that they talk to the car.
link |
One of those can buses is the radar can bus.
link |
One of them is the main car can bus.
link |
And the other one is the proxy camera can bus.
link |
We leave the existing camera in place.
link |
So we don't turn AEB off.
link |
Right now we still turn AEB off
link |
if you're using our longitudinal,
link |
but we're gonna fix that before 1.0.
link |
So in its can both ways.
link |
So how are you able to control vehicles?
link |
So we proxy the vehicles that we work with
link |
already have a lane keeping assist system.
link |
So lane keeping assist can mean a huge variety of things.
link |
It can mean it will apply a small torque
link |
to the wheel after you've already crossed a lane line
link |
by a foot, which is the system in the older Toyotas.
link |
Versus like, I think Tesla still calls it lane keeping assist
link |
where it'll keep you perfectly in the center of the lane
link |
You can control like you with the joystick, the cars.
link |
So these cars already have the capability of drive by wire.
link |
So is it, is it trivial to convert a car
link |
that it operates with?
link |
It open pilot is able to control the steering.
link |
Oh, a new car or a car that we,
link |
so we have support now for 45 different makes of cars.
link |
What are the cars in general?
link |
Mostly Honda's and Toyotas.
link |
We support almost every Honda and Toyota made this year.
link |
And then bunch of GM's, bunch of Subaru's.
link |
But it doesn't have to be like a Prius.
link |
It could be Corolla as well.
link |
Oh, the 2020 Corolla is the best car with open pilot.
link |
It just came out there.
link |
The actuator has less lag than the older Corolla.
link |
I think I started watching a video with you.
link |
I mean, the way you make videos is awesome.
link |
It's just literally at the dealership streaming.
link |
I had my friend to follow him.
link |
I probably want to stream for an hour.
link |
Yeah, and basically like if stuff goes a little wrong,
link |
you just like, you just go with it.
link |
That's so beautiful and it's so in contrast to the way
link |
other companies would put together a video like that.
link |
Kind of why I like to do it like that.
link |
I mean, if you become super rich one day and successful,
link |
I hope you keep it that way because I think that's actually
link |
what people love, that kind of genuine.
link |
Oh, it's all that has value to me.
link |
Money has no, if I sell out to like make money,
link |
It doesn't matter.
link |
Yacht, I don't want a yacht.
link |
And I think Tesla actually has a small inkling of that
link |
as well with autonomy day.
link |
They did reveal more than, I mean, of course,
link |
there's marketing communications, you could tell,
link |
but it's more than most companies would reveal,
link |
which is I hope they go towards that direction
link |
more other companies, GM, Ford.
link |
Oh, Tesla's going to win level five.
link |
So let's talk about it.
link |
You think, you're focused on level two currently, currently.
link |
We're going to be one to two years behind Tesla
link |
getting to level five.
link |
We're Android, right?
link |
I'm just saying once Tesla gets it,
link |
we're one to two years behind.
link |
I'm not making any timeline on when Tesla's going to get it.
link |
I'm sorry, Tesla investors, if you
link |
think you're going to have an autonomous robot taxi
link |
fleet by the end of the year, I'll bet against that.
link |
So what do you think about this?
link |
The most level four companies are kind of just
link |
doing their usual safety driver, doing full autonomy kind
link |
And then Tesla does basically trying
link |
to go from lane keeping to full autonomy.
link |
What do you think about that approach?
link |
How successful would it be?
link |
It's a ton better approach.
link |
Because Tesla is gathering data on a scale
link |
that none of them are.
link |
They're putting real users behind the wheel of the cars.
link |
It's, I think, the only strategy that works, the incremental.
link |
Well, so there's a few components to Tesla approach
link |
that's more than just the incremental.
link |
What you spoke with is the software,
link |
so over the air software updates.
link |
I mean, Waymo crews have those too.
link |
Those differentiate from the automakers.
link |
No lane keeping systems have no cars with lane keeping system
link |
have that except Tesla.
link |
And the other one is the data, the other direction,
link |
which is the ability to query the data.
link |
I don't think they're actually collecting
link |
as much data as people think, but the ability
link |
to turn on collection and turn it off.
link |
So I'm both in the robotics world, in the psychology,
link |
human factors world.
link |
Many people believe that level two autonomy
link |
is problematic because of the human factor.
link |
Like the more the task is automated,
link |
the more there's a vigilance decrement.
link |
You start to fall asleep.
link |
You start to become complacent, start texting more and so on.
link |
Do you worry about that?
link |
Because if you're talking about transition from lane keeping
link |
to full autonomy, if you're spending 80% of the time
link |
not supervising the machine, do you
link |
worry about what that means for the safety of the drivers?
link |
One, we don't consider OpenPilot to be 1.0
link |
until we have 100% driver monitoring.
link |
You can cheat right now, our driver monitoring system.
link |
There's a few ways to cheat it.
link |
They're pretty obvious.
link |
We're working on making that better.
link |
Before we ship a consumer product that can drive cars,
link |
I want to make sure that I have driver monitoring
link |
that you can't cheat.
link |
What's a successful driver monitoring system look like?
link |
Is it all about just keeping your eyes on the road?
link |
Well, a few things.
link |
So that's what we went with at first for driver monitoring.
link |
I'm actually looking at where your head is looking.
link |
The camera's not that high.
link |
Resolution eyes are a little bit hard to get.
link |
Well, head is big.
link |
I mean, that's just.
link |
And actually, a lot of it, just psychology wise,
link |
to have that monitor constantly there,
link |
it reminds you that you have to be paying attention.
link |
But we want to go further.
link |
We just hired someone full time to come on
link |
to do the driver monitoring.
link |
I want to detect phone in frame, and I
link |
want to make sure you're not sleeping.
link |
How much does the camera see of the body?
link |
This one, not enough.
link |
The next one, everything.
link |
What's interesting, FishEye, is we're
link |
doing just data collection, not real time.
link |
But FishEye is a beautiful being able to capture the body.
link |
And the smartphone is really the biggest problem.
link |
I can show you one of the pictures from our new system.
link |
So you're basically saying the driver monitoring
link |
will be the answer to that.
link |
I think the other point that you raised in your paper
link |
You're not asking a human to supervise a machine
link |
without giving them the they can take over at any time.
link |
Our safety model, you can take over.
link |
We disengage on both the gas or the brake.
link |
We don't disengage on steering.
link |
I don't feel you have to.
link |
But we disengage on gas or brake.
link |
So it's very easy for you to take over.
link |
And it's very easy for you to reengage.
link |
That switching should be super cheap.
link |
The cars that require, even autopilot,
link |
requires a double press.
link |
That's almost, I see, I don't like that.
link |
And then the cancel.
link |
To cancel in autopilot, you either
link |
have to press cancel, which no one knows where that is.
link |
So they press the brake.
link |
But a lot of times you don't want to press the brake.
link |
You want to press the gas.
link |
So you should cancel on gas or wiggle the steering wheel,
link |
which is bad as well.
link |
Wow, that's brilliant.
link |
I haven't heard anyone articulate that point.
link |
Oh, there's a lot I think about.
link |
Because I think actually Tesla has done a better job
link |
than most automakers at making that frictionless.
link |
But you just described that it could be even better.
link |
I love Super Cruise as an experience.
link |
Once it's engaged.
link |
I don't know if you've used it, but getting the thing
link |
Yeah, I've used the driven Super Cruise a lot.
link |
So what's your thoughts on the Super Cruise system in general?
link |
You disengage Super Cruise, and it falls back to ACC.
link |
So my car is still accelerating.
link |
Otherwise, when you actually have Super Cruise engaged
link |
on the highway, it is phenomenal.
link |
We bought that Cadillac.
link |
But we bought it just to experience this.
link |
And I wanted everyone in the office to be like,
link |
this is what we're striving to build.
link |
GM pioneering with the driver monitoring.
link |
You like their driver monitoring system?
link |
If there's a sun shining back here, it'll be blind to you.
link |
But overall, mostly, yeah.
link |
That's so cool that you know all this stuff.
link |
I don't often talk to people that because it's such a rare car,
link |
unfortunately, currently.
link |
We bought one explicitly for that.
link |
We lost like $25K in the deprecation,
link |
but it feels worth it.
link |
I was very pleasantly surprised that our GM system
link |
was so innovative and really wasn't advertised much,
link |
wasn't talked about much.
link |
And I was nervous that it would die, that it would disappear.
link |
Well, they put it on the wrong car.
link |
They should have put it on the bolt and not some weird Cadillac
link |
that nobody bought.
link |
I think that's going to be into, they're saying at least
link |
it's going to be into their entire fleet.
link |
So what do you think about, as long as we're
link |
on the driver monitoring, what do you think
link |
about Elon Musk's claim that driver monitoring is not needed?
link |
Normally, I love his claims.
link |
That one is stupid.
link |
That one is stupid.
link |
And he's not going to have his level five fleet
link |
by the end of the year.
link |
Hopefully, he's like, OK, I was wrong.
link |
I'm going to add driver monitoring.
link |
Because when these systems get to the point
link |
that they're only messing up once every 1,000 miles,
link |
you absolutely need driver monitoring.
link |
So let me play, because I agree with you,
link |
but let me play devil's advocate.
link |
One possibility is that without driver monitoring,
link |
people are able to self regulate, monitor themselves.
link |
So your idea is, I'm just.
link |
You're seeing all the people sleeping in Teslas?
link |
Well, I'm a little skeptical of all the people sleeping
link |
in Teslas because I've stopped paying attention to that kind
link |
of stuff because I want to see real data.
link |
It's too much glorified.
link |
It doesn't feel scientific to me.
link |
So I want to know how many people are really sleeping
link |
in Teslas versus sleeping.
link |
I was driving here, sleep deprived,
link |
in a car with no automation.
link |
I was falling asleep.
link |
I agree that it's hypey.
link |
It's just like, you know what?
link |
If Elon put driver monitoring, my last autopilot experience
link |
was I rented a Model 3 in March and drove it around.
link |
The wheel thing is annoying.
link |
And the reason the wheel thing is annoying.
link |
We use the wheel thing as well, but we
link |
don't disengage on wheel.
link |
For Tesla, you have to touch the wheel just enough
link |
to trigger the torque sensor to tell it that you're there,
link |
but not enough as to disengage it, which don't use it
link |
Don't disengage on wheel.
link |
You don't have to.
link |
That whole experience, wow, beautifully put.
link |
All those elements, even if you don't have driver monitoring,
link |
that whole experience needs to be better.
link |
Driver monitoring, I think would make,
link |
I mean, I think supercruise is a better experience
link |
once it's engaged over autopilot.
link |
I think supercruise is a transition to engagement
link |
and disengagement are significantly worse.
link |
There's a tricky thing, because if I were to criticize
link |
supercruise, it's a little too crude.
link |
And I think it's like six seconds or something.
link |
If you look off road, it'll start warning you.
link |
It's some ridiculously long period of time.
link |
And just the way, I think it's basically, it's a binary.
link |
It should be adapted.
link |
Yeah, it needs to learn more about you.
link |
It needs to communicate what it sees about you more.
link |
I'm not, you know, Tesla shows what it sees
link |
about the external world.
link |
It would be nice if supercruise would tell us
link |
what it sees about the internal world.
link |
It's even worse than that.
link |
You press the button to engage
link |
and it just says supercruise unavailable.
link |
Yeah, that transparency is good.
link |
We've renamed the driver monitoring packet
link |
We have car state packet, which has the state of the car
link |
and driver state packet, which has state of the driver.
link |
Estimate their BAC.
link |
Blood alcohol, kind of.
link |
You think that's possible with computer vision?
link |
It's a, to me, it's an open question.
link |
I haven't looked into too much.
link |
Actually, I quite seriously looked at the literature.
link |
It's not obvious to me that from the eyes and so on,
link |
You might need stuff from the car as well.
link |
You might need how they're controlling the car, right?
link |
And that's fundamentally at the end of the day
link |
what you care about.
link |
But I think, especially when people are really drunk,
link |
they're not controlling the car nearly as smoothly
link |
as they would look at them walking, right?
link |
They're, the car is like an extension of the body.
link |
So I think you could totally detect.
link |
And if you could fix people who are drunk,
link |
distracted, asleep, if you fix those three.
link |
Yeah, that's a huge, that's huge.
link |
So what are the current limitations of OpenPilot?
link |
What are the main problems that still need to be solved?
link |
We're hopefully fixing a few of them in zero six.
link |
We're not as good as autopilot at stop cars.
link |
So if you're coming up to a red light at like 55,
link |
so it's the radar stopped car problem,
link |
which is responsible for two autopilot accidents,
link |
it's hard to differentiate a stopped car
link |
from a like signpost.
link |
Yeah, static object.
link |
So you have to fuse, you have to do this visually.
link |
There's no way from the radar data to tell the difference.
link |
Maybe you can make a map,
link |
but I don't really believe in mapping at all anymore.
link |
Wait, wait, wait, what?
link |
You don't believe in mapping?
link |
So you're basically, the OpenPilot solution is saying,
link |
react to the environment as you see it,
link |
just like human doing beings do.
link |
And then eventually when you want to do navigate
link |
on OpenPilot, I'll train the net to look at ways.
link |
I'll run ways in the background,
link |
I'll train and come down a way.
link |
Are you using GPS at all?
link |
We use it to ground truth.
link |
We use it to very carefully ground truth the paths.
link |
We have a stack which can recover relative
link |
to 10 centimeters over one minute.
link |
And then we use that to ground truth
link |
exactly where the car went in that local part
link |
of the environment, but it's all local.
link |
How are you testing in general?
link |
Just for yourself, like experiments and stuff.
link |
Where are you located?
link |
So you basically drive around there,
link |
collect some data and watch the performance?
link |
We have a simulator now and we have,
link |
our simulator is really cool.
link |
Our simulator is not,
link |
it's not like a Unity based simulator.
link |
Our simulator lets us load in real estate.
link |
We can load in a drive and simulate
link |
what the system would have done on the historical data.
link |
Right now we're only using it for testing,
link |
but as soon as we start using it for training.
link |
That's all set up for us.
link |
What's your feeling about the real world versus simulation?
link |
Do you like simulation for training?
link |
If this moves to training?
link |
So we have to distinguish two types of simulators, right?
link |
There's a simulator that like is completely fake.
link |
I could get my car to drive around in GTA.
link |
I feel that this kind of simulator is useless.
link |
You're never, there's so many.
link |
My analogy here is like, okay, fine.
link |
You're not solving the computer vision problem,
link |
but you're solving the computer graphics problem.
link |
And you don't think you can get very far
link |
by creating ultra realistic graphics?
link |
No, because you can create ultra realistic graphics
link |
or the road, now create ultra realistic behavioral models
link |
of the other cars.
link |
Oh, well, I'll just use myself driving.
link |
You need real, you need actual human behavior
link |
because that's what you're trying to learn.
link |
The driving does not have a spec.
link |
The definition of driving is what humans do when they drive.
link |
Whatever Waymo does, I don't think it's driving.
link |
Well, I think actually Waymo and others,
link |
if there's any use for reinforcement learning,
link |
I've seen it used quite well.
link |
I studied pedestrians a lot too,
link |
is try to train models from real data
link |
of how pedestrians move and try to use reinforcement learning
link |
models to make pedestrians move in human like ways.
link |
By that point, you've already gone so many layers,
link |
you detected a pedestrian.
link |
Did you hand code the feature vector of their state?
link |
Did you guys learn anything from computer vision
link |
before deep learning?
link |
Well, okay, I feel like this is...
link |
So perception to you is the sticking point.
link |
I mean, what's the hardest part of the stack here?
link |
There is no human understandable feature vector
link |
separating perception and planning.
link |
That's the best way I can put that.
link |
So it's all together and it's a joint problem.
link |
So you can take localization.
link |
Localization and planning,
link |
there is a human understandable feature vector
link |
between these two things.
link |
I mean, okay, so I have like three degrees position,
link |
three degrees orientation and those derivatives,
link |
maybe those second derivatives, right?
link |
That's human understandable, that's physical.
link |
The between perception and planning.
link |
So like Waymo has a perception stack and then a planner.
link |
And one of the things Waymo does right
link |
is they have a simulator that can separate those two.
link |
They can like replay their perception data
link |
and test their system,
link |
which is what I'm talking about
link |
about like the two different kinds of simulators.
link |
There's the kind that can work on real data
link |
and there's the kind that can't work on real data.
link |
Now, the problem is that I don't think
link |
you can hand code a feature vector, right?
link |
Like you have some list of like,
link |
well, here's my list of cars in the scenes.
link |
Here's my list of pedestrians in the scene.
link |
This isn't what humans are doing.
link |
What are humans doing?
link |
You're saying that's too difficult to hand engineer.
link |
I'm saying that there is no state vector.
link |
Given a perfect, I could give you the best team
link |
of engineers in the world to build a perception system
link |
and the best team to build a planner.
link |
All you have to do is define the state vector
link |
that separates those two.
link |
I'm missing the state vector that separates those two.
link |
So what is the output of your perception system?
link |
Output of the perception system.
link |
It's, there's, okay, well, there's several ways to do it.
link |
One is the slam component is localization.
link |
The other is drivable area, drivable space.
link |
Drivable space, yep.
link |
And then there's the different objects in the scene.
link |
And different objects in the scene over time maybe
link |
to give you input to then try to start
link |
modeling the trajectories of those objects.
link |
I can give you a concrete example of something you missed.
link |
So say there's a bush in the scene.
link |
Humans understand that when they see this bush
link |
that there may or may not be a car behind that bush.
link |
Drivable area and a list of objects does not include that.
link |
Humans are doing this constantly
link |
at the simplest intersections.
link |
So now you have to talk about occluded area.
link |
Right, but even that, what do you mean by occluded?
link |
Okay, so I can't see it.
link |
Well, if it's the other side of a house, I don't care.
link |
What's the likelihood that there's a car
link |
in that occluded area, right?
link |
And if you say, okay, we'll add that,
link |
I can come up with 10 more examples that you can't add.
link |
Certainly occluded area would be something
link |
that simulator would have because it's simulating
link |
the entire, you know, occlusion is part of it.
link |
Occlusion is part of a vision stack.
link |
But what I'm saying is if you have a hand engineered,
link |
if your perception system output can be written
link |
in a spec document, it is incomplete.
link |
Yeah, I mean, I certainly, it's hard to argue with that
link |
because in the end, that's going to be true.
link |
Yeah, and I'll tell you what the output
link |
of our perception system is.
link |
It's a 1024 dimensional vector.
link |
Transparent neural net.
link |
Oh, you know that.
link |
No, that's the 1024 dimensions of who knows what.
link |
Because it's operating on real data.
link |
And that's the perception.
link |
That's the perception state, right?
link |
Think about an autoencoder for faces, right?
link |
If you have an autoencoder for faces
link |
and you say it has 256 dimensions in the middle,
link |
and I'm taking a face over here
link |
and projecting it to a face over here.
link |
Can you hand label all 256 of those dimensions?
link |
Well, no, but those are generated automatically.
link |
But even if you tried to do it by hand,
link |
could you come up with a spec between your encoder
link |
No, no, because it wasn't designed, but they're...
link |
No, no, no, but if you could design it,
link |
if you could design a face reconstructor system,
link |
could you come up with a spec?
link |
No, but I think we're missing here a little bit.
link |
I think you're just being very poetic
link |
about expressing a fundamental problem of simulators,
link |
that they are going to be missing so much
link |
that the feature of actually
link |
would just look fundamentally different
link |
from in the simulated world than the real world.
link |
I'm not making a claim about simulators.
link |
I'm making a claim about the spec division
link |
between perception and planning.
link |
Even in your system.
link |
Right, just in general.
link |
If you're trying to build a car that drives,
link |
if you're trying to hand code
link |
the output of your perception system,
link |
like saying, here's a list of all the cars in the scene.
link |
Here's a list of all the people.
link |
Here's a list of the occluded areas.
link |
Here's a vector of drivable areas.
link |
It's insufficient.
link |
And if you start to believe that,
link |
you realize that what Waymo and Cruz are doing is impossible.
link |
Currently, what we're doing is the perception problem
link |
is converting the scene into a chessboard.
link |
And then you reason some basic reasoning
link |
around that chessboard.
link |
And you're saying that really there's a lot missing there.
link |
First of all, why are we talking about this?
link |
Because isn't this a full autonomy?
link |
Is this something you think about?
link |
Oh, I want to win self driving cars.
link |
So your definition of win includes the full five.
link |
I don't think level four is a real thing.
link |
I want to build the AlphaGo of driving.
link |
So AlphaGo is really end to end.
link |
Is, yeah, it's end to end.
link |
And do you think this whole problem,
link |
is that also kind of what you're getting at
link |
with the perception and the planning?
link |
Is that this whole problem, the right way to do it,
link |
is really to learn the entire thing?
link |
I'll argue that not only is it the right way,
link |
it's the only way that's going to exceed human performance.
link |
Well, it's certainly true for Go.
link |
Everyone who tried to hand code Go things
link |
built human inferior things.
link |
And then someone came along and wrote some 10,000 line thing
link |
that doesn't know anything about Go that beat everybody.
link |
It's 10,000 lines.
link |
True, in that sense.
link |
The open question then that maybe I can ask you
link |
is driving is much harder than Go.
link |
The open question is how much harder?
link |
So how, because I think the Elon Musk approach here
link |
with planning and perception is similar
link |
to what you're describing,
link |
which is really turning into not some kind of modular thing,
link |
but really do formulate as a learning problem
link |
and solve the learning problem with scale.
link |
So how many years, put one,
link |
how many years would it take to solve this problem
link |
or just how hard is this freaking problem?
link |
Well, the cool thing is,
link |
I think there's a lot of value
link |
that we can deliver along the way.
link |
I think that you can build lame keeping assist
link |
actually plus adaptive cruise control plus, okay,
link |
looking at ways extends to like all of driving.
link |
Yeah, most of driving, right?
link |
Oh, your adaptive cruise control treats red lights
link |
So let's jump around with you mentioned
link |
that you didn't like navigate an autopilot.
link |
What advice, how would you make it better?
link |
Do you think as a feature that if it's done really well,
link |
it's a good feature?
link |
I think that it's too reliant on like hand coded hacks
link |
for like, how does navigate an autopilot do a lane change?
link |
It actually does the same lane change every time
link |
and it feels mechanical.
link |
Humans do different lane changes.
link |
Humans, sometimes we'll do a slow one,
link |
sometimes do a fast one.
link |
Navigate an autopilot at least every time I use it
link |
is it the identical lane change?
link |
I mean, this is a fundamental thing actually
link |
is the breaking and accelerating,
link |
something that still, Tesla probably does it better
link |
than most cars, but it still doesn't do a great job
link |
of creating a comfortable natural experience
link |
and navigate an autopilot is just lane changes
link |
and extension of that.
link |
So how do you learn to do natural lane change?
link |
So we have it and I can talk about how it works.
link |
So I feel that we have the solution for lateral
link |
but we don't yet have the solution for longitudinal.
link |
There's a few reasons longitudinal is harder than lateral.
link |
The lane change component, the way that we train on it
link |
very simply is like our model has an input
link |
for whether it's doing a lane change or not.
link |
And then when we train the end to end model,
link |
we hand label all the lane changes because you have to.
link |
I've struggled a long time about not wanting to do that
link |
but I think you have to.
link |
Or the training data.
link |
For the training data, right?
link |
We actually have an automatic ground truth
link |
or which automatically labels all the lane changes.
link |
Was that possible?
link |
To automatically label lane changes?
link |
And detect the lane I see when it crosses it, right?
link |
And I don't have to get that high percent accuracy
link |
but it's like 95 good enough.
link |
Now I set the bit when it's doing the lane change
link |
in the end to end learning.
link |
And then I set it to zero when it's not doing a lane change.
link |
So now if I want us to do a lane change a test time,
link |
I just put the bit to a one and it'll do a lane change.
link |
Yeah, but so if you look at the space of lane change,
link |
you know some percentage, not a hundred percent,
link |
that we make as humans is not a pleasant experience
link |
because we messed some part of it up.
link |
It's nerve wracking to change.
link |
If you look, you have to see,
link |
it has to accelerate.
link |
How do we label the ones that are natural and feel good?
link |
You know, that's the,
link |
because that's your ultimate criticism,
link |
the current navigate and autopilot just doesn't feel good.
link |
Well, the current navigate and autopilot
link |
is a hand coded policy written by an engineer in a room
link |
who probably went out and tested it a few times on the 280.
link |
Probably a more, a better version of that.
link |
That's how we would have written it.
link |
Maybe Tesla did a Tesla, they tested it in.
link |
That might have been two engineers.
link |
No, but so if you learn the lane change,
link |
if you learn how to do a lane change from data,
link |
just like you have a label that says lane change
link |
and then you put it in when you want it to do the lane change,
link |
it'll automatically do the lane change
link |
that's appropriate for the situation.
link |
Now, to get at the problem of some humans
link |
do bad lane changes,
link |
we haven't worked too much on this problem yet.
link |
It's not that much of a problem in practice.
link |
My theory is that all good drivers are good in the same way
link |
and all bad drivers are bad in different ways.
link |
And we've seen some data to back this up.
link |
Well, beautifully put.
link |
So you just basically, if that's true hypothesis,
link |
then your task is to discover the good drivers.
link |
The good drivers stand out
link |
because they're in one cluster
link |
and the bad drivers are scattered all over the place
link |
and your net learns the cluster.
link |
So you just learn from the good drivers
link |
and they're easy to cluster.
link |
In fact, we learned from all of them
link |
and the net automatically learns the policy
link |
that's like the majority.
link |
But we'll eventually probably have to build some out.
link |
So if that theory is true, I hope it's true
link |
because the counter theory is there is many clusters,
link |
maybe arbitrarily many clusters of good drivers.
link |
Because if there's one cluster of good drivers,
link |
you can at least discover a set of policies.
link |
You can learn a set of policies
link |
which would be good universally.
link |
That would be nice if it's true.
link |
And you're saying that there is some evidence that...
link |
Let's say lane changes can be clustered into four clusters.
link |
There's a finite level of...
link |
I would argue that all four of those are good clusters.
link |
All the things that are random are noise and probably bad.
link |
And which one of the four you pick?
link |
Or maybe it's 10 or maybe it's 20.
link |
You can learn that.
link |
It's context dependent.
link |
It depends on the scene.
link |
And the hope is it's not too dependent on the driver.
link |
Yeah, the hope is that it all washes out.
link |
The hope is that the distribution is not bimodal.
link |
The hope is that it's a nice Gaussian.
link |
So what advice would you give to Tesla?
link |
How to fix, how to improve, navigate an autopilot?
link |
That's the lessons that you've learned from Kamii.
link |
The only real advice I would give to Tesla
link |
is please put driver monitoring in your cars.
link |
With respect to improving it.
link |
You can't do that anymore.
link |
I started to interrupt.
link |
But there's a practical nature of many of hundreds of thousands
link |
of cars being produced that don't have a good driver facing camera.
link |
The Model 3 has a selfie cam.
link |
Is it not good enough?
link |
Did they not have put IR LEDs for night?
link |
That's a good question.
link |
But I do know that it's fish eye
link |
and it's relatively low resolution.
link |
So it's really not designed.
link |
It wasn't designed for driver monitoring.
link |
You can hope that you can kind of scrape up
link |
and have something from it.
link |
But why didn't they put it in today?
link |
Every time I've heard Carpathian talk about the problem
link |
and talking about like software 2.0
link |
and how the machine learning is gobbling up everything,
link |
I think this is absolutely the right strategy.
link |
I think that he didn't write and navigate on autopilot.
link |
I think somebody else did and kind of hacked it on top of that stuff.
link |
I think when Carpathian says, wait a second,
link |
why did we hand code this lane change policy
link |
with all these magic numbers?
link |
We're going to learn it from data.
link |
They already know what to do there.
link |
Well, that's Andre's job is to turn everything
link |
into a learning problem and collect a huge amount of data.
link |
The reality is, though, not every problem
link |
can be turned into a learning problem in the short term.
link |
In the end, everything will be a learning problem.
link |
The reality is, like if you want to build L5 vehicles today,
link |
it will likely involve no learning.
link |
And that's the reality is, so at which point does learning start?
link |
It's the crutch statement that LiDAR is a crutch.
link |
Which point will learning get up to part of human performance?
link |
It's over human performance on ImageNet, classification,
link |
on driving, it's a question still.
link |
I'll say this, I'm here to play for 10 years.
link |
I'm not here to try to.
link |
I'm here to play for 10 years and make money along the way.
link |
I'm not here to try to promise people
link |
that I'm going to have my L5 taxi network up and working
link |
Do you think that was a mistake?
link |
What do you think was the motivation behind saying
link |
that other companies are also promising L5 vehicles
link |
with their different approaches in 2020, 2021, 2022?
link |
If anybody would like to bet me that those things do not pan out,
link |
Even money, even money, I'll bet you as much as you want.
link |
So are you worried about what's going to happen?
link |
Because you're not in full agreement on that.
link |
What's going to happen when 2022, 2021 come around
link |
and nobody has fleets of autonomous vehicles?
link |
Well, you can look at the history.
link |
If you go back five years ago, they
link |
were all promised by 2018 and 2017.
link |
But they weren't that strong of promises.
link |
I mean, Ford really declared.
link |
I think not many have declared as definitively
link |
as they have now these dates.
link |
So let's separate L4 and L5.
link |
Do I think that it's possible for Waymo
link |
to continue to hack on their system
link |
until it gets to level four in Chandler, Arizona?
link |
Chandler, Arizona?
link |
By which year are we talking about?
link |
Oh, I even think that's possible by like 2020, 2021.
link |
But level four, Chandler, Arizona, not level five,
link |
Level four, meaning some very defined streets.
link |
It works out really well.
link |
Very defined streets.
link |
And then practically, these streets are pretty empty.
link |
If most of the streets are covered in Waymos,
link |
Waymo can kind of change the definition of what driving is.
link |
If your self driving network is the majority
link |
of cars in an area, they only need
link |
to be safe with respect to each other,
link |
and all the humans will need to learn to adapt to them.
link |
Now go drive in downtown New York.
link |
You can talk about autonomy and like on farms,
link |
it already works great, because you can really just
link |
follow the GPS line.
link |
So what does success look like for Kama AI?
link |
What are the milestones like where
link |
you can sit back with some champagne
link |
and say, we did it, boys and girls?
link |
Well, it's never over.
link |
Yeah, but don't be so.
link |
You must drink champagne every time you celebrate.
link |
What are some wins?
link |
A big milestone that we're hoping for by mid next year
link |
is profitability of the company.
link |
And we're going to have to revisit the idea of selling
link |
a consumer product.
link |
But it's not going to be like the Kama One.
link |
When we do it, it's going to be perfect.
link |
OpenPilot has gotten so much better in the last two years.
link |
We're going to have a few features.
link |
We're going to have 100% driver monitoring.
link |
We're going to disable no safety features in the car.
link |
Actually, I think it'd be really cool what we're doing right
link |
now, our project this week is we're analyzing the data set
link |
and looking for all the AEB triggers
link |
from the manufacturer systems.
link |
We have better data set on that than the manufacturers.
link |
How much does Toyota have 10 million miles of real world
link |
driving to know how many times they're AEB triggered?
link |
So let me give you, because you asked, financial advice.
link |
Because I work with a lot of automakers
link |
and one possible source of money for you,
link |
which I'll be excited to see you take on, is basically
link |
selling the data, which is something that most people,
link |
and not selling in a way where here, here at Automaker,
link |
We've done this actually at MIT, not for money purposes,
link |
but you could do it for significant money purposes
link |
and make the world a better place
link |
by creating a consortia where automakers would pay in
link |
and then they get to have free access to the data.
link |
And I think a lot of people are really hungry for that
link |
and would pay significant amount of money for it.
link |
Here's the problem with that.
link |
I like this idea all in theory.
link |
It'd be very easy for me to give them access to my servers.
link |
And we already have all open source tools to access this data.
link |
It's in a great format.
link |
We have a great pipeline.
link |
But they're going to put me in the room
link |
with some business development guy.
link |
And I'm going to have to talk to this guy.
link |
And he's not going to know most of the words I'm saying.
link |
I'm not willing to tolerate that.
link |
But I think I agree with you.
link |
But you just tell them the terms
link |
and there's no discussion needed.
link |
If I could just tell them the terms, then like, all right.
link |
Who wants access to my data?
link |
I will sell it to you for, let's say,
link |
you want a subscription?
link |
I'll sell you for 100k a month.
link |
I'll give you access to the data subscription?
link |
Yeah, I think that's kind of fair.
link |
Came up with that number off the top of my head.
link |
If somebody sends me like a three line email where it's like,
link |
we would like to pay 100k a month to get access to your data.
link |
We would agree to like reasonable privacy terms
link |
of the people who are in the data set.
link |
I would be happy to do it.
link |
But that's not going to be the email.
link |
The email is going to be, hey, do you
link |
have some time in the next month where we can sit down
link |
and we can, I don't have time for that.
link |
We're moving too fast.
link |
You could politely respond to that email,
link |
but not saying I don't have any time for your bullshit.
link |
You say, oh, well, unfortunately, these are the terms.
link |
And so this is what we try to, we brought the cost down
link |
for you in order to minimize the friction, the communication.
link |
Here's the whatever it is, $1, $2 million a year.
link |
And you have access.
link |
And it's not like I get that email from like,
link |
but OK, am I going to reach out?
link |
Am I going to hire a business development person
link |
who's going to reach out to the automakers?
link |
If they reached into me, I'm not
link |
going to ignore the email.
link |
I'll come back with something like, yeah,
link |
if you're willing to pay $100,000 for access to the data,
link |
I'm happy to set that up.
link |
That's worth my engineering time.
link |
That's actually quite insightful of you.
link |
Probably because many of the automakers
link |
are quite a bit old school, there
link |
will be a need to reach out.
link |
And they want it, but there will need
link |
to be some communication.
link |
Mobileye circa 2015 had the lowest R&D spend of any chipmaker.
link |
Like per, and you look at all the people who work for them,
link |
and it's all business development people
link |
because the car companies are impossible to work with.
link |
Yeah, so you have no patience for that,
link |
and you're a legit Android, huh?
link |
I have something to do, right?
link |
Like, it's not like I don't mean to be a dick and say,
link |
I don't have patience for that, but it's like,
link |
that stuff doesn't help us with our goal of winning
link |
self driving cars.
link |
If I want money in the short term,
link |
if I showed off the actual learning tech that we have,
link |
it's somewhat sad.
link |
It's years and years ahead of everybody else's.
link |
Maybe not Tesla's.
link |
I think Tesla has similar stuff to us, actually.
link |
I think Tesla has similar stuff, but when you compare it
link |
to what the Toyota Research Institute has,
link |
you're not even close to what we have.
link |
No comments, but I also can't.
link |
I have to take your comments.
link |
I intuitively believe you, but I have
link |
to take it with a grain of salt because,
link |
I mean, you are an inspiration because you basically
link |
don't care about a lot of things that other companies care
link |
You don't try to bullshit, in a sense, like make up stuff,
link |
so to drive up valuation.
link |
You're really very real, and you're
link |
trying to solve the problem, and I admire that a lot.
link |
What I don't necessarily fully can't trust you on about your
link |
respect is how good it is, right?
link |
I can only, but I also know how bad others are.
link |
I'll say two things about, trust, but verify, right?
link |
I'll say two things about that.
link |
One is try, get in a 2020 Corolla,
link |
and try OpenPilot 0.6 when it comes out next month.
link |
I think already, you'll look at this,
link |
and you'll be like, this is already really good.
link |
And then, I could be doing that all with hand labelers
link |
and all with the same approach that Mobileye uses.
link |
When we release a model that no longer
link |
has the lanes in it, that only outputs a path,
link |
then think about how we did that machine learning,
link |
and then right away, when you see,
link |
and that's going to be an OpenPilot,
link |
that's going to be an OpenPilot before 1.0,
link |
when you see that model, you'll know
link |
that everything I'm saying is true,
link |
because how else did I get that model?
link |
You know what I'm saying is true about the simulator.
link |
Yeah, yeah, yeah, this is super exciting.
link |
That's super exciting.
link |
But I listened to your talk with Kyle,
link |
and Kyle was originally building the aftermarket system,
link |
and he gave up on it because of technical challenges,
link |
because of the fact that he's going
link |
to have to support 20 to 50 cars.
link |
We support 45, because what is he
link |
going to do when the manufacturer ABS system triggers?
link |
We have alerts and warnings to deal with all of that
link |
and all the cars, and how is he going to formally verify it?
link |
Well, I got 10 million miles of data.
link |
It's probably better verified than the spec.
link |
Yeah, I'm glad you're here talking to me.
link |
I'll remember this day, because it's interesting.
link |
If you look at Kyle's from Cruise,
link |
I'm sure they have a large number of business development
link |
folks, and he's working with GM.
link |
He could work with Argo AI, worked with Ford.
link |
It's interesting, because chances that you fail businesswise,
link |
like bankrupt, are pretty high.
link |
And yet, it's the Android model,
link |
is you're actually taking on the problem.
link |
So that's really inspiring.
link |
Well, I have a long term way for comedy to make money, too.
link |
And one of the nice things when you really take on the problem,
link |
which is my hope for autopilot, for example,
link |
is things you don't expect, ways to make money,
link |
or create value that you don't expect will pop up.
link |
I've known how to do it since 2017 is the first time I said it.
link |
Which part to know how to do which part?
link |
Our long term plan is to be a car insurance company.
link |
I make driving twice as safe.
link |
Not only that, I have the best data
link |
such to know who statistically is the safest drivers.
link |
And oh, oh, we see you.
link |
We see you driving unsafely.
link |
We're not going to insure you.
link |
And that causes a bifurcation in the market,
link |
because the only people who can't get common insurance
link |
or the bad drivers, Geico can insure them.
link |
Their premiums are crazy high, our premiums are crazy low.
link |
We win car insurance.
link |
Take over that whole market.
link |
OK, so if we win, if we win, but that's
link |
I'm saying like how do you turn comma into a $10 billion
link |
So you Elon Musk, who else?
link |
Who else is thinking like this and working like this
link |
Who are the competitors?
link |
Are there people seriously?
link |
I don't think anyone that I'm aware of is seriously
link |
taking on lane keeping, like to where it's a huge business that
link |
turns eventually to full autonomy that then creates
link |
other businesses on top of it and so on.
link |
Thinks insurance, thinks all kinds of ideas like that.
link |
Do you know anyone else thinking like this?
link |
That's interesting.
link |
I mean, my sense is everybody turns to that in like four
link |
Like Ford, once the autonomy doesn't fall through.
link |
By the way, he paved the way for all of us.
link |
It's not iOS, true.
link |
I would not be doing comma AI today if it was not
link |
for those conversations with Elon.
link |
And if it were not for him saying like,
link |
I think he said like, well, obviously we're not
link |
going to use LiDAR, we use cameras, humans use cameras.
link |
So what do you think about that?
link |
How important is LiDAR?
link |
Everybody else's on L5 is using LiDAR.
link |
What are your thoughts on his provocative statement
link |
that LiDAR is a crutch?
link |
See, sometimes they'll say dumb things like the driver
link |
monitoring thing, but sometimes they'll say absolutely
link |
completely 100% obviously true things.
link |
Of course LiDAR is a crutch.
link |
It's not even a good crutch.
link |
You're not even using it.
link |
They're using it for localization,
link |
which isn't good in the first place.
link |
If you have to localize your car to centimeters
link |
in order to drive, that's not driving.
link |
Currently not doing much machine learning.
link |
I thought LiDAR data, meaning like to help you
link |
in the task of general task of perception.
link |
The main goal of those LiDARs on those cars
link |
I think is actually localization more than perception,
link |
or at least that's what they use them for.
link |
Yeah, that's true.
link |
If you want to localize to centimeters,
link |
you can't use GPS.
link |
The fancies GPS in the world can't do it,
link |
especially if you're under tree cover and stuff.
link |
LiDAR you can do this pretty easily.
link |
So really they're not taking on,
link |
I mean in some research they're using it for perception,
link |
but and they're certainly not, which is sad,
link |
they're not fusing it well with vision.
link |
They do use it for perception.
link |
I'm not saying they don't use it for perception,
link |
but the thing that they have vision based
link |
and radar based perception systems as well.
link |
You could remove the LiDAR and keep around
link |
a lot of the dynamic object perception.
link |
You want to get centimeter accurate localization.
link |
Good luck doing that with anything else.
link |
So what should a cruise Waymo do?
link |
Like what would be your advice to them now?
link |
I mean Waymo is actually, they're serious.
link |
Waymo out of the ball of them,
link |
are quite serious about the long game.
link |
If L5 is a lot, is requires 50 years,
link |
I think Waymo will be the only one left standing at the end
link |
with a given the financial backing that they have.
link |
They're boo Google box.
link |
I'll say nice things about both Waymo and cruise.
link |
Waymo is by far the furthest along with technology.
link |
Waymo has a three to five year lead
link |
on all the competitors.
link |
If the Waymo looking stack works,
link |
maybe three year lead.
link |
If the Waymo looking stack works,
link |
they have a three year lead.
link |
Now, I argue that Waymo has spent too much money
link |
to recapitalize, to gain back their losses
link |
in those three years.
link |
Also self driving cars have no network effect like that.
link |
Uber has a network effect.
link |
You have a market, you have drivers and you have riders.
link |
Self driving cars, you have capital and you have riders.
link |
There's no network effect.
link |
If I want to blanket a new city in self driving cars,
link |
I buy the off the shelf Chinese knockoff self driving cars
link |
and I buy enough of them in the city.
link |
I can't do that with drivers.
link |
And that's why Uber has a first mover advantage
link |
that no self driving car company will.
link |
Can you just a thing, let a little bit.
link |
Uber, you're not talking about Uber,
link |
the autonomous vehicle Uber.
link |
You're talking about the Uber cars.
link |
I open for business in Austin, Texas, let's say.
link |
I need to attract both sides of the market.
link |
I need to both get drivers on my platform
link |
and riders on my platform.
link |
And I need to keep them both sufficiently happy, right?
link |
Riders aren't going to use it
link |
if it takes more than five minutes for an Uber to show up.
link |
Drivers aren't going to use it
link |
if they have to sit around all day and there's no riders.
link |
So you have to carefully balance a market.
link |
And whenever you have to carefully balance a market,
link |
there's a great first mover advantage
link |
because there's a switching cost for everybody, right?
link |
The drivers and the riders
link |
would have to switch at the same time.
link |
Let's even say that, let's say, Uber shows up.
link |
And Uber somehow agrees to do things at a bigger,
link |
we've done it more efficiently, right?
link |
Uber only takes 5% of a car
link |
instead of the 10% that Uber takes.
link |
No one is going to switch
link |
because the switching cost is higher than that 5%.
link |
So you actually can, in markets like that,
link |
you have a first mover advantage.
link |
Autonomous vehicles of the level five variety
link |
have no first mover advantage.
link |
If the technology becomes commoditized,
link |
say I want to go to a new city, look at the scooters.
link |
It's going to look a lot more like scooters.
link |
Every person with a checkbook
link |
can blanket a city in scooters
link |
and that's why you have 10 different scooter companies.
link |
Which one's going to win?
link |
It's a race to the bottom.
link |
It's a terrible market to be in
link |
because there's no market for scooters.
link |
And the scooters don't get a say
link |
in whether they want to be bought
link |
and deployed to a city or not.
link |
We're going to entice the scooters with subsidies
link |
So whenever you have to invest that capital,
link |
It doesn't come back.
link |
They can't be your main criticism of the Waymo approach.
link |
Oh, I'm saying even if it does technically work.
link |
Even if it does technically work, that's a problem.
link |
I don't know if I were to say, I would say,
link |
you're already there.
link |
I haven't even thought about that.
link |
But I would say the bigger challenge
link |
is the technical approach.
link |
So Waymo's cruise is...
link |
And not just the technical approach,
link |
but of creating value.
link |
I still don't understand how you beat Uber,
link |
the human driven cars.
link |
In terms of financially,
link |
it doesn't make sense to me
link |
that people want to get an autonomous vehicle.
link |
I don't understand how you make money.
link |
In the long term, yes, like real long term,
link |
but it just feels like there's too much
link |
capital investment needed.
link |
Oh, and they're going to be worse than Ubers
link |
because they're going to stop
link |
for every little thing everywhere.
link |
I'll say a nice thing about cruise.
link |
That was my nice thing about Waymo.
link |
They're three years ahead of me.
link |
Oh, because they're three years.
link |
They're three years technically ahead of everybody.
link |
Their tech stack is great.
link |
My nice thing about cruise is GM buying them
link |
was a great move for GM.
link |
GM bought an insurance policy against Waymo.
link |
They put cruise is three years behind Waymo.
link |
That means Google will get a monopoly
link |
on the technology for at most three years.
link |
And if technology works,
link |
you might not even be right about the three years.
link |
Cruise actually might not be that far behind.
link |
I don't know how much Waymo has waffled around
link |
or how much of it actually is just that long tail.
link |
If that's the best you could say in terms of nice things,
link |
that's more of a nice thing for GM
link |
that that's a smart insurance policy.
link |
It's a smart insurance policy.
link |
I mean, I think that's how...
link |
I can't see cruise working out any other.
link |
For cruise to leapfrog Waymo would really surprise me.
link |
Yeah, so let's talk about the underlying assumptions
link |
of everything is...
link |
We're not gonna leapfrog Tesla.
link |
Tesla would have to seriously mess up
link |
for us to leapfrog them.
link |
Okay, so the way you leapfrog, right,
link |
is you come up with an idea
link |
or you take a direction, perhaps secretly,
link |
that the other people aren't taking.
link |
And so cruise, Waymo, even Aurora...
link |
I don't know, Aurora, Zooks is the same stack as well.
link |
They're all the same code base even.
link |
They're all the same DARPA Urban Challenge code base.
link |
So the question is, do you think there's a room
link |
for brilliance and innovation there
link |
that will change everything?
link |
Like say, okay, so I'll give you examples.
link |
It could be if revolution and mapping, for example,
link |
that allow you to map things,
link |
do HD maps of the whole world,
link |
all weather conditions somehow really well,
link |
or revolution and simulation,
link |
to where all the way you said before becomes incorrect.
link |
That kind of thing.
link |
Any room for breakthrough innovation?
link |
What I said before about,
link |
oh, they actually get the whole thing, well,
link |
I'll say this about we divide driving into three problems.
link |
And I actually haven't solved the third yet,
link |
but I haven't had any idea how to do it.
link |
So there's the static.
link |
The static driving problem is assuming
link |
you are the only car on the road, right?
link |
And this problem can be solved 100%
link |
with mapping and localization.
link |
This is why farms work the way they do.
link |
If all you have to deal with is the static problem,
link |
and you can statically schedule your machines, right?
link |
It's the same as like statically scheduling processes.
link |
You can statically schedule your tractors
link |
to never hit each other on their paths, right?
link |
Because then you know the speed they go at.
link |
So that's the static driving problem.
link |
Maps only helps you with the static driving problem.
link |
Yeah, the question about static driving,
link |
you've just made it sound like it's really easy.
link |
Static driving is really easy.
link |
How, well, because the whole drifting out of lane,
link |
when Tesla drifts out of lane,
link |
it's failing on the fundamental static driving problem.
link |
Tesla is drifting out of lane?
link |
The static driving problem is not easy for the world.
link |
The static driving problem is easy for one route.
link |
One route in one weather condition
link |
with one state of lane markings
link |
and like no deterioration, no cracks in the road.
link |
Well, I'm assuming you have a perfect localizer.
link |
So that's all for the weather condition
link |
and the lane marking condition.
link |
But that's the problem.
link |
How do you have a perfect localizer?
link |
You can build, perfect localizers are not that hard to build.
link |
Okay, come on now, with LIDAR.
link |
LIDAR, yeah, but you use LIDAR, right?
link |
Like you use LIDAR, build a perfect localizer.
link |
Building a perfect localizer without LIDAR,
link |
it's gonna be hard.
link |
You can get 10 centimeters without LIDAR,
link |
you can get one centimeter with LIDAR.
link |
I'm not even concerned about the one or 10 centimeters.
link |
I'm concerned if every once in a while you just weigh off.
link |
Yeah, so this is why you have to carefully
link |
make sure you're always tracking your position.
link |
You wanna use LIDAR camera fusion,
link |
but you can get the reliability of that system
link |
up to 100,000 miles
link |
and then you write some fallback condition
link |
where it's not that bad if you're way off, right?
link |
I think that you can get it to the point,
link |
it's like ASL D that you're never in a case
link |
where you're way off and you don't know it.
link |
Yeah, okay, so this is brilliant.
link |
So that's the static.
link |
We can, especially with LIDAR and good HD maps,
link |
you can solve that problem.
link |
The static, the static problem is so easy.
link |
It's very typical for you to say something's easy.
link |
It's not as challenging as the other ones, okay.
link |
Well, okay, maybe it's obvious how to solve it.
link |
The third one's the hardest.
link |
And a lot of people don't even think about the third one
link |
and even see it as different from the second one.
link |
So the second one is dynamic.
link |
The second one is like, say there's an obvious example,
link |
it's like a car stopped at a red light, right?
link |
You can't have that car in your map
link |
because you don't know whether that car
link |
is gonna be there or not.
link |
So you have to detect that car in real time
link |
and then you have to do the appropriate action, right?
link |
Also, that car is not a fixed object.
link |
That car may move and you have to predict
link |
what that car will do, right?
link |
So this is the dynamic problem.
link |
So you have to deal with this.
link |
This involves, again, like you're gonna need models
link |
of other people's behavior.
link |
Do you, are you including in that?
link |
I don't wanna step on the third one.
link |
Oh, but are you including in that your influence
link |
Ah, that's the third one.
link |
That's the third one.
link |
We call it the counterfactual.
link |
I just talked to Judea Pro who's obsessed
link |
with counterfactuals.
link |
Counterfactual, oh yeah, yeah, I read his books.
link |
So the static and the dynamic are our approach right now
link |
for lateral will scale completely to the static and dynamic.
link |
The counterfactual, the only way I have to do it yet,
link |
the thing that I wanna do once we have all of these cars
link |
is I wanna do reinforcement learning on the world.
link |
I'm always gonna turn the exploiter up to max.
link |
I'm not gonna have them explore.
link |
But the only real way to get at the counterfactual
link |
is to do reinforcement learning
link |
because the other agents are humans.
link |
So that's fascinating that you break it down like that.
link |
I agree completely.
link |
I've spent my life thinking about this problem.
link |
This is beautiful.
link |
And part of it, cause you're slightly insane,
link |
because not my life, just the last four years.
link |
No, no, you have some non zero percent of your brain
link |
has a madman in it, which is a really good feature.
link |
But there's a safety component to it
link |
that I think when there's sort of
link |
with counterfactuals and so on,
link |
that would just freak people out.
link |
How do you even start to think about this in general?
link |
I mean, you've had some friction with NHTSA and so on.
link |
I am frankly exhausted by safety engineers.
link |
The prioritization on safety over innovation
link |
to a degree where it kills, in my view,
link |
kills safety in the longterm.
link |
So the counterfactual thing,
link |
they just actually exploring this world
link |
of how do you interact with dynamic objects and so on?
link |
How do you think about safety?
link |
You can do reinforcement learning without ever exploring.
link |
And I said that, like,
link |
so you can think about your, in like reinforcement learning,
link |
it's usually called like a temperature parameter.
link |
And your temperature parameter
link |
is how often you deviate from the argmax.
link |
I could always set that to zero and still learn.
link |
And I feel that you'd always want that set to zero
link |
on your actual system.
link |
But the problem is you first don't know very much
link |
and so you're going to make mistakes.
link |
So the learning, the exploration happens through mistakes.
link |
We're all ready, yeah, but.
link |
Okay, so the consequences of a mistake.
link |
OpenPilot and Autopilot are making mistakes left and right.
link |
We have 700 daily active users,
link |
1,000 weekly active users.
link |
OpenPilot makes tens of thousands of mistakes a week.
link |
These mistakes have zero consequences.
link |
These mistakes are,
link |
oh, I wanted to take this exit and it went straight.
link |
So I'm just going to carefully touch the wheel.
link |
The humans catch them.
link |
The humans catch them.
link |
And the human disengagement is labeling
link |
that reinforcement learning in a completely
link |
consequence free way.
link |
So driver monitoring is the way you ensure they keep.
link |
They keep paying attention.
link |
How's your messaging?
link |
Say I gave you a billion dollars,
link |
so you would be scaling it now.
link |
Oh, if I could scale, I couldn't scale with any amount of money.
link |
I'd raise money if I could, if I had a way to scale it.
link |
Yeah, you're not, no, I'm not focused on scale.
link |
I don't know how to do.
link |
Oh, like, I guess I could sell it to more people,
link |
but I want to make the system better.
link |
And I don't know how to.
link |
But what's the messaging here?
link |
I got a chance to talk to Elon.
link |
And he basically said that the human factor doesn't matter.
link |
You know, the human doesn't matter
link |
because the system will perform.
link |
There'll be sort of a, sorry to use the term,
link |
but like a singular, like a point
link |
where it gets just much better.
link |
And so the human, it won't really matter.
link |
But it seems like that human catching the system
link |
when it gets into trouble is like the thing
link |
which will make something like reinforcement learning work.
link |
So how do you, how do you think messaging for Tesla,
link |
for you, for the industry in general, should change?
link |
I think my messaging is pretty clear,
link |
at least like our messaging wasn't that clear
link |
in the beginning and I do kind of fault myself for that.
link |
We are proud right now to be a level two system.
link |
We are proud to be level two.
link |
If we talk about level four,
link |
it's not with the current hardware.
link |
It's not going to be just a magical OTA upgrade.
link |
It's going to be new hardware.
link |
It's going to be very carefully thought out right now.
link |
We are proud to be level two.
link |
And we have a rigorous safety model.
link |
I mean, not like, like, okay, rigorous.
link |
Who knows what that means?
link |
But we at least have a safety model
link |
and we make it explicit.
link |
It's in safety.md and open pilot.
link |
And it says, seriously though.
link |
This is really, this is so Android.
link |
So, well, this is, this is the safety model
link |
and I like to have conversations like if, like, you know,
link |
sometimes people will come to you and they're like,
link |
your system's not safe.
link |
Have you read my safety docs?
link |
Would you like to have an intelligent conversation
link |
And the answer is always no.
link |
They just like scream about, it runs Python.
link |
So you're saying that, that because Python's not real time,
link |
Python not being real time never causes disengagement.
link |
Disengagement's are caused by, you know, the model is QM.
link |
But safety.md says the following.
link |
First and foremost,
link |
the driver must be paying attention at all times.
link |
I don't consider, I do,
link |
I still consider the software to be alpha software
link |
until we can actually enforce that statement.
link |
But I feel it's very well communicated to our users.
link |
One is the user must be able to easily take control
link |
of the vehicle at all times.
link |
So if you step on the gas or brake with open pilot,
link |
it gives full manual control back to the user
link |
or press the cancel button.
link |
Step two, the car will never react so quickly.
link |
We define so quickly to be about one second
link |
that you can't react in time.
link |
And we do this by enforcing torque limits,
link |
braking limits and acceleration limits.
link |
So we have like our torque limits way lower than Tesla's.
link |
This is another potential.
link |
If I could tweak autopilot,
link |
I would lower their torque limit
link |
and I would add driver monitoring.
link |
Because autopilot can jerk the wheel hard.
link |
It's, we limit and all this code is open source, readable.
link |
And I believe now it's all MISRA C compliant.
link |
MISRA is like the automotive coding standard.
link |
At first, I've come to respect,
link |
I've been reading like the standards lately
link |
and I've come to respect them.
link |
They're actually written by very smart people.
link |
Yeah, they're brilliant people actually.
link |
They have a lot of experience.
link |
They're sometimes a little too cautious,
link |
but in this case, it pays off.
link |
MISRA is written by like computer scientists
link |
and you can tell by the language they use.
link |
You can tell by the language they use.
link |
They talk about like whether certain conditions in MISRA
link |
are decidable or undecidable.
link |
And you mean like the halting problem?
link |
And yes, all right, you've earned my respect.
link |
I will read carefully what you have to say
link |
and we want to make our code compliant with that.
link |
All right, so you're proud level two, beautiful.
link |
So you were the founder and I think CEO of Kama AI,
link |
then you were the head of research.
link |
What the heck are you now?
link |
What's your connection to Kama AI?
link |
I'm the president, but I'm one of those like
link |
unelected presidents of like a small dictatorship country,
link |
not one of those like elected presidents.
link |
Oh, so you're like Putin when he was like the, yeah,
link |
So there's, what's the governance structure?
link |
What's the future of Kama AI finance?
link |
I mean, yeah, as a business, do you want,
link |
are you just focused on getting things right now,
link |
making some small amount of money in the meantime
link |
and then when it works, it works a new scale.
link |
Our burn rate is about 200 K a month
link |
and our revenue is about 100 K a month.
link |
So we need to forex our revenue,
link |
but we haven't like tried very hard at that yet.
link |
And the revenue is basically selling stuff online.
link |
Yeah, we sell stuff shop.com.ai.
link |
Is there other, well, okay.
link |
So you'll have to figure out.
link |
That's our only, see, but to me,
link |
that's like respectable revenues.
link |
We make it by selling products to consumers
link |
for honest and transparent about what they are.
link |
Most actually level four companies, right?
link |
Cause you could easily start blowing up like smoke,
link |
like overselling the hype and feeding into,
link |
getting some fundraisers.
link |
Oh, you're the guy, you're a genius
link |
because you hacked the iPhone.
link |
Yeah, I can trade my social capital for more money.
link |
I almost regret it doing it the first time.
link |
Well, on a small tangent,
link |
what's your, you seem to not like fame
link |
and yet you're also drawn to fame.
link |
What's, where have you on, where are you on that currently?
link |
Have you had some introspection, some soul searching?
link |
I actually, I've come to a pretty stable position on that.
link |
Like after the first time,
link |
I realized that I don't want attention from the masses.
link |
I want attention from people who I respect.
link |
Who do you respect?
link |
I can give a list of people.
link |
So are these like Elon Musk type characters?
link |
Actually, you know what?
link |
I'll make it more broad than that.
link |
I won't make it about a person.
link |
I respect people who have skills, right?
link |
And I would like to like be,
link |
I'm not gonna say famous,
link |
but be like known among more people
link |
who have like real skills.
link |
Who in cars, do you think have skill?
link |
Not do you respect?
link |
Oh, Kyle Voat has skill.
link |
A lot of people at Waymo have skill.
link |
And I respect them.
link |
I respect them as engineers.
link |
Like I can think, I mean,
link |
I think about all the times in my life
link |
where I've been like dead set on approaches
link |
and they turn out to be wrong.
link |
So I mean, this might, I might be wrong.
link |
I accept that, I accept that there's a decent chance
link |
And actually, I mean, having talked to Chris Armson,
link |
Sterling Anderson, those guys,
link |
I mean, I deeply respect Chris.
link |
I just admire the guy.
link |
When you drive a car through the desert
link |
when everybody thinks it's impossible, that's legit.
link |
And then I also really respect the people
link |
who are like writing the infrastructure of the world,
link |
like the Linus Torvalds and the Chris Ladin.
link |
Oh yeah, they were doing the real work.
link |
I know they're doing the real work.
link |
Having talked to Chris Ladin,
link |
you realize, especially when they're humble,
link |
it's like, you realize, oh, you guys,
link |
we're just using your...
link |
All the hard work that you did.
link |
Yeah, that's incredible.
link |
What do you think, Mr. Anthony Lewandowski?
link |
What do you, he's a, he's another mad genius.
link |
What, do you think he might long term become a competitor?
link |
Well, so I think that he has the other right approach.
link |
I think that right now, there's two right approaches.
link |
One is what we're doing and one is what he's doing.
link |
Can you describe, I think it's called Pronto AI,
link |
he's starting using, do you know what the approach is?
link |
I actually don't know.
link |
Embark is also doing the same sort of thing.
link |
The idea is almost that you want to,
link |
so if you're, I can't partner with Honda and Toyota.
link |
Honda and Toyota are like 400,000 person companies.
link |
It's not even a company at that point.
link |
Like I don't think of it like, I don't personify it.
link |
I think of it like an object, but a trucker drives for a fleet.
link |
Maybe that has like, some truckers are independent.
link |
Some truckers drive for fleets with a hundred trucks.
link |
There are tons of independent trucking companies out there.
link |
Start a trucking company and drive your costs down
link |
or figure out how to drive down the cost of trucking.
link |
Another company that I really respect is Nauto.
link |
Actually, I respect their business model.
link |
Nauto sells a driver monitoring camera
link |
and they sell it to fleet owners.
link |
If I owned a fleet of cars and I could pay 40 bucks a month
link |
to monitor my employees,
link |
this is gonna like reduces accidents 18%.
link |
It's so like that in the space,
link |
that is like the business model that I like most respect
link |
because they're creating value today.
link |
Yeah, which is, that's a huge one.
link |
How do we create value today with some of this?
link |
And the length keeping thing is huge.
link |
And it sounds like you're creeping in
link |
or full steam ahead on the drive of monitoring too.
link |
Which I think actually where the short term value,
link |
if you can get right.
link |
I still, I'm not a huge fan of the statement
link |
that everything is to have drive of monitoring.
link |
I agree with that completely,
link |
but that statement usually misses the point
link |
that to get the experience of it right is not trivial.
link |
Oh, no, not at all.
link |
In fact, like, so right now we have,
link |
I think the timeout depends on speed of the car,
link |
but we want to depend on like the scene state.
link |
If you're on like an empty highway,
link |
it's very different if you don't pay attention
link |
than if like you're like coming up to a traffic light.
link |
And long term, it should probably learn from the driver
link |
because that's to do, I watched a lot of video.
link |
We've built a smartphone detector
link |
just to analyze how people are using smartphones
link |
and people are using it very differently.
link |
And there's this, it's a texting styles.
link |
We haven't watched nearly enough of the videos.
link |
We haven't, I got millions of miles
link |
of people driving cars.
link |
In this moment, I spent a large fraction of my time
link |
just watching videos because it's never fails to learn.
link |
Like I've never failed from a video watching session
link |
to learn something I didn't know before.
link |
In fact, I usually, like when I eat lunch, I'll sit,
link |
especially when the weather is good
link |
and just watch pedestrians.
link |
With an eye to understand like from a computer vision eye,
link |
just to see, can this model, can you predict
link |
what are the decisions made?
link |
And there's so many things that we don't understand.
link |
This is what I mean about state vector.
link |
Yeah, it's, I'm trying to always think like,
link |
because I'm understanding in my human brain,
link |
how do we convert that into,
link |
how hard is the learning problem here?
link |
I guess is the fundamental question.
link |
So something that's from a hacking perspective,
link |
this is always comes up, especially with folks.
link |
Well, first, the most popular question is
link |
the trolley problem, right?
link |
So that's not a sort of a serious problem.
link |
There are some ethical questions, I think that arise.
link |
Maybe you wanna, do you think there's any ethical,
link |
serious ethical questions?
link |
We have a solution to the trolley problem at com.ai.
link |
Well, so there is actually an alert
link |
in our code, ethical dilemma detected.
link |
It's not triggered yet.
link |
We don't know how yet to detect the ethical dilemmas,
link |
but we're a level two system.
link |
So we're going to disengage and leave
link |
that decision to the human.
link |
You're such a troll.
link |
No, but the trolley problem deserves to be trolled.
link |
Yeah, that's a beautiful answer actually.
link |
I know, I gave it to someone who was like,
link |
sometimes people ask like you asked about the trolley problem.
link |
Like you can have a kind of discussion about it.
link |
Like when you get someone who's like really like
link |
earnest about it, because it's the kind of thing where
link |
if you ask a bunch of people in an office,
link |
whether we should use a SQL stack or no SQL stack,
link |
if they're not that technical, they have no opinion.
link |
But if you ask them what color they want to paint the office,
link |
everyone has an opinion on that.
link |
And that's why the trolley problem is.
link |
I mean, that's a beautiful answer.
link |
Yeah, we're able to detect the problem
link |
and we're able to pass it on to the human.
link |
Wow, I've never heard anyone say it.
link |
This is your nice escape route.
link |
I'm proud level two.
link |
So the other thing that people have some concern about
link |
with AI in general is hacking.
link |
So how hard is it, do you think, to hack an autonomous vehicle
link |
either through physical access or through the more sort of
link |
popular now, these adversarial examples on the sensors?
link |
Okay, the adversarial examples one.
link |
You want to see some adversarial examples
link |
that affect humans, right?
link |
Oh, well, there used to be a stop sign here,
link |
but I put a black bag over the stop sign
link |
and then people ran it, adversarial, right?
link |
Like, there's tons of human adversarial examples too.
link |
The question in general about security, if you saw,
link |
something just came out today and there are always
link |
such hypey headlines about how navigate on autopilot
link |
was fooled by a GPS spoof to take an exit.
link |
At least that's all they could do was take an exit.
link |
If your car is relying on GPS in order
link |
to have a safe driving policy, you're doing something wrong.
link |
If you're relying, and this is why V2V
link |
is such a terrible idea, V2V now relies on both parties
link |
getting communication right.
link |
This is not even, so I think of safety,
link |
security is like a special case of safety, right?
link |
Safety is like we put a little, you know,
link |
piece of caution tape around the hole
link |
so that people won't walk into it by accident.
link |
Security is like put a 10 foot fence around the hole
link |
so you actually physically cannot climb into it
link |
with barbed wire on the top and stuff, right?
link |
So like if you're designing systems
link |
that are like unreliable, they're definitely not secure.
link |
Your car should always do something safe
link |
using its local sensors.
link |
And then the local sensor should be hardwired.
link |
And then could somebody hack into your can boss
link |
and turn your steering wheel on your brakes?
link |
Yes, but they could do it before comma AI too, so.
link |
Let's think out of the box and some things.
link |
So do you think teleoperation has a role in any of this?
link |
So remotely stepping in and controlling the cars?
link |
No, I think that if the safety operation
link |
by design requires a constant link to the cars,
link |
I think it doesn't work.
link |
So that's the same argument used for V2I, V2V.
link |
Well, there's a lot of non safety critical stuff
link |
you can do with V2I.
link |
I like V2I, I like V2I way more than V2V
link |
because V2I is already like,
link |
I already have internet in the car, right?
link |
There's a lot of great stuff you can do with V2I.
link |
Like for example, you can,
link |
well, where I already have V2V, Waze is V2I, right?
link |
Waze can route me around traffic jams.
link |
That's a great example of V2I.
link |
And then, okay, the car automatically talks
link |
to that same service, like it works.
link |
So it's improving the experience,
link |
but it's not a fundamental fallback for safety.
link |
No, if any of your things that require
link |
wireless communication are more than QM,
link |
like have an ASL rating, you shouldn't.
link |
You previously said that life is work
link |
and that you don't do anything to relax.
link |
So how do you think about hard work?
link |
What do you think it takes to accomplish great things?
link |
And there's a lot of people saying
link |
that there needs to be some balance.
link |
You know, you need to,
link |
in order to accomplish great things,
link |
you need to take some time off,
link |
you need to reflect and so on.
link |
And then some people are just insanely working,
link |
burning the candle on both ends.
link |
How do you think about that?
link |
I think I was trolling in the Suraj interview
link |
when I said that off camera,
link |
but right before I spoke to a little bit of weed,
link |
like, you know, come on, this is a joke, right?
link |
Like I do nothing to relax.
link |
Look where I am, I'm at a party, right?
link |
Yeah, yeah, yeah, sure.
link |
So no, no, of course I don't.
link |
When I say that life is work though,
link |
I mean that like, I think that
link |
what gives my life meaning is work.
link |
I don't mean that every minute of the day
link |
you should be working.
link |
I actually think this is not the best way
link |
to maximize results.
link |
I think that if you're working 12 hours a day,
link |
you should be working smarter and not harder.
link |
Well, so it gives work gives you meaning
link |
for some people, other sorts of meaning
link |
is personal relationships, like family and so on.
link |
You've also in that interview with Suraj
link |
or the trolling mentioned that one of the things
link |
you look forward to in the future is AI girlfriends.
link |
So that's a topic that I'm very much fascinated by,
link |
not necessarily girlfriends,
link |
but just forming a deep connection with AI.
link |
What kind of system do you imagine
link |
when you say AI girlfriend,
link |
whether you were trolling or not?
link |
No, that one I'm very serious about.
link |
And I'm serious about that on both a shallow level
link |
I think that VR brothels are coming soon
link |
and are gonna be really cool.
link |
It's not cheating if it's a robot.
link |
I see the slogan already.
link |
There's a, I don't know if you've watched
link |
or just watched the Black Mirror episode.
link |
I watched the latest one, yeah.
link |
Oh, the Ashley 2 one?
link |
No, where there's two friends
link |
who were having sex with each other in...
link |
Oh, in the VR game.
link |
In the VR game, it's the two guys,
link |
but one of them was a female, yeah.
link |
Which is another mind blowing concept.
link |
That in VR, you don't have to be the form.
link |
You can be two animals having sex, it's weird.
link |
I mean, I'll see how nice
link |
that the software maps the nerve endings, right?
link |
I mean, yeah, they sweep a lot of the fascinating,
link |
really difficult technical challenges under the rug,
link |
like assuming it's possible
link |
to do the mapping of the nerve endings, then...
link |
I wish, yeah, I saw that.
link |
The way they did it with the little like stim unit
link |
on the head, that'd be amazing.
link |
So, well, no, no, on a shallow level,
link |
like you could set up like almost a brothel
link |
with like real dolls and Oculus quests,
link |
write some good software.
link |
I think it'd be a cool novelty experience.
link |
But no, on a deeper, like emotional level.
link |
I mean, yeah, I would really like to fall in love
link |
Do you see yourself having a long term relationship
link |
of the kind monogamous relationship that we have now
link |
with the robot, with the AI system, even?
link |
Not even just the robot.
link |
So, I think about maybe my ideal future.
link |
When I was 15, I read Eliezer Yudkowsky's early writings
link |
on the singularity and like that AI
link |
is going to surpass human intelligence massively.
link |
He made some Moore's law based predictions
link |
that I mostly agree with.
link |
And then I really struggled
link |
for the next couple of years of my life.
link |
Like, why should I even bother to learn anything?
link |
It's all gonna be meaningless when the machine show up.
link |
Well, maybe when I was that young,
link |
I was still a little bit more pure
link |
and really like clung to that.
link |
And then I'm like, well, the machine's ain't here yet.
link |
You know, and I seem to be pretty good at this stuff.
link |
Let's try my best, you know,
link |
like what's the worst that happens?
link |
But the best possible future I see
link |
is me sort of merging with the machine.
link |
And the way that I personify this
link |
is in a longterm and augments relationship with the machine.
link |
Oh, you don't think there's room
link |
for another human in your life
link |
if you really truly merge with another machine?
link |
I mean, I see merging.
link |
I see like the best interface to my brain
link |
is like the same relationship interface
link |
to merge with an AI, right?
link |
What does that merging feel like?
link |
I've seen couples who've been together for a long time
link |
and like, I almost think of them as one person.
link |
Like couples who spend all their time together and...
link |
That's fascinating.
link |
You're actually putting,
link |
what does that merging actually looks like?
link |
It's not just a nice channel.
link |
Like a lot of people imagine it's just an efficient link,
link |
search link to Wikipedia or something.
link |
I don't believe in that.
link |
But it's more, you're saying that there's the same kind of,
link |
the same kind of relationship you have with another human
link |
as a deep relationship is that's what merging looks like.
link |
I don't believe that link is possible.
link |
I think that that link, so you're like,
link |
oh, I'm gonna download Wikipedia right to my brain.
link |
My reading speed is not limited by my eyes.
link |
My reading speed is limited by my inner processing loop.
link |
And to like bootstrap that
link |
sounds kind of unclear how to do it and horrifying.
link |
But if I am with somebody, and I'll use somebody
link |
who is making a super sophisticated model of me
link |
and then running simulations on that model,
link |
I'm not gonna get into the question
link |
whether the simulations are conscious or not.
link |
I don't really wanna know what it's doing.
link |
But using those simulations to play out hypothetical futures
link |
for me, deciding what things to say to me
link |
to guide me along a path and that's how I envision it.
link |
So on that path to AI of super human level intelligence,
link |
you've mentioned that you believe in the singularity,
link |
that singularity is coming.
link |
Again, could be trolling, could be not, could be part...
link |
All trolling has truth in it.
link |
I don't know what that means anymore.
link |
What is the singularity?
link |
So yeah, so that's really the question.
link |
How many years do you think before the singularity
link |
of what form do you think it will take?
link |
Does that mean fundamental shifts in capabilities of AI?
link |
Does it mean some other kind of ideas?
link |
Maybe that's just my roots, but...
link |
So I can buy a human being's worth of computers
link |
for things worth of compute for like a million bucks a day.
link |
It's about one TPU pod V3.
link |
I want like, I think they claim a hundred pay to flops.
link |
That's being generous.
link |
I think humans are actually more like 20.
link |
So that's like five humans.
link |
That's pretty good.
link |
Google needs to sell their TPUs.
link |
But no, I could buy GPUs.
link |
I could buy a stack of like, I buy 1080TIs,
link |
build data center full of them.
link |
And for a million bucks, I can get a human worth of compute.
link |
But when you look at the total number of flops in the world,
link |
when you look at human flops,
link |
which goes up very, very slowly with the population,
link |
and machine flops, which goes up exponentially,
link |
but it's still nowhere near.
link |
I think that's the key thing
link |
to talk about when the singularity happened.
link |
When most flops in the world are silicon
link |
and not biological, that's kind of the crossing point.
link |
Like they are now the dominant species on the planet.
link |
And just looking at how technology is progressing,
link |
when do you think that could possibly happen?
link |
Do you think it would happen in your lifetime?
link |
Oh yeah, definitely in my lifetime.
link |
I've done the math.
link |
I like 2038 because it's the UNIX timestamp roll over.
link |
Yeah, beautifully put.
link |
So you've said that the meaning of life is to win.
link |
If you look five years into the future,
link |
what does winning look like?
link |
I can go into technical depth to what I mean by that, to win.
link |
It may not mean...
link |
I was criticized for that in the comments.
link |
Like, doesn't this guy want to save the penguins in Antarctica?
link |
Or like, oh man, listen to what I'm saying.
link |
I'm not talking about like I have a yacht or something.
link |
I am put into this world.
link |
And I don't really know what my purpose is.
link |
But if you're a reinforcement, if you're an intelligent agent
link |
and you're put into a world, what is the ideal thing to do?
link |
Well, the ideal thing, mathematically,
link |
you can go back to like Schmidt Hoover theories about this,
link |
is to build a compressive model of the world.
link |
To build a maximally compressive to explore the world
link |
such that your exploration function maximizes
link |
the derivative of compression of the past.
link |
Schmidt Hoover has a paper about this.
link |
And like, I took that kind of as like a personal goal function.
link |
So what I mean to win, I mean like,
link |
maybe this is religious, but like I think that in the future
link |
I might be given a real purpose.
link |
Or I may decide this purpose myself.
link |
And then at that point, now I know what the game is
link |
and I know how to win.
link |
I think right now I'm still just trying to figure out what the game is.
link |
But once I know...
link |
You have imperfect information.
link |
You have a lot of uncertainty about the reward function
link |
and you're discovering it.
link |
But the purpose is...
link |
That's a better way to put it.
link |
The purpose is to maximize it
link |
while you have a lot of uncertainty around it.
link |
And you're both reducing the uncertainty
link |
and maximizing at the same time.
link |
And so that's at the technical level.
link |
If you believe in the universal prior,
link |
what is the universal reward function?
link |
That's the better way to put it.
link |
So that win is interesting.
link |
I think I speak for everyone in saying that
link |
I wonder what that reward function is for you.
link |
And I look forward to seeing that in five years and ten years.
link |
I think a lot of people including myself are cheering you on, man.
link |
So I'm happy you exist.
link |
And I wish you the best of luck.
link |
Thanks for talking today, man.
link |
This was a lot of fun.