back to indexGeorge Hotz: Hacking the Simulation & Learning to Drive with Neural Nets | Lex Fridman Podcast #132
link |
The following is a conversation with George Hotz,
link |
AKA Geohot, his second time on the podcast.
link |
He's the founder of Comma AI,
link |
an autonomous and semi autonomous vehicle technology company
link |
that seeks to be to Tesla Autopilot
link |
what Android is to the iOS.
link |
They sell the Comma 2 device for $1,000
link |
that when installed in many of their supported cars
link |
can keep the vehicle centered in the lane
link |
even when there are no lane markings.
link |
It includes driver sensing
link |
that ensures that the driver's eyes are on the road.
link |
As you may know, I'm a big fan of driver sensing.
link |
I do believe Tesla Autopilot and others
link |
should definitely include it in their sensor suite.
link |
Also, I'm a fan of Android and a big fan of George
link |
including his nonlinear out of the box brilliance
link |
and the fact that he's a superstar programmer
link |
of a very different style than myself.
link |
Styles make fights and styles make conversations.
link |
So I really enjoyed this chat
link |
and I'm sure we'll talk many more times on this podcast.
link |
Quick mention of a sponsor
link |
followed by some thoughts related to the episode.
link |
First is Four Sigmatic,
link |
the maker of delicious mushroom coffee.
link |
Second is The Coding Digital,
link |
a podcast on tech and entrepreneurship
link |
that I listen to and enjoy.
link |
And finally, ExpressVPN,
link |
the VPN I've used for many years to protect my privacy
link |
Please check out the sponsors in the description
link |
to get a discount and to support this podcast.
link |
As a side note, let me say that my work at MIT
link |
on autonomous and semi autonomous vehicles
link |
led me to study the human side of autonomy
link |
enough to understand that it's a beautifully complicated
link |
and interesting problem space,
link |
much richer than what can be studied in the lab.
link |
In that sense, the data that Comma AI, Tesla Autopilot
link |
and perhaps others like Cadillac Super Cruiser collecting
link |
gives us a chance to understand
link |
how we can design safe semi autonomous vehicles
link |
for real human beings in real world conditions.
link |
I think this requires bold innovation
link |
and a serious exploration of the first principles
link |
of the driving task itself.
link |
If you enjoyed this thing, subscribe on YouTube,
link |
review it with five stars and up a podcast,
link |
follow on Spotify, support on Patreon
link |
or connect with me on Twitter at Lex Friedman.
link |
And now here's my conversation with George Hotz.
link |
So last time we started talking about the simulation,
link |
this time let me ask you,
link |
do you think there's intelligent life out there
link |
I've always maintained my answer to the Fermi paradox.
link |
I think there has been intelligent life
link |
elsewhere in the universe.
link |
So intelligent civilizations existed
link |
but they've blown themselves up.
link |
So your general intuition is that
link |
intelligent civilizations quickly,
link |
like there's that parameter in the Drake equation.
link |
Your sense is they don't last very long.
link |
How are we doing on that?
link |
Like, have we lasted pretty good?
link |
I mean, not quite yet.
link |
Well, it was good to tell you,
link |
as you'd ask the IQ required to destroy the world
link |
falls by one point every year.
link |
Technology democratizes the destruction of the world.
link |
When can a meme destroy the world?
link |
It kind of is already, right?
link |
I don't think we've seen anywhere near the worst of it yet.
link |
Well, it's going to get weird.
link |
Well, maybe a meme can save the world.
link |
You thought about that?
link |
The meme Lord Elon Musk fighting on the side of good
link |
versus the meme Lord of the darkness,
link |
which is not saying anything bad about Donald Trump,
link |
but he is the Lord of the meme on the dark side.
link |
He's a Darth Vader of memes.
link |
I think in every fairy tale they always end it with,
link |
and they lived happily ever after.
link |
And I'm like, please tell me more
link |
about this happily ever after.
link |
I've heard 50% of marriages end in divorce.
link |
Why doesn't your marriage end up there?
link |
You can't just say happily ever after.
link |
So it's the thing about destruction
link |
is it's over after the destruction.
link |
We have to do everything right in order to avoid it.
link |
And one thing wrong,
link |
I mean, actually this is what I really like
link |
about cryptography.
link |
Cryptography, it seems like we live in a world
link |
where the defense wins versus like nuclear weapons.
link |
The opposite is true.
link |
It is much easier to build a warhead
link |
that splits into a hundred little warheads
link |
than to build something that can, you know,
link |
take out a hundred little warheads.
link |
The offense has the advantage there.
link |
So maybe our future is in crypto, but.
link |
So cryptography, right.
link |
The Goliath is the defense.
link |
And then all the different hackers are the Davids.
link |
And that equation is flipped for nuclear war.
link |
Cause there's so many,
link |
like one nuclear weapon destroys everything essentially.
link |
Yeah, and it is much easier to attack with a nuclear weapon
link |
than it is to like the technology required to intercept
link |
and destroy a rocket is much more complicated
link |
than the technology required to just, you know,
link |
orbital trajectory, send a rocket to somebody.
link |
Your intuition that there were intelligent civilizations
link |
out there, but it's very possible
link |
that they're no longer there.
link |
That's kind of a sad picture.
link |
They enter some steady state.
link |
They all wirehead themselves.
link |
Stimulate, stimulate their pleasure centers
link |
and just, you know, live forever in this kind of stasis.
link |
They become, well, I mean,
link |
I think the reason I believe this is because where are they?
link |
If there's some reason they stopped expanding,
link |
cause otherwise they would have taken over the universe.
link |
The universe isn't that big.
link |
Or at least, you know,
link |
let's just talk about the galaxy, right?
link |
That's 70,000 light years across.
link |
I took that number from Star Trek Voyager.
link |
I don't know how true it is, but yeah, that's not big.
link |
Right? 70,000 light years is nothing.
link |
For some possible technology that you can imagine
link |
that can leverage like wormholes or something like that.
link |
Or you don't even need wormholes.
link |
Just a von Neumann probe is enough.
link |
A von Neumann probe and a million years of sublight travel
link |
and you'd have taken over the whole universe.
link |
That clearly didn't happen.
link |
So something stopped it.
link |
So you mean if you, right,
link |
for like a few million years,
link |
if you sent out probes that travel close,
link |
You mean close to the speed of light?
link |
And it just spreads.
link |
Actually, that's an interesting calculation, huh?
link |
So what makes you think that we'd be able
link |
to communicate with them?
link |
Like, yeah, what's,
link |
why do you think we would be able to be able
link |
to comprehend intelligent lives that are out there?
link |
Like even if they were among us kind of thing,
link |
like, or even just flying around?
link |
Well, I mean, that's possible.
link |
It's possible that there is some sort of prime directive.
link |
That'd be a really cool universe to live in.
link |
And there's some reason
link |
they're not making themselves visible to us.
link |
But it makes sense that they would use the same,
link |
well, at least the same entropy.
link |
Well, you're implying the same laws of physics.
link |
I don't know what you mean by entropy in this case.
link |
I mean, if entropy is the scarce resource in the universe.
link |
So what do you think about like Stephen Wolfram
link |
and everything is a computation?
link |
And then what if they are traveling through
link |
this world of computation?
link |
So if you think of the universe
link |
as just information processing,
link |
then what you're referring to with entropy
link |
and then these pockets of interesting complex computation
link |
swimming around, how do we know they're not already here?
link |
How do we know that this,
link |
like all the different amazing things
link |
that are full of mystery on earth
link |
are just like little footprints of intelligence
link |
from light years away?
link |
I mean, I tend to think that as civilizations expand,
link |
they use more and more energy
link |
and you can never overcome the problem of waste heat.
link |
So where is there waste heat?
link |
So we'd be able to, with our crude methods,
link |
be able to see like, there's a whole lot of energy here.
link |
But it could be something we're not,
link |
I mean, we don't understand dark energy, right?
link |
It could be just stuff we don't understand at all.
link |
Or they can have a fundamentally different physics,
link |
you know, like that we just don't even comprehend.
link |
Well, I think, okay,
link |
I mean, it depends how far out you wanna go.
link |
I don't think physics is very different
link |
on the other side of the galaxy.
link |
I would suspect that they have,
link |
I mean, if they're in our universe,
link |
they have the same physics.
link |
Well, yeah, that's the assumption we have,
link |
but there could be like super trippy things
link |
like our cognition only gets to a slice,
link |
and all the possible instruments that we can design
link |
only get to a particular slice of the universe.
link |
And there's something much like weirder.
link |
Maybe we can try a thought experiment.
link |
Would people from the past
link |
be able to detect the remnants of our,
link |
or would we be able to detect our modern civilization?
link |
I think the answer is obviously yes.
link |
You mean past from a hundred years ago?
link |
Well, let's even go back further.
link |
Let's go to a million years ago, right?
link |
The humans who were lying around in the desert
link |
probably didn't even have,
link |
maybe they just barely had fire.
link |
They would understand if a 747 flew overhead.
link |
Oh, in this vicinity, but not if a 747 flew on Mars.
link |
Like, cause they wouldn't be able to see far,
link |
cause we're not actually communicating that well
link |
with the rest of the universe.
link |
Just sending out random like fifties tracks of music.
link |
And yeah, I mean, they'd have to, you know,
link |
we've only been broadcasting radio waves for 150 years.
link |
And well, there's your light cone.
link |
What do you make about all the,
link |
I recently came across this having talked to David Fravor.
link |
I don't know if you caught what the videos
link |
of the Pentagon released
link |
and the New York Times reporting of the UFO sightings.
link |
So I kind of looked into it, quote unquote.
link |
And there's actually been like hundreds
link |
of thousands of UFO sightings, right?
link |
And a lot of it you can explain away
link |
in different kinds of ways.
link |
So one is it could be interesting physical phenomena.
link |
Two, it could be people wanting to believe
link |
and therefore they conjure up a lot of different things
link |
that just, you know, when you see different kinds of lights,
link |
some basic physics phenomena,
link |
and then you just conjure up ideas
link |
of possible out there mysterious worlds.
link |
But, you know, it's also possible,
link |
like you have a case of David Fravor,
link |
who is a Navy pilot, who's, you know,
link |
as legit as it gets in terms of humans
link |
who are able to perceive things in the environment
link |
and make conclusions,
link |
whether those things are a threat or not.
link |
And he and several other pilots saw a thing,
link |
I don't know if you followed this,
link |
but they saw a thing that they've since then called TikTok
link |
that moved in all kinds of weird ways.
link |
They don't know what it is.
link |
It could be technology developed by the United States
link |
and they're just not aware of it
link |
and the surface level from the Navy, right?
link |
It could be different kind of lighting technology
link |
or drone technology, all that kind of stuff.
link |
It could be the Russians and the Chinese,
link |
all that kind of stuff.
link |
And of course their mind, our mind,
link |
can also venture into the possibility
link |
that it's from another world.
link |
Have you looked into this at all?
link |
What do you think about it?
link |
I think all the news is a psyop.
link |
I think that the most plausible.
link |
Yeah, I listened to the, I think it was Bob Lazar
link |
And like, I believe everything this guy is saying.
link |
And then I think that it's probably just some like MKUltra
link |
kind of thing, you know?
link |
Like they, you know, they made some weird thing
link |
and they called it an alien spaceship.
link |
You know, maybe it was just to like
link |
stimulate young physicists minds.
link |
We'll tell them it's alien technology
link |
and we'll see what they come up with, right?
link |
Do you find any conspiracy theories compelling?
link |
Like have you pulled at the string
link |
of the rich complex world of conspiracy theories
link |
I think that I've heard a conspiracy theory
link |
that conspiracy theories were invented by the CIA
link |
in the 60s to discredit true things.
link |
So, you know, you can go to ridiculous conspiracy theories
link |
like Flat Earth and Pizza Gate.
link |
And, you know, these things are almost to hide
link |
like conspiracy theories that like,
link |
you know, remember when the Chinese like locked up
link |
the doctors who discovered coronavirus?
link |
Like I tell people this and I'm like,
link |
no, no, no, that's not a conspiracy theory.
link |
That actually happened.
link |
Do you remember the time that the money used to be backed
link |
by gold and now it's backed by nothing?
link |
This is not a conspiracy theory.
link |
This actually happened.
link |
Well, that's one of my worries today
link |
with the idea of fake news is that when nothing is real,
link |
then like you dilute the possibility of anything being true
link |
by conjuring up all kinds of conspiracy theories.
link |
And then you don't know what to believe.
link |
And then like the idea of truth of objectivity
link |
is lost completely.
link |
Everybody has their own truth.
link |
So you used to control information by censoring it.
link |
And then the internet happened and governments were like,
link |
oh shit, we can't censor things anymore.
link |
I know what we'll do.
link |
You know, it's the old story of the story of like
link |
tying a flag with a leprechaun tells you his gold is buried
link |
and you tie one flag and you make the leprechaun swear
link |
to not remove the flag.
link |
And you come back to the field later with a shovel
link |
and there's flags everywhere.
link |
That's one way to maintain privacy, right?
link |
It's like in order to protect the contents
link |
of this conversation, for example,
link |
we could just generate like millions of deep,
link |
fake conversations where you and I talk
link |
and say random things.
link |
So this is just one of them
link |
and nobody knows which one was the real one.
link |
This could be fake right now.
link |
Classic steganography technique.
link |
Okay, another absurd question about intelligent life.
link |
Cause you know, you're an incredible programmer
link |
outside of everything else we'll talk about
link |
just as a programmer.
link |
Do you think intelligent beings out there,
link |
the civilizations that were out there,
link |
had computers and programming?
link |
Did they, do we naturally have to develop something
link |
where we engineer machines and are able to encode
link |
both knowledge into those machines
link |
and instructions that process that knowledge,
link |
process that information to make decisions
link |
and actions and so on?
link |
And would those programming languages,
link |
if you think they exist, be at all similar
link |
to anything we've developed?
link |
So I don't see that much of a difference
link |
between quote unquote natural languages
link |
and programming languages.
link |
I think there's so many similarities.
link |
So when asked the question,
link |
what do alien languages look like?
link |
I imagine they're not all that dissimilar from ours.
link |
And I think translating in and out of them
link |
wouldn't be that crazy.
link |
Well, it's difficult to compile like DNA to Python
link |
There's a little bit of a gap in the kind of languages
link |
we use for touring machines
link |
and the kind of languages nature seems to use a little bit.
link |
Maybe that's just, we just haven't understood
link |
the kind of language that nature uses well yet.
link |
DNA is a CAD model.
link |
It's not quite a programming language.
link |
It has no sort of a serial execution.
link |
It's not quite a, yeah, it's a CAD model.
link |
So I think in that sense,
link |
we actually completely understand it.
link |
The problem is, well, simulating on these CAD models,
link |
I played with it a bit this year,
link |
is super computationally intensive.
link |
If you wanna go down to like the molecular level
link |
where you need to go to see a lot of these phenomenon
link |
like protein folding.
link |
So yeah, it's not that we don't understand it.
link |
It just requires a whole lot of compute to kind of compile it.
link |
For our human minds, it's inefficient,
link |
both for the data representation and for the programming.
link |
Yeah, it runs well on raw nature.
link |
It runs well on raw nature.
link |
And when we try to build emulators or simulators for that,
link |
well, they're mad slow, but I've tried it.
link |
It runs in that, yeah, you've commented elsewhere,
link |
I don't remember where,
link |
that one of the problems is simulating nature is tough.
link |
And if you want to sort of deploy a prototype,
link |
I forgot how you put it, but it made me laugh,
link |
but animals or humans would need to be involved
link |
in order to try to run some prototype code on,
link |
like if we're talking about COVID and viruses and so on,
link |
if you were trying to engineer
link |
some kind of defense mechanisms,
link |
like a vaccine against COVID and all that kind of stuff
link |
that doing any kind of experimentation,
link |
like you can with like autonomous vehicles
link |
would be very technically and ethically costly.
link |
I'm not sure about that.
link |
I think you can do tons of crazy biology and test tubes.
link |
I think my bigger complaint is more,
link |
oh, the tools are so bad.
link |
Like literally, you mean like libraries and?
link |
I don't know, I'm not pipetting shit.
link |
Like you're handing me a, I got a, no, no, no,
link |
there has to be some.
link |
Like automating stuff.
link |
And like the, yeah, but human biology is messy.
link |
But like, look at those Toronto's videos.
link |
It's like a little gantry.
link |
It's like little X, Y gantry,
link |
high school science project with the pipette.
link |
Gotta be something better.
link |
You can't build like nice microfluidics
link |
and I can program the computation to bio interface.
link |
I mean, this is gonna happen.
link |
But like right now, if you are asking me
link |
to pipette 50 milliliters of solution, I'm out.
link |
Okay, let's get all the crazy out of the way.
link |
So a bunch of people asked me,
link |
since we talked about the simulation last time,
link |
we talked about hacking the simulation.
link |
Do you have any updates, any insights
link |
about how we might be able to go about hacking simulation
link |
if we indeed do live in a simulation?
link |
I think a lot of people misinterpreted
link |
the point of that South by talk.
link |
The point of the South by talk
link |
was not literally to hack the simulation.
link |
I think that this is an idea is literally just,
link |
I think theoretical physics.
link |
I think that's the whole goal, right?
link |
You want your grand unified theory, but then, okay,
link |
build a grand unified theory search for exploits, right?
link |
I think we're nowhere near actually there yet.
link |
My hope with that was just more to like,
link |
are you people kidding me
link |
with the things you spend time thinking about?
link |
Do you understand like kind of how small you are?
link |
You are bytes and God's computer, really?
link |
And the things that people get worked up about, you know?
link |
So basically, it was more a message
link |
of we should humble ourselves.
link |
That we get to, like what are we humans in this byte code?
link |
Yeah, and not just humble ourselves,
link |
but like I'm not trying to like make people guilty
link |
or anything like that.
link |
I'm trying to say like, literally,
link |
look at what you are spending time on, right?
link |
What are you referring to?
link |
You're referring to the Kardashians?
link |
What are we talking about?
link |
No, the Kardashians, everyone knows that's kind of fun.
link |
I'm referring more to like the economy, you know?
link |
This idea that we gotta up our stock price.
link |
Or what is the goal function of humanity?
link |
You don't like the game of capitalism?
link |
Like you don't like the games we've constructed
link |
for ourselves as humans?
link |
I'm a big fan of capitalism.
link |
I don't think that's really the game we're playing right now.
link |
I think we're playing a different game
link |
where the rules are rigged.
link |
Okay, which games are interesting to you
link |
that we humans have constructed and which aren't?
link |
Which are productive and which are not?
link |
Actually, maybe that's the real point of the talk.
link |
It's like, stop playing these fake human games.
link |
There's a real game here.
link |
We can play the real game.
link |
The real game is, you know, nature wrote the rules.
link |
This is a real game.
link |
There still is a game to play.
link |
But if you look at, sorry to interrupt,
link |
I don't know if you've seen the Instagram account,
link |
The game that nature seems to be playing
link |
is a lot more cruel than we humans want to put up with.
link |
Or at least we see it as cruel.
link |
It's like the bigger thing eats the smaller thing
link |
and does it to impress another big thing
link |
so it can mate with that thing.
link |
That seems to be the entirety of it.
link |
Well, there's no art, there's no music,
link |
there's no comma AI, there's no comma one,
link |
no comma two, no George Hots with his brilliant talks
link |
at South by Southwest.
link |
I disagree, though.
link |
I disagree that this is what nature is.
link |
I think nature just provided basically a open world MMORPG.
link |
And, you know, here it's open world.
link |
I mean, if that's the game you want to play,
link |
you can play that game.
link |
But isn't that beautiful?
link |
I don't know if you played Diablo.
link |
They used to have, I think, cow level where it's...
link |
So everybody will go just, they figured out this,
link |
like the best way to gain like experience points
link |
is to just slaughter cows over and over and over.
link |
And so they figured out this little sub game
link |
within the bigger game that this is the most efficient way
link |
to get experience points.
link |
And everybody somehow agreed
link |
that getting experience points in RPG context
link |
where you always want to be getting more stuff,
link |
more skills, more levels, keep advancing.
link |
That seems to be good.
link |
So might as well sacrifice actual enjoyment
link |
of playing a game, exploring a world,
link |
and spending like hundreds of hours of your time
link |
I mean, the number of hours I spent in cow level,
link |
I'm not like the most impressive person
link |
because people have spent probably thousands of hours there,
link |
but it's ridiculous.
link |
So that's a little absurd game that brought me joy
link |
in some weird dopamine drug kind of way.
link |
So you don't like those games.
link |
You don't think that's us humans feeling the nature.
link |
And that was the point of the talk.
link |
So how do we hack it then?
link |
Well, I want to live forever.
link |
And I want to live forever.
link |
And this is the goal.
link |
Well, that's a game against nature.
link |
Yeah, immortality is the good objective function to you?
link |
I mean, start there and then you can do whatever else
link |
you want because you got a long time.
link |
What if immortality makes the game just totally not fun?
link |
I mean, like, why do you assume immortality
link |
is somehow a good objective function?
link |
It's not immortality that I want.
link |
A true immortality where I could not die,
link |
I would prefer what we have right now.
link |
But I want to choose my own death, of course.
link |
I don't want nature to decide when I die,
link |
I'm going to be you.
link |
And then at some point, if you choose commit suicide,
link |
like how long do you think you'd live?
link |
Until I get bored.
link |
See, I don't think people like brilliant people like you
link |
that really ponder living a long time
link |
are really considering how meaningless life becomes.
link |
Well, I want to know everything and then I'm ready to die.
link |
As long as there's...
link |
Yeah, but why do you want,
link |
isn't it possible that you want to know everything
link |
because it's finite?
link |
Like the reason you want to know quote unquote everything
link |
is because you don't have enough time to know everything.
link |
And once you have unlimited time,
link |
then you realize like, why do anything?
link |
Like why learn anything?
link |
I want to know everything and then I'm ready to die.
link |
So you have, yeah.
link |
It's not a, like, it's a terminal value.
link |
It's not in service of anything else.
link |
I'm conscious of the possibility, this is not a certainty,
link |
but the possibility of that engine of curiosity
link |
that you're speaking to is actually
link |
a symptom of the finiteness of life.
link |
Like without that finiteness, your curiosity would vanish.
link |
Like a morning fog.
link |
Bukowski talked about love like that.
link |
Then let me solve immortality
link |
and let me change the thing in my brain
link |
that reminds me of the fact that I'm immortal,
link |
tells me that life is finite shit.
link |
Maybe I'll have it tell me that life ends next week.
link |
I'm okay with some self manipulation like that.
link |
I'm okay with deceiving myself.
link |
Oh, Rika, changing the code.
link |
Yeah, well, if that's the problem, right?
link |
If the problem is that I will no longer have that,
link |
that curiosity, I'd like to have backup copies of myself,
link |
which I check in with occasionally
link |
to make sure they're okay with the trajectory
link |
and they can kind of override it.
link |
Maybe a nice, like, I think of like those wave nets,
link |
those like logarithmic go back to the copies.
link |
Yeah, but sometimes it's not reversible.
link |
Like I've done this with video games.
link |
Once you figure out the cheat code
link |
or like you look up how to cheat old school,
link |
like single player, it ruins the game for you.
link |
It ruins that feeling.
link |
But again, that just means our brain manipulation
link |
technology is not good enough yet.
link |
Remove that cheat code from your brain.
link |
So it's also possible that if we figure out immortality,
link |
that all of us will kill ourselves
link |
before we advance far enough
link |
to be able to revert the change.
link |
I'm not killing myself till I know everything, so.
link |
That's what you say now, because your life is finite.
link |
You know, I think yes, self modifying systems gets,
link |
comes up with all these hairy complexities
link |
and can I promise that I'll do it perfectly?
link |
No, but I think I can put good safety structures in place.
link |
So that talk and your thinking here
link |
is not literally referring to a simulation
link |
and that our universe is a kind of computer program
link |
running on a computer.
link |
That's more of a thought experiment.
link |
Do you also think of the potential of the sort of Bostrom,
link |
Elon Musk and others that talk about an actual program
link |
that simulates our universe?
link |
Oh, I don't doubt that we're in a simulation.
link |
I just think that it's not quite that important.
link |
I mean, I'm interested only in simulation theory
link |
as far as like it gives me power over nature.
link |
If it's totally unfalsifiable, then who cares?
link |
I mean, what do you think that experiment would look like?
link |
Like somebody on Twitter asks,
link |
asks George what signs we would look for
link |
to know whether or not we're in the simulation,
link |
which is exactly what you're asking is like,
link |
the step that precedes the step of knowing
link |
how to get more power from this knowledge
link |
is to get an indication that there's some power to be gained.
link |
So get an indication that there,
link |
you can discover and exploit cracks in the simulation
link |
or it doesn't have to be in the physics of the universe.
link |
Show me, I mean, like a memory leak could be cool.
link |
Like some scrying technology, you know?
link |
What kind of technology?
link |
Oh, that's a weird,
link |
scrying is the paranormal ability to like remote viewing,
link |
like being able to see somewhere where you're not.
link |
So, you know, I don't think you can do it
link |
by chanting in a room,
link |
but if we could find, it's a memory leak, basically.
link |
It's a memory leak.
link |
Yeah, you're able to access parts you're not supposed to.
link |
And thereby discover a shortcut.
link |
Yeah, maybe memory leak means the other thing as well,
link |
but I mean like, yeah,
link |
like an ability to read arbitrary memory, right?
link |
And that one's not that horrifying, right?
link |
The right ones start to be horrifying.
link |
It's the reading is not the problem.
link |
Yeah, it's like Heartfleet for the universe.
link |
Oh boy, the writing is a big, big problem.
link |
It's a big problem.
link |
It's the moment you can write anything,
link |
even if it's just random noise.
link |
That's terrifying.
link |
I mean, even without that,
link |
like even some of the, you know,
link |
the nanotech stuff that's coming, I think is.
link |
I don't know if you're paying attention,
link |
but actually Eric Weistand came out
link |
with the theory of everything.
link |
I mean, that came out.
link |
He's been working on a theory of everything
link |
in the physics world called geometric unity.
link |
And then for me, from computer science person like you,
link |
Steven Wolfram's theory of everything,
link |
of like hypergraphs is super interesting and beautiful,
link |
but not from a physics perspective,
link |
but from a computational perspective.
link |
I don't know, have you paid attention to any of that?
link |
So again, like what would make me pay attention
link |
and like why like I hate string theory is,
link |
okay, make a testable prediction, right?
link |
I'm only interested in,
link |
I'm not interested in theories for their intrinsic beauty.
link |
I'm interested in theories
link |
that give me power over the universe.
link |
So if these theories do, I'm very interested.
link |
Can I just say how beautiful that is?
link |
Because a lot of physicists say,
link |
I'm interested in experimental validation
link |
and they skip out the part where they say
link |
to give me more power in the universe.
link |
No, I want. The clarity of that.
link |
I want 100 gigahertz processors.
link |
I want transistors that are smaller than atoms.
link |
I want like power.
link |
And that's where people from aliens
link |
to this kind of technology where people are worried
link |
that governments, like who owns that power?
link |
Is it George Harts?
link |
Is it thousands of distributed hackers across the world?
link |
Is it governments?
link |
Is it Mark Zuckerberg?
link |
There's a lot of people that,
link |
I don't know if anyone trusts any one individual with power.
link |
So they're always worried.
link |
It's the beauty of blockchains.
link |
That's the beauty of blockchains, which we'll talk about.
link |
On Twitter, somebody pointed me to a story,
link |
a bunch of people pointed me to a story a few months ago
link |
where you went into a restaurant in New York.
link |
And you can correct me if any of this is wrong.
link |
And ran into a bunch of folks from a company
link |
in a crypto company who are trying to scale up Ethereum.
link |
And they had a technical deadline
link |
related to solidity to OVM compiler.
link |
So these are all Ethereum technologies.
link |
So you stepped in, they recognized you,
link |
pulled you aside, explained their problem.
link |
And you stepped in and helped them solve the problem,
link |
thereby creating legend status story.
link |
So can you tell me the story in a little more detail?
link |
It seems kind of incredible.
link |
Yeah, yeah, it's a true story, it's a true story.
link |
I mean, they wrote a very flattering account of it.
link |
So Optimism is the company called Optimism,
link |
spin off of Plasma.
link |
They're trying to build L2 solutions on Ethereum.
link |
So right now, every Ethereum node
link |
has to run every transaction on the Ethereum network.
link |
And this kind of doesn't scale, right?
link |
Because if you have N computers,
link |
well, if that becomes two N computers,
link |
you actually still get the same amount of compute, right?
link |
This is like all of one scaling
link |
because they all have to run it.
link |
Okay, fine, you get more blockchain security,
link |
but like, blockchain is already so secure.
link |
Can we trade some of that off for speed?
link |
So that's kind of what these L2 solutions are.
link |
They built this thing, which kind of,
link |
kind of sandbox for Ethereum contracts.
link |
So they can run it in this L2 world
link |
and it can't do certain things in L world, in L1.
link |
Can I ask you for some definitions?
link |
Oh, L2 is layer two.
link |
So L1 is like the base Ethereum chain.
link |
And then layer two is like a computational layer
link |
that runs elsewhere,
link |
but still is kind of secured by layer one.
link |
And I'm sure a lot of people know,
link |
but Ethereum is a cryptocurrency,
link |
probably one of the most popular cryptocurrency
link |
second to Bitcoin.
link |
And a lot of interesting technological innovation there.
link |
Maybe you could also slip in whenever you talk about this
link |
and things that are exciting to you in the Ethereum space.
link |
Well, I mean, Bitcoin is not Turing complete.
link |
Ethereum is not technically Turing complete
link |
with the gas limit, but close enough.
link |
With the gas limit?
link |
What's the gas limit, resources?
link |
Yeah, I mean, no computer is actually Turing complete.
link |
You're gonna find out RAM, you know?
link |
I can actually solve the whole thing.
link |
What's the word gas limit?
link |
You just have so many brilliant words.
link |
I'm not even gonna ask.
link |
That's not my word, that's Ethereum's word.
link |
Ethereum, you have to spend gas per instruction.
link |
So like different op codes use different amounts of gas
link |
and you buy gas with ether to prevent people
link |
from basically DDoSing the network.
link |
So Bitcoin is proof of work.
link |
And then what's Ethereum?
link |
It's also proof of work.
link |
They're working on some proof of stake,
link |
Ethereum 2.0 stuff.
link |
But right now it's proof of work.
link |
It uses a different hash function from Bitcoin.
link |
That's more ASIC resistance, because you need RAM.
link |
So we're all talking about Ethereum 1.0.
link |
So what were they trying to do to scale this whole process?
link |
So they were like, well, if we could run contracts elsewhere
link |
and then only save the results of that computation,
link |
well, we don't actually have to do the compute on the chain.
link |
We can do the compute off chain
link |
and just post what the results are.
link |
Now, the problem with that is,
link |
well, somebody could lie about what the results are.
link |
So you need a resolution mechanism.
link |
And the resolution mechanism can be really expensive
link |
because you just have to make sure
link |
that the person who is saying,
link |
look, I swear that this is the real computation.
link |
I'm staking $10,000 on that fact.
link |
And if you prove it wrong,
link |
yeah, it might cost you $3,000 in gas fees to prove wrong,
link |
but you'll get the $10,000 bounty.
link |
So you can secure using those kinds of systems.
link |
So it's effectively a sandbox, which runs contracts.
link |
And like, it's like any kind of normal sandbox,
link |
you have to like replace syscalls
link |
with calls into the hypervisor.
link |
Sandbox, syscalls, hypervisor.
link |
What do these things mean?
link |
As long as it's interesting to talk about.
link |
Yeah, I mean, you can take like the Chrome sandbox
link |
is maybe the one to think about, right?
link |
So the Chrome process that's doing a rendering,
link |
can't, for example, read a file from the file system.
link |
It has, if it tries to make an open syscall in Linux,
link |
the open syscall, you can't make it open syscall,
link |
You have to request from the kind of hypervisor process
link |
or like, I don't know what it's called in Chrome,
link |
but the, hey, could you open this file for me?
link |
And then it does all these checks
link |
and then it passes the file handle back in
link |
So what's the, in the context of Ethereum,
link |
what are the boundaries of the sandbox
link |
that we're talking about?
link |
Well, like one of the calls that you,
link |
actually reading and writing any state
link |
to the Ethereum contract,
link |
or to the Ethereum blockchain.
link |
Writing state is one of those calls
link |
that you're going to have to sandbox in layer two,
link |
because if you let layer two just arbitrarily write
link |
to the Ethereum blockchain.
link |
So layer two is really sitting on top of layer one.
link |
So you're going to have a lot of different kinds of ideas
link |
that you can play with.
link |
And they're all, they're not fundamentally changing
link |
the source code level of Ethereum.
link |
Well, you have to replace a bunch of calls
link |
with calls into the hypervisor.
link |
So instead of doing the syscall directly,
link |
you replace it with a call to the hypervisor.
link |
So originally they were doing this
link |
by first running the, so Solidity is the language
link |
that most Ethereum contracts are written in.
link |
It compiles to a bytecode.
link |
And then they wrote this thing they called the transpiler.
link |
And the transpiler took the bytecode
link |
and it transpiled it into OVM safe bytecode.
link |
Basically bytecode that didn't make any
link |
of those restricted syscalls
link |
and added the calls to the hypervisor.
link |
This transpiler was a 3000 line mess.
link |
And it's hard to do.
link |
It's hard to do if you're trying to do it like that,
link |
because you have to kind of like deconstruct the bytecode,
link |
change things about it, and then reconstruct it.
link |
And I mean, as soon as I hear this, I'm like,
link |
well, why don't you just change the compiler, right?
link |
Why not the first place you build the bytecode,
link |
just do it in the compiler.
link |
So yeah, I asked them how much they wanted it.
link |
Of course, measured in dollars and I'm like, well, okay.
link |
And you wrote the compiler.
link |
Yeah, I modified, I wrote a 300 line diff to the compiler.
link |
It's open source, you can look at it.
link |
Yeah, it's, yeah, I looked at the code last night.
link |
It's, yeah, exactly.
link |
Cute is a good word for it.
link |
So when asked how you were able to do it,
link |
you said, you just gotta think and then do it right.
link |
So can you break that apart a little bit?
link |
What's your process of one, thinking and two, doing it right?
link |
You know, the people that I was working for
link |
were amused that I said that.
link |
It doesn't really mean anything.
link |
I mean, is there some deep, profound insights
link |
to draw from like how you problem solve from that?
link |
This is always what I say.
link |
I'm like, do you wanna be a good programmer?
link |
Do it for 20 years.
link |
Yeah, there's no shortcuts.
link |
What are your thoughts on crypto in general?
link |
So what parts technically or philosophically
link |
do you find especially beautiful maybe?
link |
Oh, I'm extremely bullish on crypto longterm.
link |
Not any specific crypto project, but this idea of,
link |
One, the Nakamoto Consensus Algorithm
link |
is I think one of the greatest innovations
link |
of the 21st century.
link |
This idea that people can reach consensus.
link |
You can reach a group consensus.
link |
Using a relatively straightforward algorithm is wild.
link |
And like, you know, Satoshi Nakamoto,
link |
people always ask me who I look up to.
link |
It's like, whoever that is.
link |
Who do you think it is?
link |
I mean, Elon Musk?
link |
It is definitely not me.
link |
And I do not think it's Elon Musk.
link |
But yeah, this idea of groups reaching consensus
link |
in a decentralized yet formulaic way
link |
is one extremely powerful idea from crypto.
link |
Maybe the second idea is this idea of smart contracts.
link |
When you write a contract between two parties,
link |
any contract, this contract, if there are disputes,
link |
it's interpreted by lawyers.
link |
Lawyers are just really shitty overpaid interpreters.
link |
Imagine you had, let's talk about them in terms of a,
link |
in terms of like, let's compare a lawyer to Python, right?
link |
So lawyer, well, okay.
link |
That's really, I never thought of it that way.
link |
So Python, I'm paying even 10 cents an hour.
link |
I'll use the nice Azure machine.
link |
I can run Python for 10 cents an hour.
link |
Lawyers cost $1,000 an hour.
link |
So Python is 10,000 X better on that axis.
link |
Lawyers don't always return the same answer.
link |
Python almost always does.
link |
Yeah, I mean, just cost, reliability,
link |
everything about Python is so much better than lawyers.
link |
So if you can make smart contracts,
link |
this whole concept of code is law.
link |
I love, and I would love to live in a world
link |
where everybody accepted that fact.
link |
So maybe you can talk about what smart contracts are.
link |
So let's say, let's say, you know,
link |
we have a, even something as simple
link |
as a safety deposit box, right?
link |
Safety deposit box that holds a million dollars.
link |
I have a contract with the bank that says
link |
two out of these three parties must be present
link |
to open the safety deposit box and get the money out.
link |
So that's a contract for the bank,
link |
and it's only as good as the bank and the lawyers, right?
link |
Let's say, you know, somebody dies and now,
link |
oh, we're gonna go through a big legal dispute
link |
about whether, oh, well, was it in the will,
link |
was it not in the will?
link |
What, like, it's just so messy,
link |
and the cost to determine truth is so expensive.
link |
Versus a smart contract, which just uses cryptography
link |
to check if two out of three keys are present.
link |
Well, I can look at that, and I can have certainty
link |
in the answer that it's going to return.
link |
And that's what, all businesses want is certainty.
link |
You know, they say businesses don't care.
link |
Viacom, YouTube, YouTube's like,
link |
look, we don't care which way this lawsuit goes.
link |
Just please tell us so we can have certainty.
link |
Yeah, I wonder how many agreements in this,
link |
because we're talking about financial transactions only
link |
in this case, correct, the smart contracts?
link |
Oh, you can go to anything.
link |
You can put a prenup in the Ethereum blockchain.
link |
A married smart contract?
link |
Sorry, divorce lawyer, sorry.
link |
You're going to be replaced by Python.
link |
Okay, so that's another beautiful idea.
link |
Do you think there's something that's appealing to you
link |
about any one specific implementation?
link |
So if you look 10, 20, 50 years down the line,
link |
do you see any, like, Bitcoin, Ethereum,
link |
any of the other hundreds of cryptocurrencies winning out?
link |
Is there, like, what's your intuition about the space?
link |
Or are you just sitting back and watching the chaos
link |
and look who cares what emerges?
link |
I don't speculate.
link |
I don't really care.
link |
I don't really care which one of these projects wins.
link |
I'm kind of in the Bitcoin as a meme coin camp.
link |
I mean, why does Bitcoin have value?
link |
It's technically kind of, you know,
link |
not great, like the block size debate.
link |
When I found out what the block size debate was,
link |
I'm like, are you guys kidding?
link |
What's the block size debate?
link |
It's really, it's too stupid to even talk.
link |
People can look it up, but I'm like, wow.
link |
You know, Ethereum seems,
link |
the governance of Ethereum seems much better.
link |
I've come around a bit on proof of stake ideas.
link |
You know, very smart people thinking about some things.
link |
Yeah, you know, governance is interesting.
link |
It does feel like Vitalik,
link |
like it does feel like an open,
link |
even in these distributed systems,
link |
leaders are helpful
link |
because they kind of help you drive the mission
link |
and the vision and they put a face to a project.
link |
It's a weird thing about us humans.
link |
Geniuses are helpful, like Vitalik.
link |
Leaders are not necessarily, yeah.
link |
So you think the reason he's the face of Ethereum
link |
is because he's a genius.
link |
That's interesting.
link |
it's interesting to think about
link |
that we need to create systems
link |
in which the quote unquote leaders that emerge
link |
are the geniuses in the system.
link |
I mean, that's arguably why
link |
the current state of democracy is broken
link |
is the people who are emerging as the leaders
link |
are not the most competent,
link |
are not the superstars of the system.
link |
And it seems like at least for now
link |
in the crypto world oftentimes
link |
the leaders are the superstars.
link |
Imagine at the debate they asked,
link |
what's the sixth amendment?
link |
What are the four fundamental forces in the universe?
link |
What's the integral of two to the X?
link |
I'd love to see those questions asked
link |
and that's what I want as our leader.
link |
It's a little bit.
link |
What's Bayes rule?
link |
Yeah, I mean, even, oh wow, you're hurting my brain.
link |
It's that my standard was even lower
link |
but I would have loved to see
link |
just this basic brilliance.
link |
Like I've talked to historians.
link |
There's just these, they're not even like
link |
they don't have a PhD or even education history.
link |
They just like a Dan Carlin type character
link |
who just like, holy shit.
link |
How did all this information get into your head?
link |
They're able to just connect Genghis Khan
link |
to the entirety of the history of the 20th century.
link |
They know everything about every single battle that happened
link |
and they know the game of Thrones
link |
of the different power plays and all that happened there.
link |
And they know like the individuals
link |
and all the documents involved
link |
and they integrate that into their regular life.
link |
It's not like they're ultra history nerds.
link |
They're just, they know this information.
link |
That's what competence looks like.
link |
Cause I've seen that with programmers too, right?
link |
That's what great programmers do.
link |
But yeah, it would be, it's really unfortunate
link |
that those kinds of people aren't emerging as our leaders.
link |
But for now, at least in the crypto world
link |
that seems to be the case.
link |
I don't know if that always, you could imagine
link |
that in a hundred years, it's not the case, right?
link |
Crypto world has one very powerful idea going for it
link |
and that's the idea of forks, right?
link |
I mean, imagine, we'll use a less controversial example.
link |
This was actually in my joke app in 2012.
link |
I was like, Barack Obama, Mitt Romney,
link |
let's let them both be president, right?
link |
Like imagine we could fork America
link |
and just let them both be president.
link |
And then the Americas could compete
link |
and people could invest in one,
link |
pull their liquidity out of one, put it in the other.
link |
You have this in the crypto world.
link |
Ethereum forks into Ethereum and Ethereum classic.
link |
And you can pull your liquidity out of one
link |
and put it in another.
link |
And people vote with their dollars,
link |
which forks, companies should be able to fork.
link |
I'd love to fork Nvidia, you know?
link |
Yeah, like different business strategies
link |
and then try them out and see what works.
link |
Like even take, yeah, take comma AI that closes its source
link |
and then take one that's open source and see what works.
link |
Take one that's purchased by GM
link |
and one that remains Android Renegade
link |
and all these different versions and see.
link |
The beauty of comma AI is someone can actually do that.
link |
Please take comma AI and fork it.
link |
That's right, that's the beauty of open source.
link |
So you're, I mean, we'll talk about autonomous vehicle space,
link |
but it does seem that you're really knowledgeable
link |
about a lot of different topics.
link |
So the natural question a bunch of people ask this,
link |
which is how do you keep learning new things?
link |
Do you have like practical advice
link |
if you were to introspect, like taking notes,
link |
allocate time, or do you just mess around
link |
and just allow your curiosity to drive?
link |
I'll write these people a self help book
link |
and I'll charge $67 for it.
link |
And I will write, I will write,
link |
I will write on the cover of the self help book.
link |
All of this advice is completely meaningless.
link |
You're gonna be a sucker and buy this book anyway.
link |
And the one lesson that I hope they take away from the book
link |
is that I can't give you a meaningful answer to that.
link |
That's interesting.
link |
Let me translate that.
link |
Is you haven't really thought about what it is you do
link |
systematically because you could reduce it.
link |
And there's some people, I mean, I've met brilliant people
link |
that this is really clear with athletes.
link |
Some are just, you know, the best in the world
link |
at something and they have zero interest
link |
in writing like a self help book,
link |
or how to master this game.
link |
And then there's some athletes who become great coaches
link |
and they love the analysis, perhaps the over analysis.
link |
And you right now, at least at your age,
link |
which is an interesting, you're in the middle of the battle.
link |
You're like the warriors that have zero interest
link |
So you're in the middle of the battle.
link |
So you have, yeah.
link |
This is a fair point.
link |
I do think I have a certain aversion
link |
to this kind of deliberate intentional way of living life.
link |
You're eventually, the hilarity of this,
link |
especially since this is recorded,
link |
it will reveal beautifully the absurdity
link |
when you finally do publish this book.
link |
I guarantee you, you will.
link |
The story of comma AI, maybe it'll be a biography
link |
written about you.
link |
That'll be better, I guess.
link |
And you might be able to learn some cute lessons
link |
if you're starting a company like comma AI from that book.
link |
But if you're asking generic questions,
link |
like how do I be good at things?
link |
How do I be good at things?
link |
Dude, I don't know.
link |
But the interesting thing here is learning things
link |
outside of your current trajectory,
link |
which is what it feels like from an outsider's perspective.
link |
I don't know if there's advice on that,
link |
but it is an interesting curiosity.
link |
When you become really busy, you're running a company.
link |
But there's a natural inclination and trend.
link |
Just the momentum of life carries you
link |
into a particular direction of wanting to focus.
link |
And this kind of dispersion that curiosity can lead to
link |
gets harder and harder with time.
link |
Because you get really good at certain things
link |
and it sucks trying things that you're not good at,
link |
like trying to figure them out.
link |
When you do this with your live streams,
link |
you're on the fly figuring stuff out.
link |
You don't mind looking dumb.
link |
You just figure it out pretty quickly.
link |
Sometimes I try things and I don't figure them out quickly.
link |
My chest rating is like a 1400,
link |
despite putting like a couple of hundred hours in.
link |
I mean, to be fair, I know that I could do it better
link |
if I did it better.
link |
Like don't play five minute games,
link |
play 15 minute games at least.
link |
Like I know these things, but it just doesn't,
link |
it doesn't stick nicely in my knowledge stream.
link |
All right, let's talk about Comma AI.
link |
What's the mission of the company?
link |
Let's like look at the biggest picture.
link |
Oh, I have an exact statement.
link |
Solve self driving cars
link |
while delivering shippable intermediaries.
link |
So longterm vision is have fully autonomous vehicles
link |
and make sure you're making money along the way.
link |
I think it doesn't really speak to money,
link |
but I can talk about what solve self driving cars means.
link |
Solve self driving cars of course means
link |
you're not building a new car,
link |
you're building a person replacement.
link |
That person can sit in the driver's seat
link |
and drive you anywhere a person can drive
link |
with a human or better level of safety,
link |
speed, quality, comfort.
link |
And what's the second part of that?
link |
Delivering shippable intermediaries is well,
link |
it's a way to fund the company, that's true.
link |
But it's also a way to keep us honest.
link |
If you don't have that, it is very easy
link |
with this technology to think you're making progress
link |
I've heard it best described on Hacker News as
link |
you can set any arbitrary milestone,
link |
meet that milestone and still be infinitely far away
link |
from solving self driving cars.
link |
So it's hard to have like real deadlines
link |
when you're like Cruz or Waymo when you don't have revenue.
link |
Is that, I mean, is revenue essentially
link |
the thing we're talking about here?
link |
Revenue is, capitalism is based around consent.
link |
Capitalism, the way that you get revenue
link |
is real capitalism comes in the real capitalism camp.
link |
There's definitely scams out there,
link |
but real capitalism is based around consent.
link |
It's based around this idea that like,
link |
if we're getting revenue, it's because we're providing
link |
at least that much value to another person.
link |
When someone buys $1,000 comma two from us,
link |
we're providing them at least $1,000 of value
link |
or they wouldn't buy it.
link |
Brilliant, so can you give a whirlwind overview
link |
of the products that Comma AI provides,
link |
like throughout its history and today?
link |
I mean, yeah, the past ones aren't really that interesting.
link |
It's kind of just been refinement of the same idea.
link |
The real only product we sell today is the Comma 2.
link |
Which is a piece of hardware with cameras.
link |
Mm, so the Comma 2, I mean, you can think about it
link |
kind of like a person.
link |
Future hardware will probably be
link |
even more and more personlike.
link |
So it has eyes, ears, a mouth, a brain,
link |
and a way to interface with the car.
link |
Does it have consciousness?
link |
Just kidding, that was a trick question.
link |
I don't have consciousness either.
link |
Me and the Comma 2 are the same.
link |
I have a little more compute than it.
link |
It only has like the same compute as a B, you know.
link |
You're more efficient energy wise
link |
for the compute you're doing.
link |
Far more efficient energy wise.
link |
20 petaflops, 20 watts, crazy.
link |
Do you lack consciousness?
link |
Do you fear death?
link |
You do, you want immortality.
link |
Of course I fear death.
link |
Does Comma AI fear death?
link |
Of course it does.
link |
It very much fears, well, it fears negative loss.
link |
Okay, so Comma 2, when did that come out?
link |
That was a year ago?
link |
Wow, time, it feels like, yeah.
link |
2020 feels like it's taken 10 years to get to the end.
link |
So what's the sexiest thing about Comma 2 feature wise?
link |
So, I mean, maybe you can also link on like, what is it?
link |
Like what's its purpose?
link |
Cause there's a hardware, there's a software component.
link |
You've mentioned the sensors,
link |
but also like what is it, its features and capabilities?
link |
I think our slogan summarizes it well.
link |
Comma slogan is make driving chill.
link |
Yeah, I mean, it is, you know,
link |
if you like cruise control, imagine cruise control,
link |
but much, much more.
link |
So it can do adaptive cruise control things,
link |
which is like slow down for cars in front of it,
link |
maintain a certain speed.
link |
And it can also do lane keeping.
link |
So staying in the lane and doing it better
link |
and better and better over time.
link |
It's very much machine learning based.
link |
So this camera is, there's a driver facing camera too.
link |
What else is there?
link |
What am I thinking?
link |
So the hardware versus software.
link |
So open pilot versus the actual hardware of the device.
link |
What's, can you draw that distinction?
link |
What's one, what's the other?
link |
I mean, the hardware is pretty much a cell phone
link |
with a few additions.
link |
A cell phone with a cooling system
link |
and with a car interface connected to it.
link |
And by cell phone, you mean like Qualcomm Snapdragon.
link |
Yeah, the current hardware is a Snapdragon 821.
link |
It has wifi radio, it has an LTE radio, it has a screen.
link |
We use every part of the cell phone.
link |
And then the interface with the car
link |
is specific to the car.
link |
So you keep supporting more and more cars.
link |
Yeah, so the interface to the car,
link |
I mean, the device itself just has four CAN buses.
link |
It has four CAN interfaces on it
link |
that are connected through the USB port to the phone.
link |
And then, yeah, on those four CAN buses,
link |
you connect it to the car.
link |
And there's a little harness to do this.
link |
Cars are actually surprisingly similar.
link |
So CAN is the protocol by which cars communicate.
link |
And then you're able to read stuff and write stuff
link |
to be able to control the car depending on the car.
link |
So what's the software side?
link |
So I mean, OpenPilot is,
link |
the hardware is pretty simple compared to OpenPilot.
link |
OpenPilot is, well, so you have a machine learning model,
link |
which it's in OpenPilot, it's a blob.
link |
It's just a blob of weights.
link |
It's not like people are like, oh, it's closed source.
link |
I'm like, it's a blob of weights.
link |
What do you expect?
link |
So it's primarily neural network based.
link |
You, well, OpenPilot is all the software
link |
kind of around that neural network.
link |
That if you have a neural network that says,
link |
here's where you wanna send the car,
link |
OpenPilot actually goes and executes all of that.
link |
It cleans up the input to the neural network.
link |
It cleans up the output and executes on it.
link |
So it connects, it's the glue
link |
that connects everything together.
link |
Runs the sensors, does a bunch of calibration
link |
for the neural network, deals with like,
link |
if the car is on a banked road,
link |
you have to counter steer against that.
link |
And the neural network can't necessarily know that
link |
by looking at the picture.
link |
So you do that with other sensors
link |
and Fusion and Localizer.
link |
OpenPilot also is responsible
link |
for sending the data up to our servers.
link |
So we can learn from it, logging it, recording it,
link |
running the cameras, thermally managing the device,
link |
managing the disk space on the device,
link |
managing all the resources on the device.
link |
So what, since we last spoke,
link |
I don't remember when, maybe a year ago,
link |
maybe a little bit longer,
link |
how has OpenPilot improved?
link |
We did exactly what I promised you.
link |
I promised you that by the end of the year,
link |
where you'd be able to remove the lanes.
link |
The lateral policy is now almost completely end to end.
link |
You can turn the lanes off and it will drive,
link |
drive slightly worse on the highway
link |
if you turn the lanes off,
link |
but you can turn the lanes off and it will drive well,
link |
trained completely end to end on user data.
link |
And this year we hope to do the same
link |
for the longitudinal policy.
link |
So that's the interesting thing is you're not doing,
link |
you don't appear to be, maybe you can correct me,
link |
you don't appear to be doing lane detection
link |
or lane marking detection or kind of the segmentation task
link |
or any kind of object detection task.
link |
You're doing what's traditionally more called
link |
like end to end learning.
link |
So, and trained on actual behavior of drivers
link |
when they're driving the car manually.
link |
And this is hard to do.
link |
It's not supervised learning.
link |
Yeah, but so the nice thing is there's a lot of data.
link |
So it's hard and easy, right?
link |
We have a lot of high quality data, yeah.
link |
Like more than you need in the second.
link |
We have way more than we do.
link |
We have way more data than we need.
link |
I mean, it's an interesting question actually,
link |
because in terms of amount, you have more than you need,
link |
but the driving is full of edge cases.
link |
So how do you select the data you train on?
link |
I think this is an interesting open question.
link |
Like what's the cleverest way to select data?
link |
That's the question Tesla is probably working on.
link |
That's, I mean, the entirety of machine learning can be,
link |
they don't seem to really care.
link |
They just kind of select data.
link |
But I feel like that if you want to solve,
link |
if you want to create intelligent systems,
link |
you have to pick data well, right?
link |
And so do you have any hints, ideas of how to do it well?
link |
So in some ways that is...
link |
The definition I like of reinforcement learning
link |
versus supervised learning.
link |
In supervised learning, the weights depend on the data.
link |
And this is obviously true,
link |
but in reinforcement learning,
link |
the data depends on the weights.
link |
And actually both ways.
link |
So how does it know what data to train on?
link |
Well, let it pick.
link |
We're not there yet, but that's the eventual.
link |
So you're thinking this almost like
link |
a reinforcement learning framework.
link |
We're going to do RL on the world.
link |
Every time a car makes a mistake, user disengages,
link |
we train on that and do RL on the world.
link |
Ship out a new model, that's an epoch, right?
link |
And for now you're not doing the Elon style promising
link |
that it's going to be fully autonomous.
link |
You really are sticking to level two
link |
and like it's supposed to be supervised.
link |
It is definitely supposed to be supervised
link |
and we enforce the fact that it's supervised.
link |
We look at our rate of improvement in disengagements.
link |
OpenPilot now has an unplanned disengagement
link |
about every a hundred miles.
link |
This is up from 10 miles, like maybe,
link |
maybe maybe a year ago.
link |
So maybe we've seen 10 X improvement in a year,
link |
but a hundred miles is still a far cry
link |
from the a hundred thousand you're going to need.
link |
So you're going to somehow need to get three more 10 Xs
link |
And you're, what's your intuition?
link |
You're basically hoping that there's exponential
link |
improvement built into the baked into the cake somewhere.
link |
Well, that's even, I mean, 10 X improvement,
link |
that's already assuming exponential, right?
link |
There's definitely exponential improvement.
link |
And I think when Elon talks about exponential,
link |
like these things, these systems are going to
link |
exponentially improve, just exponential doesn't mean
link |
you're getting a hundred gigahertz processors tomorrow.
link |
Right? Like it's going to still take a while
link |
because the gap between even our best system
link |
and humans is still large.
link |
So that's an interesting distinction to draw.
link |
So if you look at the way Tesla is approaching the problem
link |
and the way you're approaching the problem,
link |
which is very different than the rest of the self driving
link |
So let's put them aside is you're treating most
link |
the driving task as a machine learning problem.
link |
And the way Tesla is approaching it is with the multitask
link |
learning where you break the task of driving into hundreds
link |
of different tasks and you have this multiheaded
link |
neural network that's very good at performing each task.
link |
And there there's presumably something on top that's
link |
stitching stuff together in order to make control
link |
decisions, policy decisions about how you move the car.
link |
But what that allows you, there's a brilliance to this
link |
because it allows you to master each task,
link |
like lane detection, stop sign detection,
link |
the traffic light detection, drivable area segmentation,
link |
you know, vehicle, bicycle, pedestrian detection.
link |
There's some localization tasks in there.
link |
Also predicting of like, yeah,
link |
predicting how the entities in the scene are going to move.
link |
Like everything is basically a machine learning task.
link |
So there's a classification, segmentation, prediction.
link |
And it's nice because you can have this entire engine,
link |
data engine that's mining for edge cases for each one of
link |
And you can have people like engineers that are basically
link |
masters of that task,
link |
like become the best person in the world at,
link |
as you talk about the cone guy for Waymo,
link |
the becoming the best person in the world at cone detection.
link |
So that's a compelling notion from a supervised learning
link |
perspective, automating much of the process of edge case
link |
discovery and retraining neural network for each of the
link |
individual perception tasks.
link |
And then you're looking at the machine learning in a more
link |
holistic way, basically doing end to end learning on the
link |
driving tasks, supervised, trained on the data of the
link |
actual driving of people.
link |
They use comma AI, like actual human drivers,
link |
their manual control,
link |
plus the moments of disengagement that maybe with some
link |
labeling could indicate the failure of the system.
link |
you have a huge amount of data for positive control of the
link |
vehicle, like successful control of the vehicle,
link |
both maintaining the lane as,
link |
as I think you're also working on longitudinal control of
link |
the vehicle and then failure cases where the vehicle does
link |
something wrong that needs disengagement.
link |
why do you think you're right and Tesla is wrong on this?
link |
do you think you'll come around the Tesla way?
link |
Do you think Tesla will come around to your way?
link |
If you were to start a chess engine company,
link |
would you hire a Bishop guy?
link |
this is Monday morning.
link |
Quarterbacking is a yes, probably.
link |
Oh, we stole the Rook guy from that company.
link |
Oh, we're going to have real good Rooks.
link |
Well, there's not many pieces, right?
link |
there's not many guys and gals to hire.
link |
You just have a few that work in the Bishop,
link |
a few that work in the Rook.
link |
Is that not ludicrous today to think about
link |
in a world of AlphaZero?
link |
But AlphaZero is a chess game.
link |
So the fundamental question is,
link |
how hard is driving compared to chess?
link |
Because, so long term,
link |
will be the right solution.
link |
The question is how many years away is that?
link |
End to end is going to be the only solution for level five.
link |
For the only way we'll get there.
link |
Of course, and of course,
link |
Tesla is going to come around to my way.
link |
And if you're a Rook guy out there, I'm sorry.
link |
We're going to specialize each task.
link |
We're going to really understand Rook placement.
link |
I understand the intuition you have.
link |
that is a very compelling notion
link |
that we can learn the task end to end,
link |
like the same compelling notion you might have
link |
for natural language conversation.
link |
because one thing you sneaked in there
link |
is the assertion that it's impossible to get to level five
link |
without this kind of approach.
link |
I don't know if that's obvious.
link |
I don't know if that's obvious either.
link |
I don't actually mean that.
link |
I think that it is much easier
link |
to get to level five with an end to end approach.
link |
I think that the other approach is doable,
link |
but the magnitude of the engineering challenge
link |
may exceed what humanity is capable of.
link |
But what do you think of the Tesla data engine approach,
link |
which to me is an active learning task,
link |
is kind of fascinating,
link |
is breaking it down into these multiple tasks
link |
and mining their data constantly for like edge cases
link |
for these different tasks.
link |
Yeah, but the tasks themselves are not being learned.
link |
This is feature engineering.
link |
Yeah, I mean, it's a higher abstraction level
link |
of feature engineering for the different tasks.
link |
Task engineering in a sense.
link |
It's slightly better feature engineering,
link |
but it's still fundamentally is feature engineering.
link |
And if anything about the history of AI
link |
has taught us anything,
link |
it's that feature engineering approaches
link |
will always be replaced and lose to end to end.
link |
Now, to be fair, I cannot really make promises on timelines,
link |
but I can say that when you look at the code for Stockfish
link |
and the code for AlphaZero,
link |
one is a lot shorter than the other,
link |
a lot more elegant,
link |
required a lot less programmer hours to write.
link |
Yeah, but there was a lot more murder of bad agents
link |
on the AlphaZero side.
link |
By murder, I mean agents that played a game
link |
and failed miserably.
link |
In simulation, that failure is less costly.
link |
In real world, it's...
link |
Do you mean in practice,
link |
like AlphaZero has lost games miserably?
link |
I haven't seen that.
link |
No, but I know, but the requirement for AlphaZero is...
link |
To be able to like evolution, human evolution,
link |
not human evolution, biological evolution of life on earth
link |
from the origin of life has murdered trillions
link |
upon trillions of organisms on the path thus humans.
link |
So the question is, can we stitch together
link |
a human like object without having to go
link |
through the entirety process of evolution?
link |
Well, no, but do the evolution in simulation.
link |
Yeah, that's the question.
link |
So do you have a sense that it's possible
link |
to simulate some aspect?
link |
MuZero is exactly this.
link |
MuZero is the solution to this.
link |
MuZero I think is going to be looked back
link |
as the canonical paper.
link |
And I don't think deep learning is everything.
link |
I think that there's still a bunch of things missing
link |
to get there, but MuZero I think is going to be looked back
link |
as the kind of cornerstone paper
link |
of this whole deep learning era.
link |
And MuZero is the solution to self driving cars.
link |
You have to make a few tweaks to it,
link |
but MuZero does effectively that.
link |
It does those rollouts and those murdering
link |
in a learned simulator and a learned dynamics model.
link |
That's interesting.
link |
It doesn't get enough love.
link |
I was blown away when I read that paper.
link |
I'm like, okay, I've always said a comma.
link |
I'm going to sit and I'm going to wait for the solution
link |
to self driving cars to come along.
link |
This year I saw it.
link |
Sit back and let the winning roll in.
link |
So your sense, just to elaborate a little bit,
link |
it's a link on the topic.
link |
Your sense is neural networks will solve driving.
link |
Like we don't need anything else.
link |
I think the same way chess was maybe the chess
link |
and maybe Google are the pinnacle of like search algorithms
link |
and things that look kind of like a star.
link |
The pinnacle of this era is going to be self driving cars.
link |
But on the path of that, you have to deliver products
link |
and it's possible that the path to full self driving cars
link |
will take decades.
link |
How long would you put on it?
link |
Like what are we, you're chasing it, Tesla's chasing it.
link |
What are we talking about?
link |
Five years, 10 years, 50 years.
link |
Let's say in the 2020s.
link |
The later part of the 2020s.
link |
With the neural network.
link |
Well, that would be nice to see.
link |
And then the path to that, you're delivering products,
link |
which is a nice L2 system.
link |
That's what Tesla's doing, a nice L2 system.
link |
Just gets better every time.
link |
L2, the only difference between L2 and the other levels
link |
is who takes liability.
link |
And I'm not a liability guy, I don't wanna take liability.
link |
I'm gonna level two forever.
link |
Now on that little transition,
link |
I mean, how do you make the transition work?
link |
Is this where driver sensing comes in?
link |
Like how do you make the, cause you said a hundred miles,
link |
like, is there some sort of human factor psychology thing
link |
where people start to overtrust the system,
link |
all those kinds of effects,
link |
once it gets better and better and better and better,
link |
they get lazier and lazier and lazier.
link |
Is that, like, how do you get that transition right?
link |
First off, our monitoring is already adaptive.
link |
Our monitoring is already seen adaptive.
link |
Driver monitoring is just the camera
link |
that's looking at the driver.
link |
You have an infrared camera in the...
link |
Our policy for how we enforce the driver monitoring
link |
Well, for example, in one of the extreme cases,
link |
if the car is not moving,
link |
we do not actively enforce driver monitoring, right?
link |
If you are going through a,
link |
like a 45 mile an hour road with lights
link |
and stop signs and potentially pedestrians,
link |
we enforce a very tight driver monitoring policy.
link |
If you are alone on a perfectly straight highway,
link |
and this is, it's all machine learning.
link |
None of that is hand coded.
link |
Actually, the stop is hand coded, but...
link |
So there's some kind of machine learning
link |
estimation of risk.
link |
I mean, I've always been a huge fan of that.
link |
It's difficult to do every step into that direction
link |
is a worthwhile step to take.
link |
It might be difficult to do really well.
link |
Like us humans are able to estimate risk pretty damn well,
link |
whatever the hell that is.
link |
That feels like one of the nice features of us humans.
link |
Cause like we humans are really good drivers
link |
when we're really like tuned in
link |
and we're good at estimating risk.
link |
Like when are we supposed to be tuned in?
link |
And, you know, people are like,
link |
oh, well, you know,
link |
why would you ever make the driver monitoring policy
link |
Why would you always not keep it at its most aggressive?
link |
Because then people are just going to get fatigued from it.
link |
When they get annoyed.
link |
You want the experience to be pleasant.
link |
Obviously I want the experience to be pleasant,
link |
but even just from a straight up safety perspective,
link |
if you alert people when they look around and they're like,
link |
why is this thing alerting me?
link |
There's nothing I could possibly hit right now.
link |
People will just learn to tune it out.
link |
People will just learn to tune it out,
link |
to put weights on the steering wheel,
link |
to do whatever to overcome it.
link |
And remember that you're always part
link |
of this adaptive system.
link |
So all I can really say about, you know,
link |
how this scales going forward is yeah,
link |
it's something we have to monitor for.
link |
Ooh, we don't know.
link |
This is a great psychology experiment at scale.
link |
Yeah, it's fascinating.
link |
And making sure you have a good understanding of attention
link |
is a very key part of that psychology problem.
link |
I think you and I probably have a different,
link |
come to it differently, but to me,
link |
it's a fascinating psychology problem
link |
to explore something much deeper than just driving.
link |
It's such a nice way to explore human attention
link |
and human behavior, which is why, again,
link |
we've probably both criticized Mr. Elon Musk
link |
on this one topic from different avenues.
link |
So both offline and online,
link |
I had little chats with Elon and like,
link |
I love human beings as a computer vision problem,
link |
as an AI problem, it's fascinating.
link |
He wasn't so much interested in that problem.
link |
It's like in order to solve driving,
link |
the whole point is you want to remove the human
link |
And it seems like you can't do that quite yet.
link |
Eventually, yes, but you can't quite do that yet.
link |
So this is the moment where you can't yet say,
link |
I told you so to Tesla, but it's getting there
link |
because I don't know if you've seen this,
link |
there's some reporting that they're in fact
link |
starting to do driver monitoring.
link |
Yeah, they shift the model in shadow mode.
link |
With, I believe, only a visible light camera,
link |
it might even be fisheye.
link |
It's like a low resolution.
link |
Low resolution, visible light.
link |
I mean, to be fair, that's what we have in the Eon as well,
link |
our last generation product.
link |
This is the one area where I can say
link |
our hardware is ahead of Tesla.
link |
The rest of our hardware, way, way behind,
link |
but our driver monitoring camera.
link |
So you think, I think on the third row Tesla podcast,
link |
or somewhere else, I've heard you say that obviously,
link |
eventually they're gonna have driver monitoring.
link |
I think what I've said is Elon will definitely ship
link |
driver monitoring before he ships level five.
link |
Before level five.
link |
And I'm willing to bet 10 grand on that.
link |
And you bet 10 grand on that.
link |
I mean, now I don't wanna take the bet,
link |
but before, maybe someone would have,
link |
oh, I should have got my money in.
link |
It's an interesting bet.
link |
I think you're right.
link |
I'm actually on a human level
link |
because he's been, he's made the decision.
link |
Like he said that driver monitoring is the wrong way to go.
link |
But like, you have to think of as a human, as a CEO,
link |
I think that's the right thing to say when,
link |
like sometimes you have to say things publicly
link |
that are different than when you actually believe,
link |
because when you're producing a large number of vehicles
link |
and the decision was made not to include the camera,
link |
like what are you supposed to say?
link |
Like our cars don't have the thing
link |
that I think is right to have.
link |
It's an interesting thing.
link |
But like on the other side, as a CEO,
link |
I mean, something you could probably speak to as a leader,
link |
I think about me as a human
link |
to publicly change your mind on something.
link |
Especially when assholes like George Haas say,
link |
All I will say is I am not a leader
link |
and I am happy to change my mind.
link |
You think Elon will?
link |
I think he'll come up with a good way
link |
to make it psychologically okay for him.
link |
Well, it's such an important thing, man.
link |
Especially for a first principles thinker,
link |
because he made a decision that driver monitoring
link |
is not the right way to go.
link |
And I could see that decision.
link |
And I could even make that decision.
link |
Like I was on the fence too.
link |
driver monitoring is such an obvious,
link |
simple solution to the problem of attention.
link |
It's not obvious to me that just by putting a camera there,
link |
You have to create an incredible, compelling experience.
link |
Just like you're talking about.
link |
I don't know if it's easy to do that.
link |
It's not at all easy to do that, in fact, I think.
link |
So as a creator of a car that's trying to create a product
link |
that people love, which is what Tesla tries to do, right?
link |
It's not obvious to me that as a design decision,
link |
whether adding a camera is a good idea.
link |
From a safety perspective either,
link |
like in the human factors community,
link |
everybody says that you should obviously
link |
have driver sensing, driver monitoring.
link |
But that's like saying it's obvious as parents,
link |
you shouldn't let your kids go out at night.
link |
But okay, but like,
link |
they're still gonna find ways to do drugs.
link |
Like, you have to also be good parents.
link |
So like, it's much more complicated than just the,
link |
you need to have driver monitoring.
link |
I totally disagree on, okay, if you have a camera there
link |
and the camera's watching the person,
link |
but never throws an alert, they'll never think about it.
link |
The driver monitoring policy that you choose to,
link |
how you choose to communicate with the user
link |
is entirely separate from the data collection perspective.
link |
So, you know, like, there's one thing to say,
link |
like, you know, tell your teenager they can't do something.
link |
There's another thing to like, you know, gather the data.
link |
So you can make informed decisions.
link |
That's really interesting.
link |
But you have to make that,
link |
that's the interesting thing about cars.
link |
But even true with common AI,
link |
like you don't have to manufacture the thing
link |
into the car, is you have to make a decision
link |
that anticipates the right strategy longterm.
link |
So like, you have to start collecting the data
link |
and start making decisions.
link |
Started it three years ago.
link |
I believe that we have the best driver monitoring solution
link |
I think that when you compare it to Super Cruise
link |
is the only other one that I really know that shipped.
link |
And ours is better.
link |
What do you like and not like about Super Cruise?
link |
I mean, I had a few Super Cruise,
link |
the sun would be shining through the window,
link |
would blind the camera,
link |
and it would say I wasn't paying attention.
link |
When I was looking completely straight,
link |
I couldn't reset the attention with a steering wheel touch
link |
and Super Cruise would disengage.
link |
Like I was communicating to the car, I'm like, look,
link |
I am here, I am paying attention.
link |
Why are you really gonna force me to disengage?
link |
So it's a constant conversation with the user.
link |
And yeah, there's no way to ship a system
link |
like this if you can OTA.
link |
We're shipping a new one every month.
link |
Sometimes we balance it with our users on Discord.
link |
Like sometimes we make the driver monitoring
link |
a little more aggressive and people complain.
link |
Sometimes they don't.
link |
We want it to be as aggressive as possible
link |
where people don't complain and it doesn't feel intrusive.
link |
So being able to update the system over the air
link |
is an essential component.
link |
I mean, that's probably to me, you mentioned,
link |
I mean, to me that is the biggest innovation of Tesla,
link |
that it made people realize that over the air updates
link |
I mean, was that not obvious from the iPhone?
link |
The iPhone was the first real product that OTA'd, I think.
link |
Was it actually, that's brilliant, you're right.
link |
I mean, the game consoles used to not, right?
link |
The game consoles were maybe the second thing that did.
link |
Wow, I didn't really think about one of the amazing features
link |
of a smartphone isn't just like the touchscreen
link |
isn't the thing, it's the ability to constantly update.
link |
Yeah, it gets better.
link |
Well, one thing that I probably disagree with you
link |
on driver monitoring is you said that it's easy.
link |
I mean, you tend to say stuff is easy.
link |
I'm sure the, I guess you said it's easy
link |
relative to the external perception problem.
link |
Can you elaborate why you think it's easy?
link |
Feature engineering works for driver monitoring.
link |
Feature engineering does not work for the external.
link |
So human faces are not, human faces and the movement
link |
of human faces and head and body is not as variable
link |
as the external environment, is your intuition?
link |
Yes, and there's another big difference as well.
link |
Your reliability of a driver monitoring system
link |
doesn't actually need to be that high.
link |
The uncertainty, if you have something that's detecting
link |
whether the human's paying attention and it only works
link |
92% of the time, you're still getting almost all
link |
the benefit of that because the human,
link |
like you're training the human, right?
link |
You're dealing with a system that's really helping you out.
link |
It's a conversation.
link |
It's not like the external thing where guess what?
link |
If you swerve into a tree, you swerve into a tree, right?
link |
Like you get no margin for error there.
link |
Yeah, I think that's really well put.
link |
I think that's the right, exactly the place
link |
where comparing to the external perception,
link |
the control problem, the driver monitoring is easier
link |
because you don't, the bar for success is much lower.
link |
Yeah, but I still think like the human face
link |
is more complicated actually than the external environment,
link |
but for driving, you don't give a damn.
link |
I don't need, yeah, I don't need something,
link |
I don't need something that complicated
link |
to have to communicate the idea to the human
link |
that I want to communicate, which is,
link |
yo, system might mess up here.
link |
You gotta pay attention.
link |
Yeah, see, that's my love and fascination is the human face.
link |
And it feels like this is a nice place to create products
link |
that create an experience in the car.
link |
So like, it feels like there should be
link |
more richer experiences in the car, you know?
link |
Like that's an opportunity for like something like On My Eye
link |
or just any kind of system like a Tesla
link |
or any of the autonomous vehicle companies
link |
is because software is, there's much more sensors
link |
and so much is on our software
link |
and you're doing machine learning anyway,
link |
there's an opportunity to create totally new experiences
link |
that we're not even anticipating.
link |
You don't think so?
link |
You think it's a box that gets you from A to B
link |
and you want to do it chill?
link |
Yeah, I mean, I think as soon as we get to level three
link |
on highways, okay, enjoy your candy crush,
link |
enjoy your Hulu, enjoy your, you know, whatever, whatever.
link |
Sure, you get this, you can look at screens basically
link |
versus right now where you have music and audio books.
link |
So level three is where you can kind of disengage
link |
in stretches of time.
link |
Well, you think level three is possible?
link |
Like on the highway going for 100 miles
link |
and you can just go to sleep?
link |
So again, I think it's really all on a spectrum.
link |
I think that being able to use your phone
link |
while you're on the highway and like this all being okay
link |
and being aware that the car might alert you
link |
and you have five seconds to basically.
link |
So the five second thing is you think is possible?
link |
Yeah, I think it is, oh yeah.
link |
Not in all scenarios, right?
link |
Some scenarios it's not.
link |
It's the whole risk thing that you mentioned is nice
link |
is to be able to estimate like how risky is this situation?
link |
That's really important to understand.
link |
One other thing you mentioned comparing KAMA
link |
and Autopilot is that something about the haptic feel
link |
of the way KAMA controls the car when things are uncertain.
link |
Like it behaves a little bit more uncertain
link |
when things are uncertain.
link |
That's kind of an interesting point.
link |
And then Autopilot is much more confident always
link |
even when it's uncertain until it runs into trouble.
link |
That's a funny thing.
link |
I actually mentioned that to Elon, I think.
link |
And then the first time we talked, he wasn't biting.
link |
It's like communicating uncertainty.
link |
I guess KAMA doesn't really communicate uncertainty
link |
explicitly, it communicates it through haptic feel.
link |
Like what's the role of communicating uncertainty
link |
Oh, we do some stuff explicitly.
link |
Like we do detect the lanes when you're on the highway
link |
and we'll show you how many lanes we're using to drive with.
link |
You can look at where it thinks the lanes are.
link |
You can look at the path.
link |
And we want to be better about this.
link |
We're actually hiring, want to hire some new UI people.
link |
UI people, you mentioned this.
link |
Cause it's such an, it's a UI problem too, right?
link |
We have a great designer now, but you know,
link |
we need people who are just going to like build this
link |
and debug these UIs, QT people.
link |
Is that what the UI is done with, is QT?
link |
The new UI is in QT.
link |
Tesla uses it too.
link |
We had some React stuff in there.
link |
React JS or just React?
link |
React is his own language, right?
link |
React Native, React is a JavaScript framework.
link |
So it's all based on JavaScript, but it's, you know,
link |
What do you think about Dojo with Tesla
link |
and their foray into what appears to be
link |
specialized hardware for training your own nets?
link |
I guess it's something, maybe you can correct me,
link |
from my shallow looking at it,
link |
it seems like something like Google did with TPUs,
link |
but specialized for driving data.
link |
I don't think it's specialized for driving data.
link |
It's just legit, just TPU.
link |
They want to go the Apple way,
link |
basically everything required in the chain is done in house.
link |
Well, so you have a problem right now,
link |
and this is one of my concerns.
link |
I really would like to see somebody deal with this.
link |
If anyone out there is doing it,
link |
I'd like to help them if I can.
link |
You basically have two options right now to train.
link |
One, your options are NVIDIA or Google.
link |
So Google is not even an option.
link |
Their TPUs are only available in Google Cloud.
link |
Google has absolutely onerous
link |
terms of service restrictions.
link |
They may have changed it,
link |
but back in Google's terms of service,
link |
it said explicitly you are not allowed to use Google Cloud ML
link |
for training autonomous vehicles
link |
or for doing anything that competes with Google
link |
without Google's prior written permission.
link |
I mean, Google is not a platform company.
link |
I wouldn't touch TPUs with a 10 foot pole.
link |
So that leaves you with the monopoly.
link |
That you're not a fan of.
link |
Well, look, I was a huge fan of in 2016 NVIDIA.
link |
Jensen came sat in the car.
link |
When the stock was $30 a share.
link |
NVIDIA stock has skyrocketed.
link |
I witnessed a real change
link |
in who was in management over there in like 2018.
link |
And now they are, let's exploit.
link |
Let's take every dollar we possibly can
link |
out of this ecosystem.
link |
Let's charge $10,000 for A100s
link |
because we know we got the best shit in the game.
link |
And let's charge $10,000 for an A100
link |
when it's really not that different from a 3080,
link |
The margins that they are making
link |
off of those high end chips are so high
link |
that, I mean, I think they're shooting themselves
link |
in the foot just from a business perspective.
link |
Because there's a lot of people talking like me now
link |
who are like, somebody's gotta take NVIDIA down.
link |
Where they could dominate it.
link |
NVIDIA could be the new Intel.
link |
Yeah, to be inside everything essentially.
link |
And yet the winners in certain spaces
link |
like autonomous driving, the winners,
link |
only the people who are like desperately falling back
link |
and trying to catch up and have a ton of money,
link |
like the big automakers are the ones
link |
interested in partnering with NVIDIA.
link |
Oh, and I think a lot of those things
link |
are gonna fall through.
link |
If I were NVIDIA, sell chips.
link |
Sell chips at a reasonable markup.
link |
Without any restrictions.
link |
Without any restrictions.
link |
They had a great long run.
link |
NVIDIA is trying to turn their,
link |
they're like trying to productize their chips
link |
They're trying to extract way more value
link |
than they can sustainably.
link |
Sure, you can do it tomorrow.
link |
Is it gonna up your share price?
link |
Sure, if you're one of those CEOs
link |
who's like, how much can I strip mine this company?
link |
And I think, you know, and that's what's weird about it too.
link |
Like the CEO is the founder.
link |
It's the same guy.
link |
I mean, I still think Jensen's a great guy.
link |
You have a choice.
link |
You have a choice right now.
link |
Are you trying to cash out?
link |
Are you trying to buy a yacht?
link |
But if you're trying to be
link |
the next huge semiconductor company, sell chips.
link |
Well, the interesting thing about Jensen
link |
is he is a big vision guy.
link |
So he has a plan like for 50 years down the road.
link |
So it makes me wonder like.
link |
How does price gouging fit into it?
link |
Yeah, how does that, like it's,
link |
it doesn't seem to make sense as a plan.
link |
I worry that he's listening to the wrong people.
link |
Yeah, that's the sense I have too sometimes.
link |
Because I, despite everything, I think NVIDIA
link |
is an incredible company.
link |
Well, one, so I'm deeply grateful to NVIDIA
link |
for the products they've created in the past.
link |
The 1080 Ti was a great GPU.
link |
Still have a lot of them.
link |
But at the same time, it just feels like,
link |
feels like you don't want to put all your stock in NVIDIA.
link |
And so like Elon is doing, what Tesla is doing
link |
with Autopilot and Dojo is the Apple way is,
link |
because they're not going to share Dojo with George Hott's.
link |
They should sell that chip.
link |
Oh, they should sell that.
link |
Even their accelerator.
link |
The accelerator that's in all the cars, the 30 watt one.
link |
Like make, why does Tesla have to be a car company?
link |
Well, if you sell the chip, here's what you get.
link |
Make some money off the chips.
link |
It doesn't take away from your chip.
link |
You're going to make some money, free money.
link |
And also the world is going to build an ecosystem
link |
of tooling for you.
link |
You're not going to have to fix the bug in your 10H layer.
link |
Someone else already did.
link |
Well, the question, that's an interesting question.
link |
I mean, that's the question Steve Jobs asked.
link |
That's the question Elon Musk is perhaps asking is,
link |
do you want Tesla stuff inside other vehicles?
link |
Inside, potentially inside like a iRobot vacuum cleaner.
link |
I think you should decide where your advantages are.
link |
I'm not saying Tesla should start selling battery packs
link |
Because battery packs to automakers,
link |
they are straight up in competition with you.
link |
If I were Tesla, I'd keep the battery technology totally.
link |
As far as we make batteries.
link |
But the thing about the Tesla TPU is anybody can build that.
link |
It's just a question of, you know,
link |
are you willing to spend the money?
link |
It could be a huge source of revenue potentially.
link |
Are you willing to spend a hundred million dollars?
link |
Anyone can build it.
link |
And a bunch of companies now are starting
link |
trying to build AI accelerators.
link |
Somebody is going to get the idea right.
link |
And yeah, hopefully they don't get greedy
link |
because they'll just lose to the next guy who finally,
link |
and then eventually the Chinese are going to make knockoff
link |
and video chips and that's.
link |
From your perspective,
link |
I don't know if you're also paying attention
link |
to stay on Tesla for a moment.
link |
Dave, Elon Musk has talked about a complete rewrite
link |
of the neural net that they're using.
link |
That seems to, again, I'm half paying attention,
link |
but it seems to involve basically a kind of integration
link |
of all the sensors to where it's a four dimensional view.
link |
You know, you have a 3D model of the world over time.
link |
And then you can, I think it's done both for the,
link |
for the actually, you know,
link |
so the neural network is able to,
link |
in a more holistic way,
link |
deal with the world and make predictions and so on,
link |
but also to make the annotation task more, you know, easier.
link |
Like you can annotate the world in one place
link |
and then kind of distribute itself across the sensors
link |
and across a different,
link |
like the hundreds of tasks that are involved
link |
What are your thoughts about this rewrite?
link |
Is it just like some details that are kind of obvious
link |
that are steps that should be taken,
link |
or is there something fundamental
link |
that could challenge your idea
link |
that end to end is the right solution?
link |
We're in the middle of a big rewrite now as well.
link |
We haven't shipped a new model in a bit.
link |
We're going from 2D to 3D.
link |
Right now, all our stuff, like for example,
link |
when the car pitches back,
link |
the lane lines also pitch back
link |
because we're assuming the flat world hypothesis.
link |
The new models do not do this.
link |
The new models output everything in 3D.
link |
But there's still no annotation.
link |
So the 3D is, it's more about the output.
link |
We have Zs in everything.
link |
We unified a lot of stuff as well.
link |
We switched from TensorFlow to PyTorch.
link |
My understanding of what Tesla's thing is,
link |
is that their annotator now annotates
link |
across the time dimension.
link |
Why are you building an annotator?
link |
I find their entire pipeline.
link |
I find your vision, I mean,
link |
the vision of end to end very compelling,
link |
but I also like the engineering of the data engine
link |
that they've created.
link |
In terms of supervised learning pipelines,
link |
that thing is damn impressive.
link |
You're basically, the idea is that you have
link |
hundreds of thousands of people
link |
that are doing data collection for you
link |
by doing their experience.
link |
So that's kind of similar to the Comma AI model.
link |
And you're able to mine that data
link |
based on the kind of edge cases you need.
link |
I think it's harder to do in the end to end learning.
link |
The mining of the right edge cases.
link |
Like that's where feature engineering
link |
is actually really powerful
link |
because like us humans are able to do
link |
this kind of mining a little better.
link |
But yeah, there's obvious, as we know,
link |
there's obvious constraints and limitations to that idea.
link |
Carpathia just tweeted, he's like,
link |
you get really interesting insights
link |
if you sort your validation set by loss
link |
and look at the highest loss examples.
link |
So yeah, I mean, you can do,
link |
we have a little data engine like thing.
link |
We're training a segment.
link |
I know it's not fancy.
link |
It's just like, okay, train the new segment,
link |
run it on 100,000 images
link |
and now take the thousand with highest loss.
link |
Select a hundred of those by human,
link |
put those, get those ones labeled, retrain, do it again.
link |
And so it's a much less well written data engine.
link |
And yeah, you can take these things really far
link |
and it is impressive engineering.
link |
And if you truly need supervised data for a problem,
link |
yeah, things like data engine are at the high end
link |
of what is attention?
link |
Is a human paying attention?
link |
I mean, we're going to probably build something
link |
that looks like data engine
link |
to push our driver monitoring further.
link |
But for driving itself,
link |
you have it all annotated beautifully by what the human does.
link |
Yeah, that's interesting.
link |
I mean, that applies to driver attention as well.
link |
Do you want to detect the eyes?
link |
Do you want to detect blinking and pupil movement?
link |
Do you want to detect all the like face alignments
link |
or landmark detection and so on,
link |
and then doing kind of reasoning based on that?
link |
Or do you want to take the entirety of the face over time
link |
and do end to end?
link |
I mean, it's obvious that eventually you have to do end
link |
to end with some calibration, some fixes and so on,
link |
but it's like, I don't know when that's the right move.
link |
Even if it's end to end, there actually is,
link |
there is no kind of, you have to supervise that with humans.
link |
Whether a human is paying attention or not
link |
is a completely subjective judgment.
link |
Like you can try to like automatically do it
link |
with some stuff, but you don't have,
link |
if I record a video of a human,
link |
I don't have true annotations anywhere in that video.
link |
The only way to get them is with,
link |
you know, other humans labeling it really.
link |
Well, I don't know.
link |
If you think deeply about it,
link |
you could, you might be able to just,
link |
depending on the task,
link |
maybe a discover self annotating things like,
link |
you know, you can look at like steering wheel reverse
link |
or something like that.
link |
You can discover little moments of lapse of attention.
link |
I mean, that's where psychology comes in.
link |
Is there indicate,
link |
cause you have so much data to look at.
link |
So you might be able to find moments when there's like,
link |
just inattention that even with smartphone,
link |
if you want to detect smartphone use,
link |
you can start to zoom in.
link |
I mean, that's the gold mine, sort of the comma AI.
link |
I mean, Tesla is doing this too, right?
link |
Is they're doing annotation based on,
link |
it's like a self supervised learning too.
link |
It's just a small part of the entire picture.
link |
That's kind of the challenge of solving a problem
link |
in machine learning.
link |
If you can discover self annotating parts of the problem,
link |
Our driver monitoring team is half a person right now.
link |
I would, you know, once we have,
link |
once we have two, three people on that team,
link |
I definitely want to look at self annotating stuff
link |
Let's go back for a sec to a comma and what,
link |
you know, for people who are curious to try it out,
link |
how do you install a comma in say a 2020 Toyota Corolla
link |
or like, what are the cars that are supported?
link |
What are the cars that you recommend?
link |
And what does it take?
link |
You have a few videos out, but maybe through words,
link |
can you explain what's it take to actually install a thing?
link |
So we support, I think it's 91 cars, 91 makes the models.
link |
We've got to 100 this year.
link |
The, yeah, the 2020 Corolla, great choice.
link |
The 2020 Sonata, it's using the stock longitudinal.
link |
It's using just our lateral control,
link |
but it's a very refined car.
link |
Their longitudinal control is not bad at all.
link |
So yeah, Corolla, Sonata,
link |
or if you're willing to get your hands a little dirty
link |
and look in the right places on the internet,
link |
the Honda Civic is great,
link |
but you're going to have to install a modified EPS firmware
link |
in order to get a little bit more torque.
link |
And I can't help you with that.
link |
Comma does not officially endorse that,
link |
but we have been doing it.
link |
We didn't ever release it.
link |
We waited for someone else to discover it.
link |
And then, you know.
link |
And you have a Discord server where people,
link |
there's a very active developer community, I suppose.
link |
So depending on the level of experimentation
link |
you're willing to do, that's the community.
link |
If you just want to buy it and you have a supported car,
link |
it's 10 minutes to install.
link |
There's YouTube videos.
link |
It's Ikea furniture level.
link |
If you can set up a table from Ikea,
link |
you can install a Comma 2 in your supported car
link |
and it will just work.
link |
Now you're like, oh, but I want this high end feature
link |
or I want to fix this bug.
link |
Okay, well, welcome to the developer community.
link |
So what, if I wanted to,
link |
this is something I asked you offline like a few months ago.
link |
If I wanted to run my own code to,
link |
so use Comma as a platform
link |
and try to run something like OpenPilot,
link |
what does it take to do that?
link |
So there's a toggle in the settings called enable SSH.
link |
And if you toggle that, you can SSH into your device.
link |
You can modify the code.
link |
You can upload whatever code you want to it.
link |
There's a whole lot of people.
link |
So about 60% of people are running stock comma.
link |
About 40% of people are running forks.
link |
And there's a community of,
link |
there's a bunch of people who maintain these forks
link |
and these forks support different cars
link |
or they have different toggles.
link |
We try to keep away from the toggles
link |
that are like disabled driver monitoring,
link |
but there's some people might want that kind of thing
link |
and like, yeah, you can, it's your car.
link |
I'm not here to tell you.
link |
We have some, we ban,
link |
if you're trying to subvert safety features,
link |
you're banned from our Discord.
link |
I don't want anything to do with you,
link |
but there's some forks doing that.
link |
So you encourage responsible forking.
link |
We encourage, some people, yeah, some people,
link |
like there's forks that will do,
link |
some people just like having a lot of readouts on the UI,
link |
like a lot of like flashing numbers.
link |
So there's forks that do that.
link |
Some people don't like the fact that it disengages
link |
when you press the gas pedal.
link |
There's forks that disable that.
link |
Now the stock experience is what like,
link |
so it does both lane keeping
link |
and longitudinal control all together.
link |
So it's not separate like it is in autopilot.
link |
Some cars we use the stock longitudinal control.
link |
We don't do the longitudinal control in all the cars.
link |
Some cars, the ACCs are pretty good in the cars.
link |
It's the lane keep that's atrocious in anything
link |
except for autopilot and super cruise.
link |
But, you know, you just turn it on and it works.
link |
What does this engagement look like?
link |
Yeah, so we have, I mean,
link |
I'm very concerned about mode confusion.
link |
I've experienced it on super cruise and autopilot
link |
where like autopilot, like autopilot disengages.
link |
I don't realize that the ACC is still on.
link |
The lead car moves slightly over
link |
and then the Tesla accelerates
link |
to like whatever my set speed is super fast.
link |
I'm like, what's going on here?
link |
We have engaged and disengaged.
link |
And this is similar to my understanding, I'm not a pilot,
link |
but my understanding is either the pilot is in control
link |
or the copilot is in control.
link |
And we have the same kind of transition system.
link |
Either open pilot is engaged or open pilot is disengaged.
link |
Engage with cruise control,
link |
disengage with either gas brake or cancel.
link |
Let's talk about money.
link |
What's the business strategy for Kama?
link |
So congratulations.
link |
What, so basically selling,
link |
so we should say Kama cost a thousand bucks, Kama two?
link |
200 for the interface to the car as well.
link |
It's 1200, I'll send that.
link |
Nobody's usually upfront like this.
link |
Yeah, you gotta add the tack on, right?
link |
I'm not gonna lie to you.
link |
Trust me, it will add $1,200 of value to your life.
link |
Yes, it's still super cheap.
link |
30 days, no questions asked, money back guarantee,
link |
and prices are only going up.
link |
If there ever is future hardware,
link |
it could cost a lot more than $1,200.
link |
So Kama three is in the works.
link |
All I will say is future hardware
link |
is going to cost a lot more than the current hardware.
link |
Yeah, the people that use,
link |
the people I've spoken with that use Kama,
link |
that use open pilot,
link |
first of all, they use it a lot.
link |
So people that use it, they fall in love with it.
link |
Oh, our retention rate is insane.
link |
It's a really good sign.
link |
70% of Kama two buyers are daily active users.
link |
Yeah, it's amazing.
link |
Oh, also, we don't plan on stopping selling the Kama two.
link |
Like it's, you know.
link |
So whatever you create that's beyond Kama two,
link |
it would be potentially a phase shift.
link |
Like it's so much better that,
link |
like you could use Kama two
link |
and you can use Kama whatever.
link |
Depends what you want.
link |
You know, autopilot hardware one versus hardware two.
link |
The Kama two is kind of like hardware one.
link |
You can still use both.
link |
I think I heard you talk about retention rate
link |
with the VR headsets that the average is just once.
link |
I mean, it's such a fascinating way
link |
to think about technology.
link |
And this is a really, really good sign.
link |
And the other thing that people say about Kama
link |
is like they can't believe they're getting this 4,000 bucks.
link |
It seems like some kind of steal.
link |
So, but in terms of like longterm business strategies
link |
that basically to put,
link |
so it's currently in like a thousand plus cars.
link |
So yeah, dailies is about, dailies is about 2,000.
link |
Weeklys is about 2,500, monthlys is over 3,000.
link |
We've grown a lot since we last talked.
link |
Is the goal, like can we talk crazy for a second?
link |
I mean, what's the goal to overtake Tesla?
link |
Let's talk, okay, so.
link |
I mean, Android did overtake iOS.
link |
That's exactly it, right?
link |
I actually don't know the timeline of that one.
link |
But let's talk, because everything is in alpha now.
link |
The autopilot you could argue is in alpha
link |
in terms of towards the big mission
link |
of autonomous driving, right?
link |
And so what, yeah, is your goal to overtake
link |
millions of cars essentially?
link |
Where would it stop?
link |
Like it's open source software.
link |
It might not be millions of cars
link |
with a piece of comma hardware, but yeah.
link |
I think open pilot at some point
link |
will cross over autopilot in users,
link |
just like Android crossed over iOS.
link |
How does Google make money from Android?
link |
Their own devices make money.
link |
Google, Google makes money
link |
by just kind of having you on the internet.
link |
Google search is built in, Gmail is built in.
link |
Android is just a shill
link |
for the rest of Google's ecosystem.
link |
Yeah, but the problem is Android is not,
link |
is a brilliant thing.
link |
I mean, Android arguably changed the world.
link |
That's, you can feel good ethically speaking.
link |
But as a business strategy, it's questionable.
link |
I mean, it took Google a long time to come around to it,
link |
but they are now making money on the Pixel.
link |
You're not about money, you're more about winning.
link |
No, but if only 10% of open pilot devices
link |
come from comma AI.
link |
They still make a lot.
link |
That is still, yes.
link |
That is a ton of money for our company.
link |
But can't somebody create a better comma using open pilot?
link |
Or are you basically saying, well, I'll compete them?
link |
Well, I'll compete you.
link |
Can you create a better Android phone than the Google Pixel?
link |
I mean, you can, but like, you know.
link |
So you're confident, like, you know
link |
what the hell you're doing.
link |
It's confidence and merit.
link |
I mean, our money comes from, we're
link |
a consumer electronics company.
link |
And put it this way.
link |
So we sold like 3,000 comma twos.
link |
And like, OK, we're probably going
link |
to sell 10,000 units next year.
link |
10,000 units, even just $1,000 a unit, OK,
link |
we're at 10 million in revenue.
link |
Get that up to 100,000, maybe double the price of the unit.
link |
Now we're talking like 200 million revenue.
link |
We're talking like series.
link |
Yeah, actually making money.
link |
One of the rare semi autonomous or autonomous vehicle companies
link |
that are actually making money.
link |
You know, if you look at a model,
link |
and we were just talking about this yesterday.
link |
If you look at a model, and like you're AB testing your model,
link |
and if you're one branch of the AB test,
link |
the losses go down very fast in the first five epochs.
link |
That model is probably going to converge
link |
to something considerably better than the one
link |
where the losses are going down slower.
link |
Why do people think this is going to stop?
link |
Why do people think one day there's
link |
going to be a great like, well, Waymo's eventually
link |
going to surpass you guys?
link |
Well, they're not.
link |
Do you see like a world where like a Tesla or a car
link |
like a Tesla would be able to basically press a button
link |
and you like switch to open pilot?
link |
You know, you load in.
link |
No, so I think so first off, I think
link |
that we may surpass Tesla in terms of users.
link |
I do not think we're going to surpass Tesla ever
link |
in terms of revenue.
link |
I think Tesla can capture a lot more revenue per user
link |
But this mimics the Android iOS model exactly.
link |
There may be more Android devices,
link |
but there's a lot more iPhones than Google Pixels.
link |
So I think there'll be a lot more Tesla cars sold
link |
than pieces of common hardware.
link |
And then as far as a Tesla owner being
link |
able to switch to open pilot, does iPhones run Android?
link |
No, but it doesn't make sense.
link |
You can if you really want to do it,
link |
but it doesn't really make sense.
link |
It doesn't make sense.
link |
What about if a large company like automakers, Ford, GM,
link |
Toyota came to George Hots?
link |
Or on the tech space, Amazon, Facebook, Google
link |
came with a large pile of cash?
link |
Would you consider being purchased?
link |
Do you see that as a one possible?
link |
Not seriously, no.
link |
I would probably see how much shit they'll entertain for me.
link |
And if they're willing to jump through a bunch of my hoops,
link |
But no, not the way that M&A works today.
link |
I mean, we've been approached.
link |
And I laugh in these people's faces.
link |
I'm like, are you kidding?
link |
Because it's so demeaning.
link |
The M&A people are so demeaning to companies.
link |
They treat the startup world as their innovation ecosystem.
link |
And they think that I'm cool with going along with that,
link |
so I can have some of their scam fake Fed dollars.
link |
What am I going to do with more Fed coin?
link |
So that's the cool thing about podcasting,
link |
actually, is people criticize.
link |
I don't know if you're familiar with Spotify giving Joe Rogan
link |
I don't know about that.
link |
And they respect, despite all the shit
link |
that people are talking about Spotify,
link |
people understand that podcasters like Joe Rogan
link |
know what the hell they're doing.
link |
So they give them money and say, just do what you do.
link |
And the equivalent for you would be like,
link |
George, do what the hell you do, because you're good at it.
link |
Try not to murder too many people.
link |
There's some kind of common sense things,
link |
like just don't go on a weird rampage of it.
link |
It comes down to what companies I could respect, right?
link |
Could I respect GM?
link |
I mean, could I respect a Hyundai?
link |
That's a lot closer.
link |
Korean is the way.
link |
I think that the Japanese, the Germans, the US, they're all
link |
too, they're all too, they all think they're too great.
link |
What about the tech companies?
link |
Apple is, of the tech companies that I could respect,
link |
Apple's the closest.
link |
I mean, I could never.
link |
It would be ironic.
link |
It would be ironic if Comma AI is acquired by Apple.
link |
I mean, Facebook, look, I quit Facebook 10 years ago
link |
because I didn't respect the business model.
link |
Google has declined so fast in the last five years.
link |
What are your thoughts about Waymo and its present
link |
Let me start by saying something nice, which is I've
link |
visited them a few times and have ridden in their cars.
link |
And the engineering that they're doing,
link |
both the research and the actual development
link |
and the engineering they're doing
link |
and the scale they're actually achieving
link |
by doing it all themselves is really impressive.
link |
And the balance of safety and innovation.
link |
And the cars work really well for the routes they drive.
link |
It drives fast, which was very surprising to me.
link |
It drives the speed limit or faster than the speed limit.
link |
And it works really damn well.
link |
And the interface is nice.
link |
In Chandler, Arizona, yeah.
link |
Yeah, in Chandler, Arizona, very specific environment.
link |
So it gives me enough material in my mind
link |
to push back against the madmen of the world,
link |
like George Hotz, to be like, because you kind of imply
link |
there's zero probability they're going to win.
link |
And after I've used, after I've ridden in it, to me,
link |
Oh, it's not for technology reasons.
link |
No, it's worse than that.
link |
It's actually for product reasons, I think.
link |
Oh, you think they're just not capable of creating
link |
an amazing product?
link |
No, I think that the product that they're building
link |
doesn't make sense.
link |
You say the Waymo's are fast.
link |
Benchmark a Waymo against a competent Uber driver.
link |
The Uber driver's faster.
link |
It's not even about speed.
link |
It's the thing you said.
link |
It's about the experience of being stuck at a stop sign
link |
because pedestrians are crossing nonstop.
link |
I like when my Uber driver doesn't come to a full stop
link |
And so let's say the Waymo's are 20% slower than an Uber.
link |
You can argue that they're going to be cheaper.
link |
And I argue that users already have the choice
link |
to trade off money for speed.
link |
It's called UberPool.
link |
I think it's like 15% of rides are UberPools.
link |
Users are not willing to trade off money for speed.
link |
So the whole product that they're building
link |
is not going to be competitive with traditional ride sharing
link |
And also, whether there's profit to be made
link |
depends entirely on one company having a monopoly.
link |
I think that the level four autonomous ride sharing
link |
vehicles market is going to look a lot like the scooter market
link |
if even the technology does come to exist, which I question.
link |
Who's doing well in that market?
link |
It's a race to the bottom.
link |
Well, it could be closer like an Uber and a Lyft,
link |
where it's just one or two players.
link |
Well, the scooter people have given up
link |
trying to market scooters as a practical means
link |
of transportation.
link |
And they're just like, they're super fun to ride.
link |
I love those things.
link |
And they're great on that front.
link |
But from an actual transportation product
link |
perspective, I do not think scooters are viable.
link |
And I do not think level four autonomous cars are viable.
link |
If you, let's play a fun experiment.
link |
If you ran, let's do a Tesla and let's do Waymo.
link |
If Elon Musk took a vacation for a year, he just said,
link |
screw it, I'm going to go live on an island, no electronics.
link |
And the board decides that we need to find somebody
link |
to run the company.
link |
And they did decide that you should run the company
link |
How do you run Tesla differently?
link |
I wouldn't change much.
link |
Do you think they're on the right track?
link |
I wouldn't change.
link |
I mean, I'd have some minor changes.
link |
But even my debate with Tesla about end
link |
to end versus SegNets, that's just software.
link |
It's not like you're doing something terrible with SegNets.
link |
You're probably building something that's
link |
at least going to help you debug the end to end system a lot.
link |
It's very easy to transition from what they have
link |
to an end to end kind of thing.
link |
And then I presume you would, in the Model Y
link |
or maybe in the Model 3, start adding driver
link |
sensing with infrared.
link |
Yes, I would add infrared camera, infrared lights
link |
right away to those cars.
link |
And start collecting that data and do all that kind of stuff,
link |
I think they're already kind of doing it.
link |
It's an incredibly minor change.
link |
If I actually were CEO of Tesla, first off,
link |
I'd be horrified that I wouldn't be able to do
link |
a better job as Elon.
link |
And then I would try to understand
link |
the way he's done things before.
link |
You would also have to take over his Twitter.
link |
Yeah, what's your Twitter situation?
link |
Why are you so quiet on Twitter?
link |
Since Dukama is like what's your social network presence like?
link |
Because on Instagram, you do live streams.
link |
You understand the music of the internet,
link |
but you don't always fully engage into it.
link |
Well, I used to have a Twitter.
link |
Yeah, I mean, Instagram is a pretty place.
link |
Instagram is a beautiful place.
link |
It glorifies beauty.
link |
I like Instagram's values as a network.
link |
Twitter glorifies conflict, glorifies shots,
link |
taking shots of people.
link |
And it's like, you know, Twitter and Donald Trump
link |
are perfectly, they're perfect for each other.
link |
So Tesla's on the right track in your view.
link |
OK, so let's try, let's really try this experiment.
link |
If you ran Waymo, let's say they're,
link |
I don't know if you agree, but they
link |
seem to be at the head of the pack of the kind of,
link |
what would you call that approach?
link |
Like it's not necessarily lighter based
link |
because it's not about lighter.
link |
Level four robotaxi.
link |
Level four robotaxi, all in before making any revenue.
link |
So they're probably at the head of the pack.
link |
If you were said, hey, George, can you
link |
please run this company for a year, how would you change it?
link |
I would get Anthony Levandowski out of jail,
link |
and I would put him in charge of the company.
link |
Well, let's try to break that apart.
link |
Why do you want to destroy the company by doing that?
link |
Or do you mean you like renegade style thinking that pushes,
link |
that throws away bureaucracy and goes
link |
to first principle thinking?
link |
What do you mean by that?
link |
I think Anthony Levandowski is a genius,
link |
and I think he would come up with a much better idea of what
link |
to do with Waymo than me.
link |
So you mean that unironically.
link |
I mean, I'm not saying there's no shortcomings,
link |
but in the interactions I've had with him, yeah.
link |
He's also willing to take, like, who knows
link |
what he would do with Waymo?
link |
I mean, he's also out there, like far more out there
link |
Yeah, there's big risks.
link |
What do you make of him?
link |
I was going to talk to him on this podcast,
link |
and I was going back and forth.
link |
I'm such a gullible, naive human.
link |
Like, I see the best in people.
link |
And I slowly started to realize that there
link |
might be some people out there that, like,
link |
have multiple faces to the world.
link |
They're, like, deceiving and dishonest.
link |
I still refuse to, like, I just, I trust people,
link |
and I don't care if I get hurt by it.
link |
But, like, you know, sometimes you
link |
have to be a little bit careful, especially platform
link |
wise and podcast wise.
link |
What do you, what am I supposed to think?
link |
So you think, you think he's a good person?
link |
I don't really make moral judgments.
link |
It's difficult to.
link |
Oh, I mean this about the Waymo.
link |
I actually, I mean that whole idea very nonironically
link |
about what I would do.
link |
The problem with putting me in charge of Waymo
link |
is Waymo is already $10 billion in the hole, right?
link |
Whatever idea Waymo does, look, commas profitable, commas
link |
raised $8.1 million.
link |
That's small, you know, that's small money.
link |
Like, I can build a reasonable consumer electronics company
link |
and succeed wildly at that and still never be able to pay back
link |
Waymo's $10 billion.
link |
So I think the basic idea with Waymo, well,
link |
forget the $10 billion because they have some backing,
link |
but your basic thing is, like, what can we do
link |
to start making some money?
link |
Well, no, I mean, my bigger idea is, like,
link |
whatever the idea is that's gonna save Waymo,
link |
It's gonna have to be a big risk idea
link |
and I cannot think of a better person
link |
than Anthony Levandowski to do it.
link |
So that is completely what I would do as CEO of Waymo.
link |
I would call myself a transitionary CEO,
link |
do everything I can to fix that situation up.
link |
Because I can't do it, right?
link |
Like, I can't, I mean, I can talk about how
link |
what I really wanna do is just apologize
link |
for all those corny, you know, ad campaigns
link |
and be like, here's the real state of the technology.
link |
Yeah, that's, like, I have several criticism.
link |
I'm a little bit more bullish on Waymo
link |
than you seem to be, but one criticism I have
link |
is it went into corny mode too early.
link |
Like, it's still a startup.
link |
It hasn't delivered on anything.
link |
So it should be, like, more renegade
link |
and show off the engineering that they're doing,
link |
which just can be impressive,
link |
as opposed to doing these weird commercials
link |
of, like, your friendly car company.
link |
I mean, that's my biggest snipe at Waymo is always,
link |
that guy's a paid actor.
link |
That guy's not a Waymo user.
link |
He's a paid actor.
link |
Look here, I found his call sheet.
link |
Do kind of like what SpaceX is doing
link |
with the rocket launches.
link |
Just put the nerds up front, put the engineers up front,
link |
and just, like, show failures too, just.
link |
I love SpaceX's, yeah.
link |
Yeah, the thing that they're doing is right,
link |
and it just feels like the right.
link |
We're all so excited to see them succeed.
link |
I can't wait to see when it won't fail, you know?
link |
Like, you lie to me, I want you to fail.
link |
You tell me the truth, you be honest with me,
link |
I want you to succeed.
link |
Ah, yeah, and that requires the renegade CEO, right?
link |
I'm with you, I'm with you.
link |
I still have a little bit of faith in Waymo
link |
for the renegade CEO to step forward, but.
link |
It's not, it's not John Kraftik.
link |
Yeah, it's, you can't.
link |
It's not Chris Hormiston.
link |
And those people may be very good at certain things.
link |
But they're not renegades.
link |
Yeah, because these companies are fundamentally,
link |
even though we're talking about billion dollars,
link |
all these crazy numbers,
link |
they're still, like, early stage startups.
link |
I mean, and I just, if you are pre revenue
link |
and you've raised 10 billion dollars,
link |
I have no idea, like, this just doesn't work.
link |
You know, it's against everything Silicon Valley.
link |
Where's your minimum viable product?
link |
You know, where's your users?
link |
Where's your growth numbers?
link |
This is traditional Silicon Valley.
link |
Why do you not apply it to what you think
link |
you're too big to fail already, like?
link |
How do you think autonomous driving will change society?
link |
So the mission is, for comma, to solve self driving.
link |
Do you have, like, a vision of the world
link |
of how it'll be different?
link |
Is it as simple as A to B transportation?
link |
Or is there, like, cause these are robots.
link |
It's not about autonomous driving in and of itself.
link |
It's what the technology enables.
link |
It's, I think it's the coolest applied AI problem.
link |
I like it because it has a clear path to monetary value.
link |
But as far as that being the thing that changes the world,
link |
I mean, no, like, there's cute things we're doing in common.
link |
Like, who'd have thought you could stick a phone
link |
on the windshield and it'll drive.
link |
But like, really, the product that you're building
link |
is not something that people were not capable
link |
of imagining 50 years ago.
link |
So no, it doesn't change the world on that front.
link |
Could people have imagined the internet 50 years ago?
link |
Only true genius visionaries.
link |
Everyone could have imagined autonomous cars 50 years ago.
link |
It's like a car, but I don't drive it.
link |
See, I have this sense, and I told you, like,
link |
my longterm dream is robots with which you have deep,
link |
with whom you have deep connections, right?
link |
And there's different trajectories towards that.
link |
And I've been thinking,
link |
so I've been thinking of launching a startup.
link |
I see autonomous vehicles
link |
as a potential trajectory to that.
link |
That's not where the direction I would like to go,
link |
but I also see Tesla or even Comma AI,
link |
like, pivoting into robotics broadly defined
link |
at some stage in the way, like you're mentioning,
link |
the internet didn't expect.
link |
Let's solve, you know, when I say a comma about this,
link |
we could talk about this,
link |
but let's solve self driving cars first.
link |
You gotta stay focused on the mission.
link |
Don't, don't, don't, you're not too big to fail.
link |
For however much I think Comma's winning,
link |
like, no, no, no, no, no, you're winning
link |
when you solve level five self driving cars.
link |
And until then, you haven't won.
link |
And you know, again, you wanna be arrogant
link |
in the face of other people, great.
link |
You wanna be arrogant in the face of nature, you're an idiot.
link |
Stay mission focused, brilliantly put.
link |
Like I mentioned, thinking of launching a startup,
link |
I've been considering, actually, before COVID,
link |
I've been thinking of moving to San Francisco.
link |
Ooh, ooh, I wouldn't go there.
link |
So why is, well, and now I'm thinking
link |
about potentially Austin and we're in San Diego now.
link |
San Diego, come here.
link |
So why, what, I mean, you're such an interesting human.
link |
You've launched so many successful things.
link |
What, why San Diego?
link |
What do you recommend?
link |
Why not San Francisco?
link |
Have you thought, so in your case,
link |
San Diego with Qualcomm and Snapdragon,
link |
I mean, that's an amazing combination.
link |
That wasn't really why.
link |
That wasn't the why?
link |
No, I mean, Qualcomm was an afterthought.
link |
Qualcomm was, it was a nice thing to think about.
link |
It's like, you can have a tech company here.
link |
And a good one, I mean, you know, I like Qualcomm, but.
link |
Well, so why San Diego better than San Francisco?
link |
Why does San Francisco suck?
link |
Well, so, okay, so first off,
link |
we all kind of said like, we wanna stay in California.
link |
People like the ocean.
link |
You know, California, for its flaws,
link |
it's like a lot of the flaws of California
link |
are not necessarily California as a whole,
link |
and they're much more San Francisco specific.
link |
San Francisco, so I think first tier cities in general
link |
have stopped wanting growth.
link |
Well, you have like in San Francisco, you know,
link |
the voting class always votes to not build more houses
link |
because they own all the houses.
link |
And they're like, well, you know,
link |
once people have figured out how to vote themselves
link |
more money, they're gonna do it.
link |
It is so insanely corrupt.
link |
It is not balanced at all, like political party wise,
link |
you know, it's a one party city and.
link |
For all the discussion of diversity,
link |
it stops lacking real diversity of thought,
link |
of background, of approaches, of strategies, of ideas.
link |
It's kind of a strange place
link |
that it's the loudest people about diversity
link |
and the biggest lack of diversity.
link |
I mean, that's what they say, right?
link |
It's the projection.
link |
Yeah, it's interesting.
link |
And even people in Silicon Valley tell me
link |
that's like high up people,
link |
everybody is like, this is a terrible place.
link |
It doesn't make sense.
link |
I mean, and coronavirus is really what killed it.
link |
San Francisco was the number one exodus
link |
during coronavirus.
link |
We still think San Diego is a good place to be.
link |
Yeah, I mean, we'll see.
link |
We'll see what happens with California a bit longer term.
link |
Like Austin's an interesting choice.
link |
I wouldn't, I don't have really anything bad to say
link |
about Austin either,
link |
except for the extreme heat in the summer,
link |
which, but that's like very on the surface, right?
link |
I think as far as like an ecosystem goes, it's cool.
link |
I personally love Colorado.
link |
Yeah, I mean, you have these states that are,
link |
like just way better run.
link |
California is, you know, it's especially San Francisco.
link |
It's not a tie horse and like, yeah.
link |
Can I ask you for advice to me and to others
link |
about what's it take to build a successful startup?
link |
I haven't done that.
link |
Talk to someone who did that.
link |
Well, you've, you know,
link |
this is like another book of years
link |
that I'll buy for $67, I suppose.
link |
One of these days I'll sell out.
link |
Yeah, that's right.
link |
Jailbreaks are going to be a dollar
link |
and books are going to be 67.
link |
How I jailbroke the iPhone by George Hots.
link |
How I jail broke the iPhone and you can too.
link |
Okay, I can't wait.
link |
But quite, so you have an introspective,
link |
you have built a very unique company.
link |
I mean, not you, but you and others.
link |
There's no, there's nothing.
link |
You have an introspective,
link |
you haven't really sat down and thought about like,
link |
well, like if you and I were having a bunch of,
link |
we're having some beers
link |
and you're seeing that I'm depressed
link |
and whatever, I'm struggling.
link |
There's no advice you can give?
link |
Um, yeah, I think it's all very like situation dependent.
link |
Here's, okay, if I can give a generic piece of advice,
link |
it's the technology always wins.
link |
The better technology always wins.
link |
And lying always loses.
link |
Build technology and don't lie.
link |
I agree very much.
link |
The long run, long run.
link |
That's the long run, yeah.
link |
The market can remain irrational longer
link |
than you can remain solvent.
link |
Well, this is an interesting point
link |
because I ethically and just as a human believe that
link |
like hype and smoke and mirrors is not
link |
at any stage of the company is a good strategy.
link |
I mean, there's some like, you know,
link |
PR magic kind of like, you know.
link |
Oh, hype around a new product, right?
link |
If there's a call to action,
link |
if there's like a call to action,
link |
like buy my new GPU, look at it.
link |
It takes up three slots and it's this big.
link |
Yeah, that's great.
link |
If you look at, you know,
link |
especially in the AI space broadly,
link |
but autonomous vehicles,
link |
like you can raise a huge amount of money on nothing.
link |
And the question to me is like, I'm against that.
link |
I'll never be part of that.
link |
I don't think, I hope not, willingly not.
link |
But like, is there something to be said
link |
to essentially lying to raise money,
link |
like fake it till you make it kind of thing?
link |
I mean, this is Billy McFarland in the Fyre Festival.
link |
Like we all experienced, you know,
link |
what happens with that.
link |
No, no, don't fake it till you make it.
link |
Be honest and hope you make it the whole way.
link |
The technology wins.
link |
Right, the technology wins.
link |
And like, there is, I'm not used to like the anti hype,
link |
you know, that's a Slava KPSS reference,
link |
but hype isn't necessarily bad.
link |
I loved camping out for the iPhones, you know,
link |
and as long as the hype is backed by like substance,
link |
as long as it's backed by something I can actually buy,
link |
and like it's real, then hype is great
link |
and it's a great feeling.
link |
It's when the hype is backed by lies
link |
that it's a bad feeling.
link |
I mean, a lot of people call Elon Musk a fraud.
link |
How could he be a fraud?
link |
I've noticed this, this kind of interesting effect,
link |
which is he does tend to over promise
link |
and deliver, what's the better way to phrase it?
link |
Promise a timeline that he doesn't deliver on,
link |
he delivers much later on.
link |
What do you think about that?
link |
Cause I do that, I think that's a programmer thing too.
link |
I do that as well.
link |
You think that's a really bad thing to do or is that okay?
link |
I think that's, again, as long as like,
link |
you're working toward it and you're gonna deliver on it,
link |
it's not too far off, right?
link |
Like, you know, the whole autonomous vehicle thing,
link |
it's like, I mean, I still think Tesla's on track
link |
I still think even with their missteps,
link |
they have advantages we don't have.
link |
You know, Elon is better than me
link |
at like marshaling massive amounts of resources.
link |
So, you know, I still think given the fact
link |
they're maybe making some wrong decisions,
link |
they'll end up winning.
link |
And like, it's fine to hype it
link |
if you're actually gonna win, right?
link |
Like if Elon says, look, we're gonna be landing rockets
link |
back on earth in a year and it takes four,
link |
like, you know, he landed a rocket back on earth
link |
and he was working toward it the whole time.
link |
I think there's some amount of like,
link |
I think when it becomes wrong is if you know
link |
you're not gonna meet that deadline.
link |
Yeah, that's brilliantly put.
link |
Like this is what people don't understand, I think.
link |
Like Elon believes everything he says.
link |
He does, as far as I can tell, he does.
link |
And I detected that in myself too.
link |
Like if I, it's only bullshit
link |
if you're like conscious of yourself lying.
link |
Now you can't take that to such an extreme, right?
link |
Like in a way, I think maybe Billy McFarland
link |
believed everything he said too.
link |
Right, that's how you start a cult
link |
and everybody kills themselves.
link |
Yeah, like it's, you need, you need,
link |
if there's like some factor on it, it's fine.
link |
And you need some people to like, you know,
link |
keep you in check, but like,
link |
if you deliver on most of the things you say
link |
and just the timelines are off, yeah.
link |
It does piss people off though.
link |
I wonder, but who cares?
link |
In a long arc of history, the people,
link |
everybody gets pissed off at the people who succeed,
link |
which is one of the things
link |
that frustrates me about this world,
link |
is they don't celebrate the success of others.
link |
Like there's so many people that want Elon to fail.
link |
It's so fascinating to me.
link |
Like what is wrong with you?
link |
Like, so Elon Musk talks about like people shorting,
link |
like they talk about financial,
link |
but I think it's much bigger than the financials.
link |
I've seen like the human factors community,
link |
they want, they want other people to fail.
link |
Like even people, the harshest thing is like,
link |
you know, even people that like seem
link |
to really hate Donald Trump, they want him to fail
link |
or like the other president
link |
or they want Barack Obama to fail.
link |
Yeah, we're all on the same boat, man.
link |
It's weird, but I want that,
link |
I would love to inspire that part of the world to change
link |
because damn it, if the human species is gonna survive,
link |
we should celebrate success.
link |
Like it seems like the efficient thing to do
link |
in this objective function that we're all striving for
link |
is to celebrate the ones that like figure out
link |
how to like do better at that objective function
link |
as opposed to like dragging them down back into the mud.
link |
I think there is, this is the speech I always give
link |
about the commenters on Hacker News.
link |
So first off, something to remember
link |
about the internet in general is commenters
link |
are not representative of the population.
link |
I don't comment on anything.
link |
You know, commenters are representative
link |
of a certain sliver of the population.
link |
And on Hacker News, a common thing I'll see
link |
is when you'll see something that's like,
link |
you know, promises to be wild out there and innovative.
link |
There is some amount of, you know,
link |
checking them back to earth,
link |
but there's also some amount of if this thing succeeds,
link |
well, I'm 36 and I've worked
link |
at large tech companies my whole life.
link |
They can't succeed because if they succeed,
link |
that would mean that I could have done something different
link |
with my life, but we know that I couldn't have,
link |
we know that I couldn't have,
link |
and that's why they're gonna fail.
link |
And they have to root for them to fail
link |
to kind of maintain their world image.
link |
And they comment, well, it's hard, I, so one of the things,
link |
one of the things I'm considering startup wise
link |
is to change that.
link |
Cause I think the, I think it's also a technology problem.
link |
It's a platform problem.
link |
It's like, because the thing you said,
link |
most people don't comment.
link |
I think most people want to comment.
link |
They just don't because it's all the assholes
link |
who are commenting.
link |
Exactly, I don't want to be grouped in with them.
link |
You don't want to be at a party
link |
where everyone is an asshole.
link |
And so they, but that's a platform problem.
link |
I can't believe what Reddit's become.
link |
I can't believe the group thinking, Reddit comments.
link |
There's a, Reddit is an interesting one
link |
because they're subreddits.
link |
And so you can still see, especially small subreddits
link |
that like, that are a little like havens
link |
of like joy and positivity and like deep,
link |
even disagreement, but like nuanced discussion.
link |
But it's only like small little pockets,
link |
but that's emergent.
link |
The platform is not helping that or hurting that.
link |
So I guess naturally something about the internet,
link |
if you don't put in a lot of effort to encourage
link |
nuance and positive, good vibes,
link |
it's naturally going to decline into chaos.
link |
I would love to see someone do this well.
link |
I think it's, yeah, very doable.
link |
I think actually, so I feel like Twitter
link |
could be overthrown.
link |
Yashua Bach talked about how like,
link |
if you have like and retweet,
link |
like that's only positive wiring, right?
link |
The only way to do anything like negative there
link |
is with a comment.
link |
And that's like that asymmetry is what gives,
link |
you know, Twitter its particular toxicness.
link |
Whereas I find YouTube comments to be much better
link |
because YouTube comments have an up and a down
link |
and they don't show the downvotes.
link |
Without getting into depth of this particular discussion,
link |
the point is to explore possibilities
link |
and get a lot of data on it.
link |
Because I mean, I could disagree with what you just said.
link |
The point is it's unclear.
link |
It hasn't been explored in a really rich way.
link |
Like these questions of how to create platforms
link |
that encourage positivity.
link |
Yeah, I think it's a technology problem.
link |
And I think we'll look back at Twitter as it is now.
link |
Maybe it'll happen within Twitter,
link |
but most likely somebody overthrows them
link |
is we'll look back at Twitter and say,
link |
can't believe we put up with this level of toxicity.
link |
You need a different business model too.
link |
Any social network that fundamentally has advertising
link |
as a business model, this was in The Social Dilemma,
link |
which I didn't watch, but I liked it.
link |
It's like, you know, there's always the, you know,
link |
you're the product, you're not the,
link |
but they had a nuanced take on it that I really liked.
link |
And it said, the product being sold is influence over you.
link |
The product being sold is literally your,
link |
you know, influence on you.
link |
Like that can't be, if that's your idea, okay.
link |
Well, you know, guess what?
link |
It can't not be toxic.
link |
Yeah, maybe there's ways to spin it,
link |
like with giving a lot more control to the user
link |
and transparency to see what is happening to them
link |
as opposed to in the shadows, it's possible,
link |
but that can't be the primary source of.
link |
But the users aren't, no one's gonna use that.
link |
It depends, it depends, it depends.
link |
I think that the, you're not going to,
link |
you can't depend on self awareness of the users.
link |
It's a longer discussion because you can't depend on it,
link |
but you can reward self awareness.
link |
Like if for the ones who are willing to put in the work
link |
of self awareness, you can reward them and incentivize
link |
and perhaps be pleasantly surprised how many people
link |
are willing to be self aware on the internet.
link |
Like we are in real life.
link |
Like I'm putting in a lot of effort with you right now,
link |
being self aware about if I say something stupid or mean,
link |
I'll like look at your like body language.
link |
Like I'm putting in that effort.
link |
It's costly for an introvert, very costly.
link |
But on the internet, fuck it.
link |
Like most people are like, I don't care if this hurts
link |
somebody, I don't care if this is not interesting
link |
or if this is, yeah, it's a mean or whatever.
link |
I think so much of the engagement today on the internet
link |
is so disingenuine too.
link |
You're not doing this out of a genuine,
link |
this is what you think.
link |
You're doing this just straight up to manipulate others.
link |
Whether you're in, you just became an ad.
link |
Yeah, okay, let's talk about a fun topic,
link |
which is programming.
link |
Here's another book idea for you.
link |
What's your perfect programming setup?
link |
So like this by George Hots.
link |
So like what, listen, you're.
link |
Give me a MacBook Air, sit me in a corner of a hotel room
link |
and you know I'll still ask you.
link |
So you really don't care.
link |
You don't fetishize like multiple monitors, keyboard.
link |
Those things are nice and I'm not gonna say no to them,
link |
but did they automatically unlock tons of productivity?
link |
I have definitely been more productive on a MacBook Air
link |
in a corner of a hotel room.
link |
So which operating system do you love?
link |
What text editor do you use IDE?
link |
What, is there something that is like the perfect,
link |
if you could just say the perfect productivity setup
link |
It doesn't matter.
link |
It literally doesn't matter.
link |
You know, I guess I code most of the time in Vim.
link |
Like literally I'm using an editor from the 70s.
link |
You know, you didn't make anything better.
link |
Okay, VS code is nice for reading code.
link |
There's a few things that are nice about it.
link |
I think that you can build much better tools.
link |
How like IDA's xrefs work way better than VS codes, why?
link |
Yeah, actually that's a good question, like why?
link |
I still use, sorry, Emacs for most.
link |
I've actually never, I have to confess something dark.
link |
So I've never used Vim.
link |
I think maybe I'm just afraid
link |
that my life has been like a waste.
link |
I'm so, I'm not evangelical about Emacs.
link |
This is how I feel about TensorFlow versus PyTorch.
link |
Having just like, we've switched everything to PyTorch now.
link |
Put months into the switch.
link |
I have felt like I've wasted years on TensorFlow.
link |
I can't believe it.
link |
I can't believe how much better PyTorch is.
link |
I've used Emacs and Vim, doesn't matter.
link |
Yeah, it's still just my heart.
link |
Somehow I fell in love with Lisp.
link |
You can't, the heart wants what the heart wants.
link |
I don't understand it, but it just connected with me.
link |
Maybe it's the functional language
link |
that first I connected with.
link |
Maybe it's because so many of the AI courses
link |
before the deep learning revolution
link |
were taught with Lisp in mind.
link |
I don't know what it is, but I'm stuck with it.
link |
But at the same time, like,
link |
why am I not using a modern ID
link |
for some of these programming?
link |
They're not that much better.
link |
I've used modern IDs too.
link |
But at the same time, so to just,
link |
well, not to disagree with you,
link |
but like, I like multiple monitors.
link |
Like I have to do work on a laptop
link |
and it's a pain in the ass.
link |
And also I'm addicted to the Kinesis weird keyboard.
link |
You could see there.
link |
Yeah, so you don't have any of that.
link |
You can just be on a MacBook.
link |
I mean, look at work.
link |
I have three 24 inch monitors.
link |
I have a happy hacking keyboard.
link |
I have a Razer Death Hatter mouse, like.
link |
But it's not essential for you.
link |
Let's go to a day in the life of George Hots.
link |
What is the perfect day productivity wise?
link |
So we're not talking about like Hunter S. Thompson drugs.
link |
And let's look at productivity.
link |
Like what's the day look like, like hour by hour?
link |
Is there any regularities that create
link |
a magical George Hots experience?
link |
I can remember three days in my life.
link |
And I remember these days vividly
link |
when I've gone through kind of radical transformations
link |
to the way I think.
link |
And what I would give, I would pay $100,000
link |
if I could have one of these days tomorrow.
link |
The days have been so impactful.
link |
And one was first discovering Eliezer Yudkowsky
link |
on the singularity and reading that stuff.
link |
And like, you know, my mind was blown.
link |
The next was discovering the Hutter Prize
link |
and that AI is just compression.
link |
Like finally understanding AIXI and what all of that was.
link |
You know, I like read about it when I was 18, 19,
link |
I didn't understand it.
link |
And then the fact that like lossless compression
link |
implies intelligence, the day that I was shown that.
link |
And then the third one is controversial.
link |
The day I found a blog called Unqualified Reservations.
link |
And read that and I was like.
link |
Wait, which one is that?
link |
That's, what's the guy's name?
link |
So many people tell me I'm supposed to talk to him.
link |
He looks, he sounds insane.
link |
Definitely. Or brilliant,
link |
but insane or both, I don't know.
link |
The day I found that blog was another like,
link |
this was during like Gamergate
link |
and kind of the run up to the 2016 election.
link |
And I'm like, wow, okay, the world makes sense now.
link |
This is like, I had a framework now to interpret this.
link |
Just like I got the framework for AI
link |
and a framework to interpret technological progress.
link |
Like those days when I discovered these new frameworks were.
link |
So it's not about, but what was special about those days?
link |
How did those days come to be?
link |
Is it just, you got lucky?
link |
Like, you just encountered a hotter prize
link |
on Hacker News or something like that?
link |
But you see, I don't think it's just,
link |
see, I don't think it's just that like,
link |
I could have gotten lucky at any point.
link |
I think that in a way.
link |
You were ready at that moment.
link |
To receive the information.
link |
But is there some magic to the day today
link |
of like eating breakfast?
link |
And it's the mundane things.
link |
Nah, I drift through life.
link |
Without structure.
link |
I drift through life hoping and praying
link |
that I will get another day like those days.
link |
And there's nothing in particular you do
link |
to be a receptacle for another, for day number four.
link |
No, I didn't do anything to get the other ones.
link |
So I don't think I have to really do anything now.
link |
I took a month long trip to New York
link |
and the Ethereum thing was the highlight of it,
link |
but the rest of it was pretty terrible.
link |
I did a two week road trip
link |
and I got, I had to turn around.
link |
I had to turn around driving in Gunnison, Colorado.
link |
I passed through Gunnison
link |
and the snow starts coming down.
link |
There's a pass up there called Monarch Pass
link |
in order to get through to Denver,
link |
you gotta get over the Rockies.
link |
And I had to turn my car around.
link |
I couldn't, I watched a F150 go off the road.
link |
I'm like, I gotta go back.
link |
And like that day was meaningful.
link |
Cause like, it was real.
link |
Like I actually had to turn my car around.
link |
It's rare that anything even real happens in my life.
link |
Even as, you know, mundane as the fact that,
link |
yeah, there was snow, I had to turn around,
link |
stay in Gunnison and leave the next day.
link |
Something about that moment felt real.
link |
Okay, so actually it's interesting to break apart
link |
the three moments you mentioned, if it's okay.
link |
So I always have trouble pronouncing his name,
link |
but Alousa Yurkowski.
link |
So what, how did your worldview change
link |
in starting to consider the exponential growth of AI
link |
and AGI that he thinks about
link |
and the threats of artificial intelligence
link |
and all that kind of ideas?
link |
Can you, is it just like, can you maybe break apart
link |
like what exactly was so magical to you?
link |
Is it transformational experience?
link |
Today, everyone knows him for threats and AI safety.
link |
This was pre that stuff.
link |
There was, I don't think a mention of AI safety on the page.
link |
This is, this is old Yurkowski stuff.
link |
He'd probably denounce it all now.
link |
He'd probably be like,
link |
that's exactly what I didn't want to happen.
link |
Is there something specific you can take from his work
link |
that you can remember?
link |
Yeah, it was this realization
link |
that computers double in power every 18 months
link |
and humans do not, and they haven't crossed yet.
link |
But if you have one thing that's doubling every 18 months
link |
and one thing that's staying like this, you know,
link |
here's your log graph, here's your line, you know,
link |
And then the data opened the door
link |
to the exponential thinking, like thinking that like,
link |
you know what, with technology,
link |
we can actually transform the world.
link |
It opened the door to human obsolescence.
link |
It opened the door to realize that in my lifetime,
link |
humans are going to be replaced.
link |
And then the matching idea to that of artificial intelligence
link |
with the Hutter prize, you know, I'm torn.
link |
I go back and forth on what I think about it.
link |
But the basic thesis is it's a nice compelling notion
link |
that we can reduce the task of creating
link |
an intelligent system, a generally intelligent system
link |
into the task of compression.
link |
So you can think of all of intelligence in the universe,
link |
in fact, as a kind of compression.
link |
Do you find that, was that just at the time
link |
you found that as a compelling idea
link |
or do you still find that a compelling idea?
link |
I still find that a compelling idea.
link |
I think that it's not that useful day to day,
link |
but actually one of maybe my quests before that
link |
was a search for the definition of the word intelligence.
link |
And I never had one.
link |
And I definitely have a definition of the word compression.
link |
It's a very simple, straightforward one.
link |
And you know what compression is,
link |
you know what lossless, it's lossless compression,
link |
not lossy, lossless compression.
link |
And that that is equivalent to intelligence,
link |
which I believe, I'm not sure how useful
link |
that definition is day to day,
link |
but like I now have a framework to understand what it is.
link |
And he just 10X the prize for that competition
link |
like recently a few months ago.
link |
You ever thought of taking a crack at that?
link |
I spent the next, after I found the prize,
link |
I spent the next six months of my life trying it.
link |
And well, that's when I started learning everything about AI.
link |
And then I worked at Vicarious for a bit
link |
and then I read all the deep learning stuff.
link |
And I'm like, okay, now I like I'm caught up to modern AI.
link |
And I had a really good framework to put it all in
link |
from the compression stuff, right?
link |
Like some of the first deep learning models I played with
link |
were GPT basically, but before transformers,
link |
before it was still RNNs to do character prediction.
link |
But by the way, on the compression side,
link |
I mean, especially with neural networks,
link |
what do you make of the lossless requirement
link |
with the Hutter prize?
link |
So, you know, human intelligence and neural networks
link |
can probably compress stuff pretty well,
link |
but it would be lossy.
link |
You can turn a lossy compression
link |
to a lossless compressor pretty easily
link |
using an arithmetic encoder, right?
link |
You can take an arithmetic encoder
link |
and you can just encode the noise with maximum efficiency.
link |
So even if you can't predict exactly
link |
what the next character is,
link |
the better a probability distribution,
link |
you can put over the next character.
link |
You can then use an arithmetic encoder to, right?
link |
You don't have to know whether it's an E or an I,
link |
you just have to put good probabilities on them
link |
and then, you know, code those.
link |
And if you have, it's a bits of entropy thing, right?
link |
So let me, on that topic,
link |
it'd be interesting as a little side tour.
link |
What are your thoughts in this year about GPT3
link |
and these language models and these transformers?
link |
Is there something interesting to you as an AI researcher,
link |
or is there something interesting to you
link |
as an autonomous vehicle developer?
link |
Nah, I think it's overhyped.
link |
I mean, it's not, like, it's cool.
link |
It's cool for what it is, but no,
link |
we're not just gonna be able to scale up to GPT12
link |
and get general purpose intelligence.
link |
Like, your loss function is literally just,
link |
you know, cross entropy loss on the character, right?
link |
Like, that's not the loss function of general intelligence.
link |
Is that obvious to you?
link |
Can you imagine that, like,
link |
to play devil's advocate on yourself,
link |
is it possible that you can,
link |
the GPT12 will achieve general intelligence
link |
with something as dumb as this kind of loss function?
link |
I guess it depends what you mean by general intelligence.
link |
So there's another problem with the GPTs,
link |
and that's that they don't have a,
link |
they don't have longterm memory.
link |
So, like, just GPT12,
link |
a scaled up version of GPT2 or GPT3,
link |
I find it hard to believe.
link |
Well, you can scale it in,
link |
so it's a hard coded length,
link |
but you can make it wider and wider and wider.
link |
You're gonna get cool things from those systems,
link |
but I don't think you're ever gonna get something
link |
that can, like, you know, build me a rocket ship.
link |
What about solved driving?
link |
So, you know, you can use Transformer with video,
link |
You think, is there something in there?
link |
No, because, I mean, look, we use a GRU.
link |
We could change that GRU out to a Transformer.
link |
I think driving is much more Markovian than language.
link |
So, Markovian, you mean, like, the memory,
link |
which aspect of Markovian?
link |
I mean that, like, most of the information
link |
in the state at T minus one is also in state T.
link |
Right, and it kind of, like, drops off nicely like this,
link |
whereas sometime with language,
link |
you have to refer back to the third paragraph
link |
on the second page.
link |
There's not many, like, you can say, like,
link |
speed limit signs, but there's really not many things
link |
in autonomous driving that look like that.
link |
But if you look at, to play devil's advocate,
link |
is the risk estimation thing that you've talked about
link |
is kind of interesting.
link |
Is, it feels like there might be some longer term
link |
aggregation of context necessary to be able to figure out,
link |
like, the context.
link |
Yeah, I'm not even sure I'm believing my devil's advocate.
link |
We have a nice, like, vision model,
link |
which outputs, like, a one or two,
link |
four dimensional perception space.
link |
Can I try Transformers on it?
link |
Sure, I probably will.
link |
At some point, we'll try Transformers,
link |
and then we'll just see.
link |
Do they do better?
link |
But it might not be a game changer, you're saying?
link |
No, well, I'm not.
link |
Like, might Transformers work better than GRUs
link |
for autonomous driving?
link |
Is this some radical change?
link |
Okay, we use a slightly different,
link |
you know, we switch from RNNs to GRUs.
link |
Like, okay, maybe it's GRUs to Transformers,
link |
Well, on the topic of general intelligence,
link |
I don't know how much I've talked to you about it.
link |
Like, what, do you think we'll actually build
link |
Like, if you look at Ray Kurzweil with Singularity,
link |
do you have like an intuition about,
link |
you're kind of saying driving is easy.
link |
And I tend to personally believe that solving driving
link |
will have really deep, important impacts
link |
on our ability to solve general intelligence.
link |
Like, I think driving doesn't require general intelligence,
link |
but I think they're going to be neighbors
link |
in a way that it's like deeply tied.
link |
Cause it's so, like driving is so deeply connected
link |
to the human experience that I think solving one
link |
will help solve the other.
link |
But, so I don't see, I don't see driving as like easy
link |
and almost like separate than general intelligence,
link |
but like, what's your vision of a future with a Singularity?
link |
Do you see there'll be a single moment,
link |
like a Singularity where it'll be a phase shift?
link |
Are we in the Singularity now?
link |
Like what, do you have crazy ideas about the future
link |
We're definitely in the Singularity now.
link |
Of course, of course.
link |
Look at the bandwidth between people.
link |
The bandwidth between people goes up, right?
link |
The Singularity is just, you know, when the bandwidth, but.
link |
What do you mean by the bandwidth of people?
link |
Communications, tools, the whole world is networked.
link |
The whole world is networked
link |
and we raise the speed of that network, right?
link |
Oh, so you think the communication of information
link |
in a distributed way is an empowering thing
link |
for collective intelligence?
link |
Oh, I didn't say it's necessarily a good thing,
link |
but I think that's like,
link |
when I think of the definition of the Singularity,
link |
yeah, it seems kind of right.
link |
I see, like it's a change in the world
link |
beyond which like the world be transformed
link |
in ways that we can't possibly imagine.
link |
No, I mean, I think we're in the Singularity now
link |
in the sense that there's like, you know,
link |
one world and a monoculture and it's also linked.
link |
Yeah, I mean, I kind of share the intuition
link |
that the Singularity will originate
link |
from the collective intelligence of us ands
link |
versus the like some single system AGI type thing.
link |
Oh, I totally agree with that.
link |
Yeah, I don't really believe in like a hard take off AGI
link |
Yeah, I don't even think AI is all that different in kind
link |
from what we've already been building.
link |
With respect to driving,
link |
I think driving is a subset of general intelligence
link |
and I think it's a pretty complete subset.
link |
I think the tools we develop at Kama
link |
will also be extremely helpful
link |
to solving general intelligence
link |
and that's I think the real reason why I'm doing it.
link |
I don't care about self driving cars.
link |
It's a cool problem to beat people at.
link |
But yeah, I mean, yeah, you're kind of, you're of two minds.
link |
So one, you do have to have a mission
link |
and you wanna focus and make sure you get there.
link |
You can't forget that but at the same time,
link |
there is a thread that's much bigger
link |
than that connects the entirety of your effort.
link |
That's much bigger than just driving.
link |
With AI and with general intelligence,
link |
it is so easy to delude yourself
link |
into thinking you've figured something out when you haven't.
link |
If we build a level five self driving car,
link |
we have indisputably built something.
link |
Is it general intelligence?
link |
I'm not gonna debate that.
link |
I will say we've built something
link |
that provides huge financial value.
link |
Yeah, beautifully put.
link |
That's the engineering credo.
link |
Like just build the thing.
link |
It's like, that's why I'm with Elon
link |
Yeah, that's a great one.
link |
You can argue like who the hell cares about going to Mars.
link |
But the reality is set that as a mission, get it done.
link |
And then you're going to crack some problem
link |
that you've never even expected
link |
in the process of doing that, yeah.
link |
Yeah, I mean, no, I think if I had a choice
link |
between humanity going to Mars
link |
and solving self driving cars,
link |
I think going to Mars is better, but I don't know.
link |
I'm more suited for self driving cars.
link |
I'm an information guy.
link |
I'm not a modernist, I'm a postmodernist.
link |
Postmodernist, all right, beautifully put.
link |
Let me drag you back to programming for a sec.
link |
What three, maybe three to five programming languages
link |
should people learn, do you think?
link |
Like if you look at yourself,
link |
what did you get the most out of from learning?
link |
Well, so everybody should learn C and assembly.
link |
We'll start with those two, right?
link |
Yeah, if you can't code an assembly,
link |
you don't know what the computer's doing.
link |
You don't understand like,
link |
you don't have to be great in assembly,
link |
but you have to code in it.
link |
And then like, you have to appreciate assembly
link |
in order to appreciate all the great things C gets you.
link |
And then you have to code in C
link |
in order to appreciate all the great things Python gets you.
link |
So I'll just say assembly C and Python,
link |
we'll start with those three.
link |
The memory allocation of C and the fact that,
link |
so assembly gives you a sense
link |
of just how many levels of abstraction
link |
you get to work on in modern day programming.
link |
Yeah, yeah, yeah, yeah, graph coloring for assignment,
link |
register assignment and compilers.
link |
Like, you know, you gotta do,
link |
you know, the compiler,
link |
the computer only has a certain number of registers,
link |
yet you can have all the variables you want in a C function.
link |
So you get to start to build intuition about compilation,
link |
like what a compiler gets you.
link |
Well, then there's kind of a,
link |
so those are all very imperative programming languages.
link |
Then there's two other paradigms for programming
link |
that everybody should be familiar with.
link |
And one of them is functional.
link |
You should learn Haskell and take that all the way through,
link |
learn a language with dependent types like Coq,
link |
learn that whole space,
link |
like the very PL theory, heavy languages.
link |
And Haskell is your favorite functional?
link |
Is that the go to, you'd say?
link |
Yeah, I'm not a great Haskell programmer.
link |
I wrote a compiler in Haskell once.
link |
There's another paradigm,
link |
and actually there's one more paradigm
link |
that I'll even talk about after that,
link |
that I never used to talk about
link |
when I would think about this,
link |
but the next paradigm is learn Verilog of HDL.
link |
Understand this idea of all of the instructions
link |
execute at once. If I have a block in Verilog
link |
and I write stuff in it, it's not sequential.
link |
They all execute at once.
link |
And then think like that, that's how hardware works.
link |
To be, so I guess assembly doesn't quite get you that.
link |
Assembly is more about compilation,
link |
and Verilog is more about the hardware,
link |
like giving a sense of what actually
link |
is the hardware is doing.
link |
Assembly, C, Python are straight,
link |
like they sit right on top of each other.
link |
In fact, C is, well, C is kind of coded in C,
link |
but you could imagine the first C was coded in assembly,
link |
and Python is actually coded in C.
link |
So you can straight up go on that.
link |
Got it, and then Verilog gives you, that's brilliant.
link |
And then I think there's another one now.
link |
Everyone, Carpathia calls it programming 2.0,
link |
which is learn a, I'm not even gonna,
link |
don't learn TensorFlow, learn PyTorch.
link |
So machine learning.
link |
We've got to come up with a better term
link |
than programming 2.0, or, but yeah.
link |
It's a programming language, learn it.
link |
I wonder if it can be formalized a little bit better.
link |
It feels like we're in the early days
link |
of what that actually entails.
link |
Data driven programming?
link |
Data driven programming, yeah.
link |
But it's so fundamentally different
link |
as a paradigm than the others.
link |
Like it almost requires a different skillset.
link |
But you think it's still, yeah.
link |
And PyTorch versus TensorFlow, PyTorch wins.
link |
It's the fourth paradigm.
link |
It's the fourth paradigm that I've kind of seen.
link |
There's like this, you know,
link |
imperative functional hardware.
link |
I don't know a better word for it.
link |
Do you have advice for people that wanna,
link |
you know, get into programming, wanna learn programming?
link |
what is programming noob lessons, exclamation point.
link |
And I think the top comment is like,
link |
warning, this is not for noobs.
link |
Do you have a noob, like a TLDW for that video,
link |
but also a noob friendly advice
link |
on how to get into programming?
link |
We're never going to learn programming
link |
by watching a video called Learn Programming.
link |
The only way to learn programming, I think,
link |
and the only one is that the only way
link |
everyone I've ever met who can program well,
link |
learned it all in the same way.
link |
They had something they wanted to do
link |
and then they tried to do it.
link |
And then they were like, oh, well, okay.
link |
This is kind of, you know, it'd be nice
link |
if the computer could kind of do this.
link |
And then, you know, that's how you learn.
link |
You just keep pushing on a project.
link |
So the only advice I have for learning programming
link |
Somebody wrote to me a question like,
link |
we don't really, they're looking to learn
link |
about recurring neural networks.
link |
And he's saying, like, my company's thinking
link |
of using recurring neural networks for time series data,
link |
but we don't really have an idea of where to use it yet.
link |
We just want to, like, do you have any advice
link |
on how to learn about, these are these kind of
link |
general machine learning questions.
link |
And I think the answer is, like,
link |
actually have a problem that you're trying to solve.
link |
Oh my God, when people talk like that,
link |
they're like, I heard machine learning is important.
link |
Could you help us integrate machine learning
link |
with macaroni and cheese production?
link |
You just, I don't even, you can't help these people.
link |
Like, who lets you run anything?
link |
Who lets that kind of person run anything?
link |
I think we're all, we're all beginners at some point.
link |
It's not like they're a beginner.
link |
It's like, my problem is not that they don't know
link |
about machine learning.
link |
My problem is that they think that machine learning
link |
has something to say about macaroni and cheese production.
link |
Or like, I heard about this new technology.
link |
How can I use it for why?
link |
Like, I don't know what it is, but how can I use it for why?
link |
You have to build up an intuition of how,
link |
cause you might be able to figure out a way,
link |
but like the prerequisites,
link |
you should have a macaroni and cheese problem to solve first.
link |
And then two, you should have more traditional,
link |
like the learning process should involve
link |
more traditionally applicable problems
link |
in the space of whatever that is, machine learning,
link |
and then see if it can be applied to mac and cheese.
link |
At least start with, tell me about a problem.
link |
Like if you have a problem, you're like,
link |
you know, some of my boxes aren't getting
link |
enough macaroni in them.
link |
Can we use machine learning to solve this problem?
link |
That's much, much better than how do I apply
link |
machine learning to macaroni and cheese?
link |
One big thing, maybe this is me talking
link |
to the audience a little bit, cause I get these days
link |
so many messages, advice on how to like learn stuff, okay?
link |
My, this is not me being mean.
link |
I think this is quite profound actually,
link |
is you should Google it.
link |
Like one of the like skills that you should really acquire
link |
as an engineer, as a researcher, as a thinker,
link |
like one, there's two complementary skills.
link |
Like one is with a blank sheet of paper
link |
with no internet to think deeply.
link |
And then the other is to Google the crap
link |
out of the questions you have.
link |
Like that's actually a skill people often talk about,
link |
but like doing research, like pulling at the thread,
link |
like looking up different words,
link |
going into like GitHub repositories with two stars
link |
and like looking how they did stuff,
link |
like looking at the code or going on Twitter,
link |
seeing like there's little pockets of brilliant people
link |
that are like having discussions.
link |
Like if you're a neuroscientist,
link |
go into signal processing community.
link |
If you're an AI person going into the psychology community,
link |
like switch communities.
link |
I keep searching, searching, searching,
link |
because it's so much better to invest
link |
in like finding somebody else who already solved your problem
link |
than it is to try to solve the problem.
link |
And because they've often invested years of their life,
link |
like entire communities are probably already out there
link |
who have tried to solve your problem.
link |
I think they're the same thing.
link |
I think you go try to solve the problem.
link |
And then in trying to solve the problem,
link |
if you're good at solving problems,
link |
you'll stumble upon the person who solved it already.
link |
But the stumbling is really important.
link |
I think that's a skill that people should really put,
link |
especially in undergrad, like search.
link |
If you ask me a question,
link |
how should I get started in deep learning, like especially?
link |
Like that is just so Googleable.
link |
Like the whole point is you Google that
link |
and you get a million pages and just start looking at them.
link |
Start pulling at the threads, start exploring,
link |
start taking notes, start getting advice
link |
from a million people that already like spent their life
link |
answering that question, actually.
link |
Oh, well, yeah, I mean, that's definitely also, yeah,
link |
when people like ask me things like that, I'm like, trust me,
link |
the top answer on Google is much, much better
link |
than anything I'm going to tell you, right?
link |
People ask, it's an interesting question.
link |
Let me know if you have any recommendations.
link |
What three books, technical or fiction or philosophical,
link |
had an impact on your life or you would recommend perhaps?
link |
Maybe we'll start with the least controversial,
link |
Infinite Jest, Infinite Jest is a...
link |
David Foster Wallace.
link |
Yeah, it's a book about wireheading, really.
link |
Very enjoyable to read, very well written.
link |
You know, you will grow as a person reading this book,
link |
its effort, and I'll set that up for the second book,
link |
which is pornography, it's called Atlas Shrugged,
link |
Atlas Shrugged is pornography.
link |
I mean, it is, I will not defend the,
link |
I will not say Atlas Shrugged is a well written book.
link |
It is entertaining to read, certainly, just like pornography.
link |
The production value isn't great.
link |
You know, there's a 60 page monologue in there
link |
that Ann Rand's editor really wanted to take out.
link |
And she paid, she paid out of her pocket
link |
to keep that 60 page monologue in the book.
link |
But it is a great book for a kind of framework
link |
of human relations.
link |
And I know a lot of people are like,
link |
yeah, but it's a terrible framework.
link |
Yeah, but it's a framework.
link |
Just for context, in a couple of days,
link |
I'm speaking for probably four plus hours
link |
with Yaron Brook, who's the main living,
link |
remaining objectivist, objectivist.
link |
So I've always found this philosophy quite interesting
link |
One of how repulsive some percent of,
link |
large percent of the population find it,
link |
which is always, always funny to me
link |
when people are like unable to even read a philosophy
link |
because of some, I think that says more
link |
about their psychological perspective on it.
link |
But there is something about objectivism
link |
and Ann Rand's philosophy that's deeply connected
link |
to this idea of capitalism,
link |
of the ethical life is the productive life
link |
that was always compelling to me.
link |
It didn't seem as, like I didn't seem to interpret it
link |
in the negative sense that some people do.
link |
To be fair, I read that book when I was 19.
link |
So you had an impact at that point, yeah.
link |
Yeah, and the bad guys in the book have this slogan
link |
from each according to their ability
link |
to each according to their need.
link |
And I'm looking at this and I'm like,
link |
these are the most cart,
link |
this is team rocket level cartoonishness, right?
link |
And then when I realized that was actually the slogan
link |
of the communist party, I'm like, wait a second.
link |
Wait, no, no, no, no, no.
link |
You're telling me this really happened?
link |
Yeah, it's interesting.
link |
I mean, one of the criticisms of her work
link |
is she has a cartoonish view of good and evil.
link |
Like the reality, as Jordan Peterson says,
link |
is that each of us have the capacity for good and evil
link |
in us as opposed to like, there's some characters
link |
who are purely evil and some characters that are purely good.
link |
And that's in a way why it's pornographic.
link |
The production value, I love it.
link |
Like evil is punished and there's very clearly,
link |
there's no, just like porn doesn't have character growth.
link |
Well, you know, neither does Alice Shrugged, like.
link |
But at 19 year old George Hots, it was good enough.
link |
Yeah, yeah, yeah, yeah.
link |
You have something?
link |
I could give, these two I'll just throw out.
link |
Great thing to start thinking about copies of yourself.
link |
Sorry, I didn't catch that.
link |
That is Greg Egan.
link |
He's a, that might not be his real name.
link |
Some Australian guy, might not be Australian.
link |
And then this one's online.
link |
It's called The Metamorphosis of Prime Intellect.
link |
It's a story set in a post singularity world.
link |
Is there, can you, either of the worlds,
link |
do you find something philosophical interesting in them
link |
that you can comment on?
link |
I mean, it is clear to me that
link |
Metamorphosis of Prime Intellect is like written by
link |
an engineer, which is,
link |
it's very almost a pragmatic take on a utopia, in a way.
link |
Positive or negative?
link |
That's up to you to decide reading the book.
link |
And the ending of it is very interesting as well.
link |
And I didn't realize what it was.
link |
I first read that when I was 15.
link |
I've reread that book several times in my life.
link |
And it's short, it's 50 pages.
link |
Everyone should go read it.
link |
What's, sorry, it's a little tangent.
link |
I've been working through the foundation.
link |
I've been, I haven't read much sci fi my whole life
link |
and I'm trying to fix that the last few months.
link |
That's been a little side project.
link |
What's to you as the greatest sci fi novel
link |
that people should read?
link |
I mean, I would, yeah, I would say like, yeah,
link |
Permutation City, Metamorphosis of Prime Intellect.
link |
I didn't like Foundation.
link |
I thought it was way too modernist.
link |
You like Dune and all of those.
link |
I've never read Dune.
link |
I've never read Dune.
link |
I have to read it.
link |
Fire Upon the Deep is interesting.
link |
Okay, I mean, look, everyone should read,
link |
everyone should read Neuromancer.
link |
Everyone should read Snow Crash.
link |
If you haven't read those, like start there.
link |
Yeah, I haven't read Snow Crash.
link |
You haven't read Snow Crash?
link |
Oh, it's, I mean, it's very entertaining.
link |
Go to Lesher Bach.
link |
And if you want the controversial one,
link |
Bronze Age Mindset.
link |
All right, I'll look into that one.
link |
Those aren't sci fi, but just to round out books.
link |
So a bunch of people asked me on Twitter
link |
and Reddit and so on for advice.
link |
So what advice would you give a young person today
link |
In other words, what, yeah, I mean, looking back,
link |
especially when you were younger, you did,
link |
and you continued it.
link |
You've accomplished a lot of interesting things.
link |
Is there some advice from those,
link |
from that life of yours that you can pass on?
link |
If college ever opens again,
link |
I would love to give a graduation speech.
link |
At that point, I will put a lot of somewhat satirical effort
link |
into this question.
link |
Yeah, at this, you haven't written anything at this point.
link |
Oh, you know what?
link |
Always wear sunscreen.
link |
Pick your plagiarizing.
link |
I mean, you know, but that's the,
link |
that's the like clean your room.
link |
You know, yeah, you can plagiarize from all of this stuff.
link |
And it's, there is no,
link |
self help books aren't designed to help you.
link |
They're designed to make you feel good.
link |
Like whatever advice I could give, you already know.
link |
Everyone already knows.
link |
Sorry, it doesn't feel good.
link |
Like, you know, you know,
link |
if I tell you that you should, you know,
link |
eat well and read more and it's not gonna do anything.
link |
I think the whole like genre
link |
of those kinds of questions is meaningless.
link |
If anything, it's don't worry so much about that stuff.
link |
Don't be so caught up in your head.
link |
I mean, you're, yeah.
link |
In a sense that your whole life,
link |
your whole existence is like moving version of that advice.
link |
There's something, I mean,
link |
there's something in you that resists
link |
that kind of thinking and that in itself is,
link |
it's just illustrative of who you are.
link |
And there's something to learn from that.
link |
I think you're clearly not overthinking stuff.
link |
And you know what?
link |
There's a gut thing.
link |
Even when I talk about my advice,
link |
I'm like, my advice is only relevant to me.
link |
It's not relevant to anybody else.
link |
I'm not saying you should go out.
link |
If you're the kind of person who overthinks things
link |
to stop overthinking things, it's not bad.
link |
It doesn't work for me.
link |
Maybe it works for you.
link |
Let me ask you about love.
link |
I think last time we talked about the meaning of life
link |
and it was kind of about winning.
link |
I don't think I've talked to you about love much,
link |
whether romantic or just love
link |
for the common humanity amongst us all.
link |
What role has love played in your life?
link |
In this quest for winning, where does love fit in?
link |
Well, the word love, I think means several different things.
link |
There's love in the sense of, maybe I could just say,
link |
there's like love in the sense of opiates
link |
and love in the sense of oxytocin
link |
and then love in the sense of,
link |
maybe like a love for math.
link |
I don't think it fits into either
link |
of those first two paradigms.
link |
So each of those, have they given something to you
link |
I'm not that big of a fan of the first two.
link |
The same reason I'm not a fan of,
link |
the same reason I don't do opiates and don't take ecstasy.
link |
And there were times, look, I've tried both.
link |
I liked opiates way more than I liked ecstasy,
link |
but they're not, the ethical life is the productive life.
link |
So maybe that's my problem with those.
link |
And then like, yeah, a sense of, I don't know,
link |
like abstract love for humanity.
link |
I mean, the abstract love for humanity,
link |
I'm like, yeah, I've always felt that.
link |
And I guess it's hard for me to imagine
link |
not feeling it and maybe there's people who don't.
link |
Yeah, that's just like a background thing that's there.
link |
I mean, since we brought up drugs, let me ask you,
link |
this is becoming more and more a part of my life
link |
because I'm talking to a few researchers
link |
that are working on psychedelics.
link |
I've eaten shrooms a couple of times
link |
and it was fascinating to me that like the mind can go,
link |
like just fascinating the mind can go to places
link |
I didn't imagine it could go.
link |
And it was very friendly and positive and exciting
link |
and everything was kind of hilarious in the place.
link |
Wherever my mind went, that's where I went.
link |
Is, what do you think about psychedelics?
link |
Do you think they have, where do you think the mind goes?
link |
Have you done psychedelics?
link |
Where do you think the mind goes?
link |
Is there something useful to learn about the places it goes
link |
once you come back?
link |
I find it interesting that this idea
link |
that psychedelics have something to teach
link |
is almost unique to psychedelics, right?
link |
People don't argue this about amphetamines.
link |
And I'm not really sure why.
link |
I think all of the drugs have lessons to teach.
link |
I think there's things to learn from opiates.
link |
I think there's things to learn from amphetamines.
link |
I think there's things to learn from psychedelics,
link |
things to learn from marijuana.
link |
But also at the same time recognize
link |
that I don't think you're learning things about the world.
link |
I think you're learning things about yourself.
link |
And, you know, what's the, even, it might've even been,
link |
might've even been a Timothy Leary quote.
link |
I don't wanna misquote him,
link |
but the idea is basically like, you know,
link |
everybody should look behind the door,
link |
but then once you've seen behind the door,
link |
you don't need to keep going back.
link |
So, I mean, and that's my thoughts on all real drug use too.
link |
Except maybe for caffeine.
link |
It's a little experience that is good to have, but.
link |
Oh yeah, no, I mean, yeah, I guess,
link |
yes, psychedelics are definitely.
link |
So you're a fan of new experiences, I suppose.
link |
Because they all contain a little,
link |
especially the first few times,
link |
it contains some lessons that can be picked up.
link |
Yeah, and I'll revisit psychedelics maybe once a year.
link |
Usually smaller doses.
link |
Maybe they turn up the learning rate of your brain.
link |
I've heard that, I like that.
link |
Yeah, that's cool.
link |
Big learning rates have pros and cons.
link |
Last question, and this is a little weird one,
link |
but you've called yourself crazy in the past.
link |
First of all, on a scale of one to 10,
link |
how crazy would you say are you?
link |
Oh, I mean, it depends how you, you know,
link |
when you compare me to Elon Musk and Anthony Levandowski,
link |
Let's go with six.
link |
Well, I like seven, seven's a good number.
link |
Seven, all right, well, I'm sure day by day it changes,
link |
right, so, but you're in that area.
link |
In thinking about that,
link |
what do you think is the role of madness?
link |
Is that a feature or a bug
link |
if you were to dissect your brain?
link |
So, okay, from like a mental health lens on crazy,
link |
I'm not sure I really believe in that.
link |
I'm not sure I really believe in like a lot of that stuff.
link |
Right, this concept of, okay, you know,
link |
when you get over to like hardcore bipolar and schizophrenia,
link |
these things are clearly real, somewhat biological.
link |
And then over here on the spectrum,
link |
you have like ADD and oppositional defiance disorder
link |
and these things that are like,
link |
wait, this is normal spectrum human behavior.
link |
Like this isn't, you know, where's the line here
link |
and why is this like a problem?
link |
So there's this whole, you know,
link |
the neurodiversity of humanity is huge.
link |
Like people think I'm always on drugs.
link |
People are saying this to me on my streams.
link |
And I'm like, guys, you know,
link |
like I'm real open with my drug use.
link |
I'd tell you if I was on drugs and yeah,
link |
I had like a cup of coffee this morning,
link |
but other than that, this is just me.
link |
You're witnessing my brain in action.
link |
So the word madness doesn't even make sense
link |
in the rich neurodiversity of humans.
link |
I think it makes sense, but only for like
link |
some insane extremes.
link |
Like if you are actually like visibly hallucinating,
link |
you know, that's okay.
link |
But there is the kind of spectrum on which you stand out.
link |
Like that's like, if I were to look, you know,
link |
at decorations on a Christmas tree or something like that,
link |
like if you were a decoration, that would catch my eye.
link |
Like that thing is sparkly, whatever the hell that thing is.
link |
There's something to that.
link |
Just like refusing to be boring
link |
or maybe boring is the wrong word,
link |
but to yeah, I mean, be willing to sparkle, you know?
link |
It's like somewhat constructed.
link |
I mean, I am who I choose to be.
link |
I'm gonna say things as true as I can see them.
link |
I'm not gonna lie.
link |
But that's a really important feature in itself.
link |
So like whatever the neurodiversity of your,
link |
whatever your brain is, not putting constraints on it
link |
that force it to fit into the mold of what society is like,
link |
defines what you're supposed to be.
link |
So you're one of the specimens
link |
that doesn't mind being yourself.
link |
Being right is super important,
link |
except at the expense of being wrong.
link |
Without breaking that apart,
link |
I think it's a beautiful way to end it.
link |
George, you're one of the most special humans I know.
link |
It's truly an honor to talk to you.
link |
Thanks so much for doing it.
link |
Thank you for having me.
link |
Thanks for listening to this conversation with George Hotz
link |
and thank you to our sponsors,
link |
Four Sigmatic, which is the maker
link |
of delicious mushroom coffee,
link |
Decoding Digital, which is a tech podcast
link |
that I listen to and enjoy,
link |
and ExpressVPN, which is the VPN I've used for many years.
link |
Please check out these sponsors in the description
link |
to get a discount and to support this podcast.
link |
If you enjoy this thing, subscribe on YouTube,
link |
review it with Five Stars and Apple Podcast,
link |
follow on Spotify, support on Patreon,
link |
or connect with me on Twitter at Lex Friedman.
link |
And now, let me leave you with some words
link |
from the great and powerful Linus Torvalds.
link |
Talk is cheap, show me the code.
link |
Thank you for listening and hope to see you next time.