back to indexGreg Brockman: OpenAI and AGI | Lex Fridman Podcast #17
link |
The following is a conversation with Greg Brockman.
link |
He's the cofounder and CTO of OpenAI,
link |
a world class research organization
link |
developing ideas in AI with a goal of eventually
link |
creating a safe and friendly artificial general
link |
intelligence, one that benefits and empowers humanity.
link |
OpenAI is not only a source of publications, algorithms, tools,
link |
Their mission is a catalyst for an important public discourse
link |
about our future with both narrow and general intelligence
link |
This conversation is part of the Artificial Intelligence
link |
podcast at MIT and beyond.
link |
If you enjoy it, subscribe on YouTube, iTunes,
link |
or simply connect with me on Twitter at Lex Friedman,
link |
spelled F R I D. And now, here's my conversation
link |
with Greg Brockman.
link |
So in high school, and right after you
link |
wrote a draft of a chemistry textbook,
link |
saw that that covers everything from basic structure
link |
of the atom to quantum mechanics.
link |
So it's clear you have an intuition and a passion
link |
for both the physical world with chemistry and now robotics
link |
to the digital world with AI, deep learning, reinforcement
link |
Do you see the physical world and the digital world
link |
And what do you think is the gap?
link |
A lot of it actually boils down to iteration speed.
link |
I think that a lot of what really motivates me
link |
is building things.
link |
I think about mathematics, for example,
link |
where you think really hard about a problem.
link |
You understand it.
link |
You write it down in this very obscure form
link |
that we call a proof.
link |
But then, this is in humanity's library.
link |
It's there forever.
link |
This is some truth that we've discovered.
link |
Maybe only five people in your field will ever read it.
link |
But somehow, you've kind of moved humanity forward.
link |
And so I actually used to really think
link |
that I was going to be a mathematician.
link |
And then I actually started writing this chemistry
link |
One of my friends told me, you'll never publish it
link |
because you don't have a PhD.
link |
So instead, I decided to build a website
link |
and try to promote my ideas that way.
link |
And then I discovered programming.
link |
And in programming, you think hard about a problem.
link |
You understand it.
link |
You write it down in a very obscure form
link |
that we call a program.
link |
But then once again, it's in humanity's library.
link |
And anyone can get the benefit from it.
link |
And the scalability is massive.
link |
And so I think that the thing that really appeals
link |
to me about the digital world is that you
link |
can have this insane leverage.
link |
A single individual with an idea is
link |
able to affect the entire planet.
link |
And that's something I think is really
link |
hard to do if you're moving around physical atoms.
link |
But you said mathematics.
link |
So if you look at the wet thing over here, our mind,
link |
do you ultimately see it as just math,
link |
as just information processing?
link |
Or is there some other magic, as you've seen,
link |
if you've seen through biology and chemistry and so on?
link |
Yeah, I think it's really interesting to think about
link |
humans as just information processing systems.
link |
And that seems like it's actually
link |
a pretty good way of describing a lot of how the world works
link |
or a lot of what we're capable of, to think that, again,
link |
if you just look at technological innovations
link |
over time, that in some ways, the most transformative
link |
innovation that we've had has been the computer.
link |
In some ways, the internet, that what has the internet done?
link |
The internet is not about these physical cables.
link |
It's about the fact that I am suddenly
link |
able to instantly communicate with any other human
link |
I'm able to retrieve any piece of knowledge
link |
that in some ways the human race has ever had,
link |
and that those are these insane transformations.
link |
Do you see our society as a whole, the collective,
link |
as another extension of the intelligence of the human being?
link |
So if you look at the human being
link |
as an information processing system,
link |
you mentioned the internet, the networking.
link |
Do you see us all together as a civilization
link |
as a kind of intelligent system?
link |
Yeah, I think this is actually
link |
a really interesting perspective to take
link |
and to think about, that you sort of have
link |
this collective intelligence of all of society,
link |
the economy itself is this superhuman machine
link |
that is optimizing something, right?
link |
And in some ways, a company has a will of its own, right?
link |
That you have all these individuals
link |
who are all pursuing their own individual goals
link |
and thinking really hard
link |
and thinking about the right things to do,
link |
but somehow the company does something
link |
that is this emergent thing
link |
and that is a really useful abstraction.
link |
And so I think that in some ways,
link |
we think of ourselves as the most intelligent things
link |
on the planet and the most powerful things on the planet,
link |
but there are things that are bigger than us
link |
that are the systems that we all contribute to.
link |
And so I think actually, it's interesting to think about
link |
if you've read Isaac Asimov's foundation, right?
link |
That there's this concept of psychohistory in there,
link |
which is effectively this,
link |
that if you have trillions or quadrillions of beings,
link |
then maybe you could actually predict what that being,
link |
that huge macro being will do
link |
and almost independent of what the individuals want.
link |
And I actually have a second angle on this
link |
that I think is interesting,
link |
which is thinking about technological determinism.
link |
One thing that I actually think a lot about with OpenAI,
link |
right, is that we're kind of coming on
link |
to this insanely transformational technology
link |
of general intelligence, right,
link |
that will happen at some point.
link |
And there's a question of how can you take actions
link |
that will actually steer it to go better rather than worse.
link |
And that I think one question you need to ask
link |
is as a scientist, as an inventor, as a creator,
link |
what impact can you have in general, right?
link |
You look at things like the telephone
link |
invented by two people on the same day.
link |
Like, what does that mean?
link |
Like, what does that mean about the shape of innovation?
link |
And I think that what's going on
link |
is everyone's building on the shoulders of the same giants.
link |
And so you can kind of, you can't really hope
link |
to create something no one else ever would.
link |
You know, if Einstein wasn't born,
link |
someone else would have come up with relativity.
link |
You know, he changed the timeline a bit, right,
link |
that maybe it would have taken another 20 years,
link |
but it wouldn't be that fundamentally humanity
link |
would never discover these fundamental truths.
link |
So there's some kind of invisible momentum
link |
that some people like Einstein or OpenAI is plugging into
link |
that anybody else can also plug into
link |
and ultimately that wave takes us into a certain direction.
link |
That's what he means by digital.
link |
That's right, that's right.
link |
And you know, this kind of seems to play out
link |
in a bunch of different ways,
link |
that there's some exponential that is being written
link |
and that the exponential itself, which one it is, changes.
link |
Think about Moore's Law, an entire industry
link |
set its clock to it for 50 years.
link |
Like, how can that be, right?
link |
How is that possible?
link |
And yet somehow it happened.
link |
And so I think you can't hope to ever invent something
link |
that no one else will.
link |
Maybe you can change the timeline a little bit.
link |
But if you really want to make a difference,
link |
I think that the thing that you really have to do,
link |
the only real degree of freedom you have
link |
is to set the initial conditions
link |
under which a technology is born.
link |
And so you think about the internet, right?
link |
That there are lots of other competitors
link |
trying to build similar things.
link |
And the internet won.
link |
And that the initial conditions
link |
were that it was created by this group
link |
that really valued people being able to be,
link |
anyone being able to plug in
link |
this very academic mindset of being open and connected.
link |
And I think that the internet for the next 40 years
link |
really played out that way.
link |
You know, maybe today things are starting
link |
to shift in a different direction.
link |
But I think that those initial conditions
link |
were really important to determine
link |
the next 40 years worth of progress.
link |
That's really beautifully put.
link |
So another example that I think about,
link |
you know, I recently looked at it.
link |
I looked at Wikipedia, the formation of Wikipedia.
link |
And I wondered what the internet would be like
link |
if Wikipedia had ads.
link |
You know, there's an interesting argument
link |
that why they chose not to make it,
link |
put advertisement on Wikipedia.
link |
I think Wikipedia's one of the greatest resources
link |
we have on the internet.
link |
It's extremely surprising how well it works
link |
and how well it was able to aggregate
link |
all this kind of good information.
link |
And essentially the creator of Wikipedia,
link |
I don't know, there's probably some debates there,
link |
but set the initial conditions.
link |
And now it carried itself forward.
link |
That's really interesting.
link |
So the way you're thinking about AGI
link |
or artificial intelligence is you're focused
link |
on setting the initial conditions for the progress.
link |
Okay, so looking to the future,
link |
if you create an AGI system,
link |
like one that can ace the Turing test, natural language,
link |
what do you think would be the interactions
link |
you would have with it?
link |
What do you think are the questions you would ask?
link |
Like what would be the first question you would ask?
link |
I think that at that point,
link |
if you've really built a powerful system
link |
that is capable of shaping the future of humanity,
link |
the first question that you really should ask
link |
is how do we make sure that this plays out well?
link |
And so that's actually the first question
link |
that I would ask a powerful AGI system is.
link |
So you wouldn't ask your colleague,
link |
you wouldn't ask like Ilya,
link |
you would ask the AGI system.
link |
Oh, we've already had the conversation with Ilya, right?
link |
And everyone here.
link |
And so you want as many perspectives
link |
and a piece of wisdom as you can
link |
for answering this question.
link |
So I don't think you necessarily defer
link |
to whatever your powerful system tells you,
link |
but you use it as one input
link |
to try to figure out what to do.
link |
But, and I guess fundamentally what it really comes down to
link |
is if you built something really powerful
link |
and you think about, for example,
link |
the creation of shortly after
link |
the creation of nuclear weapons, right?
link |
The most important question in the world was
link |
what's the world order going to be like?
link |
How do we set ourselves up in a place
link |
where we're going to be able to survive as a species?
link |
With AGI, I think the question is slightly different, right?
link |
That there is a question of how do we make sure
link |
that we don't get the negative effects,
link |
but there's also the positive side, right?
link |
You imagine that, like what won't AGI be like?
link |
Like what will it be capable of?
link |
And I think that one of the core reasons
link |
that an AGI can be powerful and transformative
link |
is actually due to technological development, right?
link |
If you have something that's capable as a human
link |
and that it's much more scalable,
link |
that you absolutely want that thing
link |
to go read the whole scientific literature
link |
and think about how to create cures for all the diseases,
link |
You want it to think about how to go
link |
and build technologies to help us create material abundance
link |
and to figure out societal problems
link |
that we have trouble with.
link |
Like how are we supposed to clean up the environment?
link |
And maybe you want this to go and invent
link |
a bunch of little robots that will go out
link |
and be biodegradable and turn ocean debris
link |
into harmless molecules.
link |
And I think that that positive side
link |
is something that I think people miss
link |
sometimes when thinking about what an AGI will be like.
link |
And so I think that if you have a system
link |
that's capable of all of that,
link |
you absolutely want its advice about how do I make sure
link |
that we're using your capabilities
link |
in a positive way for humanity.
link |
So what do you think about that psychology
link |
that looks at all the different possible trajectories
link |
of an AGI system, many of which,
link |
perhaps the majority of which are positive,
link |
and nevertheless focuses on the negative trajectories?
link |
I mean, you get to interact with folks,
link |
you get to think about this, maybe within yourself as well.
link |
You look at Sam Harris and so on.
link |
It seems to be, sorry to put it this way,
link |
but almost more fun to think about
link |
the negative possibilities.
link |
Whatever that's deep in our psychology,
link |
what do you think about that?
link |
And how do we deal with it?
link |
Because we want AI to help us.
link |
So I think there's kind of two problems
link |
entailed in that question.
link |
The first is more of the question of
link |
how can you even picture what a world
link |
with a new technology will be like?
link |
Now imagine we're in 1950,
link |
and I'm trying to describe Uber to someone.
link |
Apps and the internet.
link |
Yeah, I mean, that's going to be extremely complicated.
link |
But it's imaginable.
link |
It's imaginable, right?
link |
And now imagine being in 1950 and predicting Uber, right?
link |
And you need to describe the internet,
link |
you need to describe GPS,
link |
you need to describe the fact that
link |
everyone's going to have this phone in their pocket.
link |
And so I think that just the first truth
link |
is that it is hard to picture
link |
how a transformative technology will play out in the world.
link |
We've seen that before with technologies
link |
that are far less transformative than AGI will be.
link |
And so I think that one piece is that
link |
it's just even hard to imagine
link |
and to really put yourself in a world
link |
where you can predict what that positive vision
link |
And I think the second thing is that
link |
I think it is always easier to support the negative side
link |
than the positive side.
link |
It's always easier to destroy than create.
link |
And less in a physical sense
link |
and more just in an intellectual sense, right?
link |
Because I think that with creating something,
link |
you need to just get a bunch of things right.
link |
And to destroy, you just need to get one thing wrong.
link |
And so I think that what that means
link |
is that I think a lot of people's thinking dead ends
link |
as soon as they see the negative story.
link |
But that being said, I actually have some hope, right?
link |
I think that the positive vision
link |
is something that I think can be,
link |
is something that we can talk about.
link |
And I think that just simply saying this fact of,
link |
yeah, there's positive, there's negatives,
link |
everyone likes to dwell on the negative.
link |
People actually respond well to that message and say,
link |
huh, you're right, there's a part of this
link |
that we're not talking about, not thinking about.
link |
And that's actually something that's I think really
link |
been a key part of how we think about AGI at OpenAI.
link |
You can kind of look at it as like, okay,
link |
OpenAI talks about the fact that there are risks
link |
and yet they're trying to build this system.
link |
How do you square those two facts?
link |
So do you share the intuition that some people have,
link |
I mean from Sam Harris to even Elon Musk himself,
link |
that it's tricky as you develop AGI
link |
to keep it from slipping into the existential threats,
link |
into the negative?
link |
What's your intuition about how hard is it
link |
to keep AI development on the positive track?
link |
What's your intuition there?
link |
To answer that question, you can really look
link |
at how we structure OpenAI.
link |
So we really have three main arms.
link |
We have capabilities, which is actually doing
link |
the technical work and pushing forward
link |
what these systems can do.
link |
There's safety, which is working on technical mechanisms
link |
to ensure that the systems we build
link |
are aligned with human values.
link |
And then there's policy, which is making sure
link |
that we have governance mechanisms,
link |
answering that question of, well, whose values?
link |
And so I think that the technical safety one
link |
is the one that people kind of talk about the most, right?
link |
You talk about, like think about all of the dystopic AI
link |
movies, a lot of that is about not having
link |
good technical safety in place.
link |
And what we've been finding is that,
link |
you know, I think that actually a lot of people
link |
look at the technical safety problem
link |
and think it's just intractable, right?
link |
This question of what do humans want?
link |
How am I supposed to write that down?
link |
Can I even write down what I want?
link |
And then they stop there.
link |
But the thing is, we've already built systems
link |
that are able to learn things that humans can't specify.
link |
You know, even the rules for how to recognize
link |
if there's a cat or a dog in an image.
link |
Turns out it's intractable to write that down,
link |
and yet we're able to learn it.
link |
And that what we're seeing with systems we build at OpenAI,
link |
and they're still in early proof of concept stage,
link |
is that you are able to learn human preferences.
link |
You're able to learn what humans want from data.
link |
And so that's kind of the core focus
link |
for our technical safety team,
link |
and I think that there actually,
link |
we've had some pretty encouraging updates
link |
in terms of what we've been able to make work.
link |
So you have an intuition and a hope that from data,
link |
you know, looking at the value alignment problem,
link |
from data we can build systems that align
link |
with the collective better angels of our nature.
link |
So align with the ethics and the morals of human beings.
link |
To even say this in a different way,
link |
I mean, think about how do we align humans, right?
link |
Think about like a human baby can grow up
link |
to be an evil person or a great person.
link |
And a lot of that is from learning from data, right?
link |
That you have some feedback as a child is growing up,
link |
they get to see positive examples.
link |
And so I think that just like,
link |
that the only example we have of a general intelligence
link |
that is able to learn from data
link |
to align with human values and to learn values,
link |
I think we shouldn't be surprised
link |
that we can do the same sorts of techniques
link |
or whether the same sort of techniques
link |
end up being how we solve value alignment for AGI's.
link |
So let's go even higher.
link |
I don't know if you've read the book, Sapiens,
link |
but there's an idea that, you know,
link |
that as a collective, as us human beings,
link |
we kind of develop together ideas that we hold.
link |
There's no, in that context, objective truth.
link |
We just kind of all agree to certain ideas
link |
and hold them as a collective.
link |
Did you have a sense that there is,
link |
in the world of good and evil,
link |
do you have a sense that to the first approximation,
link |
there are some things that are good
link |
and that you could teach systems to behave to be good?
link |
So I think that this actually blends into our third team,
link |
right, which is the policy team.
link |
And this is the one, the aspect I think people
link |
really talk about way less than they should, right?
link |
Because imagine that we build super powerful systems
link |
that we've managed to figure out all the mechanisms
link |
for these things to do whatever the operator wants.
link |
The most important question becomes,
link |
who's the operator, what do they want,
link |
and how is that going to affect everyone else, right?
link |
And I think that this question of what is good,
link |
what are those values, I mean,
link |
I think you don't even have to go to those,
link |
those very grand existential places
link |
to start to realize how hard this problem is.
link |
You just look at different countries
link |
and cultures across the world,
link |
and that there's a very different conception
link |
of how the world works and what kinds of ways
link |
that society wants to operate.
link |
And so I think that the really core question
link |
is actually very concrete,
link |
and I think it's not a question
link |
that we have ready answers to, right?
link |
It's how do you have a world
link |
where all of the different countries that we have,
link |
United States, China, Russia,
link |
and the hundreds of other countries out there
link |
are able to continue to not just operate
link |
in the way that they see fit,
link |
but in the world that emerges
link |
where you have these very powerful systems
link |
operating alongside humans,
link |
ends up being something that empowers humans more,
link |
that makes human existence be a more meaningful thing,
link |
and that people are happier and wealthier,
link |
and able to live more fulfilling lives.
link |
It's not an obvious thing for how to design that world
link |
once you have that very powerful system.
link |
So if we take a little step back,
link |
and we're having a fascinating conversation,
link |
and OpenAI is in many ways a tech leader in the world,
link |
and yet we're thinking about
link |
these big existential questions,
link |
which is fascinating, really important.
link |
I think you're a leader in that space,
link |
and that's a really important space
link |
of just thinking how AI affects society
link |
in a big picture view.
link |
So Oscar Wilde said, we're all in the gutter,
link |
but some of us are looking at the stars,
link |
and I think OpenAI has a charter
link |
that looks to the stars, I would say,
link |
to create intelligence, to create general intelligence,
link |
make it beneficial, safe, and collaborative.
link |
So can you tell me how that came about,
link |
how a mission like that and the path
link |
to creating a mission like that at OpenAI was founded?
link |
Yeah, so I think that in some ways
link |
it really boils down to taking a look at the landscape.
link |
So if you think about the history of AI,
link |
that basically for the past 60 or 70 years,
link |
people have thought about this goal
link |
of what could happen if you could automate
link |
human intellectual labor.
link |
Imagine you could build a computer system
link |
that could do that, what becomes possible?
link |
We have a lot of sci fi that tells stories
link |
of various dystopias, and increasingly you have movies
link |
like Her that tell you a little bit about,
link |
maybe more of a little bit utopic vision.
link |
You think about the impacts that we've seen
link |
from being able to have bicycles for our minds
link |
and computers, and I think that the impact
link |
of computers and the internet has just far outstripped
link |
what anyone really could have predicted.
link |
And so I think that it's very clear
link |
that if you can build an AGI,
link |
it will be the most transformative technology
link |
that humans will ever create.
link |
And so what it boils down to then is a question of,
link |
well, is there a path, is there hope,
link |
is there a way to build such a system?
link |
And I think that for 60 or 70 years,
link |
that people got excited and that ended up
link |
not being able to deliver on the hopes
link |
that people had pinned on them.
link |
And I think that then, that after two winters
link |
of AI development, that people I think kind of
link |
almost stopped daring to dream, right?
link |
That really talking about AGI or thinking about AGI
link |
became almost this taboo in the community.
link |
But I actually think that people took the wrong lesson
link |
And if you look back, starting in 1959
link |
is when the Perceptron was released.
link |
And this is basically one of the earliest neural networks.
link |
It was released to what was perceived
link |
as this massive overhype.
link |
So in the New York Times in 1959,
link |
you have this article saying that the Perceptron
link |
will one day recognize people, call out their names,
link |
instantly translate speech between languages.
link |
And people at the time looked at this and said,
link |
this is, your system can't do any of that.
link |
And basically spent 10 years trying to discredit
link |
the whole Perceptron direction and succeeded.
link |
And all the funding dried up.
link |
And people kind of went in other directions.
link |
And in the 80s, there was this resurgence.
link |
And I'd always heard that the resurgence in the 80s
link |
was due to the invention of backpropagation
link |
and these algorithms that got people excited.
link |
But actually the causality was due to people
link |
building larger computers.
link |
That you can find these articles from the 80s
link |
saying that the democratization of computing power
link |
suddenly meant that you could run
link |
these larger neural networks.
link |
And then people started to do all these amazing things.
link |
Backpropagation algorithm was invented.
link |
And the neural nets people were running
link |
were these tiny little 20 neuron neural nets.
link |
What are you supposed to learn with 20 neurons?
link |
And so of course, they weren't able to get great results.
link |
And it really wasn't until 2012 that this approach,
link |
that's almost the most simple, natural approach
link |
that people had come up with in the 50s,
link |
in some ways even in the 40s before there were computers,
link |
with the Pitts–McCullough neuron,
link |
suddenly this became the best way of solving problems.
link |
And I think there are three core properties
link |
that deep learning has that I think
link |
are very worth paying attention to.
link |
The first is generality.
link |
We have a very small number of deep learning tools.
link |
SGD, deep neural net, maybe some RL.
link |
And it solves this huge variety of problems.
link |
Speech recognition, machine translation,
link |
game playing, all of these problems, small set of tools.
link |
So there's the generality.
link |
There's a second piece, which is the competence.
link |
You want to solve any of those problems?
link |
Throw up 40 years worth of normal computer vision research,
link |
replace it with a deep neural net,
link |
it's going to work better.
link |
And there's a third piece, which is the scalability.
link |
One thing that has been shown time and time again
link |
is that if you have a larger neural network,
link |
throw more compute, more data at it, it will work better.
link |
Those three properties together feel like essential parts
link |
of building a general intelligence.
link |
Now it doesn't just mean that if we scale up what we have,
link |
that we will have an AGI, right?
link |
There are clearly missing pieces.
link |
There are missing ideas.
link |
We need to have answers for reasoning.
link |
But I think that the core here is that for the first time,
link |
it feels that we have a paradigm that gives us hope
link |
that general intelligence can be achievable.
link |
And so as soon as you believe that,
link |
everything else comes into focus, right?
link |
If you imagine that you may be able to,
link |
and you know that the timeline I think remains uncertain,
link |
but I think that certainly within our lifetimes
link |
and possibly within a much shorter period of time
link |
than people would expect,
link |
if you can really build the most transformative technology
link |
that will ever exist,
link |
you stop thinking about yourself so much, right?
link |
You start thinking about just like,
link |
how do you have a world where this goes well?
link |
And that you need to think about the practicalities
link |
of how do you build an organization
link |
and get together a bunch of people and resources
link |
and to make sure that people feel motivated
link |
and ready to do it.
link |
But I think that then you start thinking about,
link |
well, what if we succeed?
link |
And how do we make sure that when we succeed,
link |
that the world is actually the place
link |
that we want ourselves to exist in?
link |
And almost in the Rawlsian Veil sense of the word.
link |
And so that's kind of the broader landscape.
link |
And OpenAI was really formed in 2015
link |
with that high level picture of AGI might be possible
link |
sooner than people think,
link |
and that we need to try to do our best
link |
to make sure it's going to go well.
link |
And then we spent the next couple of years
link |
really trying to figure out what does that mean?
link |
And I think that typically with a company,
link |
you start out very small, see you in a co founder,
link |
and you build a product, you get some users,
link |
you get a product market fit.
link |
Then at some point you raise some money,
link |
you hire people, you scale, and then down the road,
link |
then the big companies realize you exist
link |
and try to kill you.
link |
And for OpenAI, it was basically everything
link |
in exactly the opposite order.
link |
Let me just pause for a second, you said a lot of things.
link |
And let me just admire the jarring aspect
link |
of what OpenAI stands for, which is daring to dream.
link |
I mean, you said it's pretty powerful.
link |
It caught me off guard because I think that's very true.
link |
The step of just daring to dream about the possibilities
link |
of creating intelligence in a positive, in a safe way,
link |
but just even creating intelligence is a very powerful
link |
is a much needed refreshing catalyst for the AI community.
link |
So that's the starting point.
link |
Okay, so then formation of OpenAI, what's that?
link |
I would just say that when we were starting OpenAI,
link |
that kind of the first question that we had is,
link |
is it too late to start a lab
link |
with a bunch of the best people?
link |
Right, is that even possible? Wow, okay.
link |
That was an actual question?
link |
That was the core question of,
link |
we had this dinner in July of 2015,
link |
and that was really what we spent the whole time
link |
And, you know, because you think about kind of where AI was
link |
is that it had transitioned from being an academic pursuit
link |
to an industrial pursuit.
link |
And so a lot of the best people were in these big
link |
research labs and that we wanted to start our own one
link |
that no matter how much resources we could accumulate
link |
would be pale in comparison to the big tech companies.
link |
And it was a question of, are we going to be actually
link |
able to get this thing off the ground?
link |
You need critical mass.
link |
You can't just do you and a cofounder build a product.
link |
You really need to have a group of five to 10 people.
link |
And we kind of concluded it wasn't obviously impossible.
link |
So it seemed worth trying.
link |
Well, you're also a dreamer, so who knows, right?
link |
Okay, so speaking of that, competing with the big players,
link |
let's talk about some of the tricky things
link |
as you think through this process of growing,
link |
of seeing how you can develop these systems
link |
at a scale that competes.
link |
So you recently formed OpenAI LP,
link |
a new cap profit company that now carries the name OpenAI.
link |
So OpenAI is now this official company.
link |
The original nonprofit company still exists
link |
and carries the OpenAI nonprofit name.
link |
So can you explain what this company is,
link |
what the purpose of this creation is,
link |
and how did you arrive at the decision to create it?
link |
OpenAI, the whole entity and OpenAI LP as a vehicle
link |
is trying to accomplish the mission
link |
of ensuring that artificial general intelligence
link |
benefits everyone.
link |
And the main way that we're trying to do that
link |
is by actually trying to build general intelligence
link |
ourselves and make sure the benefits
link |
are distributed to the world.
link |
That's the primary way.
link |
We're also fine if someone else does this, right?
link |
Doesn't have to be us.
link |
If someone else is going to build an AGI
link |
and make sure that the benefits don't get locked up
link |
in one company or with one set of people,
link |
like we're actually fine with that.
link |
And so those ideas are baked into our charter,
link |
which is kind of the foundational document
link |
that describes kind of our values and how we operate.
link |
But it's also really baked into the structure of OpenAI LP.
link |
And so the way that we've set up OpenAI LP
link |
is that in the case where we succeed, right?
link |
If we actually build what we're trying to build,
link |
then investors are able to get a return,
link |
but that return is something that is capped.
link |
And so if you think of AGI in terms of the value
link |
that you could really create,
link |
you're talking about the most transformative technology
link |
ever created, it's going to create orders of magnitude
link |
more value than any existing company.
link |
And that all of that value will be owned by the world,
link |
like legally titled to the nonprofit
link |
to fulfill that mission.
link |
And so that's the structure.
link |
So the mission is a powerful one,
link |
and it's one that I think most people would agree with.
link |
It's how we would hope AI progresses.
link |
And so how do you tie yourself to that mission?
link |
How do you make sure you do not deviate from that mission,
link |
that other incentives that are profit driven
link |
don't interfere with the mission?
link |
So this was actually a really core question for us
link |
for the past couple of years,
link |
because I'd say that like the way that our history went
link |
was that for the first year,
link |
we were getting off the ground, right?
link |
We had this high level picture,
link |
but we didn't know exactly how we wanted to accomplish it.
link |
And really two years ago is when we first started realizing
link |
in order to build AGI,
link |
we're just going to need to raise way more money
link |
than we can as a nonprofit.
link |
And we're talking many billions of dollars.
link |
And so the first question is how are you supposed to do that
link |
and stay true to this mission?
link |
And we looked at every legal structure out there
link |
and concluded none of them were quite right
link |
for what we wanted to do.
link |
And I guess it shouldn't be too surprising
link |
if you're gonna do some like crazy unprecedented technology
link |
that you're gonna have to come with
link |
some crazy unprecedented structure to do it in.
link |
And a lot of our conversation was with people at OpenAI,
link |
the people who really joined
link |
because they believe so much in this mission
link |
and thinking about how do we actually
link |
raise the resources to do it
link |
and also stay true to what we stand for.
link |
And the place you gotta start is to really align
link |
on what is it that we stand for, right?
link |
What are those values?
link |
What's really important to us?
link |
And so I'd say that we spent about a year
link |
really compiling the OpenAI charter
link |
and that determines,
link |
and if you even look at the first line item in there,
link |
it says that, look, we expect we're gonna have to marshal
link |
huge amounts of resources,
link |
but we're going to make sure that we minimize
link |
conflict of interest with the mission.
link |
And that kind of aligning on all of those pieces
link |
was the most important step towards figuring out
link |
how do we structure a company
link |
that can actually raise the resources
link |
to do what we need to do.
link |
I imagine OpenAI, the decision to create OpenAI LP
link |
was a really difficult one.
link |
And there was a lot of discussions,
link |
as you mentioned, for a year,
link |
and there was different ideas,
link |
perhaps detractors within OpenAI,
link |
sort of different paths that you could have taken.
link |
What were those concerns?
link |
What were the different paths considered?
link |
What was that process of making that decision like?
link |
Yep, so if you look actually at the OpenAI charter,
link |
there's almost two paths embedded within it.
link |
There is, we are primarily trying to build AGI ourselves,
link |
but we're also okay if someone else does it.
link |
And this is a weird thing for a company.
link |
It's really interesting, actually.
link |
There is an element of competition
link |
that you do wanna be the one that does it,
link |
but at the same time, you're okay if somebody else doesn't.
link |
We'll talk about that a little bit, that trade off,
link |
that dance that's really interesting.
link |
And I think this was the core tension
link |
as we were designing OpenAI LP,
link |
and really the OpenAI strategy,
link |
is how do you make sure that both you have a shot
link |
at being a primary actor,
link |
which really requires building an organization,
link |
raising massive resources,
link |
and really having the will to go
link |
and execute on some really, really hard vision, right?
link |
You need to really sign up for a long period
link |
to go and take on a lot of pain and a lot of risk.
link |
And to do that, normally you just import
link |
the startup mindset, right?
link |
And that you think about, okay,
link |
like how do we out execute everyone?
link |
You have this very competitive angle.
link |
But you also have the second angle of saying that,
link |
well, the true mission isn't for OpenAI to build AGI.
link |
The true mission is for AGI to go well for humanity.
link |
And so how do you take all of those first actions
link |
and make sure you don't close the door on outcomes
link |
that would actually be positive and fulfill the mission?
link |
And so I think it's a very delicate balance, right?
link |
And I think that going 100% one direction or the other
link |
is clearly not the correct answer.
link |
And so I think that even in terms of just how we talk
link |
about OpenAI and think about it,
link |
there's just like one thing that's always in the back
link |
of my mind is to make sure that we're not just saying
link |
OpenAI's goal is to build AGI, right?
link |
That it's actually much broader than that, right?
link |
That first of all, it's not just AGI,
link |
it's safe AGI that's very important.
link |
But secondly, our goal isn't to be the ones to build it.
link |
Our goal is to make sure it goes well for the world.
link |
And so I think that figuring out
link |
how do you balance all of those
link |
and to get people to really come to the table
link |
and compile a single document that encompasses all of that
link |
So part of the challenge here is your mission is,
link |
I would say, beautiful, empowering,
link |
and a beacon of hope for people in the research community
link |
and just people thinking about AI.
link |
So your decisions are scrutinized more than,
link |
I think, a regular profit driven company.
link |
Do you feel the burden of this
link |
in the creation of the charter
link |
and just in the way you operate?
link |
So why do you lean into the burden
link |
by creating such a charter?
link |
Why not keep it quiet?
link |
I mean, it just boils down to the mission, right?
link |
Like I'm here and everyone else is here
link |
because we think this is the most important mission.
link |
All right, so do you think you can be good for the world
link |
or create an AGI system that's good
link |
when you're a for profit company?
link |
From my perspective, I don't understand
link |
why profit interferes with positive impact on society.
link |
I don't understand why Google,
link |
that makes most of its money from ads,
link |
can't also do good for the world
link |
or other companies, Facebook, anything.
link |
I don't understand why those have to interfere.
link |
You know, profit isn't the thing, in my view,
link |
that affects the impact of a company.
link |
What affects the impact of the company is the charter,
link |
is the culture, is the people inside,
link |
and profit is the thing that just fuels those people.
link |
So what are your views there?
link |
Yeah, so I think that's a really good question
link |
and there's some real longstanding debates
link |
in human society that are wrapped up in it.
link |
The way that I think about it is just think about
link |
what are the most impactful non profits in the world?
link |
What are the most impactful for profits in the world?
link |
Right, it's much easier to list the for profits.
link |
That's right, and I think that there's some real truth here
link |
that the system that we set up,
link |
the system for kind of how today's world is organized,
link |
is one that really allows for huge impact.
link |
And that kind of part of that is that you need to be,
link |
that for profits are self sustaining
link |
and able to kind of build on their own momentum.
link |
And I think that's a really powerful thing.
link |
It's something that when it turns out
link |
that we haven't set the guardrails correctly,
link |
causes problems, right?
link |
Think about logging companies that go into forest,
link |
the rainforest, that's really bad, we don't want that.
link |
And it's actually really interesting to me
link |
that kind of this question of how do you get
link |
positive benefits out of a for profit company,
link |
it's actually very similar to how do you get
link |
positive benefits out of an AGI, right?
link |
That you have this like very powerful system,
link |
it's more powerful than any human,
link |
and is kind of autonomous in some ways,
link |
it's superhuman in a lot of axes,
link |
and somehow you have to set the guardrails
link |
to get good things to happen.
link |
But when you do, the benefits are massive.
link |
And so I think that when I think about
link |
nonprofit versus for profit,
link |
I think just not enough happens in nonprofits,
link |
they're very pure, but it's just kind of,
link |
it's just hard to do things there.
link |
In for profits in some ways, like too much happens,
link |
but if kind of shaped in the right way,
link |
it can actually be very positive.
link |
And so with OpenAI LP, we're picking a road in between.
link |
Now the thing that I think is really important to recognize
link |
is that the way that we think about OpenAI LP
link |
is that in the world where AGI actually happens, right,
link |
in a world where we are successful,
link |
we build the most transformative technology ever,
link |
the amount of value we're gonna create will be astronomical.
link |
And so then in that case, that the cap that we have
link |
will be a small fraction of the value we create,
link |
and the amount of value that goes back to investors
link |
and employees looks pretty similar to what would happen
link |
in a pretty successful startup.
link |
And that's really the case that we're optimizing for, right?
link |
That we're thinking about in the success case,
link |
making sure that the value we create doesn't get locked up.
link |
And I expect that in other for profit companies
link |
that it's possible to do something like that.
link |
I think it's not obvious how to do it, right?
link |
I think that as a for profit company,
link |
you have a lot of fiduciary duty to your shareholders
link |
and that there are certain decisions
link |
that you just cannot make.
link |
In our structure, we've set it up
link |
so that we have a fiduciary duty to the charter,
link |
that we always get to make the decision
link |
that is right for the charter rather than,
link |
even if it comes at the expense of our own stakeholders.
link |
And so I think that when I think about
link |
what's really important,
link |
it's not really about nonprofit versus for profit,
link |
it's really a question of if you build AGI
link |
and you kind of, humanity's now in this new age,
link |
who benefits, whose lives are better?
link |
And I think that what's really important
link |
is to have an answer that is everyone.
link |
Yeah, which is one of the core aspects of the charter.
link |
So one concern people have, not just with OpenAI,
link |
but with Google, Facebook, Amazon,
link |
anybody really that's creating impact at scale
link |
is how do we avoid, as your charter says,
link |
avoid enabling the use of AI or AGI
link |
to unduly concentrate power?
link |
Why would not a company like OpenAI
link |
keep all the power of an AGI system to itself?
link |
So how does the charter
link |
actualize itself in day to day?
link |
So I think that first, to zoom out,
link |
that the way that we structure the company
link |
is so that the power for sort of dictating the actions
link |
that OpenAI takes ultimately rests with the board,
link |
the board of the nonprofit.
link |
And the board is set up in certain ways
link |
with certain restrictions that you can read about
link |
in the OpenAI LP blog post.
link |
But effectively the board is the governing body
link |
And the board has a duty to fulfill the mission
link |
And so that's kind of how we tie,
link |
how we thread all these things together.
link |
Now there's a question of, so day to day,
link |
how do people, the individuals,
link |
who in some ways are the most empowered ones, right?
link |
Now the board sort of gets to call the shots
link |
at the high level, but the people
link |
who are actually executing are the employees, right?
link |
People here on a day to day basis
link |
who have the keys to the technical whole kingdom.
link |
And there I think that the answer looks a lot like,
link |
well, how does any company's values get actualized, right?
link |
And I think that a lot of that comes down to
link |
that you need people who are here
link |
because they really believe in that mission
link |
and they believe in the charter
link |
and that they are willing to take actions
link |
that maybe are worse for them,
link |
but are better for the charter.
link |
And that's something that's really baked into the culture.
link |
And honestly, I think it's, you know,
link |
I think that that's one of the things
link |
that we really have to work to preserve as time goes on.
link |
And that's a really important part
link |
of how we think about hiring people
link |
and bringing people into OpenAI.
link |
So there's people here, there's people here
link |
who could speak up and say, like, hold on a second,
link |
this is totally against what we stand for, culture wise.
link |
Yeah, yeah, for sure.
link |
I mean, I think that we actually have,
link |
I think that's like a pretty important part
link |
of how we operate and how we have,
link |
even again with designing the charter
link |
and designing OpenAI LP in the first place,
link |
that there has been a lot of conversation
link |
with employees here and a lot of times
link |
where employees said, wait a second,
link |
this seems like it's going in the wrong direction
link |
and let's talk about it.
link |
And so I think one thing that's I think a really,
link |
and you know, here's actually one thing
link |
that I think is very unique about us as a small company,
link |
is that if you're at a massive tech giant,
link |
that's a little bit hard for someone
link |
who's a line employee to go and talk to the CEO
link |
and say, I think that we're doing this wrong.
link |
And you know, you'll get companies like Google
link |
that have had some collective action from employees
link |
to make ethical change around things like Maven.
link |
And so maybe there are mechanisms
link |
at other companies that work.
link |
But here, super easy for anyone to pull me aside,
link |
to pull Sam aside, to pull Ilya aside,
link |
and people do it all the time.
link |
One of the interesting things in the charter
link |
is this idea that it'd be great
link |
if you could try to describe or untangle
link |
switching from competition to collaboration
link |
in late stage AGI development.
link |
It's really interesting,
link |
this dance between competition and collaboration.
link |
How do you think about that?
link |
Yeah, assuming that you can actually do
link |
the technical side of AGI development,
link |
I think there's going to be two key problems
link |
with figuring out how do you actually deploy it,
link |
The first one of these is the run up
link |
to building the first AGI.
link |
You look at how self driving cars are being developed,
link |
and it's a competitive race.
link |
And the thing that always happens in competitive race
link |
is that you have huge amounts of pressure
link |
to get rid of safety.
link |
And so that's one thing we're very concerned about,
link |
is that people, multiple teams figuring out
link |
we can actually get there,
link |
but if we took the slower path
link |
that is more guaranteed to be safe, we will lose.
link |
And so we're going to take the fast path.
link |
And so the more that we can both ourselves
link |
be in a position where we don't generate
link |
that competitive race, where we say,
link |
if the race is being run and that someone else
link |
is further ahead than we are,
link |
we're not going to try to leapfrog.
link |
We're going to actually work with them, right?
link |
We will help them succeed.
link |
As long as what they're trying to do
link |
is to fulfill our mission, then we're good.
link |
We don't have to build AGI ourselves.
link |
And I think that's a really important commitment from us,
link |
but it can't just be unilateral, right?
link |
I think that it's really important that other players
link |
who are serious about building AGI
link |
make similar commitments, right?
link |
I think that, again, to the extent that everyone believes
link |
that AGI should be something to benefit everyone,
link |
then it actually really shouldn't matter
link |
which company builds it.
link |
And we should all be concerned about the case
link |
where we just race so hard to get there
link |
that something goes wrong.
link |
So what role do you think government,
link |
our favorite entity, has in setting policy and rules
link |
about this domain, from research to the development
link |
to early stage to late stage AI and AGI development?
link |
So I think that, first of all,
link |
it's really important that government's in there, right?
link |
In some way, shape, or form.
link |
At the end of the day, we're talking about
link |
building technology that will shape how the world operates,
link |
and that there needs to be government
link |
as part of that answer.
link |
And so that's why we've done a number
link |
of different congressional testimonies,
link |
we interact with a number of different lawmakers,
link |
and that right now, a lot of our message to them
link |
is that it's not the time for regulation,
link |
it is the time for measurement, right?
link |
That our main policy recommendation is that people,
link |
and the government does this all the time
link |
with bodies like NIST, spend time trying to figure out
link |
just where the technology is, how fast it's moving,
link |
and can really become literate and up to speed
link |
with respect to what to expect.
link |
So I think that today, the answer really
link |
is about measurement, and I think that there will be a time
link |
and place where that will change.
link |
And I think it's a little bit hard to predict
link |
exactly what exactly that trajectory should look like.
link |
So there will be a point at which regulation,
link |
federal in the United States, the government steps in
link |
and helps be the, I don't wanna say the adult in the room,
link |
to make sure that there is strict rules,
link |
maybe conservative rules that nobody can cross.
link |
Well, I think there's kind of maybe two angles to it.
link |
So today, with narrow AI applications
link |
that I think there are already existing bodies
link |
that are responsible and should be responsible
link |
for regulation, you think about, for example,
link |
with self driving cars, that you want the national highway.
link |
Yeah, exactly, to be regulating that.
link |
That makes sense, right, that basically what we're saying
link |
is that we're going to have these technological systems
link |
that are going to be performing applications
link |
that humans already do, great.
link |
We already have ways of thinking about standards
link |
and safety for those.
link |
So I think actually empowering those regulators today
link |
is also pretty important.
link |
And then I think for AGI, that there's going to be a point
link |
where we'll have better answers.
link |
And I think that maybe a similar approach
link |
of first measurement and start thinking about
link |
what the rules should be.
link |
I think it's really important
link |
that we don't prematurely squash progress.
link |
I think it's very easy to kind of smother a budding field.
link |
And I think that's something to really avoid.
link |
But I don't think that the right way of doing it
link |
is to say, let's just try to blaze ahead
link |
and not involve all these other stakeholders.
link |
So you recently released a paper on GPT2 language modeling,
link |
but did not release the full model
link |
because you had concerns about the possible
link |
negative effects of the availability of such model.
link |
It's outside of just that decision,
link |
it's super interesting because of the discussion
link |
at a societal level, the discourse it creates.
link |
So it's fascinating in that aspect.
link |
But if you think that's the specifics here at first,
link |
what are some negative effects that you envisioned?
link |
And of course, what are some of the positive effects?
link |
Yeah, so again, I think to zoom out,
link |
the way that we thought about GPT2
link |
is that with language modeling,
link |
we are clearly on a trajectory right now
link |
where we scale up our models
link |
and we get qualitatively better performance.
link |
GPT2 itself was actually just a scale up
link |
of a model that we've released in the previous June.
link |
We just ran it at much larger scale
link |
and we got these results where
link |
suddenly starting to write coherent pros,
link |
which was not something we'd seen previously.
link |
And what are we doing now?
link |
Well, we're gonna scale up GPT2 by 10x, by 100x, by 1000x,
link |
and we don't know what we're gonna get.
link |
And so it's very clear that the model
link |
that we released last June,
link |
I think it's kind of like, it's a good academic toy.
link |
It's not something that we think is something
link |
that can really have negative applications
link |
or to the extent that it can,
link |
that the positive of people being able to play with it
link |
is far outweighs the possible harms.
link |
You fast forward to not GPT2, but GPT20,
link |
and you think about what that's gonna be like.
link |
And I think that the capabilities are going to be substantive.
link |
And so there needs to be a point in between the two
link |
where you say, this is something
link |
where we are drawing the line
link |
and that we need to start thinking about the safety aspects.
link |
And I think for GPT2, we could have gone either way.
link |
And in fact, when we had conversations internally
link |
that we had a bunch of pros and cons,
link |
and it wasn't clear which one outweighed the other.
link |
And I think that when we announced that,
link |
hey, we decide not to release this model,
link |
then there was a bunch of conversation
link |
where various people said,
link |
it's so obvious that you should have just released it.
link |
There are other people said,
link |
it's so obvious you should not have released it.
link |
And I think that that almost definitionally means
link |
that holding it back was the correct decision.
link |
Right, if it's not obvious
link |
whether something is beneficial or not,
link |
you should probably default to caution.
link |
And so I think that the overall landscape
link |
for how we think about it
link |
is that this decision could have gone either way.
link |
There are great arguments in both directions,
link |
but for future models down the road
link |
and possibly sooner than you'd expect,
link |
because scaling these things up
link |
doesn't actually take that long,
link |
those ones you're definitely not going to want
link |
to release into the wild.
link |
And so I think that we almost view this as a test case
link |
and to see, can we even design,
link |
you know, how do you have a society
link |
or how do you have a system
link |
that goes from having no concept
link |
of responsible disclosure,
link |
where the mere idea of not releasing something
link |
for safety reasons is unfamiliar
link |
to a world where you say, okay, we have a powerful model,
link |
let's at least think about it,
link |
let's go through some process.
link |
And you think about the security community,
link |
it took them a long time
link |
to design responsible disclosure, right?
link |
You know, you think about this question of,
link |
well, I have a security exploit,
link |
I send it to the company,
link |
the company is like, tries to prosecute me
link |
or just sit, just ignores it, what do I do, right?
link |
And so, you know, the alternatives of,
link |
oh, I just always publish your exploits,
link |
that doesn't seem good either, right?
link |
And so it really took a long time
link |
and took this, it was bigger than any individual, right?
link |
It's really about building a whole community
link |
that believe that, okay, we'll have this process
link |
where you send it to the company, you know,
link |
if they don't act in a certain time,
link |
then you can go public and you're not a bad person,
link |
you've done the right thing.
link |
And I think that in AI,
link |
part of the response at GPT2 just proves
link |
that we don't have any concept of this.
link |
So that's the high level picture.
link |
And so I think that,
link |
I think this was a really important move to make
link |
and we could have maybe delayed it for GPT3,
link |
but I'm really glad we did it for GPT2.
link |
And so now you look at GPT2 itself
link |
and you think about the substance of, okay,
link |
what are potential negative applications?
link |
So you have this model that's been trained on the internet,
link |
which, you know, it's also going to be
link |
a bunch of very biased data,
link |
a bunch of, you know, very offensive content in there,
link |
and you can ask it to generate content for you
link |
on basically any topic, right?
link |
You just give it a prompt and it'll just start writing
link |
and it writes content like you see on the internet,
link |
you know, even down to like saying advertisement
link |
in the middle of some of its generations.
link |
And you think about the possibilities
link |
for generating fake news or abusive content.
link |
And, you know, it's interesting seeing
link |
what people have done with, you know,
link |
we released a smaller version of GPT2
link |
and the people have done things like try to generate,
link |
you know, take my own Facebook message history
link |
and generate more Facebook messages like me
link |
and people generating fake politician content
link |
or, you know, there's a bunch of things there
link |
where you at least have to think,
link |
is this going to be good for the world?
link |
There's the flip side, which is I think
link |
that there's a lot of awesome applications
link |
that we really want to see,
link |
like creative applications in terms of
link |
if you have sci fi authors that can work with this tool
link |
and come up with cool ideas, like that seems awesome
link |
if we can write better sci fi through the use of these tools
link |
and we've actually had a bunch of people write into us
link |
asking, hey, can we use it for, you know,
link |
a variety of different creative applications?
link |
So the positive are actually pretty easy to imagine.
link |
They're, you know, the usual NLP applications
link |
are really interesting, but let's go there.
link |
It's kind of interesting to think about a world
link |
where, look at Twitter, where not just fake news,
link |
but smarter and smarter bots being able to spread
link |
in an interesting, complex, networking way information
link |
that just floods out us regular human beings
link |
with our original thoughts.
link |
So what are your views of this world with GPT20, right?
link |
How do we think about it?
link |
Again, it's like one of those things about in the 50s
link |
trying to describe the internet or the smartphone.
link |
What do you think about that world,
link |
the nature of information?
link |
One possibility is that we'll always try to design systems
link |
that identify robot versus human
link |
and we'll do so successfully and so we'll authenticate
link |
that we're still human and the other world is that
link |
we just accept the fact that we're swimming in a sea
link |
of fake news and just learn to swim there.
link |
Well, have you ever seen the popular meme of robot
link |
with a physical arm and pen clicking the
link |
I'm not a robot button?
link |
I think the truth is that really trying to distinguish
link |
between robot and human is a losing battle.
link |
Ultimately, you think it's a losing battle?
link |
I think it's a losing battle ultimately, right?
link |
I think that that is, in terms of the content,
link |
in terms of the actions that you can take.
link |
I mean, think about how captures have gone, right?
link |
The captures used to be a very nice, simple,
link |
you just have this image, all of our OCR is terrible,
link |
you put a couple of artifacts in it,
link |
humans are gonna be able to tell what it is.
link |
An AI system wouldn't be able to.
link |
Today, I could barely do captures.
link |
And I think that this is just kind of where we're going.
link |
I think captures were a moment in time thing
link |
and as AI systems become more powerful,
link |
that there being human capabilities that can be measured
link |
in a very easy, automated way that AIs
link |
will not be capable of.
link |
I think that's just like,
link |
it's just an increasingly hard technical battle.
link |
But it's not that all hope is lost, right?
link |
You think about how do we already authenticate ourselves,
link |
right, that we have systems, we have social security numbers
link |
if you're in the US or you have ways of identifying
link |
individual people and having real world identity
link |
tied to digital identity seems like a step
link |
towards authenticating the source of content
link |
rather than the content itself.
link |
Now, there are problems with that.
link |
How can you have privacy and anonymity
link |
in a world where the only content you can really trust is,
link |
or the only way you can trust content
link |
is by looking at where it comes from?
link |
And so I think that building out good reputation networks
link |
may be one possible solution.
link |
But yeah, I think that this question is not an obvious one.
link |
And I think that we, maybe sooner than we think,
link |
will be in a world where today I often will read a tweet
link |
and be like, hmm, do I feel like a real human wrote this?
link |
Or do I feel like this is genuine?
link |
I feel like I can kind of judge the content a little bit.
link |
And I think in the future, it just won't be the case.
link |
You look at, for example, the FCC comments on net neutrality.
link |
It came out later that millions of those were auto generated
link |
and that the researchers were able to do
link |
various statistical techniques to do that.
link |
What do you do in a world
link |
where those statistical techniques don't exist?
link |
It's just impossible to tell the difference
link |
between humans and AIs.
link |
And in fact, the most persuasive arguments
link |
are written by AI.
link |
All that stuff, it's not sci fi anymore.
link |
You look at GPT2 making a great argument
link |
for why recycling is bad for the world.
link |
You gotta read that and be like, huh, you're right.
link |
We are addressing just the symptoms.
link |
Yeah, that's quite interesting.
link |
I mean, ultimately it boils down to the physical world
link |
being the last frontier of proving,
link |
so you said like basically networks of people,
link |
humans vouching for humans in the physical world.
link |
And somehow the authentication ends there.
link |
I mean, if I had to ask you,
link |
I mean, you're way too eloquent for a human.
link |
So if I had to ask you to authenticate,
link |
like prove how do I know you're not a robot
link |
and how do you know I'm not a robot?
link |
I think that's so far where in this space,
link |
this conversation we just had,
link |
the physical movements we did,
link |
is the biggest gap between us and AI systems
link |
is the physical manipulation.
link |
So maybe that's the last frontier.
link |
Well, here's another question is why is,
link |
why is solving this problem important, right?
link |
Like what aspects are really important to us?
link |
And I think that probably where we'll end up
link |
is we'll hone in on what do we really want
link |
out of knowing if we're talking to a human.
link |
And I think that, again, this comes down to identity.
link |
And so I think that the internet of the future,
link |
I expect to be one that will have lots of agents out there
link |
that will interact with you.
link |
But I think that the question of is this
link |
flesh, real flesh and blood human
link |
or is this an automated system,
link |
may actually just be less important.
link |
Let's actually go there.
link |
It's GPT2 is impressive and let's look at GPT20.
link |
Why is it so bad that all my friends are GPT20?
link |
Why is it so important on the internet,
link |
do you think, to interact with only human beings?
link |
Why can't we live in a world where ideas can come
link |
from models trained on human data?
link |
Yeah, I think this is actually
link |
a really interesting question.
link |
This comes back to the how do you even picture a world
link |
with some new technology?
link |
And I think that one thing that I think is important
link |
is, you know, let's say honesty.
link |
And I think that if you have almost in the Turing test
link |
style sense of technology, you have AIs that are pretending
link |
to be humans and deceiving you.
link |
I think that feels like a bad thing, right?
link |
I think that it's really important that we feel like
link |
we're in control of our environment, right?
link |
That we understand who we're interacting with.
link |
And if it's an AI or a human, that's not something
link |
that we're being deceived about.
link |
But I think that the flip side of can I have as meaningful
link |
of an interaction with an AI as I can with a human?
link |
Well, I actually think here you can turn to sci fi.
link |
And her I think is a great example of asking
link |
this very question, right?
link |
One thing I really love about her is it really starts out
link |
almost by asking how meaningful
link |
are human virtual relationships, right?
link |
And then you have a human who has a relationship with an AI
link |
and that you really start to be drawn into that, right?
link |
That all of your emotional buttons get triggered
link |
in the same way as if there was a real human
link |
that was on the other side of that phone.
link |
And so I think that this is one way of thinking about it
link |
is that I think that we can have meaningful interactions
link |
and that if there's a funny joke,
link |
some sense it doesn't really matter
link |
if it was written by a human or an AI.
link |
But what you don't want and why I think
link |
we should really draw hard lines is deception.
link |
And I think that as long as we're in a world
link |
where why do we build AI systems at all, right?
link |
The reason we want to build them is to enhance human lives,
link |
to make humans be able to do more things,
link |
to have humans feel more fulfilled.
link |
And if we can build AI systems that do that, sign me up.
link |
So the process of language modeling,
link |
how far do you think it'd take us?
link |
Let's look at movie Her.
link |
Do you think a dialogue, natural language conversation
link |
is formulated by the Turing test, for example,
link |
do you think that process could be achieved
link |
through this kind of unsupervised language modeling?
link |
So I think the Turing test in its real form
link |
isn't just about language, right?
link |
It's really about reasoning too, right?
link |
To really pass the Turing test,
link |
I should be able to teach calculus
link |
to whoever's on the other side
link |
and have it really understand calculus
link |
and be able to go and solve new calculus problems.
link |
And so I think that to really solve the Turing test,
link |
we need more than what we're seeing with language models.
link |
We need some way of plugging in reasoning.
link |
Now, how different will that be from what we already do?
link |
That's an open question, right?
link |
Might be that we need some sequence
link |
of totally radical new ideas,
link |
or it might be that we just need to kind of shape
link |
our existing systems in a slightly different way.
link |
But I think that in terms of how far language modeling
link |
will go, it's already gone way further
link |
than many people would have expected, right?
link |
I think that things like,
link |
and I think there's a lot of really interesting angles
link |
to poke in terms of how much does GPT2
link |
understand physical world?
link |
Like, you read a little bit about fire underwater in GPT2.
link |
So it's like, okay, maybe it doesn't quite understand
link |
what these things are, but at the same time,
link |
I think that you also see various things
link |
like smoke coming from flame,
link |
and a bunch of these things that GPT2,
link |
it has no body, it has no physical experience,
link |
it's just statically read data.
link |
And I think that the answer is like, we don't know yet.
link |
These questions, though, we're starting to be able
link |
to actually ask them to physical systems,
link |
to real systems that exist, and that's very exciting.
link |
Do you think, what's your intuition?
link |
Do you think if you just scale language modeling,
link |
like significantly scale,
link |
that reasoning can emerge from the same exact mechanisms?
link |
I think it's unlikely that if we just scale GPT2
link |
that we'll have reasoning in the full fledged way.
link |
And I think that there's like,
link |
the type signature's a little bit wrong, right?
link |
That like, there's something we do with,
link |
that we call thinking, right?
link |
Where we spend a lot of compute,
link |
like a variable amount of compute,
link |
to get to better answers, right?
link |
I think a little bit harder, I get a better answer.
link |
And that that kind of type signature
link |
isn't quite encoded in a GPT, right?
link |
GPT will kind of like, it's been a long time,
link |
and it's like evolutionary history,
link |
baking in all this information,
link |
getting very, very good at this predictive process.
link |
And then at runtime, I just kind of do one forward pass,
link |
and I'm able to generate stuff.
link |
And so, you know, there might be small tweaks
link |
to what we do in order to get the type signature, right?
link |
For example, well, you know,
link |
it's not really one forward pass, right?
link |
You know, you generate symbol by symbol,
link |
and so maybe you generate like a whole sequence
link |
of thoughts, and you only keep like the last bit
link |
But I think that at the very least,
link |
I would expect you have to make changes like that.
link |
Yeah, just exactly how we, you said, think,
link |
is the process of generating thought by thought
link |
in the same kind of way, like you said,
link |
keep the last bit, the thing that we converge towards.
link |
And I think there's another piece which is interesting,
link |
which is this out of distribution generalization, right?
link |
That like thinking somehow lets us do that, right?
link |
That we haven't experienced a thing, and yet somehow
link |
we just kind of keep refining our mental model of it.
link |
This is, again, something that feels tied
link |
to whatever reasoning is, and maybe it's a small tweak
link |
to what we do, maybe it's many ideas,
link |
and we'll take as many decades.
link |
Yeah, so the assumption there,
link |
generalization out of distribution,
link |
is that it's possible to create new ideas.
link |
You know, it's possible that nobody's ever created
link |
any new ideas, and then with scaling GPT2 to GPT20,
link |
you would essentially generalize to all possible thoughts
link |
that us humans could have.
link |
Just to play devil's advocate.
link |
Right, right, right, I mean, how many new story ideas
link |
have we come up with since Shakespeare, right?
link |
It's just all different forms of love and drama and so on.
link |
Not sure if you read Bitter Lesson,
link |
a recent blog post by Rich Sutton.
link |
He basically says something that echoes some of the ideas
link |
that you've been talking about, which is,
link |
he says the biggest lesson that can be read
link |
from 70 years of AI research is that general methods
link |
that leverage computation are ultimately going to,
link |
ultimately win out.
link |
Do you agree with this?
link |
So basically, and OpenAI in general,
link |
but the ideas you're exploring about coming up with methods,
link |
whether it's GPT2 modeling or whether it's OpenAI 5
link |
playing Dota, or a general method is better
link |
than a more fine tuned, expert tuned method.
link |
Yeah, so I think that, well one thing that I think
link |
was really interesting about the reaction
link |
to that blog post was that a lot of people have read this
link |
as saying that compute is all that matters.
link |
And that's a very threatening idea, right?
link |
And I don't think it's a true idea either.
link |
Right, it's very clear that we have algorithmic ideas
link |
that have been very important for making progress
link |
and to really build AGI.
link |
You wanna push as far as you can on the computational scale
link |
and you wanna push as far as you can on human ingenuity.
link |
And so I think you need both.
link |
But I think the way that you phrased the question
link |
is actually very good, right?
link |
That it's really about what kind of ideas
link |
should we be striving for?
link |
And absolutely, if you can find a scalable idea,
link |
you pour more compute into it, you pour more data into it,
link |
it gets better, like that's the real holy grail.
link |
And so I think that the answer to the question,
link |
I think, is yes, that that's really how we think about it
link |
and that part of why we're excited about the power
link |
of deep learning, the potential for building AGI
link |
is because we look at the systems that exist
link |
in the most successful AI systems
link |
and we realize that you scale those up,
link |
they're gonna work better.
link |
And I think that that scalability
link |
is something that really gives us hope
link |
for being able to build transformative systems.
link |
So I'll tell you, this is partially an emotional,
link |
a response that people often have,
link |
if compute is so important for state of the art performance,
link |
individual developers, maybe a 13 year old
link |
sitting somewhere in Kansas or something like that,
link |
they're sitting, they might not even have a GPU
link |
or may have a single GPU, a 1080 or something like that,
link |
and there's this feeling like, well,
link |
how can I possibly compete or contribute
link |
to this world of AI if scale is so important?
link |
So if you can comment on that and in general,
link |
do you think we need to also in the future
link |
focus on democratizing compute resources more
link |
or as much as we democratize the algorithms?
link |
Well, so the way that I think about it
link |
is that there's this space of possible progress, right?
link |
There's a space of ideas and sort of systems
link |
that will work that will move us forward
link |
and there's a portion of that space
link |
and to some extent, an increasingly significant portion
link |
of that space that does just require
link |
massive compute resources.
link |
And for that, I think that the answer is kind of clear
link |
and that part of why we have the structure that we do
link |
is because we think it's really important
link |
to be pushing the scale and to be building
link |
these large clusters and systems.
link |
But there's another portion of the space
link |
that isn't about the large scale compute
link |
that are these ideas that, and again,
link |
I think that for the ideas to really be impactful
link |
and really shine, that they should be ideas
link |
that if you scale them up, would work way better
link |
than they do at small scale.
link |
But that you can discover them
link |
without massive computational resources.
link |
And if you look at the history of recent developments,
link |
you think about things like the GAN or the VAE,
link |
that these are ones that I think you could come up with them
link |
without having, and in practice,
link |
people did come up with them without having
link |
massive, massive computational resources.
link |
Right, I just talked to Ian Goodfellow,
link |
but the thing is the initial GAN
link |
produced pretty terrible results, right?
link |
So only because it was in a very specific,
link |
it was only because they're smart enough
link |
to know that this is quite surprising
link |
it can generate anything that they know.
link |
Do you see a world, or is that too optimistic and dreamer
link |
like to imagine that the compute resources
link |
are something that's owned by governments
link |
and provided as utility?
link |
Actually, to some extent, this question reminds me
link |
of a blog post from one of my former professors at Harvard,
link |
this guy Matt Welsh, who was a systems professor.
link |
I remember sitting in his tenure talk, right,
link |
and that he had literally just gotten tenure.
link |
He went to Google for the summer
link |
and then decided he wasn't going back to academia, right?
link |
And kind of in his blog post, he makes this point that,
link |
look, as a systems researcher,
link |
that I come up with these cool system ideas, right,
link |
and I kind of build a little proof of concept,
link |
and the best thing I can hope for
link |
is that the people at Google or Yahoo,
link |
which was around at the time,
link |
will implement it and actually make it work at scale, right?
link |
That's like the dream for me, right?
link |
I build the little thing,
link |
and they turn it into the big thing that's actually working.
link |
And for him, he said, I'm done with that.
link |
I want to be the person who's actually doing building
link |
And I think that there's a similar dichotomy here, right?
link |
I think that there are people who really actually find value,
link |
and I think it is a valuable thing to do
link |
to be the person who produces those ideas, right,
link |
who builds the proof of concept.
link |
And yeah, you don't get to generate
link |
the coolest possible GAN images,
link |
but you invented the GAN, right?
link |
And so there's a real trade off there,
link |
and I think that that's a very personal choice,
link |
but I think there's value in both sides.
link |
So do you think creating AGI or some new models,
link |
we would see echoes of the brilliance
link |
even at the prototype level?
link |
So you would be able to develop those ideas without scale,
link |
the initial seeds.
link |
So take a look at, you know,
link |
I always like to look at examples that exist, right?
link |
Look at real precedent.
link |
And so take a look at the June 2018 model that we released,
link |
that we scaled up to turn into GPT2.
link |
And you can see that at small scale,
link |
it set some records, right?
link |
This was the original GPT.
link |
We actually had some cool generations.
link |
They weren't nearly as amazing and really stunning
link |
as the GPT2 ones, but it was promising.
link |
It was interesting.
link |
And so I think it is the case
link |
that with a lot of these ideas,
link |
that you see promise at small scale.
link |
But there is an asterisk here, a very big asterisk,
link |
which is sometimes we see behaviors that emerge
link |
that are qualitatively different
link |
from anything we saw at small scale.
link |
And that the original inventor of whatever algorithm
link |
looks at and says, I didn't think it could do that.
link |
This is what we saw in Dota, right?
link |
So PPO was created by John Shulman,
link |
who's a researcher here.
link |
And with Dota, we basically just ran PPO
link |
at massive, massive scale.
link |
And there's some tweaks in order to make it work,
link |
but fundamentally, it's PPO at the core.
link |
And we were able to get this long term planning,
link |
these behaviors to really play out on a time scale
link |
that we just thought was not possible.
link |
And John looked at that and was like,
link |
I didn't think it could do that.
link |
That's what happens when you're at three orders
link |
of magnitude more scale than you tested at.
link |
Yeah, but it still has the same flavors of,
link |
you know, at least echoes of the expected billions.
link |
Although I suspect with GPT scaled more and more,
link |
you might get surprising things.
link |
So yeah, you're right, it's interesting.
link |
It's difficult to see how far an idea will go
link |
It's an open question.
link |
Well, so to that point with Dota and PPO,
link |
like, I mean, here's a very concrete one, right?
link |
It's like, it's actually one thing
link |
that's very surprising about Dota
link |
that I think people don't really pay that much attention to
link |
is the decree of generalization
link |
out of distribution that happens, right?
link |
That you have this AI that's trained against other bots
link |
for its entirety, the entirety of its existence.
link |
Sorry to take a step back.
link |
Can you talk through, you know, a story of Dota,
link |
a story of leading up to opening I5 and that past,
link |
and what was the process of self play
link |
and so on of training on this?
link |
Yeah, Dota is a complex video game
link |
and we started trying to solve Dota
link |
because we felt like this was a step towards the real world
link |
relative to other games like chess or Go, right?
link |
Those very cerebral games
link |
where you just kind of have this board,
link |
very discreet moves.
link |
Dota starts to be much more continuous time
link |
that you have this huge variety of different actions
link |
that you have a 45 minute game
link |
with all these different units
link |
and it's got a lot of messiness to it
link |
that really hasn't been captured by previous games.
link |
And famously, all of the hard coded bots for Dota
link |
were terrible, right?
link |
It's just impossible to write anything good for it
link |
because it's so complex.
link |
And so this seemed like a really good place
link |
to push what's the state of the art
link |
in reinforcement learning.
link |
And so we started by focusing
link |
on the one versus one version of the game
link |
and we're able to solve that.
link |
We're able to beat the world champions
link |
and the skill curve was this crazy exponential, right?
link |
And it was like constantly we were just scaling up
link |
that we were fixing bugs
link |
and that you look at the skill curve
link |
and it was really a very, very smooth one.
link |
This is actually really interesting
link |
to see how that human iteration loop
link |
yielded very steady exponential progress.
link |
And to one side note, first of all,
link |
it's an exceptionally popular video game.
link |
The side effect is that there's a lot of incredible
link |
human experts at that video game.
link |
So the benchmark that you're trying to reach is very high.
link |
And the other, can you talk about the approach
link |
that was used initially and throughout
link |
training these agents to play this game?
link |
Yep, and so the approach that we used is self play.
link |
And so you have two agents that don't know anything.
link |
They battle each other,
link |
they discover something a little bit good
link |
and now they both know it.
link |
And they just get better and better and better
link |
And that's a really powerful idea, right?
link |
That we then went from the one versus one version
link |
of the game and scaled up to five versus five, right?
link |
So you think about kind of like with basketball
link |
where you have this like team sport
link |
and you need to do all this coordination
link |
and we were able to push the same idea,
link |
the same self play to really get to the professional level
link |
at the full five versus five version of the game.
link |
And the things I think are really interesting here
link |
is that these agents, in some ways,
link |
they're almost like an insect like intelligence, right?
link |
Where they have a lot in common
link |
with how an insect is trained, right?
link |
An insect kind of lives in this environment
link |
for a very long time or the ancestors of this insect
link |
have been around for a long time
link |
and had a lot of experience that gets baked into this agent.
link |
And it's not really smart in the sense of a human, right?
link |
It's not able to go and learn calculus,
link |
but it's able to navigate its environment extremely well.
link |
And it's able to handle unexpected things
link |
in the environment that it's never seen before pretty well.
link |
And we see the same sort of thing with our Dota bots, right?
link |
That they're able to, within this game,
link |
they're able to play against humans,
link |
which is something that never existed
link |
in its evolutionary environment,
link |
totally different play styles from humans versus the bots.
link |
And yet it's able to handle it extremely well.
link |
And that's something that I think was very surprising to us,
link |
was something that doesn't really emerge
link |
from what we've seen with PPO at smaller scale, right?
link |
And the kind of scale we're running this stuff at was,
link |
I could say like 100,000 CPU cores
link |
running with like hundreds of GPUs.
link |
It was probably about something like hundreds
link |
of years of experience going into this bot
link |
every single real day.
link |
And so that scale is massive
link |
and we start to see very different kinds of behaviors
link |
out of the algorithms that we all know and love.
link |
Dota, you mentioned, beat the world expert one v one.
link |
And then you weren't able to win five v five this year.
link |
At the best players in the world.
link |
So what's the comeback story?
link |
First of all, talk through that.
link |
That was an exceptionally exciting event.
link |
And what's the following months and this year look like?
link |
Yeah, yeah, so one thing that's interesting
link |
is that we lose all the time.
link |
The Dota team at OpenAI.
link |
We play the bot against better players
link |
than our system all the time.
link |
Or at least we used to, right?
link |
Like the first time we lost publicly
link |
was we went up on stage at the international
link |
and we played against some of the best teams in the world
link |
and we ended up losing both games,
link |
but we gave them a run for their money, right?
link |
That both games were kind of 30 minutes, 25 minutes
link |
and they went back and forth, back and forth,
link |
And so I think that really shows
link |
that we're at the professional level
link |
and that kind of looking at those games,
link |
we think that the coin could have gone a different direction
link |
and we could have had some wins.
link |
That was actually very encouraging for us.
link |
And it's interesting because the international
link |
was at a fixed time, right?
link |
So we knew exactly what day we were going to be playing
link |
and we pushed as far as we could, as fast as we could.
link |
Two weeks later, we had a bot that had an 80% win rate
link |
versus the one that played at TI.
link |
So the march of progress, you should think of it
link |
as a snapshot rather than as an end state.
link |
And so in fact, we'll be announcing our finals pretty soon.
link |
I actually think that we'll announce our final match
link |
prior to this podcast being released.
link |
So we'll be playing against the world champions.
link |
And for us, it's really less about,
link |
like the way that we think about what's upcoming
link |
is the final milestone, the final competitive milestone
link |
for the project, right?
link |
That our goal in all of this
link |
isn't really about beating humans at Dota.
link |
Our goal is to push the state of the art
link |
in reinforcement learning.
link |
And we've done that, right?
link |
And we've actually learned a lot from our system
link |
and that we have, I think, a lot of exciting next steps
link |
that we want to take.
link |
And so kind of as a final showcase of what we built,
link |
we're going to do this match.
link |
But for us, it's not really the success or failure
link |
to see do we have the coin flip go in our direction
link |
Where do you see the field of deep learning
link |
heading in the next few years?
link |
Where do you see the work and reinforcement learning
link |
perhaps heading, and more specifically with OpenAI,
link |
all the exciting projects that you're working on,
link |
what does 2019 hold for you?
link |
I will put an asterisk on that and just say,
link |
I think that it's about ideas plus scale.
link |
So that's a really good point.
link |
So the question, in terms of ideas,
link |
you have a lot of projects
link |
that are exploring different areas of intelligence.
link |
And the question is, when you think of scale,
link |
do you think about growing the scale
link |
of those individual projects
link |
or do you think about adding new projects?
link |
And sorry to, and if you're thinking about
link |
adding new projects, or if you look at the past,
link |
what's the process of coming up with new projects
link |
So we really have a life cycle of project here.
link |
So we start with a few people
link |
just working on a small scale idea.
link |
And language is actually a very good example of this.
link |
That it was really one person here
link |
who was pushing on language for a long time.
link |
I mean, then you get signs of life, right?
link |
And so this is like, let's say,
link |
with the original GPT, we had something that was interesting
link |
and we said, okay, it's time to scale this, right?
link |
It's time to put more people on it,
link |
put more computational resources behind it.
link |
And then we just kind of keep pushing and keep pushing.
link |
And the end state is something
link |
that looks like Dota or robotics,
link |
where you have a large team of 10 or 15 people
link |
that are running things at very large scale
link |
and that you're able to really have material engineering
link |
and sort of machine learning science coming together
link |
to make systems that work and get material results
link |
that just would have been impossible otherwise.
link |
So we do that whole life cycle.
link |
We've done it a number of times, typically end to end.
link |
It's probably two years or so to do it.
link |
The organization has been around for three years,
link |
so maybe we'll find that we also have
link |
longer life cycle projects, but we'll work up to those.
link |
So one team that we were actually just starting,
link |
Ilya and I are kicking off a new team
link |
called the Reasoning Team,
link |
and that this is to really try to tackle
link |
how do you get neural networks to reason?
link |
And we think that this will be a long term project.
link |
It's one that we're very excited about.
link |
In terms of reasoning, super exciting topic,
link |
what kind of benchmarks, what kind of tests of reasoning
link |
What would, if you sat back with whatever drink
link |
and you would be impressed that this system
link |
is able to do something, what would that look like?
link |
So some kind of logic, and especially mathematical logic.
link |
I think that there's other problems that are dual
link |
to theorem proving in particular.
link |
You think about programming, you think about
link |
even security analysis of code,
link |
that these all kind of capture the same sorts
link |
of core reasoning and being able to do
link |
some out of distribution generalization.
link |
So it would be quite exciting if OpenAI Reasoning Team
link |
was able to prove that P equals NP.
link |
That would be very nice.
link |
It would be very, very, very exciting, especially.
link |
If it turns out that P equals NP,
link |
that'll be interesting too.
link |
It would be ironic and humorous.
link |
So what problem stands out to you
link |
as the most exciting and challenging and impactful
link |
to the work for us as a community in general
link |
and for OpenAI this year?
link |
You mentioned reasoning.
link |
I think that's a heck of a problem.
link |
Yeah, so I think reasoning's an important one.
link |
I think it's gonna be hard to get good results in 2019.
link |
Again, just like we think about the life cycle, takes time.
link |
I think for 2019, language modeling seems to be
link |
kind of on that ramp.
link |
It's at the point that we have a technique that works.
link |
We wanna scale 100x, 1,000x, see what happens.
link |
Do you think we're living in a simulation?
link |
I think it's hard to have a real opinion about it.
link |
It's actually interesting.
link |
I separate out things that I think can have like,
link |
yield materially different predictions about the world
link |
from ones that are just kind of fun to speculate about.
link |
I kind of view simulation as more like,
link |
is there a flying teapot between Mars and Jupiter?
link |
Like, maybe, but it's a little bit hard to know
link |
what that would mean for my life.
link |
So there is something actionable.
link |
So some of the best work OpenAI has done
link |
is in the field of reinforcement learning.
link |
And some of the success of reinforcement learning
link |
come from being able to simulate
link |
the problem you're trying to solve.
link |
So do you have a hope for reinforcement,
link |
for the future of reinforcement learning
link |
and for the future of simulation?
link |
Like whether it's, we're talking about autonomous vehicles
link |
or any kind of system.
link |
Do you see that scaling to where we'll be able
link |
to simulate systems and hence,
link |
be able to create a simulator that echoes our real world
link |
and proving once and for all,
link |
even though you're denying it,
link |
that we're living in a simulation?
link |
I feel like it's two separate questions, right?
link |
So kind of at the core there of like,
link |
can we use simulation for self driving cars?
link |
Take a look at our robotic system, Dactyl, right?
link |
That was trained in simulation using the Dota system,
link |
in fact, and it transfers to a physical robot.
link |
And I think everyone looks at our Dota system,
link |
they're like, okay, it's just a game.
link |
How are you ever gonna escape to the real world?
link |
And the answer is, well, we did it with a physical robot
link |
that no one could program.
link |
And so I think the answer is simulation
link |
goes a lot further than you think
link |
if you apply the right techniques to it.
link |
Now, there's a question of,
link |
are the beings in that simulation gonna wake up
link |
and have consciousness?
link |
I think that one seems a lot harder to, again,
link |
I think that you really should think about
link |
where exactly does human consciousness come from
link |
in our own self awareness?
link |
And is it just that once you have a complicated enough
link |
neural net, you have to worry about
link |
the agents feeling pain?
link |
And I think there's interesting speculation to do there,
link |
but again, I think it's a little bit hard to know for sure.
link |
Well, let me just keep with the speculation.
link |
Do you think to create intelligence, general intelligence,
link |
you need, one, consciousness, and two, a body?
link |
Do you think any of those elements are needed,
link |
or is intelligence something that's orthogonal to those?
link |
I'll stick to the non grand answer first, right?
link |
So the non grand answer is just to look at,
link |
what are we already making work?
link |
You look at GPT2, a lot of people would have said
link |
that to even get these kinds of results,
link |
you need real world experience.
link |
You need a body, you need grounding.
link |
How are you supposed to reason about any of these things?
link |
How are you supposed to like even kind of know
link |
about smoke and fire and those things
link |
if you've never experienced them?
link |
And GPT2 shows that you can actually go way further
link |
than that kind of reasoning would predict.
link |
So I think that in terms of, do we need consciousness?
link |
Do we need a body?
link |
It seems the answer is probably not, right?
link |
That we could probably just continue to push
link |
kind of the systems we have.
link |
They already feel general.
link |
They're not as competent or as general
link |
or able to learn as quickly as an AGI would,
link |
but they're at least like kind of proto AGI in some way,
link |
and they don't need any of those things.
link |
Now let's move to the grand answer,
link |
which is, are our neural nets conscious already?
link |
Would we ever know?
link |
How can we tell, right?
link |
And here's where the speculation starts to become
link |
at least interesting or fun
link |
and maybe a little bit disturbing
link |
depending on where you take it.
link |
But it certainly seems that when we think about animals,
link |
that there's some continuum of consciousness.
link |
You know, my cat I think is conscious in some way, right?
link |
Not as conscious as a human.
link |
And you could imagine that you could build
link |
a little consciousness meter, right?
link |
You point at a cat, it gives you a little reading.
link |
Point at a human, it gives you much bigger reading.
link |
What would happen if you pointed one of those
link |
at a donor neural net?
link |
And if you're training in this massive simulation,
link |
do the neural nets feel pain?
link |
You know, it becomes pretty hard to know
link |
that the answer is no.
link |
And it becomes pretty hard to really think about
link |
what that would mean if the answer were yes.
link |
And it's very possible, you know, for example,
link |
you could imagine that maybe the reason
link |
that humans have consciousness
link |
is because it's a convenient computational shortcut, right?
link |
If you think about it, if you have a being
link |
that wants to avoid pain,
link |
which seems pretty important to survive in this environment
link |
and wants to like, you know, eat food,
link |
then that maybe the best way of doing it
link |
is to have a being that's conscious, right?
link |
That, you know, in order to succeed in the environment,
link |
you need to have those properties
link |
and how are you supposed to implement them
link |
and maybe this consciousness's way of doing that.
link |
If that's true, then actually maybe we should expect
link |
that really competent reinforcement learning agents
link |
will also have consciousness.
link |
But you know, that's a big if.
link |
And I think there are a lot of other arguments
link |
they can make in other directions.
link |
I think that's a really interesting idea
link |
that even GPT2 has some degree of consciousness.
link |
That's something, it's actually not as crazy
link |
to think about, it's useful to think about
link |
as we think about what it means
link |
to create intelligence of a dog, intelligence of a cat,
link |
and the intelligence of a human.
link |
So last question, do you think
link |
we will ever fall in love, like in the movie Her,
link |
with an artificial intelligence system
link |
or an artificial intelligence system
link |
falling in love with a human?
link |
If there's any better way to end it is on love.
link |
So Greg, thanks so much for talking today.
link |
Thank you for having me.