back to indexGreg Brockman: OpenAI and AGI | Lex Fridman Podcast #17
link |
The following is a conversation with Greg Brockman.
link |
He's the cofounder and CTO of OpenAI,
link |
a world class research organization
link |
developing ideas in AI with a goal of eventually
link |
creating a safe and friendly artificial general
link |
intelligence, one that benefits and empowers humanity.
link |
OpenAI is not only a source of publications, algorithms,
link |
tools, and data sets.
link |
Their mission is a catalyst for an important public discourse
link |
about our future with both narrow and general intelligence
link |
This conversation is part of the Artificial Intelligence
link |
Podcast at MIT and beyond.
link |
If you enjoy it, subscribe on YouTube, iTunes,
link |
or simply connect with me on Twitter
link |
at Lex Friedman, spelled F R I D.
link |
And now here's my conversation with Greg Brockman.
link |
So in high school and right after you wrote
link |
a draft of a chemistry textbook,
link |
I saw that that covers everything
link |
from basic structure of the atom to quantum mechanics.
link |
So it's clear you have an intuition and a passion
link |
for both the physical world with chemistry and non robotics
link |
to the digital world with AI, deep learning,
link |
reinforcement learning, so on.
link |
Do you see the physical world and the digital world
link |
And what do you think is the gap?
link |
A lot of it actually boils down to iteration speed,
link |
that I think that a lot of what really motivates me
link |
is building things, right?
link |
Think about mathematics, for example,
link |
where you think really hard about a problem.
link |
You understand it.
link |
You're right down in this very obscure form
link |
that we call a proof.
link |
But then this is in humanity's library, right?
link |
It's there forever.
link |
This is some truth that we've discovered.
link |
And maybe only five people in your field will ever read it.
link |
But somehow you've kind of moved humanity forward.
link |
And so I actually used to really think
link |
that I was going to be a mathematician.
link |
And then I actually started writing this chemistry textbook.
link |
One of my friends told me, you'll never publish it
link |
because you don't have a PhD.
link |
So instead, I decided to build a website
link |
and try to promote my ideas that way.
link |
And then I discovered programming.
link |
And in programming, you think hard about a problem.
link |
You understand it.
link |
You're right down in a very obscure form
link |
that we call a program.
link |
But then once again, it's in humanity's library, right?
link |
And anyone can get the benefit from it.
link |
And the scalability is massive.
link |
And so I think that the thing that really appeals to me
link |
about the digital world is that you
link |
can have this insane leverage, right?
link |
A single individual with an idea is able to affect
link |
the entire planet.
link |
And that's something I think is really hard to do
link |
if you're moving around physical atoms.
link |
But you said mathematics.
link |
So if you look at the wet thing over here, our mind,
link |
do you ultimately see it as just math,
link |
as just information processing?
link |
Or is there some other magic if you've
link |
seen through biology and chemistry and so on?
link |
I think it's really interesting to think about humans
link |
as just information processing systems.
link |
And it seems like it's actually a pretty good way
link |
of describing a lot of how the world works or a lot of what
link |
we're capable of to think that, again, if you just
link |
look at technological innovations over time,
link |
that in some ways, the most transformative innovation
link |
that we've had has been the computer, right?
link |
In some ways, the internet, what has the internet done?
link |
The internet is not about these physical cables.
link |
It's about the fact that I am suddenly
link |
able to instantly communicate with any other human
link |
I'm able to retrieve any piece of knowledge
link |
that, in some ways, the human race has ever had,
link |
and that those are these insane transformations.
link |
Do you see our society as a whole the collective
link |
as another extension of the intelligence
link |
of the human being?
link |
So if you look at the human being as an information processing
link |
system, you mentioned the internet, the networking.
link |
Do you see us all together as a civilization
link |
as a kind of intelligent system?
link |
Yeah, I think this is actually a really interesting
link |
perspective to take and to think about
link |
that you sort of have this collective intelligence
link |
of all of society.
link |
The economy itself is this superhuman machine
link |
that is optimizing something, right?
link |
And it's almost, in some ways, a company
link |
has a will of its own, right?
link |
That you have all these individuals who are all
link |
pursuing their own individual goals
link |
and thinking really hard and thinking
link |
about the right things to do, but somehow the company does
link |
something that is this emergent thing
link |
and that it's a really useful abstraction.
link |
And so I think that in some ways,
link |
we think of ourselves as the most intelligent things
link |
on the planet and the most powerful things on the planet.
link |
But there are things that are bigger than us,
link |
that are these systems that we all contribute to.
link |
And so I think actually, it's interesting to think about,
link |
if you've read Asa Geismov's foundation, right,
link |
that there's this concept of psycho history in there,
link |
which is effectively this, that if you have trillions
link |
or quadrillions of beings, then maybe you could actually
link |
predict what that huge macro being will do
link |
and almost independent of what the individuals want.
link |
And I actually have a second angle on this
link |
that I think is interesting, which is thinking about
link |
technological determinism.
link |
One thing that I actually think a lot about with OpenAI
link |
is that we're kind of coming onto this insanely
link |
transformational technology of general intelligence
link |
that will happen at some point.
link |
And there's a question of how can you take actions
link |
that will actually steer it to go better rather than worse?
link |
And that I think one question you need to ask is,
link |
as a scientist, as an event or as a creator,
link |
what impact can you have in general?
link |
You look at things like the telephone
link |
invented by two people on the same day.
link |
Like what does that mean, like what does that mean
link |
about the shape of innovation?
link |
And I think that what's going on is everyone's building
link |
on the shoulders of the same giants.
link |
And so you can kind of, you can't really hope
link |
to create something no one else ever would.
link |
You know, if Einstein wasn't born,
link |
someone else would have come up with relativity.
link |
You know, he changed the timeline a bit, right?
link |
That maybe it would have taken another 20 years,
link |
but it wouldn't be that fundamentally humanity
link |
would never discover these fundamental truths.
link |
So there's some kind of invisible momentum
link |
that some people like Einstein or OpenAI is plugging into
link |
that anybody else can also plug into.
link |
And ultimately, that wave takes us into a certain direction.
link |
That's what you mean by digitalism?
link |
That's right, that's right.
link |
And you know, this kind of seems to play out
link |
in a bunch of different ways.
link |
That there's some exponential that is being written
link |
and that the exponential itself, which one it is,
link |
changes, think about Moore's Law,
link |
an entire industry set, it's clocked to it for 50 years.
link |
Like how can that be, right?
link |
How is that possible?
link |
And yet somehow it happened.
link |
And so I think you can't hope to ever invent something
link |
that no one else will.
link |
Maybe you can change the timeline a little bit.
link |
But if you really want to make a difference,
link |
I think that the thing that you really have to do,
link |
the only real degree of freedom you have
link |
is to set the initial conditions
link |
under which a technology is born.
link |
And so you think about the internet, right?
link |
That there are lots of other competitors
link |
trying to build similar things.
link |
And the internet one, and that the initial conditions
link |
where it was created by this group
link |
that really valued people being able to be,
link |
you know, anyone being able to plug in
link |
this very academic mindset of being open and connected.
link |
And I think that the internet for the next 40 years
link |
really played out that way.
link |
You know, maybe today,
link |
things are starting to shift in a different direction,
link |
but I think that those initial conditions
link |
were really important to determine
link |
the next 40 years worth of progress.
link |
That's really beautifully put.
link |
So another example of that I think about,
link |
you know, I recently looked at it.
link |
I looked at Wikipedia, the formation of Wikipedia.
link |
And I wonder what the internet would be like
link |
if Wikipedia had ads.
link |
You know, there's an interesting argument
link |
that why they chose not to put advertisement on Wikipedia.
link |
I think Wikipedia is one of the greatest resources
link |
we have on the internet.
link |
It's extremely surprising how well it works
link |
and how well it was able to aggregate
link |
all this kind of good information.
link |
And essentially the creator of Wikipedia,
link |
I don't know, there's probably some debates there,
link |
but set the initial conditions
link |
and now it carried itself forward.
link |
That's really interesting.
link |
So the way you're thinking about AGI
link |
or artificial intelligence is you're focused on
link |
setting the initial conditions for the progress.
link |
Okay, so look into the future.
link |
If you create an AGI system,
link |
like one that can ace the Turing test, natural language,
link |
what do you think would be the interactions
link |
you would have with it?
link |
What do you think are the questions you would ask?
link |
Like what would be the first question you would ask?
link |
I think that at that point,
link |
if you've really built a powerful system
link |
that is capable of shaping the future of humanity,
link |
the first question that you really should ask
link |
is how do we make sure that this plays out well?
link |
And so that's actually the first question
link |
that I would ask a powerful AGI system is.
link |
So you wouldn't ask your colleague,
link |
you wouldn't ask like Ilya, you would ask the AGI system.
link |
Oh, we've already had the conversation with Ilya, right?
link |
And everyone here.
link |
And so you want as many perspectives
link |
and a piece of wisdom as you can
link |
for answering this question.
link |
So I don't think you necessarily defer to
link |
whatever your powerful system tells you,
link |
but you use it as one input
link |
to try to figure out what to do.
link |
But, and I guess fundamentally,
link |
what it really comes down to is
link |
if you built something really powerful
link |
and you think about, think about, for example,
link |
the creation of, of shortly after
link |
the creation of nuclear weapons, right?
link |
The most important question in the world
link |
was what's the world we're going to be like?
link |
How do we set ourselves up in a place
link |
where we're going to be able to survive as a species?
link |
With AGI, I think the question is slightly different, right?
link |
That there is a question of how do we make sure
link |
that we don't get the negative effects?
link |
But there's also the positive side, right?
link |
You imagine that, you know, like,
link |
like what will AGI be like?
link |
Like what will it be capable of?
link |
And I think that one of the core reasons
link |
that an AGI can be powerful and transformative
link |
is actually due to technological development, right?
link |
If you have something that's capable,
link |
that's capable as a human and that it's much more scalable,
link |
that you absolutely want that thing
link |
to go read the whole scientific literature
link |
and think about how to create cures for all the diseases, right?
link |
You want it to think about how to go
link |
and build technologies to help us
link |
create material abundance and to figure out societal problems
link |
that we have trouble with,
link |
like how are we supposed to clean up the environment?
link |
And, you know, maybe you want this
link |
to go and invent a bunch of little robots that will go out
link |
and be biodegradable and turn ocean debris
link |
into harmless molecules.
link |
And I think that that positive side
link |
is something that I think people miss
link |
sometimes when thinking about what an AGI will be like.
link |
And so I think that if you have a system
link |
that's capable of all of that,
link |
you absolutely want its advice about how do I make sure
link |
that we're using your capabilities
link |
in a positive way for humanity.
link |
So what do you think about that psychology
link |
that looks at all the different possible trajectories
link |
of an AGI system, many of which,
link |
perhaps the majority of which are positive
link |
and nevertheless focuses on the negative trajectories?
link |
I mean, you get to interact with folks,
link |
you get to think about this maybe within yourself as well.
link |
You look at Sam Harris and so on.
link |
It seems to be, sorry to put it this way,
link |
but almost more fun to think about the negative possibilities.
link |
Whatever that's deep in our psychology,
link |
what do you think about that?
link |
And how do we deal with it?
link |
Because we want AI to help us.
link |
So I think there's kind of two problems
link |
entailed in that question.
link |
The first is more of the question of,
link |
how can you even picture what a world
link |
with a new technology will be like?
link |
Now imagine we're in 1950
link |
and I'm trying to describe Uber to someone.
link |
Aps and the internet.
link |
Yeah, I mean, that's going to be extremely complicated,
link |
but it's imaginable.
link |
It's imaginable, right?
link |
But, and now imagine being in 1950
link |
and predicting Uber, right?
link |
And you need to describe the internet,
link |
you need to describe GPS,
link |
you need to describe the fact
link |
that everyone's going to have this phone in their pocket.
link |
And so I think that just the first truth
link |
is that it is hard to picture
link |
how a transformative technology will play out in the world.
link |
We've seen that before with technologies
link |
that are far less transformative than AGI will be.
link |
And so I think that one piece
link |
is that it's just even hard to imagine
link |
and to really put yourself in a world
link |
where you can predict what that positive vision
link |
And I think the second thing is that it is,
link |
I think it is always easier to support
link |
the negative side than the positive side.
link |
It's always easier to destroy than create.
link |
And, you know, less in a physical sense
link |
and more just in an intellectual sense, right?
link |
Because, you know, I think that with creating something,
link |
you need to just get a bunch of things right
link |
and to destroy, you just need to get one thing wrong.
link |
And so I think that what that means
link |
is that I think a lot of people's thinking dead ends
link |
as soon as they see the negative story.
link |
But that being said, I actually have some hope, right?
link |
I think that the positive vision
link |
is something that I think can be,
link |
is something that we can talk about.
link |
I think that just simply saying this fact of,
link |
yeah, like there's positive, there's negatives,
link |
everyone likes to dwell on the negative,
link |
people actually respond well to that message and say,
link |
huh, you're right, there's a part of this
link |
that we're not talking about, not thinking about.
link |
And that's actually something that's,
link |
I think really been a key part
link |
of how we think about AGI at OpenAI, right?
link |
You can kind of look at it as like, okay,
link |
like OpenAI talks about the fact that there are risks
link |
and yet they're trying to build this system.
link |
Like how do you square those two facts?
link |
So do you share the intuition that some people have,
link |
I mean, from Sam Harris to even Elon Musk himself,
link |
that it's tricky as you develop AGI
link |
to keep it from slipping into the existential threats,
link |
into the negative.
link |
What's your intuition about,
link |
how hard is it to keep AI development
link |
on the positive track?
link |
What's your intuition there?
link |
To answer that question,
link |
you can really look at how we structure OpenAI.
link |
So we really have three main arms.
link |
So we have capabilities,
link |
which is actually doing the technical work
link |
and pushing forward what these systems can do.
link |
There's safety, which is working on technical mechanisms
link |
to ensure that the systems we build
link |
are aligned with human values.
link |
And then there's policy,
link |
which is making sure that we have governance mechanisms,
link |
answering that question of, well, whose values?
link |
And so I think that the technical safety one
link |
is the one that people kind of talk about the most, right?
link |
You talk about, like think about,
link |
you know, all of the dystopic AI movies,
link |
a lot of that is about not having good
link |
technical safety in place.
link |
And what we've been finding is that, you know,
link |
I think that actually a lot of people
link |
look at the technical safety problem
link |
and think it's just intractable, right?
link |
This question of what do humans want?
link |
How am I supposed to write that down?
link |
Can I even write down what I want?
link |
And then they stop there.
link |
But the thing is we've already built systems
link |
that are able to learn things that humans can't specify.
link |
You know, even the rules for how to recognize
link |
if there's a cat or a dog in an image.
link |
Turns out it's intractable to write that down
link |
and yet we're able to learn it.
link |
And that what we're seeing with systems we build at OpenAI
link |
and they're still in early proof of concept stage
link |
is that you are able to learn human preferences.
link |
You're able to learn what humans want from data.
link |
And so that's kind of the core focus
link |
for our technical safety team.
link |
And I think that they're actually,
link |
we've had some pretty encouraging updates
link |
in terms of what we've been able to make work.
link |
So you have an intuition and a hope that from data,
link |
you know, looking at the value alignment problem,
link |
from data we can build systems that align
link |
with the collective better angels of our nature.
link |
So align with the ethics and the morals of human beings.
link |
To even say this in a different way,
link |
I mean, think about how do we align humans, right?
link |
Think about like a human baby can grow up
link |
to be an evil person or a great person.
link |
And a lot of that is from learning from data, right?
link |
That you have some feedback as a child is growing up.
link |
They get to see positive examples.
link |
And so I think that just like the only example
link |
we have of a general intelligence
link |
that is able to learn from data
link |
to align with human values and to learn values,
link |
I think we shouldn't be surprised
link |
that we can do the same sorts of techniques
link |
or whether the same sort of techniques
link |
end up being how we solve value alignment for AGI's.
link |
So let's go even higher.
link |
I don't know if you've read the book, Sapiens.
link |
But there's an idea that, you know,
link |
that as a collective, as us human beings,
link |
we kind of develop together ideas that we hold.
link |
There's no, in that context, objective truth.
link |
We just kind of all agree to certain ideas
link |
and hold them as a collective.
link |
Did you have a sense that there is
link |
in the world of good and evil,
link |
do you have a sense that to the first approximation,
link |
there are some things that are good
link |
and that you could teach systems to behave to be good?
link |
So I think that this actually blends into our third team,
link |
which is the policy team.
link |
And this is the one, the aspect that I think people
link |
really talk about way less than they should.
link |
Because imagine that we build super powerful systems
link |
that we've managed to figure out all the mechanisms
link |
for these things to do whatever the operator wants.
link |
The most important question becomes,
link |
who's the operator, what do they want,
link |
and how is that going to affect everyone else?
link |
And I think that this question of what is good,
link |
what are those values, I mean,
link |
I think you don't even have to go
link |
to those very grand existential places
link |
to realize how hard this problem is.
link |
You just look at different countries
link |
and cultures across the world.
link |
And that there's a very different conception
link |
of how the world works and what kinds of ways
link |
that society wants to operate.
link |
And so I think that the really core question
link |
is actually very concrete.
link |
And I think it's not a question
link |
that we have ready answers to,
link |
how do you have a world where all the different countries
link |
that we have, United States, China, Russia,
link |
and the hundreds of other countries out there
link |
are able to continue to not just operate
link |
in the way that they see fit,
link |
but in the world that emerges in these,
link |
where you have these very powerful systems,
link |
operating alongside humans,
link |
ends up being something that empowers humans more,
link |
that makes human existence be a more meaningful thing
link |
and that people are happier and wealthier
link |
and able to live more fulfilling lives.
link |
It's not an obvious thing for how to design that world
link |
once you have that very powerful system.
link |
So if we take a little step back,
link |
and we're having like a fascinating conversation
link |
and open as in many ways a tech leader in the world,
link |
and yet we're thinking about these big existential questions
link |
which is fascinating, really important.
link |
I think you're a leader in that space
link |
and that's a really important space
link |
of just thinking how AI affects society
link |
in a big picture view.
link |
So Oscar Wilde said, we're all in the gutter,
link |
but some of us are looking at the stars
link |
and I think OpenAI has a charter
link |
that looks to the stars, I would say,
link |
to create intelligence, to create general intelligence,
link |
make it beneficial, safe, and collaborative.
link |
So can you tell me how that came about?
link |
How a mission like that and the path
link |
to creating a mission like that at OpenAI was founded?
link |
Yeah, so I think that in some ways
link |
it really boils down to taking a look at the landscape.
link |
So if you think about the history of AI
link |
that basically for the past 60 or 70 years,
link |
people have thought about this goal
link |
of what could happen if you could automate
link |
human intellectual labor.
link |
Imagine you can build a computer system
link |
that could do that, what becomes possible?
link |
We have a lot of sci fi that tells stories
link |
of various dystopias and increasingly you have movies
link |
like Her that tell you a little bit about
link |
maybe more of a little bit utopic vision.
link |
You think about the impacts that we've seen
link |
from being able to have bicycles for our minds
link |
and computers and that I think that the impact
link |
of computers and the internet has just far outstripped
link |
what anyone really could have predicted.
link |
And so I think that it's very clear
link |
that if you can build an AGI,
link |
it will be the most transformative technology
link |
that humans will ever create.
link |
And so what it boils down to then is a question of,
link |
well, is there a path?
link |
Is there a way to build such a system?
link |
And I think that for 60 or 70 years
link |
that people got excited and that ended up not being able
link |
to deliver on the hopes that people had pinned on them.
link |
And I think that then, that after two winters
link |
of AI development, that people,
link |
I think kind of almost stopped daring to dream, right?
link |
That really talking about AGI or thinking about AGI
link |
became almost this taboo in the community.
link |
But I actually think that people took the wrong lesson
link |
And if you look back, starting in 1959
link |
is when the Perceptron was released.
link |
And this is basically one of the earliest neural networks.
link |
It was released to what was perceived
link |
as this massive overhype.
link |
So in the New York Times in 1959,
link |
you have this article saying that the Perceptron
link |
will one day recognize people, call out their names,
link |
instantly translate speech between languages.
link |
And people at the time looked at this and said,
link |
this is, your system can't do any of that.
link |
And basically spent 10 years trying to discredit
link |
the whole Perceptron direction and succeeded.
link |
And all the funding dried up.
link |
And people kind of went in other directions.
link |
And in the 80s, there was this resurgence.
link |
And I'd always heard that the resurgence in the 80s
link |
was due to the invention of back propagation
link |
and these algorithms that got people excited.
link |
But actually the causality was due to people
link |
building larger computers.
link |
That you can find these articles from the 80s saying
link |
that the democratization of computing power
link |
suddenly meant that you could run these larger neural networks.
link |
And then people started to do all these amazing things,
link |
back propagation algorithm was invented.
link |
And the neural nets people were running
link |
were these tiny little like 20 neuron neural nets.
link |
What are you supposed to learn with 20 neurons?
link |
And so of course they weren't able to get great results.
link |
And it really wasn't until 2012 that this approach,
link |
that's almost the most simple, natural approach
link |
that people had come up with in the 50s, right?
link |
In some ways, even in the 40s before there were computers
link |
with the Pits McCullin neuron,
link |
suddenly this became the best way of solving problems, right?
link |
And I think there are three core properties
link |
that deep learning has that I think
link |
are very worth paying attention to.
link |
The first is generality.
link |
We have a very small number of deep learning tools,
link |
SGD, deep neural net, maybe some, you know, RL.
link |
And it solves this huge variety of problems,
link |
speech recognition, machine translation,
link |
game playing, all of these problems,
link |
small set of tools.
link |
So there's the generality.
link |
There's a second piece, which is the competence.
link |
You wanna solve any of those problems?
link |
Throughout 40 years worth of normal computer vision research
link |
replaced with a deep neural net, it's gonna work better.
link |
And there's a third piece, which is the scalability, right?
link |
That one thing that has been shown time and time again
link |
is that you, if you have a larger neural network,
link |
throw more compute, more data at it, it will work better.
link |
Those three properties together feel like essential parts
link |
of building a general intelligence.
link |
Now, it doesn't just mean that if we scale up
link |
what we have, that we will have an AGI, right?
link |
There are clearly missing pieces.
link |
There are missing ideas.
link |
We need to have answers for reasoning.
link |
But I think that the core here is that for the first time,
link |
it feels that we have a paradigm
link |
that gives us hope that general intelligence
link |
can be achievable.
link |
And so as soon as you believe that,
link |
everything else becomes into focus, right?
link |
If you imagine that you may be able to,
link |
and that the timeline I think remains uncertain,
link |
but I think that certainly within our lifetimes
link |
and possibly within a much shorter period of time
link |
than people would expect,
link |
if you can really build the most transformative technology
link |
that will ever exist, you stop thinking about yourself
link |
And you start thinking about just like,
link |
how do you have a world where this goes well?
link |
And that you need to think about the practicalities
link |
of how do you build an organization
link |
and get together a bunch of people and resources
link |
and to make sure that people feel motivated
link |
and ready to do it.
link |
But I think that then you start thinking about,
link |
well, what if we succeed?
link |
And how do we make sure that when we succeed,
link |
that the world is actually the place
link |
that we want ourselves to exist in?
link |
And almost in the Rawlsian Vale sense of the word.
link |
And so that's kind of the broader landscape.
link |
And Open AI was really formed in 2015
link |
with that high level picture of AGI might be possible
link |
sooner than people think
link |
and that we need to try to do our best
link |
to make sure it's going to go well.
link |
And then we spent the next couple of years
link |
really trying to figure out what does that mean?
link |
And I think that typically with a company,
link |
you start out very small.
link |
So you want a cofounder and you build a product,
link |
you get some users, you get a product market fit,
link |
then at some point you raise some money,
link |
you hire people, you scale,
link |
and then down the road, then the big companies
link |
realize you exist and try to kill you.
link |
And for Open AI, it was basically everything
link |
in exactly the opposite order.
link |
Let me just pause for a second.
link |
He said a lot of things.
link |
And let me just admire the jarring aspect
link |
of what Open AI stands for, which is daring to dream.
link |
I mean, you said it's pretty powerful.
link |
You caught me off guard because I think that's very true.
link |
The step of just daring to dream
link |
about the possibilities of creating intelligence
link |
in a positive and a safe way,
link |
but just even creating intelligence
link |
is a much needed, refreshing catalyst
link |
for the AI community.
link |
So that's the starting point.
link |
Okay, so then formation of Open AI, what's your point?
link |
I would just say that when we were starting Open AI,
link |
that kind of the first question that we had is,
link |
is it too late to start a lab with a bunch of the best people?
link |
Right, is that even possible?
link |
That was an actual question.
link |
That was the core question of,
link |
we had this dinner in July of 2015,
link |
and that was really what we spent the whole time
link |
And because you think about kind of where AI was,
link |
is that it transitioned from being an academic pursuit
link |
to an industrial pursuit.
link |
And so a lot of the best people were in these big
link |
research labs and that we wanted to start our own one
link |
that no matter how much resources we could accumulate
link |
would be pale in comparison to the big tech companies.
link |
And there's a question of,
link |
are we going to be actually able to get this thing
link |
You need critical mass.
link |
You can't just do you and a cofounder build a product, right?
link |
You really need to have a group of five to 10 people.
link |
And we kind of concluded it wasn't obviously impossible.
link |
So it seemed worth trying.
link |
Well, you're also a dreamer, so who knows, right?
link |
Okay, so speaking of that,
link |
competing with the big players,
link |
let's talk about some of the tricky things
link |
as you think through this process of growing,
link |
of seeing how you can develop these systems
link |
at a scale that competes.
link |
So you recently formed OpenAI LP,
link |
a new cap profit company that now carries the name OpenAI.
link |
So OpenAI is now this official company.
link |
The original nonprofit company still exists
link |
and carries the OpenAI nonprofit name.
link |
So can you explain what this company is,
link |
what the purpose of its creation is,
link |
and how did you arrive at the decision to create it?
link |
OpenAI, the whole entity and OpenAI LP as a vehicle
link |
is trying to accomplish the mission
link |
of ensuring that artificial general intelligence
link |
benefits everyone.
link |
And the main way that we're trying to do that
link |
is by actually trying to build
link |
general intelligence to ourselves
link |
and make sure the benefits are distributed to the world.
link |
That's the primary way.
link |
We're also fine if someone else does this, right?
link |
It doesn't have to be us.
link |
If someone else is going to build an AGI
link |
and make sure that the benefits don't get locked up
link |
in one company or with one set of people,
link |
like we're actually fine with that.
link |
And so those ideas are baked into our charter,
link |
which is kind of the foundational document
link |
that describes kind of our values and how we operate.
link |
And it's also really baked into the structure of OpenAI LP.
link |
And so the way that we've set up OpenAI LP
link |
is that in the case where we succeed, right?
link |
If we actually build what we're trying to build,
link |
then investors are able to get a return,
link |
and but that return is something that is capped.
link |
And so if you think of AGI in terms of the value
link |
that you could really create,
link |
you're talking about the most transformative technology
link |
ever created, it's gonna create,
link |
or does the magnitude more value than any existing company?
link |
And that all of that value will be owned by the world,
link |
like legally titled to the nonprofit
link |
to fulfill that mission.
link |
And so that's the structure.
link |
So the mission is a powerful one,
link |
and it's one that I think most people would agree with.
link |
It's how we would hope AI progresses.
link |
And so how do you tie yourself to that mission?
link |
How do you make sure you do not deviate from that mission
link |
that other incentives that are profit driven
link |
wouldn't don't interfere with the mission?
link |
So this was actually a really core question for us
link |
for the past couple of years,
link |
because I'd say that the way that our history went
link |
was that for the first year,
link |
we were getting off the ground, right?
link |
We had this high level picture,
link |
but we didn't know exactly how we wanted to accomplish it.
link |
And really two years ago,
link |
it's when we first started realizing
link |
in order to build AGI,
link |
we're just gonna need to raise way more money
link |
than we can as a nonprofit.
link |
We're talking many billions of dollars.
link |
And so the first question is,
link |
how are you supposed to do that
link |
and stay true to this mission?
link |
And we looked at every legal structure out there
link |
and included none of them were quite right
link |
for what we wanted to do.
link |
And I guess it shouldn't be too surprising
link |
if you're gonna do some crazy unprecedented technology
link |
that you're gonna have to come
link |
with some crazy unprecedented structure to do it in.
link |
And a lot of our conversation was with people at OpenAI,
link |
the people who really joined
link |
because they believe so much in this mission
link |
and thinking about how do we actually raise the resources
link |
to do it and also stay true to what we stand for.
link |
And the place you gotta start is to really align
link |
on what is it that we stand for, right?
link |
What are those values?
link |
What's really important to us?
link |
And so I'd say that we spent about a year
link |
really compiling the OpenAI charter.
link |
And that determines,
link |
and if you even look at the first line item in there,
link |
it says that, look, we expect we're gonna have to marshal
link |
huge amounts of resources,
link |
but we're going to make sure
link |
that we minimize conflict of interest with the mission.
link |
And that kind of aligning on all of those pieces
link |
was the most important step towards figuring out
link |
how do we structure a company
link |
that can actually raise the resources
link |
to do what we need to do.
link |
I imagine OpenAI, the decision to create OpenAI LP
link |
was a really difficult one.
link |
And there was a lot of discussions
link |
as you mentioned for a year.
link |
And there was different ideas,
link |
perhaps detractors within OpenAI,
link |
sort of different paths that you could have taken.
link |
What were those concerns?
link |
What were the different paths considered?
link |
What was that process of making that decision like?
link |
But so if you look actually at the OpenAI charter,
link |
that there's almost two paths embedded within it.
link |
There is, we are primarily trying to build AGI ourselves,
link |
but we're also okay if someone else does it.
link |
And this is a weird thing for a company.
link |
It's really interesting, actually.
link |
But there is an element of competition
link |
that you do want to be the one that does it,
link |
but at the same time, you're okay if somebody else doesn't.
link |
We'll talk about that a little bit, that trade off,
link |
that dance that's really interesting.
link |
And I think this was the core tension
link |
as we were designing OpenAI LP
link |
and really the OpenAI strategy,
link |
is how do you make sure that both you have a shot
link |
at being a primary actor,
link |
which really requires building an organization,
link |
raising massive resources,
link |
and really having the will to go
link |
and execute on some really, really hard vision, right?
link |
You need to really sign up for a long period
link |
to go and take on a lot of pain and a lot of risk.
link |
normally you just import the startup mindset, right?
link |
And that you think about, okay,
link |
like how do we out execute everyone?
link |
You have this very competitive angle.
link |
But you also have the second angle of saying that,
link |
well, the true mission isn't for OpenAI to build AGI.
link |
The true mission is for AGI to go well for humanity.
link |
And so how do you take all of those first actions
link |
and make sure you don't close the door on outcomes
link |
that would actually be positive and fulfill the mission?
link |
And so I think it's a very delicate balance, right?
link |
And I think that going 100% one direction or the other
link |
is clearly not the correct answer.
link |
And so I think that even in terms of just how we talk about
link |
OpenAI and think about it,
link |
there's just like one thing that's always
link |
in the back of my mind is to make sure
link |
that we're not just saying OpenAI's goal
link |
is to build AGI, right?
link |
That it's actually much broader than that, right?
link |
That first of all, it's not just AGI, it's safe AGI
link |
that's very important.
link |
But secondly, our goal isn't to be the ones to build it,
link |
our goal is to make sure it goes well for the world.
link |
And so I think that figuring out,
link |
how do you balance all of those
link |
and to get people to really come to the table
link |
and compile a single document that encompasses all of that
link |
So part of the challenge here is your mission is,
link |
I would say, beautiful, empowering,
link |
and a beacon of hope for people in the research community
link |
and just people thinking about AI.
link |
So your decisions are scrutinized
link |
more than, I think, a regular profit driven company.
link |
Do you feel the burden of this
link |
in the creation of the charter
link |
and just in the way you operate?
link |
So why do you lean into the burden
link |
by creating such a charter?
link |
Why not keep it quiet?
link |
I mean, it just boils down to the mission, right?
link |
Like, I'm here and everyone else is here
link |
because we think this is the most important mission, right?
link |
All right, so do you think you can be good for the world
link |
or create an AGI system that's good
link |
when you're a for profit company?
link |
From my perspective, I don't understand why profit
link |
interferes with positive impact on society.
link |
I don't understand why Google
link |
that makes most of its money from ads
link |
can't also do good for the world
link |
or other companies, Facebook, anything.
link |
I don't understand why those have to interfere.
link |
You know, you can, profit isn't the thing in my view
link |
that affects the impact of a company.
link |
What affects the impact of the company is the charter,
link |
is the culture, is the people inside
link |
and profit is the thing that just fuels those people.
link |
What are your views there?
link |
Yeah, so I think that's a really good question
link |
and there's some real like longstanding debates
link |
in human society that are wrapped up in it.
link |
The way that I think about it is just think about
link |
what are the most impactful nonprofits in the world?
link |
What are the most impactful for profits in the world?
link |
Right, it's much easier to list the for profits.
link |
And I think that there's some real truth here
link |
that the system that we set up,
link |
the system for kind of how today's world is organized
link |
is one that really allows for huge impact
link |
and that kind of part of that is that you need to be,
link |
that for profits are self sustaining
link |
and able to kind of build on their own momentum.
link |
And I think that's a really powerful thing.
link |
It's something that when it turns out
link |
that we haven't set the guardrails correctly,
link |
causes problems, right?
link |
Think about logging companies that go into the rainforest,
link |
that's really bad, we don't want that.
link |
And it's actually really interesting to me
link |
that kind of this question of
link |
how do you get positive benefits out of a for profit company?
link |
It's actually very similar to
link |
how do you get positive benefits out of an AGI, right?
link |
That you have this like very powerful system,
link |
it's more powerful than any human
link |
and it's kind of autonomous in some ways.
link |
You know, it's super human in a lot of axes
link |
and somehow you have to set the guardrails
link |
to get good things to happen.
link |
But when you do, the benefits are massive.
link |
And so I think that when I think about nonprofit
link |
versus for profit, I think just not enough happens
link |
in nonprofits, they're very pure,
link |
but it's just kind of, you know,
link |
it's just hard to do things there.
link |
And for profits in some ways, like too much happens,
link |
but if kind of shaped in the right way,
link |
it can actually be very positive.
link |
And so with OpenILP, we're picking a road in between.
link |
Now, the thing that I think is really important to recognize
link |
is that the way that we think about OpenILP
link |
is that in the world where AGI actually happens, right?
link |
In a world where we are successful,
link |
we build the most transformative technology ever,
link |
the amount of value we're going to create will be astronomical.
link |
And so then in that case, that the cap that we have
link |
will be a small fraction of the value we create.
link |
And the amount of value that goes back to investors
link |
and employees looks pretty similar to what would happen
link |
in a pretty successful startup.
link |
And that's really the case that we're optimizing for, right?
link |
That we're thinking about in the success case,
link |
making sure that the value we create doesn't get locked up.
link |
And I expect that in other for profit companies
link |
that it's possible to do something like that.
link |
I think it's not obvious how to do it, right?
link |
And I think that as a for profit company,
link |
you have a lot of fiduciary duty to your shareholders
link |
and that there are certain decisions
link |
that you just cannot make.
link |
In our structure, we've set it up
link |
so that we have a fiduciary duty to the charter,
link |
that we always get to make the decision
link |
that is right for the charter,
link |
rather than even if it comes at the expense
link |
of our own stakeholders.
link |
And so I think that when I think about
link |
what's really important,
link |
it's not really about nonprofit versus for profit.
link |
It's really a question of if you build a GI
link |
and you kind of, you know,
link |
humanity is now at this new age,
link |
who benefits, whose lives are better?
link |
And I think that what's really important
link |
is to have an answer that is everyone.
link |
Yeah, which is one of the core aspects of the charter.
link |
So one concern people have, not just with OpenAI,
link |
but with Google, Facebook, Amazon,
link |
anybody really that's creating impact at scale
link |
is how do we avoid, as your charter says,
link |
avoid enabling the use of AI or AGI
link |
to unduly concentrate power?
link |
Why would not a company like OpenAI
link |
keep all the power of an AGI system to itself?
link |
So, you know, how does the charter
link |
actualize itself in day to day?
link |
So I think that first to zoom out, right,
link |
that the way that we structure the company
link |
is so that the power for sort of, you know,
link |
dictating the actions that OpenAI takes
link |
ultimately rests with the board, right?
link |
The board of the nonprofit and the board is set up
link |
in certain ways, with certain restrictions
link |
that you can read about in the OpenAI LP blog post.
link |
But effectively the board is the governing body
link |
And the board has a duty to fulfill the mission
link |
And so that's kind of how we tie,
link |
how we thread all these things together.
link |
Now there's a question of so day to day,
link |
how do people, the individuals,
link |
who in some ways are the most empowered ones, right?
link |
You know, the board sort of gets to call the shots
link |
at the high level, but the people who are actually executing
link |
are the employees, right?
link |
The people here on a day to day basis who have the,
link |
you know, the keys to the technical kingdom.
link |
And there I think that the answer looks a lot like,
link |
well, how does any company's values get actualized, right?
link |
And I think that a lot of that comes down to
link |
that you need people who are here
link |
because they really believe in that mission
link |
and they believe in the charter
link |
and that they are willing to take actions
link |
that maybe are worse for them, but are better for the charter.
link |
And that's something that's really baked into the culture.
link |
And honestly, I think it's, you know,
link |
I think that that's one of the things
link |
that we really have to work to preserve as time goes on.
link |
And that's a really important part of how we think
link |
about hiring people and bringing people into OpenAI.
link |
So there's people here, there's people here
link |
who could speak up and say, like, hold on a second,
link |
this is totally against what we stand for, culture wise.
link |
Yeah, yeah, for sure.
link |
I mean, I think that we actually have,
link |
I think that's like a pretty important part
link |
of how we operate and how we have,
link |
even again with designing the charter
link |
and designing OpenAI in the first place,
link |
that there has been a lot of conversation
link |
with employees here and a lot of times
link |
where employees said, wait a second,
link |
this seems like it's going in the wrong direction
link |
and let's talk about it.
link |
And so I think one thing that's, I think are really,
link |
and you know, here's actually one thing
link |
that I think is very unique about us as a small company,
link |
is that if you're at a massive tech giant,
link |
that's a little bit hard for someone
link |
who's a line employee to go and talk to the CEO
link |
and say, I think that we're doing this wrong.
link |
And you know, you'll get companies like Google
link |
that have had some collective action from employees
link |
to make ethical change around things like Maven.
link |
And so maybe there are mechanisms
link |
that other companies that work,
link |
but here, super easy for anyone to pull me aside,
link |
to pull Sam aside, to pull Eli aside,
link |
and people do it all the time.
link |
One of the interesting things in the charter
link |
is this idea that it'd be great
link |
if you could try to describe or untangle
link |
switching from competition to collaboration
link |
and late stage AGI development.
link |
It's really interesting,
link |
this dance between competition and collaboration,
link |
how do you think about that?
link |
Yeah, assuming that you can actually do
link |
the technical side of AGI development,
link |
I think there's going to be two key problems
link |
with figuring out how do you actually deploy it
link |
and make it go well.
link |
The first one of these is the run up
link |
to building the first AGI.
link |
You look at how self driving cars are being developed,
link |
and it's a competitive race.
link |
And the thing that always happens in competitive race
link |
is that you have huge amounts of pressure
link |
to get rid of safety.
link |
And so that's one thing we're very concerned about, right?
link |
Is that people, multiple teams figuring out,
link |
we can actually get there,
link |
but you know, if we took the slower path
link |
that is more guaranteed to be safe, we will lose.
link |
And so we're going to take the fast path.
link |
And so the more that we can, both ourselves,
link |
be in a position where we don't generate
link |
that competitive race, where we say,
link |
if the race is being run and that someone else
link |
is further ahead than we are,
link |
we're not going to try to leapfrog.
link |
We're going to actually work with them, right?
link |
We will help them succeed.
link |
As long as what they're trying to do
link |
is to fulfill our mission, then we're good.
link |
We don't have to build AGI ourselves.
link |
And I think that's a really important commitment from us,
link |
but it can't just be unilateral, right?
link |
I think that it's really important
link |
that other players who are serious about building AGI
link |
make similar commitments, right?
link |
And I think that, you know, again,
link |
to the extent that everyone believes
link |
that AGI should be something to benefit everyone,
link |
then it actually really shouldn't matter
link |
which company builds it.
link |
And we should all be concerned about the case
link |
where we just race so hard to get there
link |
that something goes wrong.
link |
So what role do you think government,
link |
our favorite entity has in setting policy and rules
link |
about this domain, from research to the development
link |
to early stage, to late stage AI and AGI development?
link |
So I think that, first of all,
link |
it's really important that government's in there, right?
link |
In some way, shape, or form, you know,
link |
at the end of the day, we're talking about
link |
building technology that will shape how the world operates
link |
and that there needs to be government as part of that answer.
link |
And so that's why we've done a number
link |
of different congressional testimonies.
link |
We interact with a number of different lawmakers
link |
and that right now, a lot of our message to them
link |
is that it's not the time for regulation,
link |
it is the time for measurement, right?
link |
That our main policy recommendation is that people,
link |
and you know, the government does this all the time
link |
with bodies like NIST, spend time trying to figure out
link |
just where the technology is, how fast it's moving,
link |
and can really become literate and up to speed
link |
with respect to what to expect.
link |
So I think that today, the answer really
link |
is about measurement.
link |
And I think that there will be a time and place
link |
where that will change.
link |
And I think it's a little bit hard to predict exactly
link |
what exactly that trajectory should look like.
link |
So there will be a point at which regulation,
link |
federal in the United States, the government steps in
link |
and helps be the, I don't wanna say the adult in the room,
link |
to make sure that there is strict rules,
link |
maybe conservative rules that nobody can cross.
link |
Well, I think there's kind of maybe two angles to it.
link |
So today with narrow AI applications,
link |
that I think there are already existing bodies
link |
that are responsible and should be responsible for regulation.
link |
You think about, for example, with self driving cars,
link |
that you want the national highway.
link |
Yeah, exactly to be regulated in that.
link |
That makes sense, right?
link |
That basically what we're saying
link |
is that we're going to have these technological systems
link |
that are going to be performing applications
link |
that humans already do.
link |
Great, we already have ways of thinking about standards
link |
and safety for those.
link |
So I think actually empowering those regulators today
link |
is also pretty important.
link |
And then I think for AGI, that there's going to be a point
link |
where we'll have better answers.
link |
And I think that maybe a similar approach
link |
of first measurement and start thinking about
link |
what the rules should be.
link |
I think it's really important
link |
that we don't prematurely squash progress.
link |
I think it's very easy to kind of smother a budding field.
link |
And I think that's something to really avoid.
link |
But I don't think that the right way of doing it
link |
is to say, let's just try to blaze ahead
link |
and not involve all these other stakeholders.
link |
So you've recently released a paper on GPT2
link |
language modeling, but did not release the full model
link |
because you had concerns about the possible negative effects
link |
of the availability of such model.
link |
It's outside of just that decision,
link |
and it's super interesting because of the discussion
link |
at a societal level, the discourse it creates.
link |
So it's fascinating in that aspect.
link |
But if you think that's the specifics here at first,
link |
what are some negative effects that you envisioned?
link |
And of course, what are some of the positive effects?
link |
Yeah, so again, I think to zoom out,
link |
like the way that we thought about GPT2
link |
is that with language modeling,
link |
we are clearly on a trajectory right now
link |
where we scale up our models
link |
and we get qualitatively better performance, right?
link |
GPT2 itself was actually just a scale up
link |
of a model that we've released in the previous June, right?
link |
And we just ran it at much larger scale
link |
and we got these results
link |
where suddenly starting to write coherent pros,
link |
which was not something we'd seen previously.
link |
And what are we doing now?
link |
Well, we're gonna scale up GPT2 by 10x by 100x by 1000x
link |
and we don't know what we're gonna get.
link |
And so it's very clear that the model
link |
that we released last June,
link |
I think it's kind of like, it's a good academic toy.
link |
It's not something that we think is something
link |
that can really have negative applications
link |
or to the extent that it can,
link |
that the positive of people being able to play with it
link |
is far outweighs the possible harms.
link |
You fast forward to not GPT2, but GPT20,
link |
and you think about what that's gonna be like.
link |
And I think that the capabilities are going to be substantive.
link |
And so there needs to be a point in between the two
link |
where you say, this is something
link |
where we are drawing the line
link |
and that we need to start thinking about the safety aspects.
link |
And I think for GPT2, we could have gone either way.
link |
And in fact, when we had conversations internally
link |
that we had a bunch of pros and cons
link |
and it wasn't clear which one outweighed the other.
link |
And I think that when we announced
link |
that, hey, we decide not to release this model,
link |
then there was a bunch of conversation
link |
where various people said it's so obvious
link |
that you should have just released it.
link |
There are other people that said it's so obvious
link |
you should not have released it.
link |
And I think that that almost definitionally means
link |
that holding it back was the correct decision.
link |
If it's not obvious whether something is beneficial
link |
or not, you should probably default to caution.
link |
And so I think that the overall landscape
link |
for how we think about it
link |
is that this decision could have gone either way.
link |
There are great arguments in both directions.
link |
But for future models down the road,
link |
and possibly sooner than you'd expect,
link |
because scaling these things up doesn't actually
link |
take that long, those ones,
link |
you're definitely not going to want to release into the wild.
link |
And so I think that we almost view this as a test case
link |
and to see, can we even design,
link |
how do you have a society or how do you have a system
link |
that goes from having no concept of responsible disclosure
link |
where the mere idea of not releasing something
link |
for safety reasons is unfamiliar
link |
to a world where you say, okay,
link |
we have a powerful model.
link |
Let's at least think about it.
link |
Let's go through some process.
link |
And you think about the security community.
link |
It took them a long time
link |
to design responsible disclosure.
link |
You think about this question of,
link |
well, I have a security exploit.
link |
I send it to the company.
link |
The company is like, tries to prosecute me
link |
or just ignores it.
link |
And so the alternatives of,
link |
oh, I just always publish your exploits.
link |
That doesn't seem good either.
link |
And so it really took a long time
link |
and it was bigger than any individual.
link |
It's really about building a whole community
link |
that believe that, okay, we'll have this process
link |
where you send it to the company
link |
if they don't act at a certain time,
link |
then you can go public
link |
and you're not a bad person.
link |
You've done the right thing.
link |
And I think that in AI,
link |
part of the response to GPT2 just proves
link |
that we don't have any concept of this.
link |
So that's the high level picture.
link |
And so I think that,
link |
I think this was a really important move to make.
link |
And we could have maybe delayed it for GPT3,
link |
but I'm really glad we did it for GPT2.
link |
And so now you look at GPT2 itself
link |
and you think about the substance of, okay,
link |
what are potential negative applications?
link |
So you have this model that's been trained on the internet,
link |
which is also going to be a bunch of very biased data,
link |
a bunch of very offensive content in there.
link |
And you can ask it to generate content for you
link |
on basically any topic, right?
link |
You just give it a prompt
link |
and it'll just start writing
link |
and it writes content like you see on the internet,
link |
you know, even down to like saying advertisement
link |
in the middle of some of its generations.
link |
And you think about the possibilities
link |
for generating fake news or abusive content.
link |
And, you know, it's interesting
link |
seeing what people have done with, you know,
link |
we released a smaller version of GPT2
link |
and the people have done things like try to generate,
link |
you know, take my own Facebook message history
link |
and generate more Facebook messages like me
link |
and people generating fake politician content
link |
or, you know, there's a bunch of things there
link |
where you at least have to think,
link |
is this going to be good for the world?
link |
There's the flip side, which is I think
link |
that there's a lot of awesome applications
link |
that we really want to see like creative applications
link |
in terms of if you have sci fi authors
link |
that can work with this tool and come with cool ideas,
link |
like that seems awesome if we can write better sci fi
link |
through the use of these tools.
link |
And we've actually had a bunch of people right into us
link |
asking, hey, can we use it for, you know,
link |
a variety of different creative applications?
link |
So the positive are actually pretty easy to imagine.
link |
There are, you know, the usual NLP applications
link |
that are really interesting, but let's go there.
link |
It's kind of interesting to think about a world
link |
where, look at Twitter, where not just fake news
link |
but smarter and smarter bots being able to spread
link |
in an interesting complex networking way in information
link |
that just floods out us regular human beings
link |
with our original thoughts.
link |
So what are your views of this world with GPT 20?
link |
Right, how do we think about, again,
link |
it's like one of those things about in the 50s
link |
trying to describe the internet or the smartphone.
link |
What do you think about that world,
link |
the nature of information?
link |
One possibility is that we'll always try to design systems
link |
that identify a robot versus human
link |
and we'll do so successfully.
link |
And so we'll authenticate that we're still human.
link |
And the other world is that we just accept the fact
link |
that we're swimming in a sea of fake news
link |
and just learn to swim there.
link |
Well, have you ever seen the, there's a, you know,
link |
popular meme of a robot with a physical arm and pen
link |
clicking the I'm not a robot button?
link |
I think the truth is that really trying to distinguish
link |
between robot and human is a losing battle.
link |
Ultimately, you think it's a losing battle?
link |
I think it's a losing battle ultimately, right?
link |
I think that that is that in terms of the content,
link |
in terms of the actions that you can take.
link |
I mean, think about how captures have gone, right?
link |
The captures used to be a very nice, simple.
link |
You just have this image, all of our OCR is terrible.
link |
You put a couple of artifacts in it, you know,
link |
humans are gonna be able to tell what it is
link |
an AI system wouldn't be able to today.
link |
Like I could barely do captures.
link |
And I think that this is just kind of where we're going.
link |
I think captures where we're a moment in time thing.
link |
And as AI systems become more powerful,
link |
that there being human capabilities that can be measured
link |
in a very easy automated way that the AIs will not be
link |
capable of, I think that's just like,
link |
it's just an increasingly hard technical battle.
link |
But it's not that all hope is lost, right?
link |
And you think about how do we already authenticate
link |
That, you know, we have systems.
link |
We have social security numbers.
link |
If you're in the U S or, you know, you have, you have,
link |
you know, ways of identifying individual people
link |
and having real world identity tied to digital identity
link |
seems like a step towards, you know,
link |
authenticating the source of content
link |
rather than the content itself.
link |
Now, there are problems with that.
link |
How can you have privacy and anonymity in a world
link |
where the only content you can really trust is,
link |
or the only way you can trust content
link |
is by looking at where it comes from.
link |
And so I think that building out good reputation networks
link |
maybe one possible solution.
link |
But yeah, I think that this question is not
link |
And I think that we, you know,
link |
maybe sooner than we think we'll be in a world
link |
where, you know, today I often will read a tweet
link |
and be like, do I feel like a real human wrote this?
link |
Or, you know, do I feel like this was like genuine?
link |
I feel like I can kind of judge the content a little bit.
link |
And I think in the future, it just won't be the case.
link |
You look at, for example, the FCC comments on net neutrality.
link |
It came out later that millions of those were auto generated
link |
and that the researchers were able to do various
link |
statistical techniques to do that.
link |
What do you do in a world where those statistical techniques
link |
It's just impossible to tell the difference
link |
between humans and AI's.
link |
And in fact, the most persuasive arguments
link |
are written by AI, all that stuff.
link |
It's not sci fi anymore.
link |
You look at GPT2 making a great argument for why recycling
link |
is bad for the world.
link |
You got to read that and be like, huh, you're right.
link |
We are addressing just the symptoms.
link |
Yeah, that's quite interesting.
link |
I mean, ultimately it boils down to the physical world
link |
being the last frontier of proving.
link |
So you said like basically networks of people,
link |
humans vouching for humans in the physical world.
link |
And somehow the authentication ends there.
link |
I mean, if I had to ask you,
link |
I mean, you're way too eloquent for a human.
link |
So if I had to ask you to authenticate,
link |
like prove how do I know you're not a robot
link |
and how do you know I'm not a robot?
link |
I think that's so far were in this space,
link |
this conversation we just had,
link |
the physical movements we did
link |
is the biggest gap between us and AI systems
link |
is the physical manipulation.
link |
So maybe that's the last frontier.
link |
Well, here's another question is,
link |
why is solving this problem important, right?
link |
Like what aspects are really important to us?
link |
And I think that probably where we'll end up
link |
is we'll hone in on what do we really want
link |
out of knowing if we're talking to a human.
link |
And I think that again, this comes down to identity.
link |
And so I think that the internet of the future,
link |
I expect to be one that will have lots of agents out there
link |
that will interact with you.
link |
But I think that the question of,
link |
is this real flesh and blood human
link |
or is this an automated system?
link |
May actually just be less important.
link |
Let's actually go there.
link |
It's GPT2 is impressive and let's look at GPT20.
link |
Why is it so bad that all my friends are GPT20?
link |
Why is it so important on the internet?
link |
Do you think to interact with only human beings?
link |
Why can't we live in a world where ideas can come
link |
from models trained on human data?
link |
Yeah, I think this is actually a really interesting question.
link |
This comes back to the,
link |
how do you even picture a world with some new technology?
link |
And I think that one thing that I think is important
link |
is, you know, let's say honesty.
link |
And I think that if you have, you know, almost in the
link |
Turing test style sense of technology,
link |
you have AIs that are pretending to be humans
link |
and deceiving you, I think that is, you know,
link |
that feels like a bad thing, right?
link |
I think that it's really important that we feel like
link |
we're in control of our environment, right?
link |
That we understand who we're interacting with.
link |
And if it's an AI or a human,
link |
that that's not something that we're being deceived about.
link |
But I think that the flip side of,
link |
can I have as meaningful of an interaction with an AI
link |
as I can with a human?
link |
Well, I actually think here you can turn to sci fi.
link |
And her, I think is a great example of asking this very
link |
And one thing I really love about her is it really starts
link |
out almost by asking how meaningful are human
link |
virtual relationships, right?
link |
And then you have a human who has a relationship with an AI
link |
and that you really start to be drawn into that, right?
link |
And that all of your emotional buttons get triggered
link |
in the same way as if there was a real human that was on
link |
the other side of that phone.
link |
And so I think that this is one way of thinking about it,
link |
is that I think that we can have meaningful interactions
link |
and that if there's a funny joke,
link |
some sense it doesn't really matter if it was written
link |
by a human or an AI, but what you don't want in a way
link |
where I think we should really draw hard lines is deception.
link |
And I think that as long as we're in a world where,
link |
you know, why do we build AI systems at all, right?
link |
The reason we want to build them is to enhance human lives,
link |
to make humans be able to do more things,
link |
to have humans feel more fulfilled.
link |
And if we can build AI systems that do that,
link |
you know, sign me up.
link |
So the process of language modeling,
link |
how far do you think it take us?
link |
Let's look at movie HER.
link |
Do you think a dialogue, natural language conversation
link |
is formulated by the Turing test, for example,
link |
do you think that process could be achieved through
link |
this kind of unsupervised language modeling?
link |
So I think the Turing test in its real form
link |
isn't just about language, right?
link |
It's really about reasoning too, right?
link |
That to really pass the Turing test,
link |
I should be able to teach calculus
link |
to whoever's on the other side
link |
and have it really understand calculus
link |
and be able to, you know, go and solve
link |
new calculus problems.
link |
And so I think that to really solve the Turing test,
link |
we need more than what we're seeing with language models.
link |
We need some way of plugging in reasoning.
link |
Now, how different will that be from what we already do?
link |
That's an open question, right?
link |
It might be that we need some sequence
link |
of totally radical new ideas,
link |
or it might be that we just need to kind of shape
link |
our existing systems in a slightly different way.
link |
But I think that in terms of how far
link |
language modeling will go,
link |
it's already gone way further
link |
than many people would have expected, right?
link |
I think that things like,
link |
and I think there's a lot of really interesting angles
link |
to poke in terms of how much does GPT2
link |
understand physical world?
link |
Like, you know, you read a little bit
link |
about fire underwater in GPT2.
link |
So it's like, okay, maybe it doesn't quite understand
link |
what these things are.
link |
But at the same time, I think that you also see
link |
various things like smoke coming from flame,
link |
and you know, a bunch of these things that GPT2,
link |
it has no body, it has no physical experience,
link |
it's just statically read data.
link |
And I think that if the answer is like,
link |
we don't know yet, and these questions though,
link |
we're starting to be able to actually ask them
link |
to physical systems, to real systems that exist,
link |
and that's very exciting.
link |
Do you think, what's your intuition?
link |
Do you think if you just scale language modeling,
link |
like significantly scale, that reasoning can emerge
link |
from the same exact mechanisms?
link |
I think it's unlikely that if we just scale GPT2,
link |
that we'll have reasoning in the full fledged way.
link |
And I think that there's like,
link |
the type signature is a little bit wrong, right?
link |
That like, there's something we do with,
link |
that we call thinking, right?
link |
Where we spend a lot of compute,
link |
like a variable amount of compute
link |
to get to better answers, right?
link |
I think a little bit harder, I get a better answer.
link |
And that that kind of type signature
link |
isn't quite encoded in a GPT, right?
link |
GPT will kind of like, it's spent a long time
link |
in it's like evolutionary history,
link |
baking and all this information,
link |
getting very, very good at this predictive process.
link |
And then at runtime, I just kind of do one forward pass
link |
and am able to generate stuff.
link |
And so, there might be small tweaks
link |
to what we do in order to get the type signature, right?
link |
For example, well, it's not really one forward pass, right?
link |
You generate symbol by symbol.
link |
And so, maybe you generate like a whole sequence of thoughts
link |
and you only keep like the last bit or something.
link |
But I think that at the very least,
link |
I would expect you have to make changes like that.
link |
Yeah, just exactly how we, you said think
link |
is the process of generating thought by thought
link |
in the same kind of way, like you said,
link |
keep the last bit, the thing that we converge towards.
link |
And I think there's another piece which is interesting,
link |
which is this out of distribution generalization, right?
link |
That like thinking somehow lets us do that, right?
link |
That we have an experience of thing
link |
and yet somehow we just kind of keep refining
link |
our mental model of it.
link |
This is again, something that feels tied to
link |
whatever reasoning is.
link |
And maybe it's a small tweak to what we do.
link |
Maybe it's many ideas and we'll take as many decades.
link |
Yeah, so the assumption there, generalization
link |
out of distribution is that it's possible
link |
to create new ideas.
link |
It's possible that nobody's ever created any new ideas.
link |
And then with scaling GPT2 to GPT20,
link |
you would essentially generalize to all possible thoughts
link |
as humans can have, just to play devil's advocate.
link |
Right, I mean, how many new story ideas
link |
have we come up with since Shakespeare, right?
link |
It's just all different forms of love and drama and so on.
link |
Not sure if you read Biddle Lesson,
link |
a recent blog post by Rich Sutton.
link |
He basically says something that echoes
link |
some of the ideas that you've been talking about,
link |
which is, he says the biggest lesson
link |
that can be read from 70 years of AI research
link |
is that general methods that leverage computation
link |
are ultimately going to ultimately win out.
link |
Do you agree with this?
link |
So basically open AI in general about the ideas
link |
you're exploring about coming up with methods,
link |
whether it's GPT2 modeling or whether it's open AI5,
link |
playing Dota, where a general method
link |
is better than a more fine tuned, expert tuned method.
link |
Yeah, so I think that, well, one thing that I think
link |
was really interesting about the reaction
link |
to that blog post was that a lot of people have read this
link |
as saying that compute is all that matters.
link |
And that's a very threatening idea, right?
link |
And I don't think it's a true idea either, right?
link |
It's very clear that we have algorithmic ideas
link |
that have been very important for making progress.
link |
And to really build AI, you wanna push as far as you can
link |
on the computational scale, and you wanna push
link |
as far as you can on human ingenuity.
link |
And so I think you need both.
link |
But I think the way that you phrase the question
link |
is actually very good, right?
link |
That it's really about what kind of ideas
link |
should we be striving for?
link |
And absolutely, if you can find a scalable idea,
link |
you pour more compute into it,
link |
you pour more data into it, it gets better.
link |
Like that's the real Holy Grail.
link |
And so I think that the answer to the question,
link |
I think is yes, that's really how we think about it.
link |
And that part of why we're excited about the power
link |
of deep learning and the potential for building AGI
link |
is because we look at the systems that exist
link |
in the most successful AI systems,
link |
and we realize that you scale those up,
link |
they're gonna work better.
link |
And I think that that scalability is something
link |
that really gives us hope
link |
for being able to build transformative systems.
link |
So I'll tell you, this is partially an emotional,
link |
you know, a thing that a response that people often have,
link |
if compute is so important for state of the art performance,
link |
you know, individual developers,
link |
maybe a 13 year old sitting somewhere in Kansas
link |
or something like that, you know, they're sitting,
link |
they might not even have a GPU
link |
and or may have a single GPU, a 1080 or something like that.
link |
And there's this feeling like, well,
link |
how can I possibly compete or contribute to this world of AI
link |
if scale is so important?
link |
So if you can comment on that,
link |
and in general, do you think we need to also
link |
in the future focus on democratizing compute resources
link |
more or as much as we democratize the algorithms?
link |
Well, so the way that I think about it
link |
is that there's this space of possible progress, right?
link |
There's a space of ideas and sort of systems
link |
that will work, that will move us forward.
link |
And there's a portion of that space,
link |
and to some extent,
link |
an increasingly significant portion of that space
link |
that does just require massive compute resources.
link |
And for that, I think that the answer is kind of clear
link |
and that part of why we have the structure that we do
link |
is because we think it's really important
link |
to be pushing the scale
link |
and to be building these large clusters and systems.
link |
But there's another portion of the space
link |
that isn't about the large scale compute,
link |
that are these ideas that, and again,
link |
I think that for the ideas to really be impactful
link |
and really shine, that they should be ideas
link |
that if you scale them up,
link |
would work way better than they do at small scale.
link |
But you can discover them without massive
link |
computational resources.
link |
And if you look at the history of recent developments,
link |
you think about things like the GAN or the VAE,
link |
that these are ones that I think you could come up with them
link |
without having, and in practice,
link |
people did come up with them without having
link |
massive, massive computational resources.
link |
Right, I just talked to Ian Goodfellow,
link |
but the thing is the initial GAN
link |
produced pretty terrible results, right?
link |
So only because it was in a very specific,
link |
only because they're smart enough to know
link |
that this is quite surprising to generate anything
link |
Do you see a world, or is that too optimistic and dreamer,
link |
like, to imagine that the compute resources
link |
are something that's owned by governments
link |
and provided as a utility?
link |
Actually, to some extent, this question reminds me
link |
of a blog post from one of my former professors
link |
at Harvard, this guy, Matt Welch,
link |
who was a systems professor.
link |
I remember sitting in his tenure talk, right,
link |
and that he had literally just gotten tenure.
link |
He went to Google for the summer,
link |
and then decided he wasn't going back to academia, right?
link |
And kind of in his blog post, he makes this point
link |
that, look, as a systems researcher,
link |
that I come up with these cool system ideas,
link |
right, and kind of build a little proof of concept,
link |
and the best thing I could hope for
link |
is that the people at Google or Yahoo,
link |
which was around at the time,
link |
will implement it and actually make it work at scale, right?
link |
That's like the dream for me, right?
link |
I build the little thing, and they turn it into
link |
the big thing that's actually working.
link |
And for him, he said, I'm done with that.
link |
I want to be the person who's actually doing
link |
building and deploying.
link |
And I think that there's a similar dichotomy here, right?
link |
I think that there are people who really actually
link |
find value, and I think it is a valuable thing to do,
link |
to be the person who produces those ideas, right,
link |
who builds the proof of concept.
link |
And yeah, you don't get to generate
link |
the coolest possible GAN images,
link |
but you invented the GAN, right?
link |
And so there's a real trade off there.
link |
And I think that that's a very personal choice,
link |
but I think there's value in both sides.
link |
So do you think creating AGI, something,
link |
or some new models, we would see echoes of the brilliance
link |
even at the prototype level.
link |
So you would be able to develop those ideas
link |
without scale, the initial seeds.
link |
So take a look at, I always like to look at examples
link |
that exist, right, look at real precedent.
link |
And so take a look at the June 2018 model
link |
that we released that we scaled up to turn to GPT2.
link |
And you can see that at small scale,
link |
it set some records, right?
link |
This was the original GPT.
link |
We actually had some cool generations.
link |
They weren't nearly as amazing and really stunning
link |
as the GPT2 ones, but it was promising.
link |
It was interesting.
link |
And so I think it is the case that with a lot
link |
of these ideas that you see promise at small scale,
link |
but there isn't an asterisk here, a very big asterisk,
link |
which is sometimes we see behaviors that emerge
link |
that are qualitatively different
link |
from anything we saw at small scale.
link |
And that the original inventor of whatever algorithm
link |
looks at and says, I didn't think it could do that.
link |
This is what we saw in Dota, right?
link |
So PPO was created by John Shulman,
link |
who's a researcher here.
link |
And with Dota, we basically just ran PPO
link |
at massive, massive scale.
link |
And there's some tweaks in order to make it work,
link |
but fundamentally it's PPO at the core.
link |
And we were able to get this longterm planning,
link |
these behaviors to really play out on a time scale
link |
that we just thought was not possible.
link |
And John looked at that and was like,
link |
I didn't think it could do that.
link |
That's what happens when you're at three orders
link |
of magnitude more scale than you tested at.
link |
Yeah, but it still has the same flavors of,
link |
you know, at least echoes of the expected billions.
link |
Although I suspect with GPT,
link |
it's scaled more and more, you might get surprising things.
link |
So yeah, you're right.
link |
It's interesting that it's difficult to see
link |
how far an idea will go when it's scaled.
link |
It's an open question.
link |
Well, so to that point with Dota and PPO,
link |
like I mean, here's a very concrete one, right?
link |
It's like, it's actually one thing
link |
that's very surprising about Dota
link |
that I think people don't really pay that much attention to.
link |
Is the decree of generalization
link |
out of distribution that happens, right?
link |
That you have this AI that's trained
link |
against other bots for its entirety,
link |
the entirety of its existence.
link |
Sorry to take a step back.
link |
Can you talk through, you know, a story of Dota,
link |
a story of leading up to opening I5 and that past,
link |
and what was the process of self playing
link |
and so on of training on this?
link |
Dota is a complex video game
link |
and we started training,
link |
we started trying to solve Dota
link |
because we felt like this was a step towards the real world
link |
relative to other games like Chess or Go, right?
link |
Those very cerebral games
link |
where you just kind of have this board
link |
of very discreet moves.
link |
Dota starts to be much more continuous time.
link |
So you have this huge variety of different actions
link |
that you have a 45 minute game
link |
with all these different units
link |
and it's got a lot of messiness to it
link |
that really hasn't been captured by previous games.
link |
And famously all of the hard coded bots for Dota
link |
were terrible, right?
link |
It's just impossible to write anything good for it
link |
because it's so complex.
link |
And so this seemed like a really good place
link |
to push what's the state of the art
link |
in reinforcement learning.
link |
And so we started by focusing on the one versus one
link |
version of the game and we're able to solve that.
link |
We're able to beat the world champions
link |
and the learning, the skill curve
link |
was this crazy exponential, right?
link |
It was like constantly we were just scaling up,
link |
that we were fixing bugs and that you look
link |
at the skill curve and it was really a very, very smooth one.
link |
So it's actually really interesting
link |
to see how that like human iteration loop
link |
yielded very steady exponential progress.
link |
And to one side note, first of all,
link |
it's an exceptionally popular video game.
link |
The side effect is that there's a lot
link |
of incredible human experts at that video game.
link |
So the benchmark that you're trying to reach is very high.
link |
And the other, can you talk about the approach
link |
that was used initially and throughout training
link |
these agents to play this game?
link |
And so the approach that we used is self play.
link |
And so you have two agents that don't know anything.
link |
They battle each other,
link |
they discover something a little bit good
link |
and now they both know it.
link |
And they just get better and better and better without bound.
link |
And that's a really powerful idea, right?
link |
That we then went from the one versus one version
link |
of the game and scaled up to five versus five, right?
link |
So you think about kind of like with basketball
link |
where you have this like team sport
link |
and you need to do all this coordination
link |
and we were able to push the same idea,
link |
the same self play to really get to the professional level
link |
at the full five versus five version of the game.
link |
And the things that I think are really interesting here
link |
is that these agents in some ways
link |
they're almost like an insect like intelligence, right?
link |
Where they have a lot in common with how an insect is trained,
link |
An insect kind of lives in this environment for a very long time
link |
or the ancestors of this insect have been around
link |
for a long time and had a lot of experience.
link |
I think it's baked into this agent.
link |
And it's not really smart in the sense of a human, right?
link |
It's not able to go and learn calculus,
link |
but it's able to navigate its environment extremely well.
link |
And it's able to handle unexpected things
link |
in the environment that's never seen before, pretty well.
link |
And we see the same sort of thing with our Dota bots, right?
link |
That they're able to, within this game,
link |
they're able to play against humans,
link |
which is something that never existed
link |
in its evolutionary environment.
link |
Totally different play styles from humans versus the bots.
link |
And yet it's able to handle it extremely well.
link |
And that's something that I think was very surprising to us
link |
was something that doesn't really emerge
link |
from what we've seen with PPO at smaller scale, right?
link |
And the kind of scale we're running this stuff at
link |
was I could take 100,000 CPU cores,
link |
running with like hundreds of GPUs.
link |
It was probably about something like hundreds of years
link |
of experience going into this bot every single real day.
link |
And so that scale is massive.
link |
And we start to see very different kinds of behaviors
link |
out of the algorithms that we all know and love.
link |
Dota, you mentioned, beat the world expert 1v1.
link |
And then you weren't able to win 5v5 this year
link |
at the best players in the world.
link |
So what's the comeback story?
link |
First of all, talk through that.
link |
That was an exceptionally exciting event.
link |
And what's the following months in this year look like?
link |
So one thing that's interesting is that we lose all the time.
link |
Because we play here.
link |
So the Dota team at OpenAI, we play the bot
link |
against better players than our system all the time.
link |
Or at least we used to, right?
link |
Like the first time we lost publicly was we went up
link |
on stage at the international and we played against some
link |
of the best teams in the world.
link |
And we ended up losing both games.
link |
But we give them a run for their money, right?
link |
That both games were kind of 30 minutes, 25 minutes.
link |
And they went back and forth, back and forth, back and forth.
link |
And so I think that really shows that we're
link |
at the professional level.
link |
And that kind of looking at those games,
link |
we think that the coin could have gone a different direction
link |
and we could have had some wins.
link |
And so that was actually very encouraging for us.
link |
And you know, it's interesting because the international was
link |
at a fixed time, right?
link |
So we knew exactly what day we were going to be playing.
link |
And we pushed as far as we could, as fast as we could.
link |
Two weeks later, we had a bot that had an 80% win rate
link |
versus the one that played at TI.
link |
So the March of Progress, you know,
link |
that you should think of as a snapshot rather
link |
than as an end state.
link |
And so in fact, we'll be announcing our finals pretty soon.
link |
I actually think that we'll announce our final match
link |
prior to this podcast being released.
link |
So there should be, we'll be playing against the world
link |
And you know, for us, it's really less about,
link |
like the way that we think about what's upcoming
link |
is the final milestone, the final competitive milestone
link |
for the project, right?
link |
That our goal in all of this isn't really
link |
about beating humans at Dota.
link |
Our goal is to push the state of the art
link |
in reinforcement learning.
link |
And we've done that, right?
link |
And we've actually learned a lot from our system
link |
and that we have, you know, I think a lot of exciting
link |
next steps that we want to take.
link |
And so, you know, kind of the final showcase
link |
of what we built, we're going to do this match.
link |
But for us, it's not really the success or failure
link |
to see, you know, do we have the coin flip go
link |
in our direction or against.
link |
Where do you see the field of deep learning
link |
heading in the next few years?
link |
Where do you see the work in reinforcement learning
link |
perhaps heading and more specifically with OpenAI,
link |
all the exciting projects that you're working on,
link |
what does 2019 hold for you?
link |
I will put an atrocious on that and just say,
link |
you know, I think that it's about ideas plus scale.
link |
So that's a really good point.
link |
So the question, in terms of ideas,
link |
you have a lot of projects that are exploring
link |
different areas of intelligence.
link |
And the question is, when you think of scale,
link |
do you think about growing the scale
link |
of those individual projects,
link |
or do you think about adding new projects?
link |
And sorry, if you were thinking about adding new projects,
link |
or if you look at the past, what's the process
link |
of coming up with new projects and new ideas?
link |
So we really have a life cycle of project here.
link |
So we start with a few people just working
link |
on a small scale idea.
link |
And language is actually a very good example of this,
link |
that it was really, you know, one person here
link |
who was pushing on language for a long time.
link |
I mean, then you get signs of life, right?
link |
And so this is like, let's say, you know,
link |
with the original GPT, we had something that was interesting.
link |
And we said, okay, it's time to scale this, right?
link |
It's time to put more people on it,
link |
put more computational resources behind it,
link |
and then we just kind of keep pushing and keep pushing.
link |
And the end state is something that looks like
link |
Dota or Robotics, where you have a large team of,
link |
you know, 10 or 15 people that are running things
link |
at very large scale, and that you're able to really have
link |
material engineering and, you know,
link |
sort of machine learning science coming together
link |
to make systems that work and get material results
link |
that just would have been impossible otherwise.
link |
So we do that whole life cycle.
link |
We've done it a number of times, you know, typically end to end.
link |
It's probably two years or so to do it.
link |
You know, the organization's been around for three years,
link |
so maybe we'll find that we also have
link |
longer life cycle projects.
link |
But, you know, we work up to those.
link |
So one team that we're actually just starting,
link |
Illy and I, are kicking off a new team
link |
called the Reasoning Team, and this is to really try to tackle
link |
how do you get neural networks to reason?
link |
And we think that this will be a long term project.
link |
It's one that we're very excited about.
link |
In terms of reasoning, super exciting topic,
link |
what kind of benchmarks, what kind of tests of reasoning
link |
What would, if you set back,
link |
whatever drink, and you would be impressed
link |
that this system is able to do something,
link |
what would that look like?
link |
So some kind of logic, and especially mathematical logic.
link |
I think so, right?
link |
And I think that there's kind of other problems
link |
that are dual to theorem proving in particular.
link |
You know, you think about programming,
link |
you think about even like security analysis of code,
link |
that these all kind of capture the same sorts of core reasoning
link |
and being able to do some out of distribution generalization.
link |
It would be quite exciting if OpenAI Reasoning Team
link |
was able to prove that P equals NP.
link |
That would be very nice.
link |
It would be very, very exciting especially.
link |
If it turns out that P equals NP,
link |
that'll be interesting too.
link |
It would be ironic and humorous.
link |
So what problem stands out to you as the most exciting
link |
and challenging impactful to the work for us as a community
link |
in general and for OpenAI this year?
link |
You mentioned reasoning.
link |
I think that's a heck of a problem.
link |
So I think reasoning is an important one.
link |
I think it's going to be hard to get good results in 2019.
link |
You know, again, just like we think about the lifecycle,
link |
I think for 2019, language modeling seems to be kind of
link |
on that ramp, right?
link |
It's at the point that we have a technique that works.
link |
We want to scale 100x, 1000x, see what happens.
link |
Do you think we're living in a simulation?
link |
I think it's hard to have a real opinion about it.
link |
It's actually interesting.
link |
I separate out things that I think can have yield
link |
materially different predictions about the world
link |
from ones that are just kind of fun to speculate about.
link |
And I kind of view simulation as more like,
link |
is there a flying teapot between Mars and Jupiter?
link |
Like, maybe, but it's a little bit hard to know
link |
what that would mean for my life.
link |
So there is something actionable.
link |
So some of the best work opening as done
link |
is in the field of reinforcement learning.
link |
And some of the success of reinforcement learning
link |
come from being able to simulate the problem you're trying
link |
So do you have a hope for reinforcement,
link |
for the future of reinforcement learning
link |
and for the future of simulation?
link |
Like, whether we're talking about autonomous vehicles
link |
or any kind of system, do you see that scaling?
link |
So we'll be able to simulate systems and, hence,
link |
be able to create a simulator that echoes our real world
link |
and proving once and for all, even though you're denying it
link |
that we're living in a simulation.
link |
I feel like I've used that for questions, right?
link |
So, you know, kind of at the core there of, like,
link |
can we use simulation for self driving cars?
link |
Take a look at our robotic system, DACTL, right?
link |
That was trained in simulation using the Dota system, in fact.
link |
And it transfers to a physical robot.
link |
And I think everyone looks at our Dota system,
link |
they're like, okay, it's just a game.
link |
How are you ever going to escape to the real world?
link |
And the answer is, well, we did it with the physical robot,
link |
the no one could program.
link |
And so I think the answer is simulation goes a lot further
link |
than you think if you apply the right techniques to it.
link |
Now, there's a question of, you know,
link |
are the beings in that simulation going to wake up
link |
and have consciousness?
link |
I think that one seems a lot harder to, again, reason about.
link |
I think that, you know, you really should think about, like,
link |
where exactly does human consciousness come from
link |
in our own self awareness?
link |
And, you know, is it just that, like,
link |
once you have, like, a complicated enough neural net,
link |
do you have to worry about the agent's feeling pain?
link |
And, you know, I think there's, like,
link |
interesting speculation to do there.
link |
But, you know, again, I think it's a little bit hard to know for sure.
link |
Well, let me just keep with the speculation.
link |
Do you think to create intelligence, general intelligence,
link |
you need one consciousness and two a body?
link |
Do you think any of those elements are needed,
link |
or is intelligence something that's orthogonal to those?
link |
I'll stick to the kind of, like, the non grand answer first,
link |
So the non grand answer is just to look at,
link |
you know, what are we already making work?
link |
You look at GPT2, a lot of people would have said
link |
that to even get these kinds of results,
link |
you need real world experience.
link |
You need a body, you need grounding.
link |
How are you supposed to reason about any of these things?
link |
How are you supposed to, like, even kind of know
link |
about smoke and fire and those things
link |
if you've never experienced them?
link |
And GPT2 shows that you can actually go way further
link |
than that kind of reasoning would predict.
link |
So I think that in terms of, do we need consciousness?
link |
Do we need a body?
link |
It seems the answer is probably not, right?
link |
That we could probably just continue to push
link |
kind of the systems we have.
link |
They already feel general.
link |
They're not as competent or as general
link |
or able to learn as quickly as an AGI would,
link |
but, you know, they're at least like kind of proto AGI
link |
in some way, and they don't need any of those things.
link |
Now, let's move to the grand answer, which is, you know,
link |
if our neural nets consciousness,
link |
nets conscious already, would we ever know?
link |
How can we tell, right?
link |
And, you know, here's where the speculation starts
link |
to become, you know, at least interesting or fun
link |
and maybe a little bit disturbing,
link |
depending on where you take it.
link |
But it certainly seems that when we think about animals,
link |
that there's some continuum of consciousness.
link |
You know, my cat, I think, is conscious in some way, right?
link |
You know, not as conscious as a human.
link |
And you could imagine that you could build
link |
a little consciousness meter, right?
link |
You point at a cat, it gives you a little reading,
link |
you point at a human, it gives you much bigger reading.
link |
What would happen if you pointed one of those
link |
at a Dota neural net?
link |
And if you're training this massive simulation,
link |
do the neural nets feel pain?
link |
You know, it becomes pretty hard to know
link |
that the answer is no, and it becomes pretty hard
link |
to really think about what that would mean
link |
if the answer were yes.
link |
And it's very possible, you know, for example,
link |
you could imagine that maybe the reason
link |
that humans have consciousness
link |
is because it's a convenient computational shortcut, right?
link |
If you think about it, if you have a being
link |
that wants to avoid pain, which seems pretty important
link |
to survive in this environment
link |
and wants to, like, you know, eat food,
link |
then maybe the best way of doing it
link |
is to have a being that's conscious, right?
link |
That, you know, in order to succeed in the environment,
link |
you need to have those properties
link |
and how are you supposed to implement them?
link |
And maybe this consciousness is a way of doing that.
link |
If that's true, then actually maybe we should expect
link |
that really competent reinforcement learning agents
link |
will also have consciousness.
link |
But, you know, that's a big if.
link |
And I think there are a lot of other arguments
link |
that you can make in other directions.
link |
I think that's a really interesting idea
link |
that even GPT2 has some degree of consciousness.
link |
That's something that's actually not as crazy
link |
It's useful to think about as we think about
link |
what it means to create intelligence of a dog,
link |
intelligence of a cat, and the intelligence of a human.
link |
So, last question, do you think we will ever fall in love,
link |
like in the movie, Her, with an artificial intelligence system
link |
or an artificial intelligence system
link |
falling in love with a human?
link |
If there's any better way to end it is on love.
link |
So, Greg, thanks so much for talking today.
link |
Thank you for having me.