back to indexNick Bostrom: Simulation and Superintelligence | Lex Fridman Podcast #83
link |
The following is a conversation with Nick Bostrom, a philosopher at University of Oxford
link |
and the director of the Future of Humanity Institute.
link |
He has worked on fascinating and important ideas in existential risk, simulation hypothesis,
link |
human enhancement ethics, and the risks of superintelligent AI systems, including in
link |
his book, Superintelligence.
link |
I can see talking to Nick multiple times in this podcast, many hours each time, because
link |
he has done some incredible work in artificial intelligence, in technology, space, science,
link |
and really philosophy in general, but we have to start somewhere.
link |
This conversation was recorded before the outbreak of the coronavirus pandemic that
link |
both Nick and I, I'm sure, will have a lot to say about next time we speak, and perhaps
link |
that is for the best, because the deepest lessons can be learned only in retrospect
link |
when the storm has passed.
link |
I do recommend you read many of his papers on the topic of existential risk, including
link |
the technical report titled Global Catastrophic Risks Survey that he coauthored with Anders
link |
For everyone feeling the medical, psychological, and financial burden of this crisis, I'm
link |
sending love your way.
link |
We're in this together.
link |
We'll beat this thing.
link |
This is the Artificial Intelligence Podcast.
link |
If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcast, support
link |
it on Patreon, or simply connect with me on Twitter at Lex Friedman, spelled F R I D M
link |
As usual, I'll do one or two minutes of ads now and never any ads in the middle that
link |
can break the flow of the conversation.
link |
I hope that works for you and doesn't hurt the listening experience.
link |
This show is presented by Cash App, the number one finance app in the App Store.
link |
When you get it, use code LEXPODCAST.
link |
Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with
link |
as little as one dollar.
link |
Since Cash App does fractional share trading, let me mention that the order execution algorithm
link |
that works behind the scenes to create the abstraction of fractional orders is an algorithmic
link |
So big props to the Cash App engineers for solving a hard problem that in the end provides
link |
an easy interface that takes a step up to the next layer of abstraction over the stock
link |
market, making trading more accessible for new investors and diversification much easier.
link |
So again, if you get Cash App from the App Store, Google Play, and use the code LEXPODCAST,
link |
you get $10, and Cash App will also donate $10 to FIRST, an organization that is helping
link |
to advance robotics and STEM education for young people around the world.
link |
And now, here's my conversation with Nick Bostrom.
link |
At the risk of asking the Beatles to play yesterday or the Rolling Stones to play Satisfaction,
link |
let me ask you the basics.
link |
What is the simulation hypothesis?
link |
That we are living in a computer simulation.
link |
What is a computer simulation?
link |
How are we supposed to even think about that?
link |
Well, so the hypothesis is meant to be understood in a literal sense, not that we can kind of
link |
metaphorically view the universe as an information processing physical system, but that there
link |
is some advanced civilization who built a lot of computers and that what we experience
link |
is an effect of what's going on inside one of those computers so that the world around
link |
us, our own brains, everything we see and perceive and think and feel would exist because
link |
this computer is running certain programs.
link |
So do you think of this computer as something similar to the computers of today, these deterministic
link |
sort of Turing machine type things?
link |
Is that what we're supposed to imagine or we're supposed to think of something more
link |
like a quantum mechanical system?
link |
Something much bigger, something much more complicated, something much more mysterious
link |
from our current perspective?
link |
The ones we have today would do fine, I mean, bigger, certainly.
link |
You'd need more memory and more processing power.
link |
I don't think anything else would be required.
link |
Now, it might well be that they do have additional, maybe they have quantum computers and other
link |
things that would give them even more of, it seems kind of plausible, but I don't think
link |
it's a necessary assumption in order to get to the conclusion that a technologically
link |
mature civilization would be able to create these kinds of computer simulations with conscious
link |
beings inside them.
link |
So do you think the simulation hypothesis is an idea that's most useful in philosophy,
link |
computer science, physics, sort of where do you see it having valuable kind of starting
link |
point in terms of a thought experiment of it?
link |
I guess it's more informative and interesting and maybe important, but it's not designed
link |
to be useful for something else.
link |
Okay, interesting, sure.
link |
But is it philosophically interesting or is there some kind of implications of computer
link |
science and physics?
link |
I think not so much for computer science or physics per se.
link |
Certainly it would be of interest in philosophy, I think also to say cosmology or physics in
link |
as much as you're interested in the fundamental building blocks of the world and the rules
link |
If we are in a simulation, there is then the possibility that say physics at the level
link |
where the computer running the simulation could be different from the physics governing
link |
phenomena in the simulation.
link |
So I think it might be interesting from point of view of religion or just for kind of trying
link |
to figure out what the heck is going on.
link |
So we mentioned the simulation hypothesis so far.
link |
There is also the simulation argument, which I tend to make a distinction.
link |
So simulation hypothesis, we are living in a computer simulation.
link |
Simulation argument, this argument that tries to show that one of three propositions is
link |
true, one of which is the simulation hypothesis, but there are two alternatives in the original
link |
simulation argument, which we can get to.
link |
Yeah, let's go there.
link |
By the way, confusing terms because people will, I think, probably naturally think simulation
link |
argument equals simulation hypothesis, just terminology wise.
link |
But let's go there.
link |
So simulation hypothesis means that we are living in a simulations, the hypothesis that
link |
we're living in a simulation, simulation argument has these three complete possibilities that
link |
cover all possibilities.
link |
So it's like a disjunction.
link |
It says at least one of these three is true, although it doesn't on its own tell us which
link |
So the first one is that almost all civilizations that are current stage of technological development
link |
go extinct before they reach technological maturity.
link |
So there is some great filter that makes it so that basically none of the civilizations
link |
throughout maybe a vast cosmos will ever get to realize the full potential of technological
link |
And this could be, theoretically speaking, this could be because most civilizations kill
link |
themselves too eagerly or destroy themselves too eagerly, or it might be super difficult
link |
to build a simulation.
link |
So the span of time.
link |
Theoretically it could be both.
link |
Now I think it looks like we would technologically be able to get there in a time span that
link |
is short compared to, say, the lifetime of planets and other sort of astronomical processes.
link |
So your intuition is to build a simulation is not...
link |
Well, so this is interesting concept of technological maturity.
link |
It's kind of an interesting concept to have other purposes as well.
link |
We can see even based on our current limited understanding what some lower bound would
link |
be on the capabilities that you could realize by just developing technologies that we already
link |
So for example, one of my research fellows here, Eric Drexler, back in the 80s, studied
link |
molecular manufacturing.
link |
That is you could analyze using theoretical tools and computer modeling the performance
link |
of various molecularly precise structures that we didn't then and still don't today
link |
have the ability to actually fabricate.
link |
But you could say that, well, if we could put these atoms together in this way, then
link |
the system would be stable and it would rotate at this speed and have all these computational
link |
And he also outlined some pathways that would enable us to get to this kind of molecularly
link |
manufacturing in the fullness of time.
link |
And you could do other studies we've done.
link |
You could look at the speed at which, say, it would be possible to colonize the galaxy
link |
if you had mature technology.
link |
We have an upper limit, which is the speed of light.
link |
We have sort of a lower current limit, which is how fast current rockets go.
link |
We know we can go faster than that by just making them bigger and have more fuel and
link |
We can then start to describe the technological affordances that would exist once a civilization
link |
has had enough time to develop, at least those technologies we already know are possible.
link |
Then maybe they would discover other new physical phenomena as well that we haven't realized
link |
that would enable them to do even more.
link |
But at least there is this kind of basic set of capabilities.
link |
Can you just link on that, how do we jump from molecular manufacturing to deep space
link |
exploration to mature technology?
link |
What's the connection there?
link |
Well, so these would be two examples of technological capability sets that we can have a high degree
link |
of confidence are physically possible in our universe and that a civilization that was
link |
allowed to continue to develop its science and technology would eventually attain.
link |
You can intuit like, we can kind of see the set of breakthroughs that are likely to happen.
link |
So you can see like, what did you call it, the technological set?
link |
With computers, maybe it's easiest.
link |
One is we could just imagine bigger computers using exactly the same parts that we have.
link |
So you can kind of scale things that way, right?
link |
But you could also make processors a bit faster.
link |
If you had this molecular nanotechnology that Eric Drexler described, he characterized a
link |
kind of crude computer built with these parts that would perform at a million times the
link |
human brain while being significantly smaller, the size of a sugar cube.
link |
And he made no claim that that's the optimum computing structure, like for all you know,
link |
we could build faster computers that would be more efficient, but at least you could
link |
do that if you had the ability to do things that were atomically precise.
link |
I mean, so you can then combine these two.
link |
You could have this kind of nanomolecular ability to build things atom by atom and then
link |
say at this as a spatial scale that would be attainable through space colonizing technology.
link |
You could then start, for example, to characterize a lower bound on the amount of computing power
link |
that a technologically mature civilization would have.
link |
If it could grab resources, you know, planets and so forth, and then use this molecular
link |
nanotechnology to optimize them for computing, you'd get a very, very high lower bound on
link |
the amount of compute.
link |
So sorry, just to define some terms, so technologically mature civilization is one that took that
link |
piece of technology to its lower bound.
link |
What is a technologically mature civilization?
link |
So that means it's a stronger concept than we really need for the simulation hypothesis.
link |
I just think it's interesting in its own right.
link |
So it would be the idea that there is some stage of technological development where you've
link |
basically maxed out, that you developed all those general purpose, widely useful technologies
link |
that could be developed, or at least kind of come very close to the, you know, 99.9%
link |
there or something.
link |
So that's an independent question.
link |
You can think either that there is such a ceiling, or you might think it just goes,
link |
the technology tree just goes on forever.
link |
Where does your sense fall?
link |
I would guess that there is a maximum that you would start to asymptote towards.
link |
So new things won't keep springing up, new ceilings.
link |
In terms of basic technological capabilities, I think that, yeah, there is like a finite
link |
set of laws that can exist in this universe.
link |
Moreover, I mean, I wouldn't be that surprised if we actually reached close to that level
link |
fairly shortly after we have, say, machine superintelligence.
link |
So I don't think it would take millions of years for a human originating civilization
link |
to begin to do this.
link |
It's more likely to happen on historical timescales.
link |
But that's an independent speculation from the simulation argument.
link |
I mean, for the purpose of the simulation argument, it doesn't really matter whether
link |
it goes indefinitely far up or whether there is a ceiling, as long as we know we can at
link |
least get to a certain level.
link |
And it also doesn't matter whether that's going to happen in 100 years or 5,000 years
link |
or 50 million years.
link |
Like the timescales really don't make any difference for this.
link |
Can you look on that a little bit?
link |
Like there's a big difference between 100 years and 10 million years.
link |
So it doesn't really not matter because you just said it doesn't matter if we jump scales
link |
to beyond historical scales.
link |
So we described that.
link |
So for the simulation argument, sort of doesn't it matter that we if it takes 10 million years,
link |
it gives us a lot more opportunity to destroy civilization in the meantime?
link |
Yeah, well, so it would shift around the probabilities between these three alternatives.
link |
That is, if we are very, very far away from being able to create these simulations, if
link |
it's like, say, billions of years into the future, then it's more likely that we will
link |
fail ever to get there.
link |
There's more time for us to kind of go extinct along the way.
link |
And so this is similarly for other civilizations.
link |
So it is important to think about how hard it is to build a simulation.
link |
In terms of figuring out which of the disjuncts.
link |
But for the simulation argument itself, which is agnostic as to which of these three alternatives
link |
It's like you don't have to like the simulation argument would be true whether or not we thought
link |
this could be done in 500 years or it would take 500 million years.
link |
The simulation argument stands.
link |
I mean, I'm sure there might be some people who oppose it, but it doesn't matter.
link |
I mean, it's very nice those three cases cover it.
link |
But the fun part is at least not saying what the probabilities are, but kind of thinking
link |
about kind of intuiting reasoning about what's more likely, what are the kind of things that
link |
would make some of the arguments less and more so like.
link |
But let's actually, I don't think we went through them.
link |
So number one is we destroy ourselves before we ever create simulation.
link |
So that's kind of sad, but we have to think not just what might destroy us.
link |
I mean, so there could be some whatever disaster, some meteor slamming the earth a few years
link |
from now that could destroy us.
link |
But you'd have to postulate in order for this first disjunct to be true that almost all
link |
civilizations throughout the cosmos also failed to reach technological maturity.
link |
And the underlying assumption there is that there is likely a very large number of other
link |
intelligent civilizations.
link |
Well, if there are, yeah, then they would virtually all have to succumb in the same
link |
I mean, then that leads off another, I guess there are a lot of little digressions that
link |
Definitely, let's go there.
link |
Keep dragging us back.
link |
Well, there are these, there is a set of basic questions that always come up in conversations
link |
with interesting people, like the Fermi paradox, like there's like, you could almost define
link |
whether a person is interesting, whether at some point the question of the Fermi paradox
link |
comes up, like, well, so for what it's worth, it looks to me that the universe is very big.
link |
I mean, in fact, according to the most popular current cosmological theories, infinitely
link |
And so then it would follow pretty trivially that it would contain a lot of other civilizations,
link |
in fact, infinitely many.
link |
If you have some local stochasticity and infinitely many, it's like, you know, infinitely many
link |
lumps of matter, one next to another, there's kind of random stuff in each one, then you're
link |
going to get all possible outcomes with probability one infinitely repeated.
link |
So then certainly there would be a lot of extraterrestrials out there.
link |
Even short of that, if the universe is very big, that might be a finite but large number.
link |
If we were literally the only one, yeah, then of course, if we went extinct, then all of
link |
civilizations at our current stage would have gone extinct before becoming technological
link |
So then it kind of becomes trivially true that a very high fraction of those went extinct.
link |
But if we think there are many, I mean, it's interesting, because there are certain things
link |
that possibly could kill us, like if you look at existential risks, and it might be a different,
link |
like the best answer to what would be most likely to kill us might be a different answer
link |
than the best answer to the question, if there is something that kills almost everyone, what
link |
Because that would have to be some risk factor that was kind of uniform overall possible
link |
So in this, for the sake of this argument, you have to think about not just us, but like
link |
every civilization dies out before they create the simulation or something very close to
link |
So what's number two in the number two is the convergence hypothesis that is that maybe
link |
like a lot of some of these civilizations do make it through to technological maturity,
link |
but out of those who do get there, they all lose interest in creating these simulations.
link |
So they just have the capability of doing it, but they choose not to.
link |
Not just a few of them decide not to, but out of a million, maybe not even a single
link |
one of them would do it.
link |
And I think when you say lose interest, that sounds like unlikely because it's like they
link |
get bored or whatever, but it could be so many possibilities within that.
link |
I mean, losing interest could be, it could be anything from it being exceptionally difficult
link |
to do to fundamentally changing the sort of the fabric of reality.
link |
If you do it is ethical concerns, all those kinds of things could be exceptionally strong
link |
Well, certainly, I mean, yeah, ethical concerns.
link |
I mean, not really too difficult to do.
link |
I mean, in a sense, that's the first assumption that you get to technological maturity where
link |
you would have the ability using only a tiny fraction of your resources to create many,
link |
So it wouldn't be the case that they would need to spend half of their GDP forever in
link |
order to create one simulation and they had this like difficult debate about whether they
link |
should invest half of their GDP for this.
link |
It would more be like, well, if any little fraction of the civilization feels like doing
link |
this at any point during maybe their millions of years of existence, then that would be
link |
millions of simulations.
link |
But certainly, there could be many conceivable reasons for why there would be this convert,
link |
many possible reasons for not running ancestor simulations or other computer simulations,
link |
even if you could do so cheaply.
link |
By the way, what's an ancestor simulation?
link |
Well, that would be the type of computer simulation that would contain people like those we think
link |
have lived on our planet in the past and like ourselves in terms of the types of experiences
link |
they have and where those simulated people are conscious.
link |
So like not just simulated in the same sense that a non player character would be simulated
link |
in the current computer game where it's kind of has like an avatar body and then a very
link |
simple mechanism that moves it forward or backwards.
link |
But something where the simulated being has a brain, let's say that's simulated at a sufficient
link |
level of granularity that it would have the same subjective experiences as we have.
link |
So where does consciousness fit into this?
link |
Do you think simulation, I guess there are different ways to think about how this can
link |
be simulated, just like you're talking about now.
link |
Do we have to simulate each brain within the larger simulation?
link |
Is it enough to simulate just the brain, just the minds and not the simulation, not the
link |
Like, is there a different ways to think about this?
link |
Yeah, I guess there is a kind of premise in the simulation argument rolled in from philosophy
link |
of mind that is that it would be possible to create a conscious mind in a computer.
link |
And that what determines whether some system is conscious or not is not like whether it's
link |
built from organic biological neurons, but maybe something like what the structure of
link |
the computation is that it implements.
link |
So we can discuss that if we want, but I think it would be more forward as far as my view
link |
that it would be sufficient, say, if you had a computation that was identical to the computation
link |
in the human brain down to the level of neurons.
link |
So if you had a simulation with 100 billion neurons connected in the same way as the human
link |
brain, and you then roll that forward with the same kind of synaptic weights and so forth,
link |
so you actually had the same behavior coming out of this as a human with that brain would
link |
have done, then I think that would be conscious.
link |
Now it's possible you could also generate consciousness without having that detailed
link |
assimilation, there I'm getting more uncertain exactly how much you could simplify or abstract
link |
Can you look on that?
link |
I missed where you're placing consciousness in the second.
link |
Well, so if you are a computationalist, do you think that what creates consciousness
link |
is the implementation of a computation?
link |
Some property, emergent property of the computation itself.
link |
Yeah, you could say that.
link |
But then the question is, what's the class of computations such that when they are run,
link |
consciousness emerges?
link |
So if you just have something that adds one plus one plus one plus one, like a simple
link |
computation, you think maybe that's not going to have any consciousness.
link |
If on the other hand, the computation is one like our human brains are performing, where
link |
as part of the computation, there is a global workspace, a sophisticated attention mechanism,
link |
there is self representations of other cognitive processes and a whole lot of other things
link |
that possibly would be conscious.
link |
And in fact, if it's exactly like ours, I think definitely it would.
link |
But exactly how much less than the full computation that the human brain is performing would be
link |
required is a little bit, I think, of an open question.
link |
He asked another interesting question as well, which is, would it be sufficient to just have
link |
say the brain or would you need the environment in order to generate the same kind of experiences
link |
And there is a bunch of stuff we don't know.
link |
I mean, if you look at, say, current virtual reality environments, one thing that's clear
link |
is that we don't have to simulate all details of them all the time in order for, say, the
link |
human player to have the perception that there is a full reality and that you can have say
link |
procedurally generated where you might only render a scene when it's actually within the
link |
view of the player character.
link |
And so similarly, if this environment that we perceive is simulated, it might be that
link |
all of the parts that come into our view are rendered at any given time.
link |
And a lot of aspects that never come into view, say the details of this microphone I'm
link |
talking into, exactly what each atom is doing at any given point in time, might not be part
link |
of the simulation, only a more coarse grained representation.
link |
So that to me is actually from an engineering perspective, why the simulation hypothesis
link |
is really interesting to think about is how difficult is it to fake sort of in a virtual
link |
reality context, I don't know if fake is the right word, but to construct a reality that
link |
is sufficiently real to us to be immersive in the way that the physical world is.
link |
I think that's actually probably an answerable question of psychology, of computer science,
link |
of how, where's the line where it becomes so immersive that you don't want to leave
link |
Yeah, or that you don't realize while you're in it that it is a virtual world.
link |
Yeah, those are two actually questions, yours is the more sort of the good question about
link |
the realism, but mine, from my perspective, what's interesting is it doesn't have to be
link |
real, but how can we construct a world that we wouldn't want to leave?
link |
Yeah, I mean, I think that might be too low a bar, I mean, if you think, say when people
link |
first had pong or something like that, I'm sure there were people who wanted to keep
link |
playing it for a long time because it was fun and they wanted to be in this little world.
link |
I'm not sure we would say it's immersive, I mean, I guess in some sense it is, but like
link |
an absorbing activity doesn't even have to be.
link |
But they left that world though, that's the thing.
link |
So like, I think that bar is deceivingly high.
link |
So they eventually left, so you can play pong or Starcraft or whatever more sophisticated
link |
games for hours, for months, you know, while the work has to be in a big addiction, but
link |
eventually they escaped that.
link |
So you mean when it's absorbing enough that you would spend your entire, you would choose
link |
to spend your entire life in there.
link |
And then thereby changing the concept of what reality is, because your reality becomes the
link |
Not because you're fooled, but because you've made that choice.
link |
Yeah, and it made, different people might have different preferences regarding that.
link |
Some might, even if you had any perfect virtual reality, might still prefer not to spend the
link |
rest of their lives there.
link |
I mean, in philosophy, there's this experience machine, thought experiment.
link |
Have you come across this?
link |
So Robert Nozick had this thought experiment where you imagine some crazy super duper neuroscientist
link |
of the future have created a machine that could give you any experience you want if
link |
you step in there.
link |
And for the rest of your life, you can kind of pre programmed it in different ways.
link |
So your fun dreams could come true, you could, whatever you dream, you want to be a great
link |
artist, a great lover, like have a wonderful life, all of these things.
link |
If you step into the experience machine will be your experiences, constantly happy.
link |
But you would kind of disconnect from the rest of reality and you would float there
link |
And so Nozick thought that most people would choose not to enter the experience machine.
link |
I mean, many might want to go there for a holiday, but they wouldn't want to have to
link |
check out of existence permanently.
link |
And so he thought that was an argument against certain views of value according to what we
link |
value is a function of what we experience.
link |
Because in the experience machine, you could have any experience you want, and yet many
link |
people would think that would not be much value.
link |
So therefore, what we value depends on other things than what we experience.
link |
So okay, can you can you take that argument further?
link |
What about the fact that maybe what we value is the up and down of life?
link |
So you could have up and downs in the experience machine, right?
link |
But what can't you have in the experience machine?
link |
Well, I mean, that then becomes an interesting question to explore.
link |
But for example, real connection with other people, if the experience machine is a solo
link |
machine where it's only you, like that's something you wouldn't have there.
link |
You would have this subjective experience that would be like fake people.
link |
But when if you gave somebody flowers, there wouldn't be anybody there who actually got
link |
It would just be a little simulation of somebody smiling.
link |
But the simulation would not be the kind of simulation I'm talking about in the simulation
link |
argument where the simulated creature is conscious, it would just be a kind of smiley face that
link |
would look perfectly real to you.
link |
So we're now drawing a distinction between appear to be perfectly real and actually being
link |
Um, so that could be one thing, I mean, like a big impact on history, maybe is also something
link |
you won't have if you check into this experience machine.
link |
So some people might actually feel the life I want to have for me is one where I have
link |
a big positive impact on history unfolds.
link |
So you could kind of explore these different possible explanations for why it is you wouldn't
link |
want to go into the experience machine if that's, if that's what you feel.
link |
And one interesting observation regarding this Nozick thought experiment and the conclusions
link |
he wanted to draw from it is how much is a kind of a status quo effect.
link |
So a lot of people might not want to get this on current reality to plug into this dream
link |
But if they instead were told, well, what you've experienced up to this point was a
link |
dream now, do you want to disconnect from this and enter the real world when you have
link |
no idea maybe what the real world is, or maybe you could say, well, you're actually a farmer
link |
in Peru, growing, you know, peanuts, and you could live for the rest of your life in this
link |
way, or would you want to continue your dream life as Alex Friedman going around the world
link |
making podcasts and doing research.
link |
So if the status quo was that they were actually in the experience machine, I think a lot of
link |
people might then prefer to live the life that they are familiar with rather than sort
link |
So that's interesting, the change itself, the leap, yeah, so it might not be so much
link |
the reality itself that we're after.
link |
But it's more that we are maybe involved in certain projects and relationships.
link |
And we have, you know, a self identity and these things that our values are kind of connected
link |
with carrying that forward.
link |
And then whether it's inside a tank or outside a tank in Peru, or whether inside a computer
link |
outside a computer, that's kind of less important to what we ultimately care about.
link |
Yeah, but still, so just to linger on it, it is interesting.
link |
I find maybe people are different, but I find myself quite willing to take the leap to the
link |
farmer in Peru, especially as the virtual reality system become more realistic.
link |
I find that possibility and I think more people would take that leap.
link |
But so in this thought experiment, just to make sure we are understanding, so in this
link |
case, the farmer in Peru would not be a virtual reality, that would be the real, your life,
link |
like before this whole experience machine started.
link |
Well, I kind of assumed from that description, you're being very specific, but that kind
link |
of idea just like washes away the concept of what's real.
link |
I'm still a little hesitant about your kind of distinction between real and illusion.
link |
Because when you can have an illusion that feels, I mean, that looks real, I don't know
link |
how you can definitively say something is real or not, like what's a good way to prove
link |
that something is real in that context?
link |
Well, so I guess in this case, it's more a stipulation.
link |
In one case, you're floating in a tank with these wires by the super duper neuroscientists
link |
plugging into your head, giving you like Friedman experiences.
link |
In the other, you're actually tilling the soil in Peru, growing peanuts, and then those
link |
peanuts are being eaten by other people all around the world who buy the exports.
link |
That's two different possible situations in the one and the same real world that you could
link |
But just to be clear, when you're in a vat with wires and the neuroscientists, you can
link |
still go farming in Peru, right?
link |
No, well, if you wanted to, you could have the experience of farming in Peru, but there
link |
wouldn't actually be any peanuts grown.
link |
But what makes a peanut, so a peanut could be grown and you could feed things with that
link |
peanut and why can't all of that be done in a simulation?
link |
I hope, first of all, that they actually have peanut farms in Peru, I guess we'll get a
link |
lot of comments otherwise from Angrit.
link |
I was way up to the point when you started talking about Peru peanuts, that's when I
link |
realized you're relying out of these.
link |
No, I mean, I think, I mean, in the simulation, I think there is a sense, the important sense
link |
in which it would all be real.
link |
Nevertheless, there is a distinction between inside the simulation and outside the simulation.
link |
Or in the case of Nozick's thought experiment, whether you're in the vat or outside the vat,
link |
and some of those differences may or may not be important.
link |
I mean, that comes down to your values and preferences.
link |
So if the, if the experience machine only gives you the experience of growing peanuts,
link |
but you're the only one in the experience machines.
link |
No, but there's other, you can, within the experience machine, others can plug in.
link |
Well, there are versions of the experience machine.
link |
So in fact, you might want to have, distinguish different thought experiments, different versions
link |
So in, like in the original thought experiment, maybe it's only you, right?
link |
And you think, I wouldn't want to go in there.
link |
Well, that tells you something interesting about what you value and what you care about.
link |
Then you could say, well, what if you add the fact that there would be other people
link |
in there and you would interact with them?
link |
Well, it starts to make it more attractive, right?
link |
Then you could add in, well, what if you could also have important longterm effects on human
link |
history and the world, and you could actually do something useful, even though you were
link |
That makes it maybe even more attractive.
link |
Like you could actually have a life that had a purpose and consequences.
link |
And so as you sort of add more into it, it becomes more similar to the baseline reality
link |
that you were comparing it to.
link |
Yeah, but I just think inside the experience machine and without taking those steps you
link |
just mentioned, you still have an impact on longterm history of the creatures that live
link |
inside that, of the quote unquote fake creatures that live inside that experience machine.
link |
And that, like at a certain point, you know, if there's a person waiting for you inside
link |
that experience machine, maybe your newly found wife and she dies, she has fear, she
link |
has hopes, and she exists in that machine when you plug out, when you unplug yourself
link |
and plug back in, she's still there going on about her life.
link |
Well, in that case, yeah, she starts to have more of an independent existence.
link |
Independent existence.
link |
But it depends, I think, on how she's implemented in the experience machine.
link |
Take one limit case where all she is is a static picture on the wall, a photograph.
link |
So you think, well, I can look at her, right?
link |
Then you think, well, it doesn't really matter much what happens to that, any more than a
link |
normal photograph if you tear it up, right?
link |
It means you can't see it anymore, but you haven't harmed the person whose picture you
link |
But if she's actually implemented, say, at a neural level of detail so that she's a fully
link |
realized digital mind with the same behavioral repertoire as you have, then very plausibly
link |
she would be a conscious person like you are.
link |
And then what you do in this experience machine would have real consequences for how this
link |
So you have to specify which of these experience machines you're talking about.
link |
I think it's not entirely obvious that it would be possible to have an experience machine
link |
that gave you a normal set of human experiences, which include experiences of interacting with
link |
other people, without that also generating consciousnesses corresponding to those other
link |
That is, if you create another entity that you perceive and interact with, that to you
link |
looks entirely realistic.
link |
Not just when you say hello, they say hello back, but you have a rich interaction, many
link |
days, deep conversations.
link |
It might be that the only possible way of implementing that would be one that also has
link |
a side effect, instantiated this other person in enough detail that you would have a second
link |
consciousness there.
link |
I think that's to some extent an open question.
link |
So you don't think it's possible to fake consciousness and fake intelligence?
link |
Well, it might be.
link |
I mean, I think you can certainly fake, if you have a very limited interaction with somebody,
link |
you could certainly fake that.
link |
If all you have to go on is somebody said hello to you, that's not enough for you to
link |
tell whether that was a real person there, or a prerecorded message, or a very superficial
link |
simulation that has no consciousness, because that's something easy to fake.
link |
We could already fake it, now you can record a voice recording.
link |
But if you have a richer set of interactions where you're allowed to ask open ended questions
link |
and probe from different angles, you couldn't give canned answer to all of the possible
link |
ways that you could probe it, then it starts to become more plausible that the only way
link |
to realize this thing in such a way that you would get the right answer from any which
link |
angle you probed it, would be a way of instantiating it, where you also instantiated a conscious
link |
Yeah, I'm with you on the intelligence part, but is there something about me that says
link |
consciousness is easier to fake?
link |
Like I've recently gotten my hands on a lot of rubas, don't ask me why or how.
link |
And I've made them, there's just a nice robotic mobile platform for experiments.
link |
And I made them scream and or moan in pain, so on, just to see when they're responding
link |
And it's just a sort of psychological experiment on myself.
link |
And I think they appear conscious to me pretty quickly.
link |
To me, at least my brain can be tricked quite easily.
link |
I said if I introspect, it's harder for me to be tricked that something is intelligent.
link |
So I just have this feeling that inside this experience machine, just saying that you're
link |
conscious and having certain qualities of the interaction, like being able to suffer,
link |
like being able to hurt, like being able to wander about the essence of your own existence,
link |
not actually, I mean, creating the illusion that you're wandering about it is enough to
link |
create the illusion of consciousness.
link |
And because of that, create a really immersive experience to where you feel like that is
link |
So you think there's a big gap between appearing conscious and being conscious?
link |
Or is it that you think it's very easy to be conscious?
link |
I'm not actually sure what it means to be conscious.
link |
All I'm saying is the illusion of consciousness is enough to create a social interaction that's
link |
as good as if the thing was conscious, meaning I'm making it about myself.
link |
I mean, I guess there are a few different things.
link |
One is how good the interaction is, which might, I mean, if you don't really care about
link |
like probing hard for whether the thing is conscious, maybe it would be a satisfactory
link |
interaction, whether or not you really thought it was conscious.
link |
Now, if you really care about it being conscious in like inside this experience machine, how
link |
easy would it be to fake it?
link |
And you say, it sounds fairly easy, but then the question is, would that also mean it's
link |
very easy to instantiate consciousness?
link |
Like it's much more widely spread in the world and we have thought it doesn't require a big
link |
human brain with a hundred billion neurons, all you need is some system that exhibits
link |
basic intentionality and can respond and you already have consciousness.
link |
Like in that case, I guess you still have a close coupling.
link |
I guess that case would be where they can come apart, where you could create the appearance
link |
of there being a conscious mind with actually not being another conscious mind.
link |
I'm somewhat agnostic exactly where these lines go.
link |
I think one observation that makes it plausible that you could have very realistic appearances
link |
relatively simply, which also is relevant for the simulation argument and in terms of
link |
thinking about how realistic would a virtual reality model have to be in order for the
link |
simulated creature not to notice that anything was awry.
link |
Well, just think of our own humble brains during the wee hours of the night when we
link |
Many times, well, dreams are very immersive, but often you also don't realize that you're
link |
And that's produced by simple primitive three pound lumps of neural matter effortlessly.
link |
So if a simple brain like this can create the virtual reality that seems pretty real
link |
to us, then how much easier would it be for a super intelligent civilization with planetary
link |
sized computers optimized over the eons to create a realistic environment for you to
link |
By the way, behind that intuition is that our brain is not that impressive relative
link |
to the possibilities of what technology could bring.
link |
It's also possible that the brain is the epitome, is the ceiling.
link |
How is that possible?
link |
Meaning like this is the smartest possible thing that the universe could create.
link |
So that seems unlikely to me.
link |
I mean, for some of these reasons we alluded to earlier in terms of designs we already
link |
have for computers that would be faster by many orders of magnitude than the human brain.
link |
We can see that the constraints, the cognitive constraints in themselves is what enables
link |
So the more powerful you make the computer, the less likely it is to become super intelligent.
link |
This is where I say dumb things to push back on that statement.
link |
I'm not sure I thought that we might.
link |
I mean, so there are different dimensions of intelligence.
link |
A simple one is just speed.
link |
Like if you can solve the same challenge faster in some sense, you're like smarter.
link |
So there I think we have very strong evidence for thinking that you could have a computer
link |
in this universe that would be much faster than the human brain and therefore have speed
link |
super intelligence, like be completely superior, maybe a million times faster.
link |
Then maybe there are other ways in which you could be smarter as well, maybe more qualitative
link |
And the concepts are a little bit less clear cut.
link |
So it's harder to make a very crisp, neat, firmly logical argument for why that could
link |
be qualitative super intelligence as opposed to just things that were faster.
link |
Although I still think it's very plausible and for various reasons that are less than
link |
watertight arguments.
link |
But when you can sort of, for example, if you look at animals and even within humans,
link |
like there seems to be like Einstein versus random person, like it's not just that Einstein
link |
was a little bit faster, but like how long would it take a normal person to invent general
link |
relativity is like, it's not 20% longer than it took Einstein or something like that.
link |
It's like, I don't know whether they would do it at all or it would take millions of
link |
years or some totally bizarre.
link |
But your intuition is that the compute size will get you go increasing the size of the
link |
computer and the speed of the computer might create some much more powerful levels of intelligence
link |
that would enable some of the things we've been talking about with like the simulation,
link |
being able to simulate an ultra realistic environment, ultra realistic perception of
link |
I mean, strictly speaking, it would not be necessary to have super intelligence in order
link |
to have say the technology to make these simulations, ancestor simulations or other kinds of simulations.
link |
As a matter of fact, I think if we are in a simulation, it would most likely be one
link |
built by a civilization that had super intelligence.
link |
It certainly would help a lot.
link |
I mean, you could build more efficient larger scale structures if you had super intelligence.
link |
I also think that if you had the technology to build these simulations, that's like a
link |
very advanced technology.
link |
It seems kind of easier to get the technology to super intelligence.
link |
I'd expect by the time they could make these fully realistic simulations of human history
link |
with human brains in there, like before that they got to that stage, they would have figured
link |
out how to create machine super intelligence or maybe biological enhancements of their
link |
own brains if there were biological creatures to start with.
link |
So we talked about the three parts of the simulation argument.
link |
One, we destroy ourselves before we ever create the simulation.
link |
Two, we somehow, everybody somehow loses interest in creating the simulation.
link |
Three, we're living in a simulation.
link |
So you've kind of, I don't know if your thinking has evolved on this point, but you kind of
link |
said that we know so little that these three cases might as well be equally probable.
link |
So probabilistically speaking, where do you stand on this?
link |
Yeah, I mean, I don't think equal necessarily would be the most supported probability assignment.
link |
So how would you, without assigning actual numbers, what's more or less likely in your
link |
Well, I mean, I've historically tended to punt on the question of like between these
link |
So maybe you ask me another way is which kind of things would make each of these more or
link |
What kind of intuition?
link |
Certainly in general terms, if you think anything that say increases or reduces the probability
link |
of one of these, we tend to slosh probability around on the other.
link |
So if one becomes less probable, like the other would have to, cause it's got to add
link |
So if we consider the first hypothesis, the first alternative that there's this filter
link |
that makes it so that virtually no civilization reaches technological maturity, in particular
link |
our own civilization, if that's true, then it's like very unlikely that we would reach
link |
technological maturity because if almost no civilization at our stage does it, then it's
link |
unlikely that we do it.
link |
Sorry, can you linger on that for a second?
link |
Well, so if it's the case that almost all civilizations at our current stage of technological
link |
development failed to reach maturity, that would give us very strong reason for thinking
link |
we will fail to reach technological maturity.
link |
Oh, and also sort of the flip side of that is the fact that we've reached it means that
link |
many other civilizations have reached this point.
link |
So that means if we get closer and closer to actually reaching technological maturity,
link |
there's less and less distance left where we could go extinct before we are there, and
link |
therefore the probability that we will reach increases as we get closer, and that would
link |
make it less likely to be true that almost all civilizations at our current stage failed
link |
Like we would have this...
link |
The one case we had started ourselves would be very close to getting there, that would
link |
be strong evidence that it's not so hard to get to technological maturity.
link |
So to the extent that we feel we are moving nearer to technological maturity, that would
link |
tend to reduce the probability of the first alternative and increase the probability of
link |
It doesn't need to be a monotonic change.
link |
Like if every once in a while some new threat comes into view, some bad new thing you could
link |
do with some novel technology, for example, that could change our probabilities in the
link |
But that technology, again, you have to think about as that technology has to be able to
link |
equally in an even way affect every civilization out there.
link |
Yeah, pretty much.
link |
I mean, that's strictly speaking, it's not true.
link |
I mean, that could be two different existential risks and every civilization, you know, one
link |
or the other, like, but none of them kills more than 50%.
link |
But incidentally, so in some of my work, I mean, on machine superintelligence, like pointed
link |
to some existential risks related to sort of super intelligent AI and how we must make
link |
sure, you know, to handle that wisely and carefully.
link |
It's not the right kind of existential catastrophe to make the first alternative true though.
link |
Like it might be bad for us if the future lost a lot of value as a result of it being
link |
shaped by some process that optimized for some completely nonhuman value.
link |
But even if we got killed by machine superintelligence, that machine superintelligence might still
link |
attain technological maturity.
link |
Oh, I see, so you're not human exclusive.
link |
This could be any intelligent species that achieves, like it's all about the technological
link |
But the humans have to attain it.
link |
So like superintelligence could replace us and that's just as well for the simulation
link |
I mean, it could interact with the second hypothesis by alternative.
link |
Like if the thing that replaced us was either more likely or less likely than we would be
link |
to have an interest in creating ancestor simulations, you know, that could affect probabilities.
link |
But yeah, to a first order, like if we all just die, then yeah, we won't produce any
link |
simulations because we are dead.
link |
But if we all die and get replaced by some other intelligent thing that then gets to
link |
technological maturity, the question remains, of course, if not that thing, then use some
link |
of its resources to do this stuff.
link |
So can you reason about this stuff, given how little we know about the universe?
link |
Is it reasonable to reason about these probabilities?
link |
So like how little, well, maybe you can disagree, but to me, it's not trivial to figure out
link |
how difficult it is to build a simulation.
link |
We kind of talked about it a little bit.
link |
We also don't know, like as we try to start building it, like start creating virtual worlds
link |
and so on, how that changes the fabric of society.
link |
Like there's all these things along the way that can fundamentally change just so many
link |
aspects of our society about our existence that we don't know anything about, like the
link |
kind of things we might discover when we understand to a greater degree the fundamental, the physics,
link |
like the theory, if we have a breakthrough, have a theory and everything, how that changes
link |
stuff, how that changes deep space exploration and so on.
link |
Like, is it still possible to reason about probabilities given how little we know?
link |
Yes, I think there will be a large residual of uncertainty that we'll just have to acknowledge.
link |
And I think that's true for most of these big picture questions that we might wonder
link |
It's just we are small, short lived, small brained, cognitively very limited humans with
link |
And it's amazing we can figure out as much as we can really about the cosmos.
link |
But okay, so there's this cognitive trick that seems to happen when I look at the simulation
link |
argument, which for me, it seems like case one and two feel unlikely.
link |
I want to say feel unlikely as opposed to sort of like, it's not like I have too much
link |
scientific evidence to say that either one or two are not true.
link |
It just seems unlikely that every single civilization destroys itself.
link |
And it seems like feels unlikely that the civilizations lose interest.
link |
So naturally, without necessarily explicitly doing it, but the simulation argument basically
link |
says it's very likely we're living in a simulation.
link |
To me, my mind naturally goes there.
link |
I think the mind goes there for a lot of people.
link |
Is that the incorrect place for it to go?
link |
Well, not necessarily.
link |
I think the second alternative, which has to do with the motivations and interests of
link |
technological and material civilizations, I think there is much we don't understand about
link |
Can you talk about that a little bit?
link |
What do you think?
link |
I mean, this is a question that pops up when you when you build an AGI system or build
link |
a general intelligence.
link |
How does that change our motivations?
link |
Do you think it'll fundamentally transform our motivations?
link |
Well, it doesn't seem that implausible that once you take this leap to to technological
link |
maturity, I mean, I think like it involves creating machine super intelligence, possibly
link |
that would be sort of on the path for basically all civilizations, maybe before they are able
link |
to create large numbers of ancestry simulations, they would that that possibly could be one
link |
of these things that quite radically changes the orientation of what a civilization is,
link |
in fact, optimizing for.
link |
There are other things as well.
link |
So at the moment, we have not perfect control over our own being our own mental states,
link |
our own experiences are not under our direct control.
link |
So for example, if if you want to experience a pleasure and happiness, you might have to
link |
do a whole host of things in the external world to try to get into the stage into the
link |
mental state where you experience pleasure, like some people get some pleasure from eating
link |
Well, they can just turn that on, they have to kind of actually go to a nice restaurant
link |
and then they have to make money.
link |
So there's like all this kind of activity that maybe arises from the fact that we are
link |
trying to ultimately produce mental states.
link |
But the only way to do that is by a whole host of complicated activities in the external
link |
Now, at some level of technological development, I think we'll become auto potent in the sense
link |
of gaining direct ability to choose our own internal configuration, and enough knowledge
link |
and insight to be able to actually do that in a meaningful way.
link |
So then it could turn out that there are a lot of instrumental goals that would drop
link |
out of the picture and be replaced by other instrumental goals, because we could now serve
link |
some of these final goals in more direct ways.
link |
And who knows how all of that shakes out after civilizations reflect on that and converge
link |
on different attractors and so on and so forth.
link |
And that could be new instrumental considerations that come into view as well, that we are just
link |
oblivious to, that would maybe have a strong shaping effect on actions, like very strong
link |
reasons to do something or not to do something, then we just don't realize they are there
link |
because we are so dumb, bumbling through the universe.
link |
But if almost inevitably en route to attaining the ability to create many ancestors simulations,
link |
you do have this cognitive enhancement, or advice from super intelligences or yourself,
link |
then maybe there's like this additional set of considerations coming into view and it's
link |
obvious that the thing that makes sense is to do X, whereas right now it seems you could
link |
X, Y or Z and different people will do different things and we are kind of random in that sense.
link |
Because at this time, with our limited technology, the impact of our decisions is minor.
link |
I mean, that's starting to change in some ways.
link |
Well, I'm not sure how it follows that the impact of our decisions is minor.
link |
Well, it's starting to change.
link |
I mean, I suppose 100 years ago it was minor.
link |
It's starting to…
link |
Well, it depends on how you view it.
link |
What people did 100 years ago still have effects on the world today.
link |
As a civilization in the togetherness.
link |
So it might be that the greatest impact of individuals is not at technological maturity
link |
It might be earlier on when there are different tracks, civilization could go down.
link |
Maybe the population is smaller, things still haven't settled out.
link |
If you count indirect effects, those could be bigger than the direct effects that people
link |
So part three of the argument says that…
link |
So that leads us to a place where eventually somebody creates a simulation.
link |
I think you had a conversation with Joe Rogan.
link |
I think there's some aspect here where you got stuck a little bit.
link |
How does that lead to we're likely living in a simulation?
link |
So this kind of probability argument, if somebody eventually creates a simulation, why does
link |
that mean that we're now in a simulation?
link |
What you get to if you accept alternative three first is there would be more simulated
link |
people with our kinds of experiences than non simulated ones.
link |
Like if you look at the world as a whole, by the end of time as it were, you just count
link |
That would be more simulated ones than non simulated ones.
link |
Then there is an extra step to get from that.
link |
If you assume that, suppose for the sake of the argument, that that's true.
link |
How do you get from that to the statement we are probably in a simulation?
link |
So here you're introducing an indexical statement like it's that this person right now is in
link |
There are all these other people that are in simulations and some that are not in the
link |
But what probability should you have that you yourself is one of the simulated ones
link |
So I call it the bland principle of indifference, which is that in cases like this, when you
link |
have two sets of observers, one of which is much larger than the other and you can't from
link |
any internal evidence you have, tell which set you belong to, you should assign a probability
link |
that's proportional to the size of these sets.
link |
So that if there are 10 times more simulated people with your kinds of experiences, you
link |
would be 10 times more likely to be one of those.
link |
Is that as intuitive as it sounds?
link |
I mean, that seems kind of, if you don't have enough information, you should rationally
link |
just assign the same probability as the size of the set.
link |
It seems pretty plausible to me.
link |
Where are the holes in this?
link |
Is it at the very beginning, the assumption that everything stretches, you have infinite
link |
You don't need infinite time.
link |
You just need, how long does the time take?
link |
However long it takes, I guess, for a universe to produce an intelligent civilization that
link |
attains the technology to run some ancestry simulations.
link |
When the first simulation is created, that stretch of time, just a little longer than
link |
they'll all start creating simulations.
link |
Well, I mean, there might be a difference.
link |
If you think of there being a lot of different planets and some subset of them have life
link |
and then some subset of those get to intelligent life and some of those maybe eventually start
link |
creating simulations, they might get started at quite different times.
link |
Maybe on some planet, it takes a billion years longer before you get monkeys or before you
link |
get even bacteria than on another planet.
link |
This might happen at different cosmological epochs.
link |
Is there a connection here to the doomsday argument and that sampling there?
link |
Yeah, there is a connection in that they both involve an application of anthropic reasoning
link |
that is reasoning about these kind of indexical propositions.
link |
But the assumption you need in the case of the simulation argument is much weaker than
link |
the assumption you need to make the doomsday argument go through.
link |
What is the doomsday argument and maybe you can speak to the anthropic reasoning in more
link |
Yeah, that's a big and interesting topic in its own right, anthropics, but the doomsday
link |
argument is this really first discovered by Brandon Carter, who was a theoretical physicist
link |
and then developed by philosopher John Leslie.
link |
I think it might have been discovered initially in the 70s or 80s and Leslie wrote this book,
link |
And there are some other versions as well by Richard Gott, who's a physicist, but let's
link |
focus on the Carter Leslie version where it's an argument that we have systematically underestimated
link |
the probability that humanity will go extinct soon.
link |
Now I should say most people probably think at the end of the day there is something wrong
link |
with this doomsday argument that it doesn't really hold.
link |
It's like there's something wrong with it, but it's proved hard to say exactly what is
link |
wrong with it and different people have different accounts.
link |
My own view is it seems inconclusive, but I can say what the argument is.
link |
Yeah, that would be good.
link |
So maybe it's easiest to explain via an analogy to sampling from urns.
link |
So imagine you have two urns in front of you and they have balls in them that have numbers.
link |
The two urns look the same, but inside one there are 10 balls.
link |
Ball number one, two, three, up to ball number 10.
link |
And then in the other urn you have a million balls numbered one to a million and somebody
link |
puts one of these urns in front of you and asks you to guess what's the chance it's the
link |
10 ball urn and you say, well, 50, 50, I can't tell which urn it is.
link |
But then you're allowed to reach in and pick a ball at random from the urn and that's suppose
link |
you find that it's ball number seven.
link |
So that's strong evidence for the 10 ball hypothesis.
link |
It's a lot more likely that you would get such a low numbered ball if there are only
link |
10 balls in the urn, like it's in fact 10% done, right?
link |
Then if there are a million balls, it would be very unlikely you would get number seven.
link |
So you perform a Bayesian update and if your prior was 50, 50 that it was the 10 ball urn,
link |
you become virtually certain after finding the random sample was seven that it's only
link |
has 10 balls in it.
link |
So in the case of the urns, this is uncontroversial, just elementary probability theory.
link |
The Doomsday Argument says that you should reason in a similar way with respect to different
link |
hypotheses about how many balls there will be in the urn of humanity as it were, how
link |
many humans there will ever have been by the time we go extinct.
link |
So to simplify, let's suppose we only consider two hypotheses, either maybe 200 billion humans
link |
in total or 200 trillion humans in total.
link |
You could fill in more hypotheses, but it doesn't change the principle here.
link |
So it's easiest to see if we just consider these two.
link |
So you start with some prior based on ordinary empirical ideas about threats to civilization
link |
And maybe you say it's a 5% chance that we will go extinct by the time there will have
link |
been 200 billion only, you're kind of optimistic, let's say, you think probably we'll make it
link |
through, colonize the universe.
link |
But then, according to this Doomsday Argument, you should take off your own birth rank as
link |
So your birth rank is your sequence in the position of all humans that have ever existed.
link |
It turns out you're about a human number of 100 billion, you know, give or take.
link |
That's like, roughly how many people have been born before you.
link |
That's fascinating, because I probably, we each have a number.
link |
We would each have a number in this, I mean, obviously, the exact number would depend on
link |
where you started counting, like which ancestors was human enough to count as human.
link |
But those are not really important, there are relatively few of them.
link |
So yeah, so you're roughly 100 billion.
link |
Now, if they're only going to be 200 billion in total, that's a perfectly unremarkable
link |
You're somewhere in the middle, right?
link |
It's a run of the mill human, completely unsurprising.
link |
Now, if they're going to be 200 trillion, you would be remarkably early, like what are
link |
the chances out of these 200 trillion human that you should be human number 100 billion?
link |
That seems it would have a much lower conditional probability.
link |
And so analogously to how in the urn case, you thought after finding this low numbered
link |
random sample, you update it in favor of the urn having few balls.
link |
Similarly, in this case, you should update in favor of the human species having a lower
link |
total number of members that is doomed soon.
link |
You said doomed soon?
link |
Well, that would be the hypothesis in this case that it will end 100 billion.
link |
I just like that term for that hypothesis.
link |
So what it kind of crucially relies on, the Doomsday Argument, is the idea that you should
link |
reason as if you were a random sample from the set of all humans that will have existed.
link |
If you have that assumption, then I think the rest kind of follows.
link |
The question then is, why should you make that assumption?
link |
In fact, you know you're 100 billion, so where do you get this prior?
link |
And then there is like a literature on that with different ways of supporting that assumption.
link |
That's just one example of anthropic reasoning, right?
link |
That seems to be kind of convenient when you think about humanity, when you think about
link |
sort of even like existential threats and so on, as it seems that quite naturally that
link |
you should assume that you're just an average case.
link |
Yeah, that you're kind of a typical randomly sample.
link |
Now, in the case of the Doomsday Argument, it seems to lead to what intuitively we think
link |
is the wrong conclusion, or at least many people have this reaction that there's got
link |
to be something fishy about this argument.
link |
Because from very, very weak premises, it gets this very striking implication that we
link |
have almost no chance of reaching size 200 trillion humans in the future.
link |
And how could we possibly get there just by reflecting on when we were born?
link |
It seems you would need sophisticated arguments about the impossibility of space colonization,
link |
So one might be tempted to reject this key assumption, I call it the self sampling assumption,
link |
the idea that you should reason as if you're a random sample from all observers or in your
link |
some reference class.
link |
However, it turns out that in other domains, it looks like we need something like this
link |
self sampling assumption to make sense of bona fide scientific inferences.
link |
In contemporary cosmology, for example, you have these multiverse theories.
link |
And according to a lot of those, all possible human observations are made.
link |
So if you have a sufficiently large universe, you will have a lot of people observing all
link |
kinds of different things.
link |
So if you have two competing theories, say about the value of some constant, it could
link |
be true according to both of these theories that there will be some observers observing
link |
the value that corresponds to the other theory, because there will be some observers that
link |
have hallucinations, so there's a local fluctuation or a statistically anomalous measurement,
link |
these things will happen.
link |
And if enough observers make enough different observations, there will be some that sort
link |
of by chance make these different ones.
link |
And so what we would want to say is, well, many more observers, a larger proportion of
link |
the observers will observe as it were the true value.
link |
And a few will observe the wrong value.
link |
If we think of ourselves as a random sample, we should expect with a probability to observe
link |
the true value and that will then allow us to conclude that the evidence we actually
link |
have is evidence for the theories we think are supported.
link |
It kind of then is a way of making sense of these inferences that clearly seem correct,
link |
that we can make various observations and infer what the temperature of the cosmic background
link |
is and the fine structure constant and all of this.
link |
But it seems that without rolling in some assumption similar to the self sampling assumption,
link |
this inference just doesn't go through.
link |
And there are other examples.
link |
So there are these scientific contexts where it looks like this kind of anthropic reasoning
link |
is needed and makes perfect sense.
link |
And yet, in the case of the Dupest argument, it has this weird consequence and people might
link |
think there's something wrong with it there.
link |
So there's then this project that would consist in trying to figure out what are the legitimate
link |
ways of reasoning about these indexical facts when observer selection effects are in play.
link |
In other words, developing a theory of anthropics.
link |
And there are different views of looking at that and it's a difficult methodological area.
link |
But to tie it back to the simulation argument, the key assumption there, this bland principle
link |
of indifference, is much weaker than the self sampling assumption.
link |
So if you think about, in the case of the Dupest argument, it says you should reason
link |
as if you are a random sample from all humans that will have lived, even though in fact
link |
you know that you are about number 100 billionth human and you're alive in the year 2020.
link |
Whereas in the case of the simulation argument, it says that, well, if you actually have no
link |
way of telling which one you are, then you should assign this kind of uniform probability.
link |
Yeah, yeah, your role as the observer in the simulation argument is different, it seems
link |
Like who's the observer?
link |
I mean, I keep assigning the individual consciousness.
link |
But a lot of observers in the context of the simulation argument, the relevant observers
link |
would be A, the people in original histories, and B, the people in simulations.
link |
So this would be the class of observers that we need, I mean, they're also maybe the simulators,
link |
but we can set those aside for this.
link |
So the question is, given that class of observers, a small set of original history observers
link |
and a large class of simulated observers, which one should you think is you?
link |
Where are you amongst this set of observers?
link |
I'm maybe having a little bit of trouble wrapping my head around the intricacies of what it
link |
means to be an observer in this, in the different instantiations of the anthropic reasoning
link |
cases that we mentioned.
link |
I mean, does it have to be...
link |
It's not the observer.
link |
Yeah, I mean, it may be an easier way of putting it is just like, are you simulated, are you
link |
not simulated, given this assumption that these two groups of people exist?
link |
In the simulation case, it seems pretty straightforward.
link |
So the key point is the methodological assumption you need to make to get the simulation argument
link |
to where it wants to go is much weaker and less problematic than the methodological assumption
link |
you need to make to get the doomsday argument to its conclusion.
link |
Maybe the doomsday argument is sound or unsound, but you need to make a much stronger and more
link |
controversial assumption to make it go through.
link |
In the case of the simulation argument, I guess one maybe way intuition pumped to support
link |
this bland principle of indifference is to consider a sequence of different cases where
link |
the fraction of people who are simulated to non simulated approaches one.
link |
So in the limiting case where everybody is simulated, obviously you can deduce with certainty
link |
that you are simulated.
link |
If everybody with your experiences is simulated and you know you've got to be one of those,
link |
you don't need a probability at all, you just kind of logically conclude it, right?
link |
So then as we move from a case where say 90% of everybody is simulated, 99%, 99.9%, it
link |
should seem plausible that the probability you assign should sort of approach one certainty
link |
as the fraction approaches the case where everybody is in a simulation.
link |
You wouldn't expect that to be a discrete, well, if there's one non simulated person,
link |
then it's 50, 50, but if we move that, then it's 100%, like it should kind of, there are
link |
other arguments as well one can use to support this bland principle of indifference, but
link |
that might be enough to.
link |
But in general, when you start from time equals zero and go into the future, the fraction
link |
of simulated, if it's possible to create simulated worlds, the fraction of simulated worlds will
link |
Well, I mean, it won't go all the way to one.
link |
In reality, that would be some ratio, although maybe a technologically mature civilization
link |
could run a lot of simulations using a small portion of its resources, it probably wouldn't
link |
be able to run infinitely many.
link |
I mean, if we take say the observed, the physics in the observed universe, if we assume that
link |
that's also the physics at the level of the simulators, that would be limits to the amount
link |
of information processing that any one civilization could perform in its future trajectory.
link |
First of all, there's limited amount of matter you can get your hands off because with a
link |
positive cosmological constant, the universe is accelerating, there's like a finite sphere
link |
of stuff, even if you traveled with the speed of light that you could ever reach, you have
link |
a finite amount of stuff.
link |
And then if you think there is like a lower limit to the amount of loss you get when you
link |
perform an erasure of a computation, or if you think, for example, just matter gradually
link |
over cosmological timescales, decay, maybe protons decay, other things, and you radiate
link |
out gravitational waves, like there's all kinds of seemingly unavoidable losses that
link |
Eventually, we'll have something like a heat death of the universe or a cold death or whatever,
link |
So it's finite, but of course, we don't know which, if there's many ancestral simulations,
link |
we don't know which level we are.
link |
So there could be, couldn't there be like an arbitrary number of simulation that spawned
link |
ours, and those had more resources, in terms of physical universe to work with?
link |
Sorry, what do you mean that that could be?
link |
Sort of, okay, so if simulations spawn other simulations, it seems like each new spawn
link |
has fewer resources to work with.
link |
But we don't know at which step along the way we are at.
link |
Any one observer doesn't know whether we're in level 42, or 100, or one, or is that not
link |
matter for the resources?
link |
I mean, it's true that there would be uncertainty as to, you could have stacked simulations,
link |
and that could then be uncertainty as to which level we are at.
link |
As you remarked also, all the computations performed in a simulation within the simulation
link |
also have to be expanded at the level of the simulation.
link |
So the computer in basement reality where all these simulations with the simulations
link |
with the simulations are taking place, like that computer, ultimately, it's CPU or whatever
link |
it is, like that has to power this whole tower, right?
link |
So if there is a finite compute power in basement reality, that would impose a limit to how
link |
tall this tower can be.
link |
And if each level kind of imposes a large extra overhead, you might think maybe the
link |
tower would not be very tall, that most people would be low down in the tower.
link |
I love the term basement reality.
link |
Let me ask one of the popularizers, you said there's many through this, when you look at
link |
sort of the last few years of the simulation hypothesis, just like you said, it comes up
link |
every once in a while, some new community discovers it and so on.
link |
But I would say one of the biggest popularizers of this idea is Elon Musk.
link |
Do you have any kind of intuition about what Elon thinks about when he thinks about simulation?
link |
Why is this of such interest?
link |
Is it all the things we've talked about, or is there some special kind of intuition about
link |
simulation that he has?
link |
I mean, you might have a better, I think, I mean, why it's of interest, I think it's
link |
like seems pretty obvious why, to the extent that one thinks the argument is credible,
link |
why it would be of interest, it would, if it's correct, tell us something very important
link |
about the world in one way or the other, whichever of the three alternatives for a simulation
link |
that seems like arguably one of the most fundamental discoveries, right?
link |
Now, interestingly, in the case of someone like Elon, so there's like the standard arguments
link |
for why you might want to take the simulation hypothesis seriously, the simulation argument,
link |
In the case that if you are actually Elon Musk, let us say, there's a kind of an additional
link |
reason in that what are the chances you would be Elon Musk?
link |
It seems like maybe there would be more interest in simulating the lives of very unusual and
link |
remarkable people.
link |
So if you consider not just simulations where all of human history or the whole of human
link |
civilization are simulated, but also other kinds of simulations, which only include some
link |
subset of people, like in those simulations that only include a subset, it might be more
link |
likely that they would include subsets of people with unusually interesting or consequential
link |
So if you're Elon Musk, it's more likely that you're an inspiration.
link |
Like if you're Donald Trump, or if you're Bill Gates, or you're like, some particularly
link |
like distinctive character, you might think that that, I mean, if you just think of yourself
link |
into the shoes, right, it's got to be like an extra reason to think that's kind of.
link |
So on a scale of like farmer in Peru to Elon Musk, the more you get towards the Elon Musk,
link |
the higher the probability.
link |
You'd imagine that would be some extra boost from that.
link |
There's an extra boost.
link |
So he also asked the question of what he would ask an AGI saying, the question being, what's
link |
outside the simulation?
link |
Do you think about the answer to this question?
link |
If we are living in a simulation, what is outside the simulation?
link |
So the programmer of the simulation?
link |
Yeah, I mean, I think it connects to the question of what's inside the simulation in that.
link |
So if you had views about the creators of the simulation, it might help you make predictions
link |
about what kind of simulation it is, what might happen, what happens after the simulation,
link |
if there is some after, but also like the kind of setup.
link |
So these two questions would be quite closely intertwined.
link |
But do you think it would be very surprising to like, is the stuff inside the simulation,
link |
is it possible for it to be fundamentally different than the stuff outside?
link |
Like, another way to put it, can the creatures inside the simulation be smart enough to even
link |
understand or have the cognitive capabilities or any kind of information processing capabilities
link |
enough to understand the mechanism that created them?
link |
They might understand some aspects of it.
link |
I mean, it's a level of, it's kind of, there are levels of explanation, like degrees to
link |
which you can understand.
link |
So does your dog understand what it is to be human?
link |
Well, it's got some idea, like humans are these physical objects that move around and
link |
And a normal human would have a deeper understanding of what it is to be a human.
link |
And maybe some very experienced psychologist or great novelist might understand a little
link |
bit more about what it is to be human.
link |
And maybe superintelligence could see right through your soul.
link |
So similarly, I do think that we are quite limited in our ability to understand all of
link |
the relevant aspects of the larger context that we exist in.
link |
But there might be hope for some.
link |
I think we understand some aspects of it.
link |
But you know, how much good is that?
link |
If there's like one key aspect that changes the significance of all the other aspects.
link |
So we understand maybe seven out of 10 key insights that you need.
link |
But the answer actually, like varies completely depending on what like number eight, nine
link |
and 10 insight is.
link |
It's like whether you want to suppose that the big task were to guess whether a certain
link |
number was odd or even, like a 10 digit number.
link |
And if it's even, the best thing for you to do in life is to go north.
link |
And if it's odd, the best thing for you is to go south.
link |
Now we are in a situation where maybe through our science and philosophy, we figured out
link |
what the first seven digits are.
link |
So we have a lot of information, right?
link |
Most of it we figured out.
link |
But we are clueless about what the last three digits are.
link |
So we are still completely clueless about whether the number is odd or even and therefore
link |
whether we should go north or go south.
link |
I feel that's an analogy, but I feel we're somewhat in that predicament.
link |
We know a lot about the universe.
link |
We've come maybe more than half of the way there to kind of fully understanding it.
link |
But the parts we're missing are plausibly ones that could completely change the overall
link |
upshot of the thing and including change our overall view about what the scheme of priorities
link |
should be or which strategic direction would make sense to pursue.
link |
I think your analogy of us being the dog trying to understand human beings is an entertaining
link |
one, and probably correct.
link |
The closer the understanding tends from the dog's viewpoint to us human psychologist viewpoint,
link |
the steps along the way there will have completely transformative ideas of what it means to be
link |
So the dog has a very shallow understanding.
link |
It's interesting to think that, to analogize that a dog's understanding of a human being
link |
is the same as our current understanding of the fundamental laws of physics in the universe.
link |
We spent an hour and 40 minutes talking about the simulation.
link |
Let's talk about super intelligence.
link |
At least for a little bit.
link |
And let's start at the basics.
link |
What to you is intelligence?
link |
I tend not to get too stuck with the definitional question.
link |
I mean, the common sense to understand, like the ability to solve complex problems, to
link |
learn from experience, to plan, to reason, some combination of things like that.
link |
Is consciousness mixed up into that or no?
link |
Is consciousness mixed up into that?
link |
Well, I think it could be fairly intelligent at least without being conscious probably.
link |
So then what is super intelligence?
link |
That would be like something that was much more, had much more general cognitive capacity
link |
than we humans have.
link |
So if we talk about general super intelligence, it would be much faster learner be able to
link |
reason much better, make plans that are more effective at achieving its goals, say in a
link |
wide range of complex challenging environments.
link |
In terms of as we turn our eye to the idea of sort of existential threats from super
link |
intelligence, do you think super intelligence has to exist in the physical world or can
link |
it be digital only?
link |
Sort of we think of our general intelligence as us humans, as an intelligence that's associated
link |
with the body, that's able to interact with the world, that's able to affect the world
link |
directly with physically.
link |
I mean, digital only is perfectly fine, I think.
link |
I mean, you could, it's physical in the sense that obviously the computers and the memories
link |
But it's capability to affect the world sort of.
link |
Could be very strong, even if it has a limited set of actuators, if it can type text on the
link |
screen or something like that, that would be, I think, ample.
link |
So in terms of the concerns of existential threat of AI, how can an AI system that's
link |
in the digital world have existential risk, sort of, and what are the attack vectors for
link |
Well, I mean, I guess maybe to take one step back, so I should emphasize that I also think
link |
there's this huge positive potential from machine intelligence, including super intelligence.
link |
And I want to stress that because some of my writing has focused on what can go wrong.
link |
And when I wrote the book Superintelligence, at that point, I felt that there was a kind
link |
of neglect of what would happen if AI succeeds, and in particular, a need to get a more granular
link |
understanding of where the pitfalls are so we can avoid them.
link |
I think that since the book came out in 2014, there has been a much wider recognition of
link |
And a number of research groups are now actually working on developing, say, AI alignment techniques
link |
and so on and so forth.
link |
So yeah, I think now it's important to make sure we bring back onto the table the upside
link |
And there's a little bit of a neglect now on the upside, which is, I mean, if you look
link |
at, I was talking to a friend, if you look at the amount of information that is available,
link |
or people talking and people being excited about the positive possibilities of general
link |
intelligence, that's not, it's far outnumbered by the negative possibilities in terms of
link |
our public discourse.
link |
It's hard to measure.
link |
But what are, can you linger on that for a little bit, what are some, to you, possible
link |
big positive impacts of general intelligence?
link |
Super intelligence?
link |
Well, I mean, super intelligence, because I tend to also want to distinguish these two
link |
different contexts of thinking about AI and AI impacts, the kind of near term and long
link |
term, if you want, both of which I think are legitimate things to think about, and people
link |
should discuss both of them, but they are different and they often get mixed up.
link |
And then, then I get, you get confusion, like, I think you get simultaneously like maybe
link |
an overhyping of the near term and then under hyping of the long term.
link |
And so I think as long as we keep them apart, we can have like, two good conversations,
link |
but or we can mix them together and have one bad conversation.
link |
Can you clarify just the two things we were talking about, the near term and the long
link |
And what are the distinctions?
link |
Well, it's a, it's a blurry distinction.
link |
But say the things I wrote about in this book, super intelligence, long term, things people
link |
are worrying about today with, I don't know, algorithmic discrimination, or even things,
link |
self driving cars and drones and stuff, more near term.
link |
And then of course, you could imagine some medium term where they kind of overlap and
link |
they one evolves into the other.
link |
But at any rate, I think both, yeah, the issues look kind of somewhat different depending
link |
on which of these contexts.
link |
So I think, I think it'd be nice if we can talk about the long term and think about a
link |
positive impact or a better world because of the existence of the long term super intelligence.
link |
Do you have views of such a world?
link |
I mean, I guess it's a little hard to articulate because it seems obvious that the world has
link |
a lot of problems as it currently stands.
link |
And it's hard to think of any one of those, which it wouldn't be useful to have like a
link |
friendly aligned super intelligence working on.
link |
So from health to the economic system to be able to sort of improve the investment and
link |
trade and foreign policy decisions, all that kind of stuff.
link |
All that kind of stuff and a lot more.
link |
I mean, what's the killer app?
link |
Well, I don't think there is one.
link |
I think AI, especially artificial general intelligence is really the ultimate general
link |
purpose technology.
link |
So it's not that there is this one problem, this one area where it will have a big impact.
link |
But if and when it succeeds, it will really apply across the board in all fields where
link |
human creativity and intelligence and problem solving is useful, which is pretty much all
link |
The thing that it would do is give us a lot more control over nature.
link |
It wouldn't automatically solve the problems that arise from conflict between humans, fundamentally
link |
political problems.
link |
Some subset of those might go away if you just had more resources and cooler tech.
link |
But some subset would require coordination that is not automatically achieved just by
link |
having more technological capability.
link |
But anything that's not of that sort, I think you just get an enormous boost with this kind
link |
of cognitive technology once it goes all the way.
link |
Now, again, that doesn't mean I'm thinking, oh, people don't recognize what's possible
link |
with current technology and like sometimes things get overhyped.
link |
But I mean, those are perfectly consistent views to hold.
link |
The ultimate potential being enormous.
link |
And then it's a very different question of how far are we from that or what can we do
link |
with near term technology?
link |
So what's your intuition about the idea of intelligence explosion?
link |
So there's this, you know, when you start to think about that leap from the near term
link |
to the long term, the natural inclination, like for me, sort of building machine learning
link |
systems today, it seems like it's a lot of work to get the general intelligence, but
link |
there's some intuition of exponential growth of exponential improvement of intelligence
link |
Can you maybe try to elucidate, try to talk about what's your intuition about the possibility
link |
of an intelligence explosion, that it won't be this gradual slow process, there might
link |
Yeah, I think it's, we don't know how explosive it will be.
link |
I think for what it's worth, it seems fairly likely to me that at some point, there will
link |
be some intelligence explosion, like some period of time, where progress in AI becomes
link |
extremely rapid, roughly, roughly in the area where you might say it's kind of humanish
link |
equivalent in core cognitive faculties, that the concept of human equivalent starts to
link |
break down when you look too closely at it.
link |
And just how explosive does something have to be for it to be called an intelligence
link |
Like, does it have to be like overnight, literally, or a few years?
link |
But overall, I guess, if you plotted the opinions of different people in the world, I guess
link |
that would be somewhat more probability towards the intelligence explosion scenario than probably
link |
the average, you know, AI researcher, I guess.
link |
So and then the other part of the intelligence explosion, or just forget explosion, just
link |
progress is once you achieve that gray area of human level intelligence, is it obvious
link |
to you that we should be able to proceed beyond it to get to super intelligence?
link |
Yeah, that seems, I mean, as much as any of these things can be obvious, given we've never
link |
had one, people have different views, smart people have different views, it's like some
link |
degree of uncertainty that always remains for any big, futuristic, philosophical grand
link |
question that just we realize humans are fallible, especially about these things.
link |
But it does seem, as far as I'm judging things based on my own impressions, that it seems
link |
very unlikely that that would be a ceiling at or near human cognitive capacity.
link |
And that's such a, I don't know, that's such a special moment, it's both terrifying and
link |
exciting to create a system that's beyond our intelligence.
link |
So maybe you can step back and say, like, how does that possibility make you feel that
link |
we can create something, it feels like there's a line beyond which it steps, it'll be able
link |
And therefore, it feels like a step where we lose control.
link |
Well, I don't think the latter follows that is you could imagine.
link |
And in fact, this is what a number of people are working towards making sure that we could
link |
ultimately project higher levels of problem solving ability while still making sure that
link |
they are aligned, like they are in the service of human values.
link |
I mean, so losing control, I think, is not a given that that would happen.
link |
Now you asked how it makes me feel, I mean, to some extent, I've lived with this for so
link |
long, since as long as I can remember, being an adult or even a teenager, it seemed to
link |
me obvious that at some point, AI will succeed.
link |
And so I actually misspoke, I didn't mean control, I meant, because the control problem
link |
is an interesting thing.
link |
And I think the hope is, at least we should be able to maintain control over systems that
link |
are smarter than us.
link |
But we do lose our specialness, it sort of will lose our place as the smartest, coolest
link |
And there's an ego involved with that, that humans aren't very good at dealing with.
link |
I mean, I value my intelligence as a human being.
link |
It seems like a big transformative step to realize there's something out there that's
link |
I mean, you don't see that as such a fundamentally...
link |
I think yes, a lot, I think it would be small, because I mean, I think there are already
link |
a lot of things out there that are, I mean, certainly, if you think the universe is big,
link |
there's going to be other civilizations that already have super intelligences, or that
link |
just naturally have brains the size of beach balls and are like, completely leaving us
link |
And we haven't come face to face with them.
link |
We haven't come face to face.
link |
But I mean, that's an open question, what would happen in a kind of post human world?
link |
Like how much day to day would these super intelligences be involved in the lives of
link |
I mean, you could imagine some scenario where it would be more like a background thing that
link |
would help protect against some things, but you wouldn't like that, they wouldn't be this
link |
intrusive kind of, like making you feel bad by like, making clever jokes on your expert,
link |
like there's like all sorts of things that maybe in the human context would feel awkward
link |
You don't want to be the dumbest kid in your class, everybody picks it, like, a lot of
link |
those things, maybe you need to abstract away from, if you're thinking about this context
link |
where we have infrastructure that is in some sense, beyond any or all humans.
link |
I mean, it's a little bit like, say, the scientific community as a whole, if you think of that
link |
as a mind, it's a little bit of a metaphor.
link |
But I mean, obviously, it's got to be like, way more capacious than any individual.
link |
So in some sense, there is this mind like thing already out there that's just vastly
link |
more intelligent than any individual is.
link |
And we think, okay, that's, you just accept that as a fact.
link |
That's the basic fabric of our existence is there's super intelligent.
link |
You get used to a lot of, I mean, there's already Google and Twitter and Facebook, these
link |
recommender systems that are the basic fabric of our, I could see them becoming, I mean,
link |
do you think of the collective intelligence of these systems as already perhaps reaching
link |
super intelligence level?
link |
Well, I mean, so here it comes to the concept of intelligence and the scale and what human
link |
The kind of vagueness and indeterminacy of those concepts starts to dominate how you
link |
would answer that question.
link |
So like, say the Google search engine has a very high capacity of a certain kind, like
link |
retrieving, remembering and retrieving information, particularly like text or images that are,
link |
you have a kind of string, a word string key, obviously superhuman at that, but a vast set
link |
of other things it can't even do at all.
link |
Not just not do well, but so you have these current AI systems that are superhuman in
link |
some limited domain and then like radically subhuman in all other domains.
link |
Same with a chess, like are just a simple computer that can multiply really large numbers,
link |
So it's going to have this like one spike of super intelligence and then a kind of a
link |
zero level of capability across all other cognitive fields.
link |
Yeah, I don't necessarily think the generalness, I mean, I'm not so attached with it, but I
link |
think it's sort of, it's a gray area and it's a feeling, but to me sort of alpha zero is
link |
somehow much more intelligent, much, much more intelligent than Deep Blue.
link |
And to say which domain, you could say, well, these are both just board games, they're both
link |
just able to play board games, who cares if they're going to do better or not, but there's
link |
something about the learning, the self play that makes it, crosses over into that land
link |
of intelligence that doesn't necessarily need to be general.
link |
In the same way, Google is much closer to Deep Blue currently in terms of its search
link |
engine than it is to sort of the alpha zero.
link |
And the moment it becomes, the moment these recommender systems really become more like
link |
alpha zero, but being able to learn a lot without the constraints of being heavily constrained
link |
by human interaction, that seems like a special moment in time.
link |
I mean, certainly learning ability seems to be an important facet of general intelligence,
link |
that you can take some new domain that you haven't seen before and you weren't specifically
link |
pre programmed for, and then figure out what's going on there and eventually become really
link |
So that's something alpha zero has much more of than Deep Blue had.
link |
And in fact, I mean, systems like alpha zero can learn not just Go, but other, in fact,
link |
probably beat Deep Blue in chess and so forth.
link |
So you do see this as general and it matches the intuition.
link |
We feel it's more intelligent and it also has more of this general purpose learning
link |
And if we get systems that have even more general purpose learning ability, it might
link |
also trigger an even stronger intuition that they are actually starting to get smart.
link |
So if you were to pick a future, what do you think a utopia looks like with AGI systems?
link |
Sort of, is it the neural link brain computer interface world where we're kind of really
link |
closely interlinked with AI systems?
link |
Is it possibly where AGI systems replace us completely while maintaining the values and
link |
the consciousness?
link |
Is it something like it's a completely invisible fabric, like you mentioned, a society where
link |
just aids and a lot of stuff that we do like curing diseases and so on.
link |
What is utopia if you get to pick?
link |
Yeah, I mean, it is a good question and a deep and difficult one.
link |
I'm quite interested in it.
link |
I don't have all the answers yet, but I might never have.
link |
But I think there are some different observations one can make.
link |
One is if this scenario actually did come to pass, it would open up this vast space
link |
of possible modes of being.
link |
On one hand, material and resource constraints would just be like expanded dramatically.
link |
So there would be a lot of a big pie, let's say.
link |
Also it would enable us to do things, including to ourselves, it would just open up this much
link |
larger design space and option space than we have ever had access to in human history.
link |
I think two things follow from that.
link |
One is that we probably would need to make a fairly fundamental rethink of what ultimately
link |
we value, like think things through more from first principles.
link |
The context would be so different from the familiar that we could have just take what
link |
we've always been doing and then like, oh, well, we have this cleaning robot that cleans
link |
the dishes in the sink and a few other small things.
link |
I think we would have to go back to first principles.
link |
So even from the individual level, go back to the first principles of what is the meaning
link |
of life, what is happiness, what is fulfillment.
link |
And then also connected to this large space of resources is that it would be possible.
link |
And I think something we should aim for is to do well by the lights of more than one
link |
That is, we wouldn't have to choose only one value criterion and say we're going to do
link |
something that scores really high on the metric of, say, hedonism, and then is like a zero
link |
by other criteria, like kind of wireheaded brain synovat, and it's like a lot of pleasure,
link |
that's good, but then like no beauty, no achievement like that.
link |
Or pick it up, I think to some significant, not unlimited sense, but the significant sense,
link |
it would be possible to do very well by many criteria, like maybe you could get like 98%
link |
of the best according to several criteria at the same time, given this great expansion
link |
of the option space.
link |
So have competing value systems, competing criteria, as a sort of forever, just like
link |
our Democrat versus Republican, there seems to be this always multiple parties that are
link |
useful for our progress in society, even though it might seem dysfunctional inside the moment,
link |
but having the multiple value system seems to be beneficial for, I guess, a balance of
link |
So that's, yeah, not exactly what I have in mind that it, well, although maybe in an indirect
link |
way it is, but that if you had the chance to do something that scored well on several
link |
different metrics, our first instinct should be to do that rather than immediately leap
link |
to the thing, which ones of these value systems are we going to screw over?
link |
Like our first, let's first try to do very well by all of them.
link |
Then it might be that you can't get 100% of all and you would have to then like have the
link |
hard conversation about which one will only get 97%.
link |
There's my cynicism that all of existence is always a trade off, but you say, maybe
link |
it's not such a bad trade off.
link |
Let's first at least try it.
link |
Well, this would be a distinctive context in which at least some of the constraints
link |
I'll leave it at that.
link |
So there's probably still be trade offs in the end.
link |
It's just that we should first make sure we at least take advantage of this abundance.
link |
So in terms of thinking about this, like, yeah, one should think, I think in this kind
link |
of frame of mind of generosity and inclusiveness to different value systems and see how far
link |
one can get there at first.
link |
And I think one could do something that would be very good according to many different criteria.
link |
We kind of talked about AGI fundamentally transforming the value system of our existence,
link |
the meaning of life.
link |
But today, what do you think is the meaning of life?
link |
The silliest or perhaps the biggest question, what's the meaning of life?
link |
What's the meaning of existence?
link |
What gives your life fulfillment, purpose, happiness, meaning?
link |
Yeah, I think these are, I guess, a bunch of different but related questions in there
link |
Happiness meaning.
link |
I mean, like you could imagine somebody getting a lot of happiness from something that they
link |
didn't think was meaningful.
link |
Like mindless, like watching reruns of some television series, waiting junk food, like
link |
maybe some people that gives pleasure, but they wouldn't think it had a lot of meaning.
link |
Whereas, conversely, something that might be quite loaded with meaning might not be
link |
very fun always, like some difficult achievement that really helps a lot of people, maybe requires
link |
self sacrifice and hard work.
link |
So these things can, I think, come apart, which is something to bear in mind also when
link |
if you're thinking about these utopia questions that you might, to actually start to do some
link |
constructive thinking about that, you might have to isolate and distinguish these different
link |
kinds of things that might be valuable in different ways.
link |
Make sure you can sort of clearly perceive each one of them and then you can think about
link |
how you can combine them.
link |
And just as you said, hopefully come up with a way to maximize all of them together.
link |
Yeah, or at least get, I mean, maximize or get like a very high score on a wide range
link |
of them, even if not literally all.
link |
You can always come up with values that are exactly opposed to one another, right?
link |
But I think for many values, they're kind of opposed with, if you place them within
link |
a certain dimensionality of your space, like there are shapes that are kind of, you can't
link |
untangle like in a given dimensionality, but if you start adding dimensions, then it might
link |
in many cases just be that they are easy to pull apart and you could.
link |
So we'll see how much space there is for that, but I think that there could be a lot in this
link |
context of radical abundance, if ever we get to that.
link |
I don't think there's a better way to end it, Nick.
link |
You've influenced a huge number of people to work on what could very well be the most
link |
important problems of our time.
link |
So it's a huge honor.
link |
Thank you so much for talking.
link |
Well, thank you for coming by, Lex.
link |
Thanks for listening to this conversation with Nick Bostrom, and thank you to our presenting
link |
sponsor, Cash App.
link |
Please consider supporting the podcast by downloading Cash App and using code LEXPodcast.
link |
If you enjoy this podcast, subscribe on YouTube, review it with five stars on Apple Podcast,
link |
subscribe on Patreon, or simply connect with me on Twitter at Lex Friedman.
link |
And now, let me leave you with some words from Nick Bostrom.
link |
Our approach to existential risks cannot be one of trial and error.
link |
There's no opportunity to learn from errors.
link |
The reactive approach, see what happens, limit damages, and learn from experience is unworkable.
link |
Rather, we must take a proactive approach.
link |
This requires foresight to anticipate new types of threats and a willingness to take
link |
decisive, preventative action and to bear the costs, moral and economic, of such actions.
link |
Thank you for listening, and hope to see you next time.