back to indexBen Goertzel: Artificial General Intelligence | Lex Fridman Podcast #103
link |
The following is a conversation with Ben Goertzel,
link |
one of the most interesting minds
link |
in the artificial intelligence community.
link |
He's the founder of SingularityNet,
link |
designer of OpenCog AI Framework,
link |
formerly a director of research
link |
at the Machine Intelligence Research Institute,
link |
and chief scientist of Hanson Robotics,
link |
the company that created the Sophia robot.
link |
He has been a central figure in the AGI community
link |
for many years, including in his organizing
link |
and contributing to the conference
link |
on artificial general intelligence,
link |
the 2020 version of which is actually happening this week,
link |
Wednesday, Thursday, and Friday.
link |
It's virtual and free.
link |
I encourage you to check out the talks,
link |
including by Yosha Bach from episode 101 of this podcast.
link |
Quick summary of the ads.
link |
Two sponsors, The Jordan Harbinger Show and Masterclass.
link |
Please consider supporting this podcast
link |
by going to jordanharbinger.com slash lex
link |
and signing up at masterclass.com slash lex.
link |
Click the links, buy all the stuff.
link |
It's the best way to support this podcast
link |
and the journey I'm on in my research and startup.
link |
This is the Artificial Intelligence Podcast.
link |
If you enjoy it, subscribe on YouTube,
link |
review it with five stars on Apple Podcast,
link |
support it on Patreon, or connect with me on Twitter
link |
at lexfriedman, spelled without the E, just F R I D M A N.
link |
As usual, I'll do a few minutes of ads now
link |
and never any ads in the middle
link |
that can break the flow of the conversation.
link |
This episode is supported by The Jordan Harbinger Show.
link |
Go to jordanharbinger.com slash lex.
link |
It's how he knows I sent you.
link |
On that page, there's links to subscribe to it
link |
on Apple Podcast, Spotify, and everywhere else.
link |
I've been binging on his podcast.
link |
He gets the best out of his guests,
link |
dives deep, calls them out when it's needed,
link |
and makes the whole thing fun to listen to.
link |
He's interviewed Kobe Bryant, Mark Cuban,
link |
Neil deGrasse Tyson, Keira Kasparov, and many more.
link |
His conversation with Kobe is a reminder
link |
how much focus and hard work is required for greatness
link |
in sport, business, and life.
link |
I highly recommend the episode if you want to be inspired.
link |
Again, go to jordanharbinger.com slash lex.
link |
It's how Jordan knows I sent you.
link |
This show is sponsored by Master Class.
link |
Sign up at masterclass.com slash lex
link |
to get a discount and to support this podcast.
link |
When I first heard about Master Class,
link |
I thought it was too good to be true.
link |
For 180 bucks a year, you get an all access pass
link |
to watch courses from to list some of my favorites.
link |
Chris Hadfield on Space Exploration,
link |
Neil deGrasse Tyson on Scientific Thinking
link |
and Communication, Will Wright, creator of
link |
the greatest city building game ever, Sim City,
link |
and Sims on Space Exploration.
link |
Ben Sims on Game Design, Carlos Santana on Guitar,
link |
Keira Kasparov, the greatest chess player ever on chess,
link |
Daniel Negrano on Poker, and many more.
link |
Chris Hadfield explaining how rockets work
link |
and the experience of being launched into space alone
link |
is worth the money.
link |
Once again, sign up at masterclass.com slash lex
link |
to get a discount and to support this podcast.
link |
Now, here's my conversation with Ben Kurtzell.
link |
What books, authors, ideas had a lot of impact on you
link |
in your life in the early days?
link |
You know, what got me into AI and science fiction
link |
and such in the first place wasn't a book,
link |
but the original Star Trek TV show,
link |
which my dad watched with me like in its first run.
link |
It would have been 1968, 69 or something,
link |
and that was incredible because every show
link |
they visited a different alien civilization
link |
with different culture and weird mechanisms.
link |
But that got me into science fiction,
link |
and there wasn't that much science fiction
link |
to watch on TV at that stage,
link |
so that got me into reading the whole literature
link |
of science fiction, you know,
link |
from the beginning of the previous century until that time.
link |
And I mean, there was so many science fiction writers
link |
who were inspirational to me.
link |
I'd say if I had to pick two,
link |
it would have been Stanisław Lem, the Polish writer.
link |
Yeah, Solaris, and then he had a bunch
link |
of more obscure writings on superhuman AIs
link |
that were engineered.
link |
Solaris was sort of a superhuman,
link |
naturally occurring intelligence.
link |
Then Philip K. Dick, who, you know,
link |
ultimately my fandom for Philip K. Dick
link |
is one of the things that brought me together
link |
with David Hansen, my collaborator on robotics projects.
link |
So, you know, Stanisław Lem was very much an intellectual,
link |
right, so he had a very broad view of intelligence
link |
going beyond the human and into what I would call,
link |
you know, open ended superintelligence.
link |
The Solaris superintelligent ocean was intelligent,
link |
in some ways more generally intelligent than people,
link |
but in a complex and confusing way
link |
so that human beings could never quite connect to it,
link |
but it was still probably very, very smart.
link |
And then the Golem 4 supercomputer
link |
in one of Lem's books, this was engineered by people,
link |
but eventually it became very intelligent
link |
in a different direction than humans
link |
and decided that humans were kind of trivial,
link |
not that interesting.
link |
So it put some impenetrable shield around itself,
link |
shut itself off from humanity,
link |
and then issued some philosophical screed
link |
about the pathetic and hopeless nature of humanity
link |
and all human thought, and then disappeared.
link |
Now, Philip K. Dick, he was a bit different.
link |
He was human focused, right?
link |
His main thing was, you know, human compassion
link |
and the human heart and soul are going to be the constant
link |
that will keep us going through whatever aliens we discover
link |
or telepathy machines or super AIs or whatever it might be.
link |
So he didn't believe in reality,
link |
like the reality that we see may be a simulation
link |
or a dream or something else we can't even comprehend,
link |
but he believed in love and compassion
link |
as something persistent
link |
through the various simulated realities.
link |
So those two science fiction writers had a huge impact on me.
link |
Then a little older than that, I got into Dostoevsky
link |
and Friedrich Nietzsche and Rimbaud
link |
and a bunch of more literary type writing.
link |
Can we talk about some of those things?
link |
So on the Solaris side, Stanislaw Lem,
link |
this kind of idea of there being intelligences out there
link |
that are different than our own,
link |
do you think there are intelligences maybe all around us
link |
that we're not able to even detect?
link |
So this kind of idea of,
link |
maybe you can comment also on Stephen Wolfram
link |
thinking that there's computations all around us
link |
and we're just not smart enough to kind of detect
link |
their intelligence or appreciate their intelligence.
link |
Yeah, so my friend Hugo de Gares,
link |
who I've been talking to about these things
link |
for many decades, since the early 90s,
link |
he had an idea he called SIPI,
link |
the Search for Intraparticulate Intelligence.
link |
So the concept there was as AIs get smarter
link |
and smarter and smarter,
link |
assuming the laws of physics as we know them now
link |
are still what these super intelligences
link |
perceived to hold and are bound by,
link |
as they get smarter and smarter,
link |
they're gonna shrink themselves littler and littler
link |
because special relativity makes it
link |
so they can communicate
link |
between two spatially distant points.
link |
So they're gonna get smaller and smaller,
link |
but then ultimately, what does that mean?
link |
The minds of the super, super, super intelligences,
link |
they're gonna be packed into the interaction
link |
of elementary particles or quarks
link |
or the partons inside quarks or whatever it is.
link |
So what we perceive as random fluctuations
link |
on the quantum or sub quantum level
link |
may actually be the thoughts
link |
of the micro, micro, micro miniaturized super intelligences
link |
because there's no way we can tell random
link |
from structured but within algorithmic information
link |
more complex than our brains, right?
link |
We can't tell the difference.
link |
So what we think is random could be the thought processes
link |
of some really tiny super minds.
link |
And if so, there is not a damn thing we can do about it,
link |
except try to upgrade our intelligences
link |
and expand our minds so that we can perceive
link |
more of what's around us.
link |
But if those random fluctuations,
link |
like even if we go to like quantum mechanics,
link |
if that's actually super intelligent systems,
link |
aren't we then part of the super of super intelligence?
link |
Aren't we just like a finger of the entirety
link |
of the body of the super intelligent system?
link |
It could be, I mean, a finger is a strange metaphor.
link |
A finger is dumb is what I mean.
link |
But the finger is also useful
link |
and is controlled with intent by the brain
link |
whereas we may be much less than that, right?
link |
I mean, yeah, we may be just some random epiphenomenon
link |
that they don't care about too much.
link |
Like think about the shape of the crowd emanating
link |
from a sports stadium or something, right?
link |
There's some emergent shape to the crowd, it's there.
link |
You could take a picture of it, it's kind of cool.
link |
It's irrelevant to the main point of the sports event
link |
or where the people are going
link |
or what's on the minds of the people
link |
making that shape in the crowd, right?
link |
So we may just be some semi arbitrary higher level pattern
link |
popping out of a lower level
link |
hyper intelligent self organization.
link |
And I mean, so be it, right?
link |
I mean, that's one thing that...
link |
Yeah, I mean, the older I've gotten,
link |
the more respect I've achieved for our fundamental ignorance.
link |
I mean, mine and everybody else's.
link |
I mean, I look at my two dogs,
link |
two beautiful little toy poodles
link |
and they watch me sitting at the computer typing.
link |
They just think I'm sitting there wiggling my fingers
link |
to exercise them maybe or guarding the monitor on the desk
link |
that they have no idea that I'm communicating
link |
with other people halfway around the world,
link |
let alone creating complex algorithms
link |
running in RAM on some computer server
link |
in St. Petersburg or something, right?
link |
Although they're right there in the room with me.
link |
So what things are there right around us
link |
that we're just too stupid or close minded to comprehend?
link |
Probably quite a lot.
link |
Your very poodle could also be communicating
link |
across multiple dimensions with other beings
link |
and you're too unintelligent to understand
link |
the kind of communication mechanism they're going through.
link |
There have been various TV shows and science fiction novels,
link |
poisoning cats, dolphins, mice and whatnot
link |
are actually super intelligences here to observe that.
link |
I would guess as one or the other quantum physics founders
link |
said, those theories are not crazy enough to be true.
link |
The reality is probably crazier than that.
link |
So on the human side, with Philip K. Dick
link |
and in general, where do you fall on this idea
link |
that love and just the basic spirit of human nature
link |
persists throughout these multiple realities?
link |
Are you on the side, like the thing that inspires you
link |
about artificial intelligence,
link |
is it the human side of somehow persisting
link |
through all of the different systems we engineer
link |
or is AI inspire you to create something
link |
that's greater than human, that's beyond human,
link |
that's almost nonhuman?
link |
I would say my motivation to create AGI
link |
comes from both of those directions actually.
link |
So when I first became passionate about AGI
link |
when I was, it would have been two or three years old
link |
after watching robots on Star Trek.
link |
I mean, then it was really a combination
link |
of intellectual curiosity, like can a machine really think,
link |
how would you do that?
link |
And yeah, just ambition to create something much better
link |
than all the clearly limited
link |
and fundamentally defective humans I saw around me.
link |
Then as I got older and got more enmeshed
link |
in the human world and got married, had children,
link |
saw my parents begin to age, I started to realize,
link |
well, not only will AGI let you go far beyond
link |
the limitations of the human,
link |
but it could also stop us from dying and suffering
link |
and feeling pain and tormenting ourselves mentally.
link |
So you can see AGI has amazing capability
link |
to do good for humans, as humans,
link |
alongside with its capability
link |
to go far, far beyond the human level.
link |
So I mean, both aspects are there,
link |
which makes it even more exciting and important.
link |
So you mentioned Dostoevsky and Nietzsche.
link |
Where did you pick up from those guys?
link |
That would probably go beyond the scope
link |
of a brief interview, certainly.
link |
I mean, both of those are amazing thinkers
link |
who one, will necessarily have
link |
a complex relationship with, right?
link |
So, I mean, Dostoevsky on the minus side,
link |
he's kind of a religious fanatic
link |
and he sort of helped squash the Russian nihilist movement,
link |
which was very interesting.
link |
Because what nihilism meant originally
link |
in that period of the mid, late 1800s in Russia
link |
was not taking anything fully 100% for granted.
link |
It was really more like what we'd call Bayesianism now,
link |
where you don't wanna adopt anything
link |
as a dogmatic certitude and always leave your mind open.
link |
And how Dostoevsky parodied nihilism
link |
was a bit different, right?
link |
He parodied as people who believe absolutely nothing.
link |
So they must assign an equal probability weight
link |
to every proposition, which doesn't really work.
link |
So on the one hand, I didn't really agree with Dostoevsky
link |
on his sort of religious point of view.
link |
On the other hand, if you look at his understanding
link |
of human nature and sort of the human mind
link |
and heart and soul, it's really unparalleled.
link |
He had an amazing view of how human beings construct a world
link |
for themselves based on their own understanding
link |
and their own mental predisposition.
link |
And I think if you look in the brothers Karamazov
link |
in particular, the Russian literary theorist Mikhail Bakhtin
link |
wrote about this as a polyphonic mode of fiction,
link |
which means it's not third person,
link |
but it's not first person from any one person really.
link |
There are many different characters in the novel
link |
and each of them is sort of telling part of the story
link |
from their own point of view.
link |
So the reality of the whole story is an intersection
link |
like synergetically of the many different characters
link |
And that really, it's a beautiful metaphor
link |
and even a reflection I think of how all of us
link |
socially create our reality.
link |
Like each of us sees the world in a certain way.
link |
Each of us in a sense is making the world as we see it
link |
based on our own minds and understanding,
link |
but it's polyphony like in music
link |
where multiple instruments are coming together
link |
to create the sound.
link |
The ultimate reality that's created
link |
comes out of each of our subjective understandings,
link |
intersecting with each other.
link |
And that was one of the many beautiful things in Dostoevsky.
link |
So maybe a little bit to mention,
link |
you have a connection to Russia and the Soviet culture.
link |
I mean, I'm not sure exactly what the nature
link |
of the connection is, but at least the spirit
link |
of your thinking is in there.
link |
Well, my ancestry is three quarters Eastern European Jewish.
link |
So I mean, my three of my great grandparents
link |
emigrated to New York from Lithuania
link |
and sort of border regions of Poland,
link |
which are in and out of Poland
link |
in around the time of World War I.
link |
And they were socialists and communists as well as Jews,
link |
mostly Menshevik, not Bolshevik.
link |
And they sort of, they fled at just the right time
link |
to the US for their own personal reasons.
link |
And then almost all, or maybe all of my extended family
link |
that remained in Eastern Europe was killed
link |
either by Hitlands or Stalin's minions at some point.
link |
So the branch of the family that emigrated to the US
link |
was pretty much the only one.
link |
So how much of the spirit of the people
link |
is in your blood still?
link |
Like, when you look in the mirror, do you see,
link |
Meat, I see a bag of meat that I want to transcend
link |
by uploading into some sort of superior reality.
link |
But very, I mean, yeah, very clearly,
link |
I mean, I'm not religious in a traditional sense,
link |
but clearly the Eastern European Jewish tradition
link |
was what I was raised in.
link |
I mean, there was, my grandfather, Leo Zwell,
link |
was a physical chemist who worked with Linus Pauling
link |
and a bunch of the other early greats in quantum mechanics.
link |
I mean, he was into X ray diffraction.
link |
He was on the material science side,
link |
an experimentalist rather than a theorist.
link |
His sister was also a physicist.
link |
And my father's father, Victor Gertzel,
link |
was a PhD in psychology who had the unenviable job
link |
of giving Soka therapy to the Japanese
link |
in internment camps in the US in World War II,
link |
like to counsel them why they shouldn't kill themselves,
link |
even though they'd had all their stuff taken away
link |
and been imprisoned for no good reason.
link |
So, I mean, yeah, there's a lot of Eastern European
link |
Jewishness in my background.
link |
One of my great uncles was, I guess,
link |
conductor of San Francisco Orchestra.
link |
So there's a lot of Mickey Salkind,
link |
bunch of music in there also.
link |
And clearly this culture was all about learning
link |
and understanding the world,
link |
and also not quite taking yourself too seriously
link |
while you do it, right?
link |
There's a lot of Yiddish humor in there.
link |
So I do appreciate that culture,
link |
although the whole idea that like the Jews
link |
are the chosen people of God
link |
never resonated with me too much.
link |
The graph of the Gertzel family,
link |
I mean, just the people I've encountered
link |
just doing some research and just knowing your work
link |
through the decades, it's kind of fascinating.
link |
Just the number of PhDs.
link |
Yeah, yeah, I mean, my dad is a sociology professor
link |
who recently retired from Rutgers University,
link |
but clearly that gave me a head start in life.
link |
I mean, my grandfather gave me
link |
all those quantum mechanics books
link |
when I was like seven or eight years old.
link |
I remember going through them,
link |
and it was all the old quantum mechanics
link |
like Rutherford Adams and stuff.
link |
So I got to the part of wave functions,
link |
which I didn't understand, although I was very bright kid.
link |
And I realized he didn't quite understand it either,
link |
but at least like he pointed me to some professor
link |
he knew at UPenn nearby who understood these things, right?
link |
So that's an unusual opportunity for a kid to have, right?
link |
My dad, he was programming Fortran
link |
when I was 10 or 11 years old
link |
on like HP 3000 mainframes at Rutgers University.
link |
So I got to do linear regression in Fortran
link |
on punch cards when I was in middle school, right?
link |
Because he was doing, I guess, analysis of demographic
link |
and sociology data.
link |
So yes, certainly that gave me a head start
link |
and a push towards science beyond what would have been
link |
the case with many, many different situations.
link |
When did you first fall in love with AI?
link |
Is it the programming side of Fortran?
link |
Is it maybe the sociology psychology
link |
that you picked up from your dad?
link |
Or is it the quantum mechanics?
link |
I fell in love with AI when I was probably three years old
link |
when I saw a robot on Star Trek.
link |
It was turning around in a circle going,
link |
error, error, error, error,
link |
because Spock and Kirk had tricked it
link |
into a mechanical breakdown by presenting it
link |
with a logical paradox.
link |
And I was just like, well, this makes no sense.
link |
This AI is very, very smart.
link |
It's been traveling all around the universe,
link |
but these people could trick it
link |
with a simple logical paradox.
link |
Like why, if the human brain can get beyond that paradox,
link |
why can't this AI?
link |
So I felt the screenwriters of Star Trek
link |
had misunderstood the nature of intelligence.
link |
And I complained to my dad about it,
link |
and he wasn't gonna say anything one way or the other.
link |
But before I was born, when my dad was at Antioch College
link |
in the middle of the US,
link |
he led a protest movement called SLAM,
link |
Student League Against Mortality.
link |
They were protesting against death,
link |
wandering across the campus.
link |
So he was into some futuristic things even back then,
link |
but whether AI could confront logical paradoxes or not,
link |
But when I, 10 years after that or something,
link |
I discovered Douglas Hofstadter's book,
link |
Gordalesh or Bach, and that was sort of to the same point of AI
link |
and paradox and logic, right?
link |
Because he was over and over
link |
with Gordal's incompleteness theorem,
link |
and can an AI really fully model itself reflexively
link |
or does that lead you into some paradox?
link |
Can the human mind truly model itself reflexively
link |
or does that lead you into some paradox?
link |
So I think that book, Gordalesh or Bach,
link |
which I think I read when it first came out,
link |
I would have been 12 years old or something.
link |
I remember it was like 16 hour day.
link |
I read it cover to cover and then reread it.
link |
I reread it after that,
link |
because there was a lot of weird things
link |
with little formal systems in there
link |
that were hard for me at the time.
link |
But that was the first book I read
link |
that gave me a feeling for AI as like a practical academic
link |
or engineering discipline that people were working in.
link |
Because before I read Gordalesh or Bach,
link |
I was into AI from the point of view of a science fiction fan.
link |
And I had the idea, well, it may be a long time
link |
before we can achieve immortality in superhuman AGI.
link |
So I should figure out how to build a spacecraft
link |
traveling close to the speed of light, go far away,
link |
then come back to the earth in a million years
link |
when technology is more advanced
link |
and we can build these things.
link |
Reading Gordalesh or Bach,
link |
while it didn't all ring true to me, a lot of it did,
link |
but I could see like there are smart people right now
link |
at various universities around me
link |
who are actually trying to work on building
link |
what I would now call AGI,
link |
although Hofstadter didn't call it that.
link |
So really it was when I read that book,
link |
which would have been probably middle school,
link |
that then I started to think,
link |
well, this is something that I could practically work on.
link |
Yeah, as opposed to flying away and waiting it out,
link |
you can actually be one of the people
link |
that actually builds the system.
link |
And if you think about, I mean,
link |
I was interested in what we'd now call nanotechnology
link |
and in the human immortality and time travel,
link |
all the same cool things as every other,
link |
like science fiction loving kid.
link |
But AI seemed like if Hofstadter was right,
link |
you just figure out the right program,
link |
sit there and type it.
link |
Like you don't need to spin stars into weird configurations
link |
or get government approval to cut people up
link |
and fiddle with their DNA or something, right?
link |
It's just programming.
link |
And then of course that can achieve anything else.
link |
There's another book from back then,
link |
which was by Gerald Feinbaum,
link |
who was a physicist at Princeton.
link |
And that was the Prometheus Project.
link |
And this book was written in the late 1960s,
link |
though I encountered it in the mid 70s.
link |
But what this book said is in the next few decades,
link |
humanity is gonna create superhuman thinking machines,
link |
molecular nanotechnology and human immortality.
link |
And then the challenge we'll have is what to do with it.
link |
Do we use it to expand human consciousness
link |
in a positive direction?
link |
Or do we use it just to further vapid consumerism?
link |
And what he proposed was that the UN
link |
should do a survey on this.
link |
And the UN should send people out to every little village
link |
in remotest Africa or South America
link |
and explain to everyone what technology
link |
was gonna bring the next few decades
link |
and the choice that we had about how to use it.
link |
And let everyone on the whole planet vote
link |
about whether we should develop super AI nanotechnology
link |
and immortality for expanded consciousness
link |
or for rampant consumerism.
link |
And needless to say, that didn't quite happen.
link |
And I think this guy died in the mid 80s,
link |
so we didn't even see his ideas start
link |
to become more mainstream.
link |
But it's interesting, many of the themes I'm engaged with now
link |
from AGI and immortality,
link |
even to trying to democratize technology
link |
as I've been pushing forward with Singularity,
link |
my work in the blockchain world,
link |
many of these themes were there in Feinbaum's book
link |
in the late 60s even.
link |
And of course, Valentin Turchin, a Russian writer
link |
and a great Russian physicist who I got to know
link |
when we both lived in New York in the late 90s
link |
I mean, he had a book in the late 60s in Russia,
link |
which was the phenomenon of science,
link |
which laid out all these same things as well.
link |
And Val died in, I don't remember,
link |
2004 or five or something of Parkinson'sism.
link |
So yeah, it's easy for people to lose track now
link |
of the fact that the futurist and Singularitarian
link |
advanced technology ideas that are now almost mainstream
link |
are on TV all the time.
link |
I mean, these are not that new, right?
link |
They're sort of new in the history of the human species,
link |
but I mean, these were all around in fairly mature form
link |
in the middle of the last century,
link |
were written about quite articulately
link |
by fairly mainstream people
link |
who were professors at top universities.
link |
It's just until the enabling technologies
link |
got to a certain point, then you couldn't make it real.
link |
And even in the 70s, I was sort of seeing that
link |
and living through it, right?
link |
From Star Trek to Douglas Hofstadter,
link |
things were getting very, very practical
link |
from the late 60s to the late 70s.
link |
And the first computer I bought,
link |
you could only program with hexadecimal machine code
link |
and you had to solder it together.
link |
And then like a few years later, there's punch cards.
link |
And a few years later, you could get like Atari 400
link |
and Commodore VIC 20, and you could type on the keyboard
link |
and program in higher level languages
link |
alongside the assembly language.
link |
So these ideas have been building up a while.
link |
And I guess my generation got to feel them build up,
link |
which is different than people coming into the field now
link |
for whom these things have just been part of the ambience
link |
of culture for their whole career
link |
or even their whole life.
link |
Well, it's fascinating to think about there being all
link |
of these ideas kind of swimming, almost with the noise
link |
all around the world, all the different generations,
link |
and then some kind of nonlinear thing happens
link |
where they percolate up
link |
and capture the imagination of the mainstream.
link |
And that seems to be what's happening with AI now.
link |
I mean, Nietzsche, who you mentioned had the idea
link |
of the Superman, right?
link |
But he didn't understand enough about technology
link |
to think you could physically engineer a Superman
link |
by piecing together molecules in a certain way.
link |
He was a bit vague about how the Superman would appear,
link |
but he was quite deep at thinking
link |
about what the state of consciousness
link |
and the mode of cognition of a Superman would be.
link |
He was a very astute analyst of how the human mind
link |
constructs the illusion of a self,
link |
how it constructs the illusion of free will,
link |
how it constructs values like good and evil
link |
out of its own desire to maintain
link |
and advance its own organism.
link |
He understood a lot about how human minds work.
link |
Then he understood a lot
link |
about how post human minds would work.
link |
I mean, the Superman was supposed to be a mind
link |
that would basically have complete root access
link |
to its own brain and consciousness
link |
and be able to architect its own value system
link |
and inspect and fine tune all of its own biases.
link |
So that's a lot of powerful thinking there,
link |
which then fed in and sort of seeded
link |
all of postmodern continental philosophy
link |
and all sorts of things have been very valuable
link |
in development of culture and indirectly even of technology.
link |
But of course, without the technology there,
link |
it was all some quite abstract thinking.
link |
So now we're at a time in history
link |
when a lot of these ideas can be made real,
link |
which is amazing and scary, right?
link |
It's kind of interesting to think,
link |
what do you think Nietzsche would do
link |
if he was born a century later or transported through time?
link |
What do you think he would say about AI?
link |
I mean. Well, those are quite different.
link |
If he's born a century later or transported through time.
link |
Well, he'd be on like TikTok and Instagram
link |
and he would never write the great works he's written.
link |
So let's transport him through time.
link |
Maybe also Sprach Zarathustra would be a music video,
link |
right? I mean, who knows?
link |
Yeah, but if he was transported through time,
link |
do you think, that'd be interesting actually to go back.
link |
You just made me realize that it's possible to go back
link |
and read Nietzsche with an eye of,
link |
is there some thinking about artificial beings?
link |
I'm sure there he had inklings.
link |
I mean, with Frankenstein before him,
link |
I'm sure he had inklings of artificial beings
link |
somewhere in the text.
link |
It'd be interesting to try to read his work
link |
to see if Superman was actually an AGI system.
link |
Like if he had inklings of that kind of thinking.
link |
No, I would say not.
link |
I mean, he had a lot of inklings of modern cognitive science,
link |
which are very interesting.
link |
If you look in like the third part of the collection
link |
that's been titled The Will to Power.
link |
I mean, in book three there,
link |
there's very deep analysis of thinking processes,
link |
but he wasn't so much of a physical tinkerer type guy,
link |
right? He was very abstract.
link |
Do you think, what do you think about the will to power?
link |
Do you think human, what do you think drives humans?
link |
Oh, an unholy mix of things.
link |
I don't think there's one pure, simple,
link |
and elegant objective function driving humans by any means.
link |
What do you think, if we look at,
link |
I know it's hard to look at humans in an aggregate,
link |
but do you think overall humans are good?
link |
Or do we have both good and evil within us
link |
that depending on the circumstances,
link |
depending on whatever can percolate to the top?
link |
Good and evil are very ambiguous, complicated
link |
and in some ways silly concepts.
link |
But if we could dig into your question
link |
from a couple of directions.
link |
So I think if you look in evolution,
link |
humanity is shaped both by individual selection
link |
and what biologists would call group selection,
link |
like tribe level selection, right?
link |
So individual selection has driven us
link |
in a selfish DNA sort of way.
link |
So that each of us does to a certain approximation
link |
what will help us propagate our DNA to future generations.
link |
I mean, that's why I've got four kids so far
link |
and probably that's not the last one.
link |
On the other hand.
link |
I like the ambition.
link |
Tribal, like group selection means humans in a way
link |
will do what will advocate for the persistence of the DNA
link |
of their whole tribe or their social group.
link |
And in biology, you have both of these, right?
link |
And you can see, say an ant colony or a beehive,
link |
there's a lot of group selection
link |
in the evolution of those social animals.
link |
On the other hand, say a big cat
link |
or some very solitary animal,
link |
it's a lot more biased toward individual selection.
link |
Humans are an interesting balance.
link |
And I think this reflects itself
link |
in what we would view as selfishness versus altruism
link |
So we just have both of those objective functions
link |
contributing to the makeup of our brains.
link |
And then as Nietzsche analyzed in his own way
link |
and others have analyzed in different ways,
link |
I mean, we abstract this as well,
link |
we have both good and evil within us, right?
link |
Because a lot of what we view as evil
link |
is really just selfishness.
link |
A lot of what we view as good is altruism,
link |
which means doing what's good for the tribe.
link |
And on that level,
link |
we have both of those just baked into us
link |
and that's how it is.
link |
Of course, there are psychopaths and sociopaths
link |
and people who get gratified by the suffering of others.
link |
And that's a different thing.
link |
Yeah, those are exceptions on the whole.
link |
But I think at core, we're not purely selfish,
link |
we're not purely altruistic, we are a mix
link |
and that's the nature of it.
link |
And we also have a complex constellation of values
link |
that are just very specific to our evolutionary history.
link |
Like we love waterways and mountains
link |
and the ideal place to put a house
link |
is in a mountain overlooking the water, right?
link |
And we care a lot about our kids
link |
and we care a little less about our cousins
link |
and even less about our fifth cousins.
link |
I mean, there are many particularities to human values,
link |
which whether they're good or evil
link |
depends on your perspective.
link |
Say, I spent a lot of time in Ethiopia in Addis Ababa
link |
where we have one of our AI development offices
link |
for my SingularityNet project.
link |
And when I walk through the streets in Addis,
link |
you know, there's people lying by the side of the road,
link |
like just living there by the side of the road,
link |
dying probably of curable diseases
link |
without enough food or medicine.
link |
And when I walk by them, you know, I feel terrible,
link |
I give them money.
link |
When I come back home to the developed world,
link |
they're not on my mind that much.
link |
I do donate some, but I mean,
link |
I also spend some of the limited money I have
link |
enjoying myself in frivolous ways
link |
rather than donating it to those people who are right now,
link |
like starving, dying and suffering on the roadside.
link |
So does that make me evil?
link |
I mean, it makes me somewhat selfish
link |
and somewhat altruistic.
link |
And we each balance that in our own way, right?
link |
So whether that will be true of all possible AGI's
link |
is a subtler question.
link |
So that's how humans are.
link |
So you have a sense, you kind of mentioned
link |
that there's a selfish,
link |
I'm not gonna bring up the whole Ayn Rand idea
link |
of selfishness being the core virtue.
link |
That's a whole interesting kind of tangent
link |
that I think we'll just distract ourselves on.
link |
I have to make one amusing comment.
link |
A comment that has amused me anyway.
link |
So the, yeah, I have extraordinary negative respect
link |
Negative, what's a negative respect?
link |
But when I worked with a company called Genescient,
link |
which was evolving flies to have extraordinary long lives
link |
in Southern California.
link |
So we had flies that were evolved by artificial selection
link |
to have five times the lifespan of normal fruit flies.
link |
But the population of super long lived flies
link |
was physically sitting in a spare room
link |
at an Ayn Rand elementary school in Southern California.
link |
So that was just like,
link |
well, if I saw this in a movie, I wouldn't believe it.
link |
Well, yeah, the universe has a sense of humor
link |
in that kind of way.
link |
That fits in, humor fits in somehow
link |
into this whole absurd existence.
link |
But you mentioned the balance between selfishness
link |
and altruism as kind of being innate.
link |
Do you think it's possible
link |
that's kind of an emergent phenomena,
link |
those peculiarities of our value system?
link |
How much of it is innate?
link |
How much of it is something we collectively
link |
kind of like a Dostoevsky novel
link |
bring to life together as a civilization?
link |
I mean, the answer to nature versus nurture
link |
And of course it's nature versus nurture
link |
versus self organization, as you mentioned.
link |
So clearly there are evolutionary roots
link |
to individual and group selection
link |
leading to a mix of selfishness and altruism.
link |
On the other hand,
link |
different cultures manifest that in different ways.
link |
Well, we all have basically the same biology.
link |
And if you look at sort of precivilized cultures,
link |
you have tribes like the Yanomamo in Venezuela,
link |
which their culture is focused on killing other tribes.
link |
And you have other Stone Age tribes
link |
that are mostly peaceful and have big taboos
link |
So you can certainly have a big difference
link |
in how culture manifests
link |
these innate biological characteristics,
link |
but still, there's probably limits
link |
that are given by our biology.
link |
I used to argue this with my great grandparents
link |
who were Marxists actually,
link |
because they believed in the withering away of the state.
link |
Like they believe that,
link |
as you move from capitalism to socialism to communism,
link |
people would just become more social minded
link |
so that a state would be unnecessary
link |
and everyone would give everyone else what they needed.
link |
Now, setting aside that
link |
that's not what the various Marxist experiments
link |
on the planet seem to be heading toward in practice.
link |
Just as a theoretical point,
link |
I was very dubious that human nature could go there.
link |
Like at that time when my great grandparents are alive,
link |
I was just like, you know, I'm a cynical teenager.
link |
I think humans are just jerks.
link |
The state is not gonna wither away.
link |
If you don't have some structure
link |
keeping people from screwing each other over,
link |
they're gonna do it.
link |
So now I actually don't quite see things that way.
link |
I mean, I think my feeling now subjectively
link |
is the culture aspect is more significant
link |
than I thought it was when I was a teenager.
link |
And I think you could have a human society
link |
that was dialed dramatically further toward,
link |
you know, self awareness, other awareness,
link |
compassion and sharing than our current society.
link |
And of course, greater material abundance helps,
link |
but to some extent material abundance
link |
is a subjective perception also
link |
because many Stone Age cultures perceive themselves
link |
as living in great material abundance
link |
that they had all the food and water they wanted,
link |
they lived in a beautiful place,
link |
that they had sex lives, that they had children.
link |
I mean, they had abundance without any factories, right?
link |
So I think humanity probably would be capable
link |
of fundamentally more positive and joy filled mode
link |
of social existence than what we have now.
link |
Clearly Marx didn't quite have the right idea
link |
about how to get there.
link |
I mean, he missed a number of key aspects
link |
of human society and its evolution.
link |
And if we look at where we are in society now,
link |
how to get there is a quite different question
link |
because there are very powerful forces
link |
pushing people in different directions
link |
than a positive, joyous, compassionate existence, right?
link |
So if we were tried to, you know,
link |
Elon Musk is dreams of colonizing Mars at the moment,
link |
so we maybe will have a chance to start a new civilization
link |
with a new governmental system.
link |
And certainly there's quite a bit of chaos.
link |
We're sitting now, I don't know what the date is,
link |
There's quite a bit of chaos in all different forms
link |
going on in the United States and all over the world.
link |
So there's a hunger for new types of governments,
link |
new types of leadership, new types of systems.
link |
And so what are the forces at play
link |
and how do we move forward?
link |
Yeah, I mean, colonizing Mars, first of all,
link |
it's a super cool thing to do.
link |
We should be doing it.
link |
So you love the idea.
link |
Yeah, I mean, it's more important than making
link |
chocolatey or chocolates and sexier lingerie
link |
and many of the things that we spend
link |
a lot more resources on as a species, right?
link |
So I mean, we certainly should do it.
link |
I think the possible futures in which a Mars colony
link |
makes a critical difference for humanity are very few.
link |
I mean, I think, I mean, assuming we make a Mars colony
link |
and people go live there in a couple of decades,
link |
I mean, their supplies are gonna come from Earth.
link |
The money to make the colony came from Earth
link |
and whatever powers are supplying the goods there
link |
from Earth are gonna, in effect, be in control
link |
of that Mars colony.
link |
Of course, there are outlier situations
link |
where Earth gets nuked into oblivion
link |
and somehow Mars has been made self sustaining by that point
link |
and then Mars is what allows humanity to persist.
link |
But I think that those are very, very, very unlikely.
link |
You don't think it could be a first step on a long journey?
link |
Of course it's a first step on a long journey,
link |
I'm guessing the colonization of the rest
link |
of the physical universe will probably be done
link |
by AGI's that are better designed to live in space
link |
than by the meat machines that we are.
link |
But I mean, who knows?
link |
We may cryopreserve ourselves in some superior way
link |
to what we know now and like shoot ourselves out
link |
to Alpha Centauri and beyond.
link |
I mean, that's all cool.
link |
It's very interesting and it's much more valuable
link |
than most things that humanity is spending its resources on.
link |
On the other hand, with AGI, we can get to a singularity
link |
before the Mars colony becomes sustaining for sure,
link |
possibly before it's even operational.
link |
So your intuition is that that's the problem
link |
if we really invest resources and we can get to faster
link |
than a legitimate full self sustaining colonization of Mars.
link |
Yeah, and it's very clear that we will to me
link |
because there's so much economic value
link |
in getting from narrow AI toward AGI,
link |
whereas the Mars colony, there's less economic value
link |
until you get quite far out into the future.
link |
So I think that's very interesting.
link |
I just think it's somewhat off to the side.
link |
I mean, just as I think, say, art and music
link |
are very, very interesting and I wanna see resources
link |
go into amazing art and music being created.
link |
And I'd rather see that than a lot of the garbage
link |
that the society spends their money on.
link |
On the other hand, I don't think Mars colonization
link |
or inventing amazing new genres of music
link |
is not one of the things that is most likely
link |
to make a critical difference in the evolution
link |
of human or nonhuman life in this part of the universe
link |
over the next decade.
link |
Do you think AGI is really?
link |
AGI is by far the most important thing
link |
that's on the horizon.
link |
And then technologies that have direct ability
link |
to enable AGI or to accelerate AGI are also very important.
link |
For example, say, quantum computing.
link |
I don't think that's critical to achieve AGI,
link |
but certainly you could see how
link |
the right quantum computing architecture
link |
could massively accelerate AGI,
link |
similar other types of nanotechnology.
link |
Right now, the quest to cure aging and end disease
link |
while not in the big picture as important as AGI,
link |
of course, it's important to all of us as individual humans.
link |
And if someone made a super longevity pill
link |
and distributed it tomorrow, I mean,
link |
that would be huge and a much larger impact
link |
than a Mars colony is gonna have for quite some time.
link |
But perhaps not as much as an AGI system.
link |
No, because if you can make a benevolent AGI,
link |
then all the other problems are solved.
link |
I mean, if then the AGI can be,
link |
once it's as generally intelligent as humans,
link |
it can rapidly become massively more generally intelligent
link |
And then that AGI should be able to solve science
link |
and engineering problems much better than human beings,
link |
as long as it is in fact motivated to do so.
link |
That's why I said a benevolent AGI.
link |
There could be other kinds.
link |
Maybe it's good to step back a little bit.
link |
I mean, we've been using the term AGI.
link |
People often cite you as the creator,
link |
or at least the popularizer of the term AGI,
link |
artificial general intelligence.
link |
Can you tell the origin story of the term maybe?
link |
So yeah, I would say I launched the term AGI upon the world
link |
for what it's worth without ever fully being in love
link |
What happened is I was editing a book,
link |
and this process started around 2001 or two.
link |
I think the book came out 2005, finally.
link |
I was editing a book which I provisionally
link |
was titling Real AI.
link |
And I mean, the goal was to gather together
link |
fairly serious academicish papers
link |
on the topic of making thinking machines
link |
that could really think in the sense like people can,
link |
or even more broadly than people can, right?
link |
So then I was reaching out to other folks
link |
that I had encountered here or there
link |
who were interested in that,
link |
which included some other folks who I knew
link |
from the transhumist and singularitarian world,
link |
like Peter Vos, who has a company, AGI Incorporated,
link |
still in California, and included Shane Legge,
link |
who had worked for me at my company, WebMind,
link |
in New York in the late 90s,
link |
who by now has become rich and famous.
link |
He was one of the cofounders of Google DeepMind.
link |
But at that time, Shane was,
link |
I think he may have just started doing his PhD
link |
with Marcus Hooter, who at that time
link |
hadn't yet published his book, Universal AI,
link |
which sort of gives a mathematical foundation
link |
for artificial general intelligence.
link |
So I reached out to Shane and Marcus and Peter Vos
link |
and Pei Wang, who was another former employee of mine
link |
who had been Douglas Hofstadter's PhD student
link |
who had his own approach to AGI,
link |
and a bunch of some Russian folks reached out to these guys
link |
and they contributed papers for the book.
link |
But that was my provisional title, but I never loved it
link |
because in the end, I was doing some,
link |
what we would now call narrow AI as well,
link |
like applying machine learning to genomics data
link |
or chat data for sentiment analysis.
link |
I mean, that work is real.
link |
And in a sense, it's really AI.
link |
It's just a different kind of AI.
link |
Ray Kurzweil wrote about narrow AI versus strong AI,
link |
but that seemed weird to me because first of all,
link |
narrow and strong are not antennas.
link |
But secondly, strong AI was used
link |
in the cognitive science literature
link |
to mean the hypothesis that digital computer AIs
link |
could have true consciousness like human beings.
link |
So there was already a meaning to strong AI,
link |
which was complexly different, but related, right?
link |
So we were tossing around on an email list
link |
whether what title it should be.
link |
And so we talked about narrow AI, broad AI, wide AI,
link |
narrow AI, general AI.
link |
And I think it was either Shane Legge or Peter Vos
link |
on the private email discussion we had.
link |
He said, but why don't we go
link |
with AGI, artificial general intelligence?
link |
And Pei Wang wanted to do GAI,
link |
general artificial intelligence,
link |
because in Chinese it goes in that order.
link |
But we figured gay wouldn't work
link |
in US culture at that time, right?
link |
So we went with the AGI.
link |
We used it for the title of that book.
link |
And part of Peter and Shane's reasoning
link |
was you have the G factor in psychology,
link |
which is IQ, general intelligence, right?
link |
So you have a meaning of GI, general intelligence,
link |
in psychology, so then you're looking like artificial GI.
link |
So then we use that for the title of the book.
link |
And so I think maybe both Shane and Peter
link |
think they invented the term,
link |
but then later after the book was published,
link |
this guy, Mark Guberd, came up to me and he's like,
link |
well, I published an essay with the term AGI
link |
in like 1997 or something.
link |
And so I'm just waiting for some Russian to come out
link |
and say they published that in 1953, right?
link |
I mean, that term is not dramatically innovative
link |
It's one of these obvious in hindsight things,
link |
which is also annoying in a way,
link |
because Joshua Bach, who you interviewed,
link |
is a close friend of mine.
link |
He likes the term synthetic intelligence,
link |
which I like much better,
link |
but it hasn't actually caught on, right?
link |
Because I mean, artificial is a bit off to me
link |
because artifice is like a tool or something,
link |
but not all AGI's are gonna be tools.
link |
I mean, they may be now,
link |
but we're aiming toward making them agents
link |
rather than tools.
link |
And in a way, I don't like the distinction
link |
between artificial and natural,
link |
because I mean, we're part of nature also
link |
and machines are part of nature.
link |
I mean, you can look at evolved versus engineered,
link |
but that's a different distinction.
link |
Then it should be engineered general intelligence, right?
link |
And then general, well,
link |
if you look at Marcus Hooter's book,
link |
universally, what he argues there is,
link |
within the domain of computation theory,
link |
which is limited, but interesting.
link |
So if you assume computable environments
link |
or computable reward functions,
link |
then he articulates what would be
link |
a truly general intelligence,
link |
a system called AIXI, which is quite beautiful.
link |
AIXI, and that's the middle name
link |
of my latest child, actually, is it?
link |
What's the first name?
link |
First name is QORXI, Q O R X I,
link |
which my wife came up with,
link |
but that's an acronym for quantum organized rational
link |
expanding intelligence, and his middle name is Xiphonies,
link |
actually, which means the former principal underlying AIXI.
link |
You're giving Elon Musk's new child a run for his money.
link |
Well, I did it first.
link |
He copied me with this new freakish name,
link |
but now if I have another baby,
link |
I'm gonna have to outdo him.
link |
It's becoming an arms race of weird, geeky baby names.
link |
We'll see what the babies think about it, right?
link |
But I mean, my oldest son, Zarathustra, loves his name,
link |
and my daughter, Sharazad, loves her name.
link |
So far, basically, if you give your kids weird names.
link |
They live up to it.
link |
Well, you're obliged to make the kids weird enough
link |
that they like the names, right?
link |
It directs their upbringing in a certain way.
link |
But yeah, anyway, I mean, what Marcus showed in that book
link |
is that a truly general intelligence
link |
theoretically is possible,
link |
but would take infinite computing power.
link |
So then the artificial is a little off.
link |
The general is not really achievable within physics
link |
And I mean, physics as we know it may be limited,
link |
but that's what we have to work with now.
link |
Infinitely general, you mean,
link |
like information processing perspective, yeah.
link |
Yeah, intelligence is not very well defined either, right?
link |
I mean, what does it mean?
link |
I mean, in AI now, it's fashionable to look at it
link |
as maximizing an expected reward over the future.
link |
But that sort of definition is pathological in various ways.
link |
And my friend David Weinbaum, AKA Weaver,
link |
he had a beautiful PhD thesis on open ended intelligence,
link |
trying to conceive intelligence in a...
link |
Yeah, he's just looking at it differently.
link |
He's looking at complex self organizing systems
link |
and looking at an intelligent system
link |
as being one that revises and grows
link |
and improves itself in conjunction with its environment
link |
without necessarily there being one objective function
link |
it's trying to maximize.
link |
Although over certain intervals of time,
link |
it may act as if it's optimizing
link |
a certain objective function.
link |
Very much Solaris from Stanislav Lem's novels, right?
link |
So yeah, the point is artificial, general and intelligence.
link |
On the other hand, everyone knows what AI is.
link |
And AGI seems immediately comprehensible
link |
to people with a technical background.
link |
So I think that the term has served
link |
as sociological function.
link |
And now it's out there everywhere, which baffles me.
link |
I mean, that's it.
link |
We're stuck with AGI probably for a very long time
link |
until AGI systems take over and rename themselves.
link |
And then we'll be biological.
link |
We're stuck with GPUs too,
link |
which mostly have nothing to do with graphics.
link |
I wonder what the AGI system will call us humans.
link |
Grandpa processing unit, yeah.
link |
Biological grandpa processing units.
link |
Okay, so maybe also just a comment on AGI representing
link |
before even the term existed,
link |
representing a kind of community.
link |
You've talked about this in the past,
link |
sort of AI is coming in waves,
link |
but there's always been this community of people
link |
who dream about creating general human level
link |
super intelligence systems.
link |
Can you maybe give your sense of the history
link |
of this community as it exists today,
link |
as it existed before this deep learning revolution
link |
all throughout the winters and the summers of AI?
link |
First, I would say as a side point,
link |
the winters and summers of AI are greatly exaggerated
link |
by Americans and in that,
link |
if you look at the publication record
link |
of the artificial intelligence community
link |
since say the 1950s,
link |
you would find a pretty steady growth
link |
in advance of ideas and papers.
link |
And what's thought of as an AI winter or summer
link |
was sort of how much money is the US military
link |
pumping into AI, which was meaningful.
link |
On the other hand, there was AI going on in Germany,
link |
UK and in Japan and in Russia, all over the place,
link |
while US military got more and less enthused about AI.
link |
That happened to be, just for people who don't know,
link |
the US military happened to be the main source
link |
of funding for AI research.
link |
So another way to phrase that is it's up and down
link |
of funding for artificial intelligence research.
link |
And I would say the correlation between funding
link |
and intellectual advance was not 100%, right?
link |
Because I mean, in Russia, as an example, or in Germany,
link |
there was less dollar funding than in the US,
link |
but many foundational ideas were laid out,
link |
but it was more theory than implementation, right?
link |
And US really excelled at sort of breaking through
link |
from theoretical papers to working implementations,
link |
which did go up and down somewhat
link |
with US military funding,
link |
but still, I mean, you can look in the 1980s,
link |
Dietrich Derner in Germany had self driving cars
link |
on the Autobahn, right?
link |
And I mean, it was a little early
link |
with regard to the car industry,
link |
so it didn't catch on such as has happened now.
link |
But I mean, that whole advancement
link |
of self driving car technology in Germany
link |
was pretty much independent of AI military summers
link |
and winters in the US.
link |
So there's been more going on in AI globally
link |
than not only most people on the planet realize,
link |
but then most new AI PhDs realize
link |
because they've come up within a certain sub field of AI
link |
and haven't had to look so much beyond that.
link |
But I would say when I got my PhD in 1989 in mathematics,
link |
I was interested in AI already.
link |
Yeah, I started at NYU, then I transferred to Philadelphia
link |
to Temple University, good old North Philly.
link |
Yeah, yeah, yeah, the pearl of the US.
link |
You never stopped at a red light then
link |
because you were afraid if you stopped at a red light,
link |
someone will carjack you.
link |
So you just drive through every red light.
link |
Every day driving or bicycling to Temple from my house
link |
was like a new adventure.
link |
But yeah, the reason I didn't do a PhD in AI
link |
was what people were doing in the academic AI field then,
link |
was just astoundingly boring and seemed wrong headed to me.
link |
It was really like rule based expert systems
link |
and production systems.
link |
And actually I loved mathematical logic.
link |
I had nothing against logic as the cognitive engine for an AI,
link |
but the idea that you could type in the knowledge
link |
that AI would need to think seemed just completely stupid
link |
and wrong headed to me.
link |
I mean, you can use logic if you want,
link |
but somehow the system has got to be...
link |
It should be learning from experience.
link |
And the AI field then was not interested
link |
in learning from experience.
link |
I mean, some researchers certainly were.
link |
I mean, I remember in mid eighties,
link |
I discovered a book by John Andreas,
link |
which was, it was about a reinforcement learning system
link |
called PURRDASHPUSS, which was an acronym
link |
that I can't even remember what it was for,
link |
but purpose anyway.
link |
But he, I mean, that was a system
link |
that was supposed to be an AGI
link |
and basically by some sort of fancy
link |
like Markov decision process learning,
link |
it was supposed to learn everything
link |
just from the bits coming into it
link |
and learn to maximize its reward
link |
and become intelligent, right?
link |
So that was there in academia back then,
link |
but it was like isolated, scattered, weird people.
link |
But all these isolated, scattered, weird people
link |
in that period, I mean, they laid the intellectual grounds
link |
for what happened later.
link |
So you look at John Andreas at University of Canterbury
link |
with his PURRDASHPUSS reinforcement learning Markov system.
link |
He was the PhD supervisor for John Cleary in New Zealand.
link |
Now, John Cleary worked with me
link |
when I was at Waikato University in 1993 in New Zealand.
link |
And he worked with Ian Whitten there
link |
and they launched WEKA,
link |
which was the first open source machine learning toolkit,
link |
which was launched in, I guess, 93 or 94
link |
when I was at Waikato University.
link |
Written in Java, unfortunately.
link |
Written in Java, which was a cool language back then.
link |
I guess it's still, well, it's not cool anymore,
link |
but it's powerful.
link |
I find, like most programmers now,
link |
I find Java unnecessarily bloated,
link |
but back then it was like Java or C++ basically.
link |
And Java was easier for students.
link |
Amusingly, a lot of the work on WEKA
link |
when we were in New Zealand was funded by a US,
link |
sorry, a New Zealand government grant
link |
to use machine learning
link |
to predict the menstrual cycles of cows.
link |
So in the US, all the grant funding for AI
link |
was about how to kill people or spy on people.
link |
In New Zealand, it's all about cows or kiwi fruits, right?
link |
So yeah, anyway, I mean, John Andreas
link |
had his probability theory based reinforcement learning,
link |
John Cleary was trying to do much more ambitious,
link |
probabilistic AGI systems.
link |
Now, John Cleary helped do WEKA,
link |
which is the first open source machine learning toolkit.
link |
So the predecessor for TensorFlow and Torch
link |
and all these things.
link |
Also, Shane Legg was at Waikato
link |
working with John Cleary and Ian Witten
link |
and this whole group.
link |
And then working with my own companies,
link |
my company, WebMind, an AI company I had in the late 90s
link |
with a team there at Waikato University,
link |
which is how Shane got his head full of AGI,
link |
which led him to go on
link |
and with Demis Hassabis found DeepMind.
link |
So what you can see through that lineage is,
link |
you know, in the 80s and 70s,
link |
John Andreas was trying to build probabilistic
link |
reinforcement learning AGI systems.
link |
The technology, the computers just weren't there to support
link |
his ideas were very similar to what people are doing now.
link |
But, you know, although he's long since passed away
link |
and didn't become that famous outside of Canterbury,
link |
I mean, the lineage of ideas passed on from him
link |
to his students, to their students,
link |
you can go trace directly from there to me
link |
and to DeepMind, right?
link |
So that there was a lot going on in AGI
link |
that did ultimately lay the groundwork
link |
for what we have today, but there wasn't a community, right?
link |
And so when I started trying to pull together
link |
an AGI community, it was in the, I guess,
link |
the early aughts when I was living in Washington, D.C.
link |
and making a living doing AI consulting
link |
for various U.S. government agencies.
link |
And I organized the first AGI workshop in 2006.
link |
And I mean, it wasn't like it was literally
link |
in my basement or something.
link |
I mean, it was in the conference room at the Marriott
link |
in Bethesda, it's not that edgy or underground,
link |
unfortunately, but still.
link |
How many people attended?
link |
About 60 or something.
link |
I mean, D.C. has a lot of AI going on,
link |
probably until the last five or 10 years,
link |
much more than Silicon Valley, although it's just quiet
link |
because of the nature of what happens in D.C.
link |
Their business isn't driven by PR.
link |
Mostly when something starts to work really well,
link |
it's taken black and becomes even more quiet, right?
link |
But yeah, the thing is that really had the feeling
link |
of a group of starry eyed mavericks huddled in a basement,
link |
like plotting how to overthrow the narrow AI establishment.
link |
And for the first time, in some cases,
link |
coming together with others who shared their passion
link |
for AGI and the technical seriousness about working on it.
link |
And that's very, very different than what we have today.
link |
I mean, now it's a little bit different.
link |
We have AGI conference every year
link |
and there's several hundred people rather than 50.
link |
Now it's more like this is the main gathering
link |
of people who want to achieve AGI
link |
and think that large scale nonlinear regression
link |
is not the golden path to AGI.
link |
AKA neural networks.
link |
Well, certain architectures for learning using neural networks.
link |
So yeah, the AGI conferences are sort of now
link |
the main concentration of people not obsessed
link |
with deep neural nets and deep reinforcement learning,
link |
but still interested in AGI, not the only ones.
link |
I mean, there's other little conferences and groupings
link |
interested in human level AI
link |
and cognitive architectures and so forth.
link |
But yeah, it's been a big shift.
link |
Like back then, you couldn't really...
link |
It'll be very, very edgy then
link |
to give a university department seminar
link |
that mentioned AGI or human level AI.
link |
It was more like you had to talk about
link |
something more short term and immediately practical
link |
than in the bar after the seminar,
link |
you could bullshit about AGI in the same breath
link |
as time travel or the simulation hypothesis or something.
link |
Whereas now, AGI is not only in the academic seminar room,
link |
like you have Vladimir Putin knows what AGI is.
link |
And he's like, Russia needs to become the leader in AGI.
link |
So national leaders and CEOs of large corporations.
link |
I mean, the CTO of Intel, Justin Ratner,
link |
this was years ago, Singularity Summit Conference,
link |
2008 or something.
link |
He's like, we believe Ray Kurzweil,
link |
the singularity will happen in 2045
link |
and it will have Intel inside.
link |
So, I mean, it's gone from being something
link |
which is the pursuit of like crazed mavericks,
link |
crackpots and science fiction fanatics
link |
to being a marketing term for large corporations
link |
and the national leaders,
link |
which is a astounding transition.
link |
But yeah, in the course of this transition,
link |
I think a bunch of sub communities have formed
link |
and the community around the AGI conference series
link |
is certainly one of them.
link |
It hasn't grown as big as I might've liked it to.
link |
On the other hand, sometimes a modest size community
link |
can be better for making intellectual progress also.
link |
Like you go to a society for neuroscience conference,
link |
you have 35 or 40,000 neuroscientists.
link |
On the one hand, it's amazing.
link |
On the other hand, you're not gonna talk to the leaders
link |
of the field there if you're an outsider.
link |
Yeah, in the same sense, the AAAI,
link |
the artificial intelligence,
link |
the main kind of generic artificial intelligence
link |
conference is too big.
link |
It's too amorphous.
link |
Like it doesn't make sense.
link |
Well, yeah, and NIPS has become a company advertising outlet
link |
in the whole of it.
link |
So, I mean, to comment on the role of AGI
link |
in the research community, I'd still,
link |
if you look at NeurIPS, if you look at CVPR,
link |
if you look at these iClear,
link |
AGI is still seen as the outcast.
link |
I would say in these main machine learning,
link |
in these main artificial intelligence conferences
link |
amongst the researchers,
link |
I don't know if it's an accepted term yet.
link |
What I've seen bravely, you mentioned Shane Legg's
link |
DeepMind and then OpenAI are the two places that are,
link |
I would say unapologetically so far,
link |
I think it's actually changing unfortunately,
link |
but so far they've been pushing the idea
link |
that the goal is to create an AGI.
link |
Well, they have billions of dollars behind them.
link |
So, I mean, they're in the public mind
link |
that certainly carries some oomph, right?
link |
But they also have really strong researchers, right?
link |
They do, they're great teams.
link |
I mean, DeepMind in particular, yeah.
link |
And they have, I mean, DeepMind has Marcus Hutter
link |
I mean, there's all these folks who basically
link |
their full time position involves dreaming
link |
about creating AGI.
link |
I mean, Google Brain has a lot of amazing
link |
AGI oriented people also.
link |
And I mean, so I'd say from a public marketing view,
link |
DeepMind and OpenAI are the two large well funded
link |
organizations that have put the term and concept AGI
link |
out there sort of as part of their public image.
link |
But I mean, they're certainly not,
link |
there are other groups that are doing research
link |
that seems just as AGI is to me.
link |
I mean, including a bunch of groups in Google's
link |
main Mountain View office.
link |
So yeah, it's true.
link |
AGI is somewhat away from the mainstream now.
link |
But if you compare it to where it was 15 years ago,
link |
there's been an amazing mainstreaming.
link |
You could say the same thing about super longevity research,
link |
which is one of my application areas that I'm excited about.
link |
I mean, I've been talking about this since the 90s,
link |
but working on this since 2001.
link |
And back then, really to say,
link |
you're trying to create therapies to allow people
link |
to live hundreds of thousands of years,
link |
you were way, way, way, way out of the industry,
link |
academic mainstream.
link |
But now, Google had Project Calico,
link |
Craig Venter had Human Longevity Incorporated.
link |
And then once the suits come marching in, right?
link |
I mean, once there's big money in it,
link |
then people are forced to take it seriously
link |
because that's the way modern society works.
link |
So it's still not as mainstream as cancer research,
link |
just as AGI is not as mainstream
link |
as automated driving or something.
link |
But the degree of mainstreaming that's happened
link |
in the last 10 to 15 years is astounding
link |
to those of us who've been at it for a while.
link |
Yeah, but there's a marketing aspect to the term,
link |
but in terms of actual full force research
link |
that's going on under the header of AGI,
link |
it's currently, I would say dominated,
link |
maybe you can disagree,
link |
dominated by neural networks research,
link |
that the nonlinear regression, as you mentioned.
link |
Like what's your sense with OpenCog, with your work,
link |
but in general, I was logic based systems
link |
and expert systems.
link |
For me, always seemed to capture a deep element
link |
of intelligence that needs to be there.
link |
Like you said, it needs to learn,
link |
it needs to be automated somehow,
link |
but that seems to be missing from a lot of research currently.
link |
So what's your sense?
link |
I guess one way to ask this question,
link |
what's your sense of what kind of things
link |
will an AGI system need to have?
link |
Yeah, that's a very interesting topic
link |
that I've thought about for a long time.
link |
And I think there are many, many different approaches
link |
that can work for getting to human level AI.
link |
So I don't think there's like one golden algorithm,
link |
or one golden design that can work.
link |
And I mean, flying machines is the much worn
link |
analogy here, right?
link |
Like, I mean, you have airplanes, you have helicopters,
link |
you have balloons, you have stealth bombers
link |
that don't look like regular airplanes.
link |
You've got all blimps.
link |
Birds, yeah, and bugs, right?
link |
And there are certainly many kinds of flying machines that.
link |
And there's a catapult that you can just launch.
link |
And there's bicycle powered like flying machines, right?
link |
Yeah, so now these are all analyzable
link |
by a basic theory of aerodynamics, right?
link |
Now, so one issue with AGI is we don't yet have the analog
link |
of the theory of aerodynamics.
link |
And that's what Marcus Hutter was trying to make
link |
with the AXI and his general theory of general intelligence.
link |
But that theory in its most clearly articulated parts
link |
really only works for either infinitely powerful machines
link |
or almost, or insanely impractically powerful machines.
link |
So I mean, if you were gonna take a theory based approach
link |
to AGI, what you would do is say, well, let's take
link |
what's called say AXE TL, which is Hutter's AXE machine
link |
that can work on merely insanely much processing power
link |
rather than infinitely much.
link |
What does TL stand for?
link |
So you're basically how it.
link |
Like constrained somehow.
link |
So how AXE works basically is each action
link |
that it wants to take, before taking that action,
link |
it looks at all its history.
link |
And then it looks at all possible programs
link |
that it could use to make a decision.
link |
And it decides like which decision program
link |
would have let it make the best decisions
link |
according to its reward function over its history.
link |
And it uses that decision program
link |
to make the next decision, right?
link |
It's not afraid of infinite resources.
link |
It's searching through the space
link |
of all possible computer programs
link |
in between each action and each next action.
link |
Now, AXE TL searches through all possible computer programs
link |
that have runtime less than T and length less than L.
link |
So it's, which is still an impractically humongous space,
link |
So what you would like to do to make an AGI
link |
and what will probably be done 50 years from now
link |
to make an AGI is say, okay, well, we have some constraints.
link |
We have these processing power constraints
link |
and we have the space and time constraints on the program.
link |
We have energy utilization constraints
link |
and we have this particular class environments,
link |
class of environments that we care about,
link |
which may be say, you know, manipulating physical objects
link |
on the surface of the earth,
link |
communicating in human language.
link |
I mean, whatever our particular, not annihilating humanity,
link |
whatever our particular requirements happen to be.
link |
If you formalize those requirements
link |
in some formal specification language,
link |
you should then be able to run
link |
automated program specializer on AXE TL,
link |
specialize it to the computing resource constraints
link |
and the particular environment and goal.
link |
And then it will spit out like the specialized version
link |
of AXE TL to your resource restrictions
link |
and your environment, which will be your AGI, right?
link |
And that I think is how our super AGI
link |
will create new AGI systems, right?
link |
But that's a very rush.
link |
It seems really inefficient.
link |
It's a very Russian approach by the way,
link |
like the whole field of program specialization
link |
came out of Russia.
link |
Can you backtrack?
link |
So what is program specialization?
link |
So it's basically...
link |
Well, take sorting, for example.
link |
You can have a generic program for sorting lists,
link |
but what if all your lists you care about
link |
are length 10,000 or less?
link |
You can run an automated program specializer
link |
on your sorting algorithm,
link |
and it will come up with the algorithm
link |
that's optimal for sorting lists of length 1,000 or less,
link |
or 10,000 or less, right?
link |
That's kind of like, isn't that the kind of the process
link |
of evolution as a program specializer to the environment?
link |
So you're kind of evolving human beings,
link |
or you're living creatures.
link |
Your Russian heritage is showing there.
link |
So with Alexander Vityaev and Peter Anokhin and so on,
link |
I mean, there's a long history
link |
of thinking about evolution that way also, right?
link |
So, well, my point is that what we're thinking of
link |
as a human level general intelligence,
link |
if you start from narrow AIs,
link |
like are being used in the commercial AI field now,
link |
then you're thinking,
link |
okay, how do we make it more and more general?
link |
On the other hand,
link |
if you start from AICSI or Schmidhuber's Gödel machine,
link |
or these infinitely powerful,
link |
but practically infeasible AIs,
link |
then getting to a human level AGI
link |
is a matter of specialization.
link |
It's like, how do you take these
link |
maximally general learning processes
link |
and how do you specialize them
link |
so that they can operate
link |
within the resource constraints that you have,
link |
but will achieve the particular things that you care about?
link |
Because we humans are not maximally general intelligence.
link |
If I ask you to run a maze in 750 dimensions,
link |
you'd probably be very slow.
link |
Whereas at two dimensions,
link |
you're probably way better, right?
link |
So, I mean, we're special because our hippocampus
link |
has a two dimensional map in it, right?
link |
And it does not have a 750 dimensional map in it.
link |
So, I mean, we're a peculiar mix
link |
of generality and specialization, right?
link |
We'll probably start quite general at birth.
link |
Not obviously still narrow,
link |
but like more general than we are
link |
at age 20 and 30 and 40 and 50 and 60.
link |
I don't think that, I think it's more complex than that
link |
because I mean, in some sense,
link |
a young child is less biased
link |
and the brain has yet to sort of crystallize
link |
into appropriate structures
link |
for processing aspects of the physical and social world.
link |
On the other hand,
link |
the young child is very tied to their sensorium.
link |
Whereas we can deal with abstract mathematics,
link |
like 750 dimensions and the young child cannot
link |
because they haven't grown what Piaget
link |
called the formal capabilities.
link |
They haven't learned to abstract yet, right?
link |
And the ability to abstract
link |
gives you a different kind of generality
link |
than what the baby has.
link |
So, there's both more specialization
link |
and more generalization that comes
link |
with the development process actually.
link |
I mean, I guess just the trajectories
link |
of the specialization are most controllable
link |
at the young age, I guess is one way to put it.
link |
They're not as controllable as you think.
link |
So, you think it's interesting.
link |
I think, honestly, I think a human adult
link |
is much more generally intelligent than a human baby.
link |
Babies are very stupid, you know what I mean?
link |
I mean, they're cute, which is why we put up
link |
with their repetitiveness and stupidity.
link |
And they have what the Zen guys would call
link |
a beginner's mind, which is a beautiful thing,
link |
but that doesn't necessarily correlate
link |
with a high level of intelligence.
link |
On the plot of cuteness and stupidity,
link |
there's a process that allows us to put up
link |
with their stupidity as they become more intelligent.
link |
So, by the time you're an ugly old man like me,
link |
you gotta get really, really smart to compensate.
link |
To compensate, okay, cool.
link |
But yeah, going back to your original question,
link |
so the way I look at human level AGI
link |
is how do you specialize, you know,
link |
unrealistically inefficient, superhuman,
link |
brute force learning processes
link |
to the specific goals that humans need to achieve
link |
and the specific resources that we have.
link |
And both of these, the goals and the resources
link |
and the environments, I mean, all this is important.
link |
And on the resources side, it's important
link |
that the hardware resources we're bringing to bear
link |
are very different than the human brain.
link |
So the way I would want to implement AGI
link |
on a bunch of neurons in a vat
link |
that I could rewire arbitrarily is quite different
link |
than the way I would want to create AGI
link |
on say a modern server farm of CPUs and GPUs,
link |
which in turn may be quite different
link |
than the way I would want to implement AGI
link |
on whatever quantum computer we'll have in 10 years,
link |
supposing someone makes a robust quantum turing machine
link |
or something, right?
link |
So I think there's been coevolution
link |
of the patterns of organization in the human brain
link |
and the physiological particulars
link |
of the human brain over time.
link |
And when you look at neural networks,
link |
that is one powerful class of learning algorithms,
link |
but it's also a class of learning algorithms
link |
that evolve to exploit the particulars of the human brain
link |
as a computational substrate.
link |
If you're looking at the computational substrate
link |
of a modern server farm,
link |
you won't necessarily want the same algorithms
link |
that you want on the human brain.
link |
And from the right level of abstraction,
link |
you could look at maybe the best algorithms on the brain
link |
and the best algorithms on a modern computer network
link |
as implementing the same abstract learning
link |
and representation processes,
link |
but finding that level of abstraction
link |
is its own AGI research project then, right?
link |
So that's about the hardware side
link |
and the software side, which follows from that.
link |
Then regarding what are the requirements,
link |
I wrote the paper years ago
link |
on what I called the embodied communication prior,
link |
which was quite similar in intent
link |
to Yoshua Bengio's recent paper on the consciousness prior,
link |
except I didn't wanna wrap up consciousness in it
link |
because to me, the qualia problem and subjective experience
link |
is a very interesting issue also,
link |
which we can chat about,
link |
but I would rather keep that philosophical debate distinct
link |
from the debate of what kind of biases
link |
do you wanna put in a general intelligence
link |
to give it human like general intelligence.
link |
And I'm not sure Yoshua Bengio is really addressing
link |
that kind of consciousness.
link |
He's just using the term.
link |
I love Yoshua to pieces.
link |
Like he's by far my favorite of the lines of deep learning.
link |
He's such a good hearted guy.
link |
He's a good human being.
link |
I am not sure he has plumbed to the depths
link |
of the philosophy of consciousness.
link |
No, he's using it as a sexy term.
link |
So what I called it was the embodied communication prior.
link |
Can you maybe explain it a little bit?
link |
What I meant was, what are we humans evolved for?
link |
You can say being human, but that's very abstract, right?
link |
I mean, our minds control individual bodies,
link |
which are autonomous agents moving around in a world
link |
that's composed largely of solid objects, right?
link |
And we've also evolved to communicate via language
link |
with other solid object agents that are going around
link |
doing things collectively with us
link |
in a world of solid objects.
link |
And these things are very obvious,
link |
but if you compare them to the scope
link |
of all possible intelligences
link |
or even all possible intelligences
link |
that are physically realizable,
link |
that actually constrains things a lot.
link |
So if you start to look at how would you realize
link |
some specialized or constrained version
link |
of universal general intelligence
link |
in a system that has limited memory
link |
and limited speed of processing,
link |
but whose general intelligence will be biased
link |
toward controlling a solid object agent,
link |
which is mobile in a solid object world
link |
for manipulating solid objects
link |
and communicating via language with other similar agents
link |
in that same world, right?
link |
Then starting from that,
link |
you're starting to get a requirements analysis
link |
for human level general intelligence.
link |
And then that leads you into cognitive science
link |
and you can look at, say, what are the different types
link |
of memory that the human mind and brain has?
link |
And this has matured over the last decades
link |
and I got into this a lot.
link |
So after getting my PhD in math,
link |
I was an academic for eight years.
link |
I was in departments of mathematics,
link |
computer science, and psychology.
link |
When I was in the psychology department
link |
at the University of Western Australia,
link |
I was focused on cognitive science of memory and perception.
link |
Actually, I was teaching neural nets and deep neural nets
link |
and it was multi layer perceptrons, right?
link |
Cognitive science, it was cross disciplinary
link |
among engineering, math, psychology, philosophy,
link |
linguistics, computer science.
link |
But yeah, we were teaching psychology students
link |
to try to model the data from human cognition experiments
link |
using multi layer perceptrons,
link |
which was the early version of a deep neural network.
link |
Very, very, yeah, recurrent back prop
link |
was very, very slow to train back then, right?
link |
So this is the study of these constraint systems
link |
that are supposed to deal with physical objects.
link |
So if you look at cognitive psychology,
link |
you can see there's multiple types of memory,
link |
which are to some extent represented
link |
by different subsystems in the human brain.
link |
So we have episodic memory,
link |
which takes into account our life history
link |
and everything that's happened to us.
link |
We have declarative or semantic memory,
link |
which is like facts and beliefs abstracted
link |
from the particular situations that they occurred in.
link |
There's sensory memory, which to some extent
link |
is sense modality specific,
link |
and then to some extent is unified across sense modalities.
link |
There's procedural memory, memory of how to do stuff,
link |
like how to swing the tennis racket, right?
link |
Which is, there's motor memory,
link |
but it's also a little more abstract than motor memory.
link |
It involves cerebellum and cortex working together.
link |
Then there's memory linkage with emotion
link |
which has to do with linkages of cortex and limbic system.
link |
There's specifics of spatial and temporal modeling
link |
connected with memory, which has to do with hippocampus
link |
and thalamus connecting to cortex.
link |
And the basal ganglia, which influences goals.
link |
So we have specific memory of what goals,
link |
subgoals and sub subgoals we want to perceive
link |
in which context in the past.
link |
Human brain has substantially different subsystems
link |
for these different types of memory
link |
and substantially differently tuned learning,
link |
like differently tuned modes of longterm potentiation
link |
to do with the types of neurons and neurotransmitters
link |
in the different parts of the brain
link |
corresponding to these different types of knowledge.
link |
And these different types of memory and learning
link |
in the human brain, I mean, you can back these all
link |
into embodied communication for controlling agents
link |
in worlds of solid objects.
link |
Now, so if you look at building an AGI system,
link |
one way to do it, which starts more from cognitive science
link |
than neuroscience is to say,
link |
okay, what are the types of memory
link |
that are necessary for this kind of world?
link |
Yeah, yeah, necessary for this sort of intelligence.
link |
What types of learning work well
link |
with these different types of memory?
link |
And then how do you connect all these things together, right?
link |
And of course the human brain did it incrementally
link |
through evolution because each of the sub networks
link |
of the brain, I mean, it's not really the lobes
link |
of the brain, it's the sub networks,
link |
each of which is widely distributed,
link |
which of the, each of the sub networks of the brain
link |
co evolves with the other sub networks of the brain,
link |
both in terms of its patterns of organization
link |
and the particulars of the neurophysiology.
link |
So they all grew up communicating
link |
and adapting to each other.
link |
It's not like they were separate black boxes
link |
that were then glommed together, right?
link |
Whereas as engineers, we would tend to say,
link |
let's make the declarative memory box here
link |
and the procedural memory box here
link |
and the perception box here and wire them together.
link |
And when you can do that, it's interesting.
link |
I mean, that's how a car is built, right?
link |
But on the other hand, that's clearly not
link |
how biological systems are made.
link |
The parts co evolve so as to adapt and work together.
link |
That's by the way, how every human engineered system
link |
that flies, that was, we were using that analogy
link |
before it's built as well.
link |
So do you find this at all appealing?
link |
Like there's been a lot of really exciting,
link |
which I find strange that it's ignored work
link |
in cognitive architectures, for example,
link |
throughout the last few decades.
link |
Yeah, I mean, I had a lot to do with that community
link |
and you know, Paul Rosenbloom, who was one of the,
link |
and John Laird who built the SOAR architecture,
link |
are friends of mine.
link |
And I learned SOAR quite well
link |
and ACTAR and these different cognitive architectures.
link |
And how I was looking at the AI world about 10 years ago
link |
before this whole commercial deep learning explosion was,
link |
on the one hand, you had these cognitive architecture guys
link |
who were working closely with psychologists
link |
and cognitive scientists who had thought a lot
link |
about how the different parts of a human like mind
link |
should work together.
link |
On the other hand, you had these learning theory guys
link |
who didn't care at all about the architecture,
link |
but we're just thinking about like,
link |
how do you recognize patterns in large amounts of data?
link |
And in some sense, what you needed to do
link |
was to get the learning that the learning theory guys
link |
were doing and put it together with the architecture
link |
that the cognitive architecture guys were doing.
link |
And then you would have what you needed.
link |
Now, you can't, unfortunately, when you look at the details,
link |
you can't just do that without totally rebuilding
link |
what is happening on both the cognitive architecture
link |
and the learning side.
link |
So, I mean, they tried to do that in SOAR,
link |
but what they ultimately did is like,
link |
take a deep neural net or something for perception
link |
and you include it as one of the black boxes.
link |
It becomes one of the boxes.
link |
The learning mechanism becomes one of the boxes
link |
as opposed to fundamental part of the system.
link |
You could look at some of the stuff DeepMind has done,
link |
like the differential neural computer or something
link |
that sort of has a neural net for deep learning perception.
link |
It has another neural net, which is like a memory matrix
link |
that stores, say, the map of the London subway or something.
link |
So probably Demis Tsabas was thinking about this
link |
like part of cortex and part of hippocampus
link |
because hippocampus has a spatial map.
link |
And when he was a neuroscientist,
link |
he was doing a bunch on cortex hippocampus interconnection.
link |
So there, the DNC would be an example of folks
link |
from the deep neural net world trying to take a step
link |
in the cognitive architecture direction
link |
by having two neural modules that correspond roughly
link |
to two different parts of the human brain
link |
that deal with different kinds of memory and learning.
link |
But on the other hand, it's super, super, super crude
link |
from the cognitive architecture view, right?
link |
Just as what John Laird and Soar did with neural nets
link |
was super, super crude from a learning point of view
link |
because the learning was like off to the side,
link |
not affecting the core representations, right?
link |
I mean, you weren't learning the representation.
link |
You were learning the data that feeds into the...
link |
You were learning abstractions of perceptual data
link |
to feed into the representation that was not learned, right?
link |
So yeah, this was clear to me a while ago.
link |
And one of my hopes with the AGI community
link |
was to sort of bring people
link |
from those two directions together.
link |
That didn't happen much in terms of...
link |
And what I was gonna say is it didn't happen
link |
in terms of bringing like the lions
link |
of cognitive architecture together
link |
with the lions of deep learning.
link |
It did work in the sense that a bunch of younger researchers
link |
have had their heads filled with both of those ideas.
link |
This comes back to a saying my dad,
link |
who was a university professor, often quoted to me,
link |
which was, science advances one funeral at a time,
link |
which I'm trying to avoid.
link |
Like I'm 53 years old and I'm trying to invent
link |
amazing, weird ass new things
link |
that nobody ever thought about,
link |
which we'll talk about in a few minutes.
link |
But there is that aspect, right?
link |
Like the people who've been at AI a long time
link |
and have made their career developing one aspect,
link |
like a cognitive architecture or a deep learning approach,
link |
it can be hard once you're old
link |
and have made your career doing one thing,
link |
it can be hard to mentally shift gears.
link |
I mean, I try quite hard to remain flexible minded.
link |
Have you been successful somewhat in changing,
link |
maybe, have you changed your mind on some aspects
link |
of what it takes to build an AGI, like technical things?
link |
The hard part is that the world doesn't want you to.
link |
The world or your own brain?
link |
The world, well, that one point
link |
is that your brain doesn't want to.
link |
The other part is that the world doesn't want you to.
link |
Like the people who have followed your ideas
link |
get mad at you if you change your mind.
link |
And the media wants to pigeonhole you as an avatar
link |
of a certain idea.
link |
But yeah, I've changed my mind on a bunch of things.
link |
I mean, when I started my career,
link |
I really thought quantum computing
link |
would be necessary for AGI.
link |
And I doubt it's necessary now,
link |
although I think it will be a super major enhancement.
link |
But I mean, I'm now in the middle of embarking
link |
on the complete rethink and rewrite from scratch
link |
of our OpenCog AGI system together with Alexey Potapov
link |
and his team in St. Petersburg,
link |
who's working with me in SingularityNet.
link |
So now we're trying to like go back to basics,
link |
take everything we learned from working
link |
with the current OpenCog system,
link |
take everything everybody else has learned
link |
from working with their proto AGI systems
link |
and design the best framework for the next stage.
link |
And I do think there's a lot to be learned
link |
from the recent successes with deep neural nets
link |
and deep reinforcement systems.
link |
I mean, people made these essentially trivial systems
link |
work much better than I thought they would.
link |
And there's a lot to be learned from that.
link |
And I wanna incorporate that knowledge appropriately
link |
in our OpenCog 2.0 system.
link |
On the other hand, I also think current deep neural net
link |
architectures as such will never get you anywhere near AGI.
link |
So I think you wanna avoid the pathology
link |
of throwing the baby out with the bathwater
link |
and like saying, well, these things are garbage
link |
because foolish journalists overblow them
link |
as being the path to AGI
link |
and a few researchers overblow them as well.
link |
There's a lot of interesting stuff to be learned there
link |
even though those are not the golden path.
link |
So maybe this is a good chance to step back.
link |
You mentioned OpenCog 2.0, but...
link |
Go back to OpenCog 0.0, which exists now.
link |
Yeah, maybe talk through the history of OpenCog
link |
and your thinking about these ideas.
link |
I would say OpenCog 2.0 is a term we're throwing around
link |
sort of tongue in cheek because the existing OpenCog system
link |
that we're working on now is not remotely close
link |
to what we'd consider a 1.0, right?
link |
I mean, it's an early...
link |
It's been around, what, 13 years or something,
link |
but it's still an early stage research system, right?
link |
And actually, we are going back to the beginning
link |
in terms of theory and implementation
link |
because we feel like that's the right thing to do,
link |
but I'm sure what we end up with is gonna have
link |
a huge amount in common with the current system.
link |
I mean, we all still like the general approach.
link |
So first of all, what is OpenCog?
link |
Sure, OpenCog is an open source software project
link |
that I launched together with several others in 2008
link |
and probably the first code written toward that
link |
was written in 2001 or two or something
link |
that was developed as a proprietary code base
link |
within my AI company, Novamente LLC.
link |
Then we decided to open source it in 2008,
link |
cleaned up the code throughout some things
link |
and added some new things and...
link |
What language is it written in?
link |
Primarily, there's a bunch of scheme as well,
link |
but most of it's C++.
link |
And it's separate from something we'll also talk about,
link |
the SingularityNet.
link |
So it was born as a non networked thing.
link |
Well, there are many levels of networks involved here.
link |
No connectivity to the internet, or no, at birth.
link |
Yeah, I mean, SingularityNet is a separate project
link |
and a separate body of code.
link |
And you can use SingularityNet as part of the infrastructure
link |
for a distributed OpenCog system,
link |
but there are different layers.
link |
So OpenCog on the one hand as a software framework
link |
could be used to implement a variety
link |
of different AI architectures and algorithms,
link |
but in practice, there's been a group of developers
link |
which I've been leading together with Linus Vepstas,
link |
Neil Geisweiler, and a few others,
link |
which have been using the OpenCog platform
link |
and infrastructure to implement certain ideas
link |
about how to make an AGI.
link |
So there's been a little bit of ambiguity
link |
about OpenCog, the software platform
link |
versus OpenCog, the AGI design,
link |
because in theory, you could use that software to do,
link |
you could use it to make a neural net.
link |
You could use it to make a lot of different AGI.
link |
What kind of stuff does the software platform provide,
link |
like in terms of utilities, tools, like what?
link |
Yeah, let me first tell about OpenCog
link |
as a software platform,
link |
and then I'll tell you the specific AGI R&D
link |
we've been building on top of it.
link |
So the core component of OpenCog as a software platform
link |
is what we call the atom space,
link |
which is a weighted labeled hypergraph.
link |
Atom space, yeah, yeah, not atom, like Adam and Eve,
link |
although that would be cool too.
link |
Yeah, so you have a hypergraph, which is like,
link |
so a graph in this sense is a bunch of nodes
link |
with links between them.
link |
A hypergraph is like a graph,
link |
but links can go between more than two nodes.
link |
So you have a link between three nodes.
link |
And in fact, OpenCog's atom space
link |
would properly be called a metagraph
link |
because you can have links pointing to links,
link |
or you could have links pointing to whole subgraphs, right?
link |
So it's an extended hypergraph or a metagraph.
link |
Is metagraph a technical term?
link |
It is now a technical term.
link |
But I don't think it was yet a technical term
link |
when we started calling this a generalized hypergraph.
link |
But in any case, it's a weighted labeled
link |
generalized hypergraph or weighted labeled metagraph.
link |
The weights and labels mean that the nodes and links
link |
can have numbers and symbols attached to them.
link |
So they can have types on them.
link |
They can have numbers on them that represent,
link |
say, a truth value or an importance value
link |
for a certain purpose.
link |
And of course, like with all things,
link |
you can reduce that to a hypergraph,
link |
and then the hypergraph can be reduced to a graph.
link |
You can reduce hypergraph to a graph,
link |
and you could reduce a graph to an adjacency matrix.
link |
So, I mean, there's always multiple representations.
link |
But there's a layer of representation
link |
that seems to work well here.
link |
Right, right, right.
link |
And so similarly, you could have a link to a whole graph
link |
because a whole graph could represent,
link |
say, a body of information.
link |
And I could say, I reject this body of information.
link |
Then one way to do that is make that link
link |
go to that whole subgraph representing
link |
the body of information, right?
link |
I mean, there are many alternate representations,
link |
but that's, anyway, what we have in OpenCOG,
link |
we have an atom space, which is this weighted, labeled,
link |
generalized hypergraph.
link |
Knowledge store, it lives in RAM.
link |
There's also a way to back it up to disk.
link |
There are ways to spread it among
link |
multiple different machines.
link |
Then there are various utilities for dealing with that.
link |
So there's a pattern matcher,
link |
which lets you specify a sort of abstract pattern
link |
and then search through a whole atom space
link |
with labeled hypergraph to see what subhypergraphs
link |
may match that pattern, for an example.
link |
So that's, then there's something called
link |
the COG server in OpenCOG,
link |
which lets you run a bunch of different agents
link |
or processes in a scheduler.
link |
And each of these agents, basically it reads stuff
link |
from the atom space and it writes stuff to the atom space.
link |
So this is sort of the basic operational model.
link |
That's the software framework.
link |
And of course that's, there's a lot there
link |
just from a scalable software engineering standpoint.
link |
So you could use this, I don't know if you've,
link |
have you looked into the Stephen Wolfram's physics project
link |
recently with the hypergraphs and stuff?
link |
Could you theoretically use like the software framework
link |
to play with it? You certainly could,
link |
although Wolfram would rather die
link |
than use anything but Mathematica for his work.
link |
Well that's, yeah, but there's a big community of people
link |
who are, you know, would love integration.
link |
Like you said, the young minds love the idea
link |
of integrating, of connecting things.
link |
Yeah, that's right.
link |
And I would add on that note,
link |
the idea of using hypergraph type models in physics
link |
Like if you look at...
link |
The Russians did it first.
link |
Well, I'm sure they did.
link |
And a guy named Ben Dribis, who's a mathematician,
link |
a professor in Louisiana or somewhere,
link |
had a beautiful book on quantum sets and hypergraphs
link |
and algebraic topology for discrete models of physics.
link |
And carried it much farther than Wolfram has,
link |
but he's not rich and famous,
link |
so it didn't get in the headlines.
link |
But yeah, Wolfram aside, yeah,
link |
certainly that's a good way to put it.
link |
The whole OpenCog framework,
link |
you could use it to model biological networks
link |
and simulate biology processes.
link |
You could use it to model physics
link |
on discrete graph models of physics.
link |
So you could use it to do, say, biologically realistic
link |
neural networks, for example.
link |
And that's a framework.
link |
What do agents and processes do?
link |
Do they grow the graph?
link |
What kind of computations, just to get a sense,
link |
are they supposed to do?
link |
So in theory, they could do anything they want to do.
link |
They're just C++ processes.
link |
On the other hand, the computation framework
link |
is sort of designed for agents
link |
where most of their processing time
link |
is taken up with reads and writes to the atom space.
link |
And so that's a very different processing model
link |
than, say, the matrix multiplication based model
link |
as underlies most deep learning systems, right?
link |
So you could create an agent
link |
that just factored numbers for a billion years.
link |
It would run within the OpenCog platform,
link |
but it would be pointless, right?
link |
I mean, the point of doing OpenCog
link |
is because you want to make agents
link |
that are cooperating via reading and writing
link |
into this weighted labeled hypergraph, right?
link |
And that has both cognitive architecture importance
link |
because then this hypergraph is being used
link |
as a sort of shared memory
link |
among different cognitive processes,
link |
but it also has software and hardware
link |
implementation implications
link |
because current GPU architectures
link |
are not so useful for OpenCog,
link |
whereas a graph chip would be incredibly useful, right?
link |
And I think Graphcore has those now,
link |
but they're not ideally suited for this.
link |
But I think in the next, let's say, three to five years,
link |
we're gonna see new chips
link |
where like a graph is put on the chip
link |
and the back and forth between multiple processes
link |
acting SIMD and MIMD on that graph is gonna be fast.
link |
And then that may do for OpenCog type architectures
link |
what GPUs did for deep neural architecture.
link |
It's a small tangent.
link |
Can you comment on thoughts about neuromorphic computing?
link |
So like hardware implementations
link |
of all these different kind of, are you interested?
link |
Are you excited by that possibility?
link |
I'm excited by graph processors
link |
because I think they can massively speed up OpenCog,
link |
which is a class of architectures that I'm working on.
link |
I think if, you know, in principle, neuromorphic computing
link |
should be amazing.
link |
I haven't yet been fully sold
link |
on any of the systems that are out.
link |
They're like, memristors should be amazing too, right?
link |
So a lot of these things have obvious potential,
link |
but I haven't yet put my hands on a system
link |
that seemed to manifest that.
link |
Mark's system should be amazing,
link |
but the current systems have not been great.
link |
Yeah, I mean, look, for example,
link |
if you wanted to make a biologically realistic
link |
hardware neural network,
link |
like making a circuit in hardware
link |
that emulated like the Hodgkin–Huxley equation
link |
or the Izhekevich equation,
link |
like differential equations
link |
for a biologically realistic neuron
link |
and putting that in hardware on the chip,
link |
that would seem that it would make more feasible
link |
to make a large scale, truly biologically realistic
link |
Now, what's been done so far is not like that.
link |
So I guess personally, as a researcher,
link |
I mean, I've done a bunch of work in computational neuroscience
link |
where I did some work with IARPA in DC,
link |
Intelligence Advanced Research Project Agency.
link |
We were looking at how do you make
link |
a biologically realistic simulation
link |
of seven different parts of the brain
link |
cooperating with each other,
link |
using like realistic nonlinear dynamical models of neurons,
link |
and how do you get that to simulate
link |
what's going on in the mind of a geo intelligence analyst
link |
while they're trying to find terrorists on a map, right?
link |
So if you want to do something like that,
link |
having neuromorphic hardware that really let you simulate
link |
like a realistic model of the neuron would be amazing.
link |
But that's sort of with my computational neuroscience
link |
With an AGI hat on, I'm just more interested
link |
in these hypergraph knowledge representation
link |
based architectures, which would benefit more
link |
from various types of graph processors
link |
because the main processing bottleneck
link |
is reading writing to RAM.
link |
It's reading writing to the graph in RAM.
link |
The main processing bottleneck for this kind of
link |
proto AGI architecture is not multiplying matrices.
link |
And for that reason, GPUs, which are really good
link |
at multiplying matrices, don't apply as well.
link |
There are frameworks like Gunrock and others
link |
that try to boil down graph processing
link |
to matrix operations, and they're cool,
link |
but you're still putting a square peg
link |
into a round hole in a certain way.
link |
The same is true, I mean, current quantum machine learning,
link |
which is very cool.
link |
It's also all about how to get matrix and vector operations
link |
in quantum mechanics, and I see why that's natural to do.
link |
I mean, quantum mechanics is all unitary matrices
link |
and vectors, right?
link |
On the other hand, you could also try
link |
to make graph centric quantum computers,
link |
which I think is where things will go.
link |
And then we can have, then we can make,
link |
like take the open cog implementation layer,
link |
implement it in a collapsed state inside a quantum computer.
link |
But that may be the singularity squared, right?
link |
I'm not sure we need that to get to human level.
link |
That's already beyond the first singularity.
link |
But can we just go back to open cog?
link |
Yeah, and the hypergraph and open cog.
link |
That's the software framework, right?
link |
So the next thing is our cognitive architecture
link |
tells us particular algorithms to put there.
link |
Can we backtrack on the kind of, is this graph designed,
link |
is it in general supposed to be sparse
link |
and the operations constantly grow and change the graph?
link |
Yeah, the graph is sparse.
link |
But is it constantly adding links and so on?
link |
It is a self modifying hypergraph.
link |
So it's not, so the write and read operations
link |
you're referring to, this isn't just a fixed graph
link |
to which you change the way, it's a constantly growing graph.
link |
Yeah, that's true.
link |
So it is different model than,
link |
say current deep neural nets
link |
and have a fixed neural architecture
link |
and you're updating the weights.
link |
Although there have been like cascade correlational
link |
neural net architectures that grow new nodes and links,
link |
but the most common neural architectures now
link |
have a fixed neural architecture,
link |
you're updating the weights.
link |
And then open cog, you can update the weights
link |
and that certainly happens a lot,
link |
but adding new nodes, adding new links,
link |
removing nodes and links is an equally critical part
link |
of the system's operations.
link |
So now when you start to add these cognitive algorithms
link |
on top of this open cog architecture,
link |
what does that look like?
link |
Yeah, so within this framework then,
link |
creating a cognitive architecture is basically two things.
link |
It's choosing what type system you wanna put
link |
on the nodes and links in the hypergraph,
link |
what types of nodes and links you want.
link |
And then it's choosing what collection of agents,
link |
what collection of AI algorithms or processes
link |
are gonna run to operate on this hypergraph.
link |
And of course those two decisions
link |
are closely connected to each other.
link |
So in terms of the type system,
link |
there are some links that are more neural net like,
link |
they're just like have weights to get updated
link |
by heavy and learning and activation spreads along them.
link |
There are other links that are more logic like
link |
and nodes that are more logic like.
link |
So you could have a variable node
link |
and you can have a node representing a universal
link |
or existential quantifier as in predicate logic
link |
So you can have logic like nodes and links,
link |
or you can have neural like nodes and links.
link |
You can also have procedure like nodes and links
link |
as in say a combinatorial logic or Lambda calculus
link |
representing programs.
link |
So you can have nodes and links representing
link |
many different types of semantics,
link |
which means you could make a horrible ugly mess
link |
or you could make a system
link |
where these different types of knowledge
link |
all interpenetrate and synergize
link |
with each other beautifully, right?
link |
So the hypergraph can contain programs.
link |
Yeah, it can contain programs,
link |
although in the current version,
link |
it is a very inefficient way
link |
to guide the execution of programs,
link |
which is one thing that we are aiming to resolve
link |
with our rewrite of the system now.
link |
So what to you is the most beautiful aspect of OpenCog?
link |
Just to you personally,
link |
some aspect that captivates your imagination
link |
from beauty or power?
link |
What fascinates me is finding a common representation
link |
that underlies abstract, declarative knowledge
link |
and sensory knowledge and movement knowledge
link |
and procedural knowledge and episodic knowledge,
link |
finding the right level of representation
link |
where all these types of knowledge are stored
link |
in a sort of universal and interconvertible
link |
yet practically manipulable way, right?
link |
So to me, that's the core,
link |
because once you've done that,
link |
then the different learning algorithms
link |
can help each other out. Like what you want is,
link |
if you have a logic engine
link |
that helps with declarative knowledge
link |
and you have a deep neural net
link |
that gathers perceptual knowledge,
link |
and you have, say, an evolutionary learning system
link |
that learns procedures,
link |
you want these to not only interact
link |
on the level of sharing results
link |
and passing inputs and outputs to each other,
link |
you want the logic engine, when it gets stuck,
link |
to be able to share its intermediate state
link |
with the neural net and with the evolutionary system
link |
and with the evolutionary learning algorithm
link |
so that they can help each other out of bottlenecks
link |
and help each other solve combinatorial explosions
link |
by intervening inside each other's cognitive processes.
link |
But that can only be done
link |
if the intermediate state of a logic engine,
link |
the evolutionary learning engine,
link |
and a deep neural net are represented in the same form.
link |
And that's what we figured out how to do
link |
by putting the right type system
link |
on top of this weighted labeled hypergraph.
link |
So is there, can you maybe elaborate
link |
on what are the different characteristics
link |
of a type system that can coexist
link |
amongst all these different kinds of knowledge
link |
that needs to be represented?
link |
And is, I mean, like, is it hierarchical?
link |
Just any kind of insights you can give
link |
on that kind of type system?
link |
Yeah, yeah, so this gets very nitty gritty
link |
and mathematical, of course,
link |
but one key part is switching
link |
from predicate logic to term logic.
link |
What is predicate logic?
link |
What is term logic?
link |
So term logic was invented by Aristotle,
link |
or at least that's the oldest recollection we have of it.
link |
But term logic breaks down basic logic
link |
into basically simple links between nodes,
link |
like an inheritance link between node A and node B.
link |
So in term logic, the basic deduction operation
link |
is A implies B, B implies C, therefore A implies C.
link |
Whereas in predicate logic,
link |
the basic operation is modus ponens,
link |
like A implies B, therefore B.
link |
So it's a slightly different way of breaking down logic,
link |
but by breaking down logic into term logic,
link |
you get a nice way of breaking logic down
link |
into nodes and links.
link |
So your concepts can become nodes,
link |
the logical relations become links.
link |
And so then inference is like,
link |
so if this link is A implies B,
link |
this link is B implies C,
link |
then deduction builds a link A implies C.
link |
And your probabilistic algorithm
link |
can assign a certain weight there.
link |
Now, you may also have like a Hebbian neural link
link |
from A to C, which is the degree to which thinking,
link |
the degree to which A being the focus of attention
link |
should make B the focus of attention, right?
link |
So you could have then a neural link
link |
and you could have a symbolic,
link |
like logical inheritance link in your term logic.
link |
And they have separate meaning,
link |
but they could be used to guide each other as well.
link |
Like if there's a large amount of neural weight
link |
on the link between A and B,
link |
that may direct your logic engine to think about,
link |
well, what is the relation?
link |
Is there an inheritance relation?
link |
Are they similar in some context?
link |
On the other hand, if there's a logical relation
link |
between A and B, that may direct your neural component
link |
to think, well, when I'm thinking about A,
link |
should I be directing some attention to B also?
link |
Because there's a logical relation.
link |
So in terms of logic,
link |
there's a lot of thought that went into
link |
how do you break down logic relations,
link |
including basic sort of propositional logic relations
link |
as Aristotelian term logic deals with,
link |
and then quantifier logic relations also.
link |
How do you break those down elegantly into a hypergraph?
link |
Because you, I mean, you can boil logic expression
link |
into a graph in many different ways.
link |
Many of them are very ugly, right?
link |
We tried to find elegant ways
link |
of sort of hierarchically breaking down
link |
complex logic expression into nodes and links.
link |
So that if you have say different nodes representing,
link |
Ben, AI, Lex, interview or whatever,
link |
the logic relations between those things
link |
are compact in the node and link representation.
link |
So that when you have a neural net acting
link |
on the same nodes and links,
link |
the neural net and the logic engine
link |
can sort of interoperate with each other.
link |
And also interpretable by humans.
link |
Is that an important?
link |
Yeah, in simple cases, it's interpretable by humans.
link |
But honestly, I would say logic systems
link |
I would say logic systems give more potential
link |
for transparency and comprehensibility
link |
than neural net systems,
link |
but you still have to work at it.
link |
Because I mean, if I show you a predicate logic proposition
link |
with like 500 nested universal and existential quantifiers
link |
and 217 variables, that's no more comprehensible
link |
than the weight metrics of a neural network, right?
link |
So I'd say the logic expressions
link |
that AI learns from its experience
link |
are mostly totally opaque to human beings
link |
and maybe even harder to understand than neural net.
link |
Because I mean, when you have multiple
link |
nested quantifier bindings,
link |
it's a very high level of abstraction.
link |
There is a difference though,
link |
in that within logic, it's a little more straightforward
link |
to pose the problem of like normalize this
link |
and boil this down to a certain form.
link |
I mean, you can do that in neural nets too.
link |
Like you can distill a neural net to a simpler form,
link |
but that's more often done to make a neural net
link |
that'll run on an embedded device or something.
link |
It's harder to distill a net to a comprehensible form
link |
than it is to simplify a logic expression
link |
to a comprehensible form, but it doesn't come for free.
link |
Like what's in the AI's mind is incomprehensible
link |
to a human unless you do some special work
link |
to make it comprehensible.
link |
So on the procedural side, there's some different
link |
and sort of interesting voodoo there.
link |
I mean, if you're familiar in computer science,
link |
there's something called the Curry Howard correspondence,
link |
which is a one to one mapping between proofs and programs.
link |
So every program can be mapped into a proof.
link |
Every proof can be mapped into a program.
link |
You can model this using category theory
link |
and a bunch of nice math,
link |
but we wanna make that practical, right?
link |
So that if you have an executable program
link |
that like moves the robot's arm or figures out
link |
in what order to say things in a dialogue,
link |
that's a procedure represented in OpenCog's hypergraph.
link |
But if you wanna reason on how to improve that procedure,
link |
you need to map that procedure into logic
link |
using Curry Howard isomorphism.
link |
So then the logic engine can reason
link |
about how to improve that procedure
link |
and then map that back into the procedural representation
link |
that is efficient for execution.
link |
So again, that comes down to not just
link |
can you make your procedure into a bunch of nodes and links?
link |
Cause I mean, that can be done trivially.
link |
A C++ compiler has nodes and links inside it.
link |
Can you boil down your procedure
link |
into a bunch of nodes and links
link |
in a way that's like hierarchically decomposed
link |
and simple enough?
link |
It can reason about.
link |
Yeah, yeah, that given the resource constraints at hand,
link |
you can map it back and forth to your term logic,
link |
and without having a bloated logic expression, right?
link |
So there's just a lot of,
link |
there's a lot of nitty gritty particulars there,
link |
but by the same token, if you ask a chip designer,
link |
like how do you make the Intel I7 chip so good?
link |
There's a long list of technical answers there,
link |
which will take a while to go through, right?
link |
And this has been decades of work.
link |
I mean, the first AI system of this nature I tried to build
link |
was called WebMind in the mid 1990s.
link |
And we had a big graph,
link |
a big graph operating in RAM implemented with Java 1.1,
link |
which was a terrible, terrible implementation idea.
link |
And then each node had its own processing.
link |
So like that there,
link |
the core loop looped through all nodes in the network
link |
and let each node enact what its little thing was doing.
link |
And we had logic and neural nets in there,
link |
but an evolutionary learning,
link |
but we hadn't done enough of the math
link |
to get them to operate together very cleanly.
link |
So it was really, it was quite a horrible mess.
link |
So as well as shifting an implementation
link |
where the graph is its own object
link |
and the agents are separately scheduled,
link |
we've also done a lot of work
link |
on how do you represent programs?
link |
How do you represent procedures?
link |
You know, how do you represent genotypes for evolution
link |
in a way that the interoperability
link |
between the different types of learning
link |
associated with these different types of knowledge
link |
And that's been quite difficult.
link |
It's taken decades and it's totally off to the side
link |
of what the commercial mainstream of the AI field is doing,
link |
which isn't thinking about representation at all really.
link |
Although you could see like in the DNC,
link |
they had to think a little bit about
link |
how do you make representation of a map
link |
in this memory matrix work together
link |
with the representation needed
link |
for say visual pattern recognition
link |
in the hierarchical neural network.
link |
But I would say we have taken that direction
link |
of taking the types of knowledge you need
link |
for different types of learning,
link |
like declarative, procedural, attentional,
link |
and how do you make these types of knowledge represent
link |
in a way that allows cross learning
link |
across these different types of memory.
link |
We've been prototyping and experimenting with this
link |
within OpenCog and before that WebMind
link |
since the mid 1990s.
link |
Now, disappointingly to all of us,
link |
this has not yet been cashed out in an AGI system, right?
link |
I mean, we've used this system
link |
within our consulting business.
link |
So we've built natural language processing
link |
and robot control and financial analysis.
link |
We've built a bunch of sort of vertical market specific
link |
proprietary AI projects.
link |
They use OpenCog on the backend,
link |
but we haven't, that's not the AGI goal, right?
link |
It's interesting, but it's not the AGI goal.
link |
So now what we're looking at with our rebuild of the system.
link |
Yeah, we're also calling it True AGI.
link |
So we're not quite sure what the name is yet.
link |
We made a website for trueagi.io,
link |
but we haven't put anything on there yet.
link |
We may come up with an even better name.
link |
It's kind of like the real AI starting point
link |
for your AGI book.
link |
Yeah, but I like True better
link |
because True has like, you can be true hearted, right?
link |
You can be true to your girlfriend.
link |
So True has a number and it also has logic in it, right?
link |
Because logic is a key part of the system.
link |
So yeah, with the True AGI system,
link |
we're sticking with the same basic architecture,
link |
but we're trying to build on what we've learned.
link |
And one thing we've learned is that,
link |
we need type checking among dependent types
link |
and among probabilistic dependent types to be much faster.
link |
you can have complex types on the nodes and links.
link |
But if you wanna put,
link |
like if you want types to be first class citizens,
link |
so that you can have the types can be variables
link |
and then you do type checking
link |
among complex higher order types.
link |
You can do that in the system now, but it's very slow.
link |
This is stuff like it's done
link |
in cutting edge program languages like Agda or something,
link |
these obscure research languages.
link |
On the other hand,
link |
we've been doing a lot tying together deep neural nets
link |
with symbolic learning.
link |
So we did a project for Cisco, for example,
link |
which was on, this was street scene analysis,
link |
but they had deep neural models
link |
for a bunch of cameras watching street scenes,
link |
but they trained a different model for each camera
link |
because they couldn't get the transfer learning
link |
to work between camera A and camera B.
link |
So we took what came out of all the deep neural models
link |
for the different cameras,
link |
we fed it into an open called symbolic representation.
link |
Then we did some pattern mining and some reasoning
link |
on what came out of all the different cameras
link |
within the symbolic graph.
link |
And that worked well for that application.
link |
I mean, Hugo Latapie from Cisco gave a talk touching on that
link |
at last year's AGI conference, it was in Shenzhen.
link |
On the other hand, we learned from there,
link |
it was kind of clunky to get the deep neural models
link |
to work well with the symbolic system
link |
because we were using torch.
link |
And torch keeps a sort of state computation graph,
link |
but you needed like real time access
link |
to that computation graph within our hypergraph.
link |
And we certainly did it,
link |
Alexey Polopov who leads our St. Petersburg team
link |
wrote a great paper on cognitive modules in OpenCog
link |
explaining sort of how do you deal
link |
with the torch compute graph inside OpenCog.
link |
But in the end we realized like,
link |
that just hadn't been one of our design thoughts
link |
when we built OpenCog, right?
link |
So between wanting really fast dependent type checking
link |
and wanting much more efficient interoperation
link |
between the computation graphs
link |
of deep neural net frameworks and OpenCog's hypergraph
link |
and adding on top of that,
link |
wanting to more effectively run an OpenCog hypergraph
link |
distributed across RAM in 10,000 machines,
link |
which is we're doing dozens of machines now,
link |
but it's just not, we didn't architect it
link |
with that sort of modern scalability in mind.
link |
So these performance requirements are what have driven us
link |
to want to rearchitect the base,
link |
but the core AGI paradigm doesn't really change.
link |
Like the mathematics is the same.
link |
It's just, we can't scale to the level that we want
link |
in terms of distributed processing
link |
or speed of various kinds of processing
link |
with the current infrastructure
link |
that was built in the phase 2001 to 2008,
link |
which is hardly shocking.
link |
Well, I mean, the three things you mentioned
link |
are really interesting.
link |
So what do you think about in terms of interoperability
link |
communicating with computational graph of neural networks?
link |
What do you think about the representations
link |
that neural networks form?
link |
They're bad, but there's many ways
link |
that you could deal with that.
link |
So I've been wrestling with this a lot
link |
in some work on supervised grammar induction,
link |
and I have a simple paper on that.
link |
They'll give it the next AGI conference,
link |
online portion of which is next week, actually.
link |
What is grammar induction?
link |
So this isn't AGI either,
link |
but it's sort of on the verge
link |
between narrow AI and AGI or something.
link |
Unsupervised grammar induction is the problem.
link |
Throw your AI system, a huge body of text,
link |
and have it learn the grammar of the language
link |
that produced that text.
link |
So you're not giving it labeled examples.
link |
So you're not giving it like a thousand sentences
link |
where the parses were marked up by graduate students.
link |
So it's just got to infer the grammar from the text.
link |
It's like the Rosetta Stone, but worse, right?
link |
Because you only have the one language,
link |
and you have to figure out what is the grammar.
link |
So that's not really AGI because,
link |
I mean, the way a human learns language is not that, right?
link |
I mean, we learn from language that's used in context.
link |
So it's a social embodied thing.
link |
We see how a given sentence is grounded in observation.
link |
There's an interactive element, I guess.
link |
On the other hand, so I'm more interested in that.
link |
I'm more interested in making an AGI system learn language
link |
from its social and embodied experience.
link |
On the other hand, that's also more of a pain to do,
link |
and that would lead us into Hanson Robotics
link |
and their robotics work I've known much.
link |
We'll talk about it in a few minutes.
link |
But just as an intellectual exercise,
link |
as a learning exercise,
link |
trying to learn grammar from a corpus
link |
is very, very interesting, right?
link |
And that's been a field in AI for a long time.
link |
No one can do it very well.
link |
So we've been looking at transformer neural networks
link |
and tree transformers, which are amazing.
link |
These came out of Google Brain, actually.
link |
And actually on that team was Lucas Kaiser,
link |
who used to work for me in the one,
link |
the period 2005 through eight or something.
link |
So it's been fun to see my former
link |
sort of AGI employees disperse and do
link |
all these amazing things.
link |
Way too many sucked into Google, actually.
link |
Well, yeah, anyway.
link |
We'll talk about that too.
link |
Lucas Kaiser and a bunch of these guys,
link |
they create transformer networks,
link |
that classic paper like attention is all you need
link |
and all these things following on from that.
link |
So we're looking at transformer networks.
link |
And like, these are able to,
link |
I mean, this is what underlies GPT2 and GPT3 and so on,
link |
which are very, very cool
link |
and have absolutely no cognitive understanding
link |
of any of the texts they're looking at.
link |
Like they're very intelligent idiots, right?
link |
So sorry to take, but this small, I'll bring this back,
link |
but do you think GPT3 understands language?
link |
No, no, it understands nothing.
link |
It's a complete idiot.
link |
But it's a brilliant idiot.
link |
You don't think GPT20 will understand language?
link |
So size is not gonna buy you understanding.
link |
And any more than a faster car is gonna get you to Mars.
link |
It's a completely different kind of thing.
link |
I mean, these networks are very cool.
link |
And as an entrepreneur,
link |
I can see many highly valuable uses for them.
link |
And as an artist, I love them, right?
link |
So I mean, we're using our own neural model,
link |
which is along those lines
link |
to control the Philip K. Dick robot now.
link |
And it's amazing to like train a neural model
link |
on the robot Philip K. Dick
link |
and see it come up with like crazed,
link |
stoned philosopher pronouncements,
link |
very much like what Philip K. Dick might've said, right?
link |
Like these models are super cool.
link |
And I'm working with Hanson Robotics now
link |
on using a similar, but more sophisticated one for Sophia,
link |
which we haven't launched yet.
link |
But so I think it's cool.
link |
But no, these are recognizing a large number
link |
of shallow patterns.
link |
They're not forming an abstract representation.
link |
And that's the point I was coming to
link |
when we're looking at grammar induction,
link |
we tried to mine patterns out of the structure
link |
of the transformer network.
link |
And you can, but the patterns aren't what you want.
link |
So I mean, if you do supervised learning,
link |
if you look at sentences where you know
link |
the correct parts of a sentence,
link |
you can learn a matrix that maps
link |
between the internal representation of the transformer
link |
and the parse of the sentence.
link |
And so then you can actually train something
link |
that will output the sentence parse
link |
from the transformer network's internal state.
link |
And we did this, I think Christopher Manning,
link |
some others have not done this also.
link |
But I mean, what you get is that the representation
link |
is hardly ugly and is scattered all over the network
link |
and doesn't look like the rules of grammar
link |
that you know are the right rules of grammar, right?
link |
It's kind of ugly.
link |
So what we're actually doing is we're using
link |
a symbolic grammar learning algorithm,
link |
but we're using the transformer neural network
link |
as a sentence probability oracle.
link |
So like if you have a rule of grammar
link |
and you aren't sure if it's a correct rule of grammar or not,
link |
you can generate a bunch of sentences
link |
using that rule of grammar
link |
and a bunch of sentences violating that rule of grammar.
link |
And you can see the transformer model
link |
doesn't think the sentences obeying the rule of grammar
link |
are more probable than the sentences
link |
disobeying the rule of grammar.
link |
So in that way, you can use the neural model
link |
as a sense probability oracle
link |
to guide a symbolic grammar learning process.
link |
And that seems to work better than trying to milk
link |
the grammar out of the neural network
link |
that doesn't have it in there.
link |
So I think the thing is these neural nets
link |
are not getting a semantically meaningful representation
link |
internally by and large.
link |
So one line of research is to try to get them to do that.
link |
And InfoGAN was trying to do that.
link |
So like if you look back like two years ago,
link |
there was all these papers on like at Edward,
link |
this probabilistic programming neural net framework
link |
that Google had, which came out of InfoGAN.
link |
So the idea there was like you could train
link |
an InfoGAN neural net model,
link |
which is a generative associative network
link |
to recognize and generate faces.
link |
And the model would automatically learn a variable
link |
for how long the nose is and automatically learn a variable
link |
for how wide the eyes are
link |
or how big the lips are or something, right?
link |
So it automatically learned these variables,
link |
which have a semantic meaning.
link |
So that was a rare case where a neural net
link |
trained with a fairly standard GAN method
link |
was able to actually learn the semantic representation.
link |
So for many years, many of us tried to take that
link |
the next step and get a GAN type neural network
link |
that would have not just a list of semantic latent variables,
link |
but would have say a Bayes net of semantic latent variables
link |
with dependencies between them.
link |
The whole programming framework Edward was made for that.
link |
I mean, no one got it to work, right?
link |
Do you think it's possible?
link |
Yeah, do you think?
link |
It might be that back propagation just won't work for it
link |
because the gradients are too screwed up.
link |
Maybe you could get it to work using CMAES
link |
or some like floating point evolutionary algorithm.
link |
We tried, we didn't get it to work.
link |
Eventually we just paused that rather than gave it up.
link |
We paused that and said, well, okay, let's try
link |
more innovative ways to learn implicit,
link |
to learn what are the representations implicit
link |
in that network without trying to make it grow
link |
inside that network.
link |
And I described how we're doing that in language.
link |
You can do similar things in vision, right?
link |
Use it as an oracle.
link |
So you can, that's one way is that you use
link |
a structure learning algorithm, which is symbolic.
link |
And then you use the deep neural net as an oracle
link |
to guide the structure learning algorithm.
link |
The other way to do it is like Infogam was trying to do
link |
and try to tweak the neural network
link |
to have the symbolic representation inside it.
link |
I tend to think what the brain is doing
link |
is more like using the deep neural net type thing
link |
I think the visual cortex or the cerebellum
link |
are probably learning a non semantically meaningful
link |
opaque tangled representation.
link |
And then when they interface with the more cognitive parts
link |
of the cortex, the cortex is sort of using those
link |
as an oracle and learning the abstract representation.
link |
So if you do sports, say take for example,
link |
serving in tennis, right?
link |
I mean, my tennis serve is okay, not great,
link |
but I learned it by trial and error, right?
link |
And I mean, I learned music by trial and error too.
link |
I just sit down and play, but then if you're an athlete,
link |
which I'm not a good athlete,
link |
I mean, then you'll watch videos of yourself serving
link |
and your coach will help you think about what you're doing
link |
and you'll then form a declarative representation,
link |
but your cerebellum maybe didn't have
link |
a declarative representation.
link |
Same way with music, like I will hear something in my head,
link |
I'll sit down and play the thing like I heard it.
link |
And then I will try to study what my fingers did
link |
to see like, what did you just play?
link |
Like how did you do that, right?
link |
Because if you're composing,
link |
you may wanna see how you did it
link |
and then declaratively morph that in some way
link |
that your fingers wouldn't think of, right?
link |
But the physiological movement may come out of some opaque,
link |
like cerebellar reinforcement learned thing, right?
link |
And so that's, I think trying to milk the structure
link |
of a neural net by treating it as an oracle,
link |
maybe more like how your declarative mind post processes
link |
what your visual or motor cortex.
link |
I mean, in vision, it's the same way,
link |
like you can recognize beautiful art
link |
much better than you can say why
link |
you think that piece of art is beautiful.
link |
But if you're trained as an art critic,
link |
you do learn to say why.
link |
And some of it's bullshit, but some of it isn't, right?
link |
Some of it is learning to map sensory knowledge
link |
into declarative and linguistic knowledge,
link |
yet without necessarily making the sensory system itself
link |
use a transparent and an easily communicable representation.
link |
Yeah, that's fascinating to think of neural networks
link |
as like dumb question answers that you can just milk
link |
to build up a knowledge base.
link |
And then it can be multiple networks, I suppose,
link |
Yeah, yeah, so I think if a group like DeepMind or OpenAI
link |
were to build AGI, and I think DeepMind is like
link |
a thousand times more likely from what I could tell,
link |
because they've hired a lot of people with broad minds
link |
and many different approaches and angles on AGI,
link |
whereas OpenAI is also awesome,
link |
but I see them as more of like a pure
link |
deep reinforcement learning shop.
link |
Yeah, this time, I got you.
link |
So far. Yeah, there's a lot of,
link |
you're right, I mean, there's so much interdisciplinary
link |
work at DeepMind, like neuroscience.
link |
And you put that together with Google Brain,
link |
which granted they're not working that closely together now,
link |
but my oldest son Zarathustra is doing his PhD
link |
in machine learning applied to automated theorem proving
link |
in Prague under Josef Urban.
link |
So the first paper, DeepMath, which applied deep neural nets
link |
to guide theorem proving was out of Google Brain.
link |
I mean, by now, the automated theorem proving community
link |
is going way, way, way beyond anything Google was doing,
link |
but still, yeah, but anyway,
link |
if that community was gonna make an AGI,
link |
probably one way they would do it was,
link |
take 25 different neural modules,
link |
architected in different ways,
link |
maybe resembling different parts of the brain,
link |
like a basal ganglia model, cerebellum model,
link |
a thalamus module, a few hippocampus models,
link |
number of different models,
link |
representing parts of the cortex, right?
link |
Take all of these and then wire them together
link |
to co train and learn them together like that.
link |
That would be an approach to creating an AGI.
link |
One could implement something like that efficiently
link |
on top of our true AGI, like OpenCog 2.0 system,
link |
once it exists, although obviously Google
link |
has their own highly efficient implementation architecture.
link |
So I think that's a decent way to build AGI.
link |
I was very interested in that in the mid 90s,
link |
but I mean, the knowledge about how the brain works
link |
sort of pissed me off, like it wasn't there yet.
link |
Like, you know, in the hippocampus,
link |
you have these concept neurons,
link |
like the so called grandmother neuron,
link |
which everyone laughed at it, it's actually there.
link |
Like I have some Lex Friedman neurons
link |
that fire differentially when I see you
link |
and not when I see any other person, right?
link |
So how do these Lex Friedman neurons,
link |
how do they coordinate with the distributed representation
link |
of Lex Friedman I have in my cortex, right?
link |
There's some back and forth between cortex and hippocampus
link |
that lets these discrete symbolic representations
link |
in hippocampus correlate and cooperate
link |
with the distributed representations in cortex.
link |
This probably has to do with how the brain
link |
does its version of abstraction and quantifier logic, right?
link |
Like you can have a single neuron in the hippocampus
link |
that activates a whole distributed activation pattern
link |
in cortex, well, this may be how the brain does
link |
like symbolization and abstraction
link |
as in functional programming or something,
link |
but we can't measure it.
link |
Like we don't have enough electrodes stuck
link |
between the cortex and the hippocampus
link |
in any known experiment to measure it.
link |
So I got frustrated with that direction,
link |
not because it's impossible.
link |
Because we just don't understand enough yet.
link |
Of course, it's a valid research direction.
link |
You can try to understand more and more.
link |
And we are measuring more and more
link |
about what happens in the brain now than ever before.
link |
So it's quite interesting.
link |
On the other hand, I sort of got more
link |
of an engineering mindset about AGI.
link |
I'm like, well, okay,
link |
we don't know how the brain works that well.
link |
We don't know how birds fly that well yet either.
link |
We have no idea how a hummingbird flies
link |
in terms of the aerodynamics of it.
link |
On the other hand, we know basic principles
link |
of like flapping and pushing the air down.
link |
And we know the basic principles
link |
of how the different parts of the brain work.
link |
So let's take those basic principles
link |
and engineer something that embodies those basic principles,
link |
but is well designed for the hardware
link |
that we have on hand right now.
link |
So do you think we can create AGI
link |
before we understand how the brain works?
link |
I think that's probably what will happen.
link |
And maybe the AGI will help us do better brain imaging
link |
that will then let us build artificial humans,
link |
which is very, very interesting to us
link |
because we are humans, right?
link |
I mean, building artificial humans is super worthwhile.
link |
I just think it's probably not the shortest path to AGI.
link |
So it's fascinating idea that we would build AGI
link |
to help us understand ourselves.
link |
A lot of people ask me if the young people
link |
interested in doing artificial intelligence,
link |
they look at sort of doing graduate level, even undergrads,
link |
but graduate level research and they see
link |
whether the artificial intelligence community stands now,
link |
it's not really AGI type research for the most part.
link |
So the natural question they ask is
link |
what advice would you give?
link |
I mean, maybe I could ask if people were interested
link |
in working on OpenCog or in some kind of direct
link |
or indirect connection to OpenCog or AGI research,
link |
what would you recommend?
link |
OpenCog, first of all, is open source project.
link |
There's a Google group discussion list.
link |
There's a GitHub repository.
link |
So if anyone's interested in lending a hand
link |
with that aspect of AGI,
link |
introduce yourself on the OpenCog email list.
link |
And there's a Slack as well.
link |
I mean, we're certainly interested to have inputs
link |
into our redesign process for a new version of OpenCog,
link |
but also we're doing a lot of very interesting research.
link |
I mean, we're working on data analysis
link |
for COVID clinical trials.
link |
We're working with Hanson Robotics.
link |
We're doing a lot of cool things
link |
with the current version of OpenCog now.
link |
So there's certainly opportunity to jump into OpenCog
link |
or various other open source AGI oriented projects.
link |
So would you say there's like masters
link |
and PhD theses in there?
link |
Plenty, yeah, plenty, of course.
link |
I mean, the challenge is to find a supervisor
link |
who wants to foster that sort of research,
link |
but it's way easier than it was when I got my PhD, right?
link |
We talked about OpenCog, which is kind of one,
link |
the software framework,
link |
but also the actual attempt to build an AGI system.
link |
And then there is this exciting idea of SingularityNet.
link |
So maybe can you say first what is SingularityNet?
link |
SingularityNet is a platform
link |
for realizing a decentralized network
link |
of artificial intelligences.
link |
So Marvin Minsky, the AI pioneer who I knew a little bit,
link |
he had the idea of a society of minds,
link |
like you should achieve an AI
link |
not by writing one algorithm or one program,
link |
but you should put a bunch of different AIs out there
link |
and the different AIs will interact with each other,
link |
each playing their own role.
link |
And then the totality of the society of AIs
link |
would be the thing
link |
that displayed the human level intelligence.
link |
And I had, when he was alive,
link |
I had many debates with Marvin about this idea.
link |
And I think he really thought the mind
link |
was more like a society than I do.
link |
Like I think you could have a mind
link |
that was as disorganized as a human society,