The following is a conversation with William McCaskill.

He's a philosopher, ethicist, and one of the originators of the effective altruism movement.

His research focuses on the fundamentals of effective altruism,

or the use of evidence and reason to help others by as much as possible with our time and money,

with a particular concentration on how to act given moral uncertainty.

He's the author of Doing Good, Better, Effective Altruism,

and a Radical New Way to Make a Difference.

He is a cofounder and the president of the Center of Effective Altruism, CEA,

that encourages people to commit to donate at least 10% of their income to the most effective charities.

He cofounded 80,000 Hours, which is a nonprofit that provides research and advice

on how you can best make a difference through your career.

This conversation was recorded before the outbreak of the coronavirus pandemic.

For everyone feeling the medical, psychological, and financial burden of this crisis,

I'm sending love your way.

Stay strong. We're in this together. We'll beat this thing.

This is the Artificial Intelligence Podcast.

If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcast,

support it on Patreon, or simply connect with me on Twitter at Lex Friedman, spelled F R I D M A N.

As usual, I'll do one or two minutes of ads now,

and never any ads in the middle that can break the flow of the conversation.

I hope that works for you and doesn't hurt the listening experience.

This show is presented by Cash App, the number one finance app in the App Store.

When you get it, use code LEXPODCAST.

Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1.

Since Cash App allows you to send and receive money digitally, peer to peer,

and security in all digital transactions is very important,

let me mention the PCI data security standard that Cash App is compliant with.

I'm a big fan of standards for safety and security.

PCI DSS is a good example of that,

where a bunch of competitors got together and agreed

that there needs to be a global standard around the security of transactions.

Now, we just need to do the same for autonomous vehicles and AI systems in general.

So again, if you get Cash App from the App Store or Google Play,

and use the code LEXPODCAST, you get $10, and Cash App will also donate $10 to FIRST,

an organization that is helping to advance robotics and STEM education for young people around the world.

And now, here's my conversation with William McCaskill.

What does utopia for humans and all life on Earth look like for you?

That's a great question.

What I want to say is that we don't know,

and the utopia we want to get to is an indirect one that I call the long reflection.

So, period of post scarcity, no longer have the kind of urgent problems we have today,

but instead can spend, perhaps it's tens of thousands of years debating,

engaging in ethical reflection in order, before we take any kind of drastic lock in,

actions like spreading to the stars,

and then we can figure out what is of kind of moral value.

The long reflection, that's a really beautiful term.

So, if we look at Twitter for just a second,

do you think human beings are able to reflect in a productive way?

I don't mean to make it sound bad,

because there is a lot of fights and politics and division in our discourse.

Maybe if you zoom out, it actually is civilized discourse.

It might not feel like it, but when you zoom out.

So, I don't want to say that Twitter is not civilized discourse.

I actually believe it.

It's more civilized than people give it credit for.

But do you think the long reflection can actually be stable,

where we as human beings with our descendant of eight brains

would be able to sort of rationally discuss things together and arrive at ideas?

I think, overall, we're pretty good at discussing things rationally,

and at least in the earlier stages of our lives being open to many different ideas,

and being able to be convinced and change our views.

I think that Twitter is designed almost to bring out all the worst tendencies.

So, if the long reflection were conducted on Twitter,

maybe it would be better just not even to bother.

But I think the challenge really is getting to a stage

where we have a society that is as conducive as possible

to rational reflection, to deliberation.

I think we're actually very lucky to be in a liberal society

where people are able to discuss a lot of ideas and so on.

I think when we look to the future,

that's not at all guaranteed that society would be like that,

rather than a society where there's a fixed canon of values

that are being imposed on all of society,

and where you aren't able to question that.

That would be very bad from my perspective,

because it means we wouldn't be able to figure out what the truth is.

I can already sense we're going to go down a million tangents,

but what do you think is the...

If Twitter is not optimal,

what kind of mechanism in this modern age of technology

can we design where the exchange of ideas could be both civilized and productive,

and yet not be too constrained

where there's rules of what you can say and can't say,

which is, as you say, is not desirable,

but yet not have some limits as to what can be said or not and so on?

Do you have any ideas, thoughts on the possible future?

Of course, nobody knows how to do it,

but do you have thoughts of what a better Twitter might look like?

I think that text based media are intrinsically going to be very hard

to be conducive to rational discussion,

because if you think about it from an informational perspective,

if I just send you a text of less than,

what is it now, 240 characters, 280 characters, I think,

that's a tiny amount of information compared to, say, you and I talking now,

where you have access to the words I say, which is the same as in text,

but also my tone, also my body language,

and we're very poorly designed to be able to assess...

I have to read all of this context into anything you say,

so maybe your partner sends you a text and has a full stop at the end.

Are they mad at you?

You don't know.

You have to infer everything about this person's mental state

from whether they put a full stop at the end of a text or not.

Well, the flip side of that is it truly text that's the problem here,

because there's a viral aspect to the text,

where you could just post text nonstop.

It's very immediate.

The times before Twitter, before the internet,

the way you would exchange texts is you would write books.

And that, while it doesn't get body language, it doesn't get tone, it doesn't...

so on, but it does actually boil down after some time of thinking,

some editing, and so on, boil down ideas.

So is the immediacy and the viral nature,

which produces the outrage mobs and so on, the potential problem?

I think that is a big issue.

I think there's going to be this strong selection effect where

something that provokes outrage, well, that's high arousal,

you're more likely to retweet that,

whereas kind of sober analysis is not as sexy, not as viral.

I do agree that long form content is much better to productive discussion.

In terms of the media that are very popular at the moment,

I think that podcasting is great where your podcasts are two hours long,

so they're much more in depth than Twitter are,

and you are able to convey so much more nuance,

so much more caveat, because it's an actual conversation.

It's more like the sort of communication that we've evolved to do,

rather than these very small little snippets of ideas that,

when also combined with bad incentives,

just clearly aren't designed for helping us get to the truth.

It's kind of interesting that it's not just the length of the podcast medium,

but it's the fact that it was started by people that don't give a damn about

quote unquote demand, that there's a relaxed,

sort of the style that Joe Rogan does,

there's a freedom to express ideas

in an unconstrained way that's very real.

It's kind of funny that it feels so refreshingly real to us today,

and I wonder what the future looks like.

It's a little bit sad now that quite a lot of sort of more popular people

are getting into podcasting,

and they try to sort of create, they try to control it,

they try to constrain it in different kinds of ways.

People I love, like Conan O Brien and so on, different comedians,

and I'd love to see where the real aspects of this podcasting medium persist,

maybe in TV, maybe in YouTube,

maybe Netflix is pushing those kind of ideas,

and it's kind of, it's a really exciting word,

that kind of sharing of knowledge.

Yeah, I mean, I think it's a double edged sword

as it becomes more popular and more profitable,

where on the one hand you'll get a lot more creativity,

people doing more interesting things with the medium,

but also perhaps you get this place to the bottom

where suddenly maybe it'll be hard to find good content on podcasts

because it'll be so overwhelmed by the latest bit of viral outrage.

So speaking of that, jumping on Effective Altruism for a second,

so much of that internet content is funded by advertisements.

Just in the context of Effective Altruism,

we're talking about the richest companies in the world,

they're funded by advertisements essentially,

Google, that's their primary source of income.

Do you see that as,

do you have any criticism of that source of income?

Do you see that source of money

as a potentially powerful source of money that could be used,

well, certainly could be used for good,

but is there something bad about that source of money?

I think there's significant worries with it,

where it means that the incentives of the company

might be quite misaligned with making people's lives better,

where again, perhaps the incentives are towards increasing drama

and debate on your social media feed

in order that more people are going to be engaged,

perhaps compulsively involved with the platform.

Whereas there are other business models

like having an opt in subscription service

where perhaps they have other issues,

but there's much more of an incentive to provide a product

that its users are just really wanting,

because now I'm paying for this product.

I'm paying for this thing that I want to buy

rather than I'm trying to use this thing

and it's going to get a profit mechanism

that is somewhat orthogonal to me

actually just wanting to use the product.

And so, I mean, in some cases it'll work better than others.

I can imagine, I can in theory imagine Facebook

having a subscription service,

but I think it's unlikely to happen anytime soon.

Well, it's interesting and it's weird

now that you bring it up that it's unlikely.

For example, I pay I think 10 bucks a month for YouTube Red

and I don't think I get it much for that

except just for no ads,

but in general it's just a slightly better experience.

And I would gladly, now I'm not wealthy,

in fact I'm operating very close to zero dollars,

but I would pay 10 bucks a month to Facebook

and 10 bucks a month to Twitter

for some kind of more control

in terms of advertisements and so on.

But the other aspect of that is data, personal data.

People are really sensitive about this

and I as one who hopes to one day

create a company that may use people's data

to do good for the world,

wonder about this.

One, the psychology of why people are so paranoid.

Well, I understand why,

but they seem to be more paranoid

than is justified at times.

And the other is how do you do it right?

So it seems that Facebook is,

it seems that Facebook is doing it wrong.

That's certainly the popular narrative.

It's unclear to me actually how wrong.

Like I tend to give them more benefit of the doubt

because it's a really hard thing to do right

and people don't necessarily realize it,

but how do we respect in your view people's privacy?

Yeah, I mean in the case of how worried are people

about using their data,

I mean there's a lot of public debate

and criticism about it.

When we look at people's revealed preferences,

people's continuing massive use

of these sorts of services.

It's not clear to me how much people really do care.

Perhaps they care a bit,

but they're happy to in effect kind of sell their data

in order to be able to kind of use a certain service.

That's a great term, revealed preferences.

So these aren't preferences you self report in the survey.

This is like your actions speak.

Yeah, exactly.

So you might say,

oh yeah, I hate the idea of Facebook having my data.

But then when it comes to it,

you actually are willing to give that data in exchange

for being able to use the service.

And if that's the case,

then I think unless we have some explanation

about why there's some negative externality from that

or why there's some coordination failure,

or if there's something that consumers

are just really misled about

where they don't realize why giving away data like this

is a really bad thing to do,

then ultimately I kind of want to,

you know, respect people's preferences.

They can give away their data if they want.

I think there's a big difference

between companies use of data

and governments having data where,

you know, looking at the track record of history,

governments knowing a lot about their people can be very bad

if the government chooses to do bad things with it.

And that's more worrying, I think.

So let's jump into it a little bit.

Most people know, but actually I, two years ago,

had no idea what effective altruism was

until I saw there was a cool looking event

in an MIT group here.

I think it's called the Effective Altruism Club or a group.

I was like, what the heck is that?

And one of my friends said,

I mean, he said that they're just

a bunch of eccentric characters.

So I was like, hell yes, I'm in.

So I went to one of their events

and looked up what's it about.

It's quite a fascinating philosophical

and just a movement of ideas.

So can you tell me what is effective altruism?

Great, so the core of effective altruism

is about trying to answer this question,

which is how can I do as much good as possible

with my scarce resources, my time and with my money?

And then once we have our best guess answers to that,

trying to take those ideas and put that into practice,

and do those things that we believe will do the most good.

And we're now a community of people,

many thousands of us around the world,

who really are trying to answer that question

as best we can and then use our time and money

to make the world better.

So what's the difference between sort of

classical general idea of altruism

and effective altruism?

So normally when people try to do good,

they often just aren't so reflective about those attempts.

So someone might approach you on the street

asking you to give to charity.

And if you're feeling altruistic,

you'll give to the person on the street.

Or if you think, oh, I wanna do some good in my life,

you might volunteer at a local place.

Or perhaps you'll decide, pursue a career

where you're working in a field

that's kind of more obviously beneficial

like being a doctor or a nurse or a healthcare professional.

But it's very rare that people apply the same level

of rigor and analytical thinking

to lots of other areas we think about.

So take the case of someone approaching you on the street.

Imagine if that person instead was saying,

hey, I've got this amazing company.

Do you want to invest in it?

It would be insane.

No one would ever think, oh, of course,

I'm just a company like you'd think it was a scam.

But somehow we don't have that same level of rigor

when it comes to doing good,

even though the stakes are more important

when it comes to trying to help others

than trying to make money for ourselves.

Well, first of all, so there is a psychology

at the individual level of doing good just feels good.

And so in some sense, on that pure psychological part,

it doesn't matter.

In fact, you don't wanna know if it does good or not

because most of the time it won't.

So like in a certain sense,

it's understandable why altruism

without the effective part is so appealing

to a certain population.

By the way, let's zoom off for a second.

Do you think most people, two questions.

Do you think most people are good?

And question number two is,

do you think most people wanna do good?

So are most people good?

I think it's just super dependent

on the circumstances that someone is in.

I think that the actions people take

and their moral worth is just much more dependent

on circumstance than it is on someone's intrinsic character.

So is there evil within all of us?

It seems like with the better angels of our nature,

there's a tendency of us as a society

to tend towards good, less war.

I mean, with all these metrics.

Is that us becoming who we want to be

or is that some kind of societal force?

What's the nature versus nurture thing here?

Yeah, so in that case, I just think,

yeah, so violence has massively declined over time.

I think that's a slow process of cultural evolution,

institutional evolution such that now the incentives

for you and I to be violent are very, very small indeed.

In contrast, when we were hunter gatherers,

the incentives were quite large.

If there was someone who was potentially disturbing

the social order and hunter gatherer setting,

there was a very strong incentive to kill that person

and people did and it was just the guarded 10% of deaths

among hunter gatherers were murders.

After hunter gatherers, when you have actual societies

is when violence can probably go up

because there's more incentive to do mass violence, right?

To take over, conquer other people's lands

and murder everybody in place and so on.

Yeah, I mean, I think total death rate

from human causes does go down,

but you're right that if you're in a hunter gatherer situation

you're kind of a group that you're part of is very small

then you can't have massive wars

that just massive communities don't exist.

But anyway, the second question,

do you think most people want to do good?

Yeah, and then I think that is true for most people.

I think you see that with the fact that most people donate,

a large proportion of people volunteer.

If you give people opportunities

to easily help other people, they will take it.

But at the same time,

we're a product of our circumstances

and if it were more socially awarded to be doing more good,

if it were more socially awarded to do good effectively

rather than not effectively,

then we would see that behavior a lot more.

So why should we do good?

Yeah, my answer to this is

there's no kind of deeper level of explanation.

So my answer to kind of why should you do good is

well, there is someone whose life is on the line,

for example, whose life you can save

via donating just actually a few thousand dollars

to an effective nonprofit

like the Against Malaria Foundation.

That is a sufficient reason to do good.

And then if you ask, well, why ought I to do that?

I'm like, I just show you the same facts again.

It's that fact that is the reason to do good.

There's nothing more fundamental than that.

I'd like to sort of make more concrete

the thing we're trying to make better.

So you just mentioned malaria.

So there's a huge amount of suffering in the world.

Are we trying to remove?

So is ultimately the goal, not ultimately,

but the first step is to remove the worst of the suffering.

So there's some kind of threshold of suffering

that we want to make sure does not exist in the world.

Or do we really naturally want to take a much further step

and look at things like income inequality?

So not just getting everybody above a certain threshold,

but making sure that there's some,

that broadly speaking,

there's less injustice in the world, unfairness,

in some definition, of course,

very difficult to define a fairness.

Yeah, so the metric I use is how many people do we affect

and by how much do we affect them?

And so that can, often that means eliminating suffering,

but it doesn't have to,

could be helping promote a flourishing life instead.

And so if I was comparing reducing income inequality

or getting people from the very pits of suffering

to a higher level,

the question I would ask is just a quantitative one

of just if I do this first thing or the second thing,

how many people am I going to benefit

and by how much am I going to benefit?

Am I going to move that one person from kind of 10%,

0% well being to 10% well being?

Perhaps that's just not as good as moving a hundred people

from 10% well being to 50% well being.

And the idea is the diminishing returns is the idea of

when you're in terrible poverty,

then the $1 that you give goes much further

than if you were in the middle class in the United States,

for example.

Absolutely.

And this fact is really striking.

So if you take even just quite a conservative estimate

of how we are able to turn money into well being,

the economists put it as like a log curve.

That's the or steeper.

But that means that any proportional increase

in your income has the same impact on your well being.

And so someone moving from $1,000 a year

to $2,000 a year has the same impact

as someone moving from $100,000 a year to $200,000 a year.

And then when you combine that with the fact that we

in middle class members of rich countries are 100 times richer

than financial terms in the global poor,

that means we can do a hundred times to benefit the poorest people

in the world as we can to benefit people of our income level.

And that's this astonishing fact.

Yeah, it's quite incredible.

A lot of these facts and ideas are just difficult to think about

because there's an overwhelming amount of suffering in the world.

And even acknowledging it is difficult.

Not exactly sure why that is.

I mean, I mean, it's difficult because you have to bring to mind,

you know, it's an unpleasant experience thinking

about other people's suffering.

It's unpleasant to be empathizing with it, firstly.

And then secondly, thinking about it means

that maybe we'd have to change our lifestyles.

And if you're very attached to the income that you've got,

perhaps you don't want to be confronting ideas or arguments

that might cause you to use some of that money to help others.

So it's quite understandable in the psychological terms,

even if it's not the right thing that we ought to be doing.

So how can we do better?

How can we be more effective?

How does data help?

Yeah, in general, how can we do better?

It's definitely hard.

And we have spent the last 10 years engaged in kind of some deep research projects,

to try and answer kind of two questions.

One is, of all the many problems the world is facing,

what are the problems we ought to be focused on?

And then within those problems that we judge to be kind of the most pressing,

where we use this idea of focusing on problems that are the biggest in scale,

that are the most tractable,

where we can make the most progress on that problem,

and that are the most neglected.

Within them, what are the things that have the kind of best evidence,

or we have the best guess, will do the most good.

And so we have a bunch of organizations.

So GiveWell, for example, is focused on global health and development,

and has a list of seven top recommended charities.

So the idea in general, and sorry to interrupt,

is, so we'll talk about sort of poverty and animal welfare and existential risk.

Those are all fascinating topics, but in general,

the idea is there should be a group,

sorry, there's a lot of groups that seek to convert money into good.

And then you also on top of that want to have a accounting

of how good they actually perform that conversion,

how well they did in converting money to good.

So ranking of these different groups,

ranking these charities.

So does that apply across basically all aspects of effective altruism?

So there should be a group of people,

and they should report on certain metrics of how well they've done,

and you should only give your money to groups that do a good job.

That's the core idea. I'd make two comments.

One is just, it's not just about money.

So we're also trying to encourage people to work in areas

where they'll have the biggest impact.

Absolutely.

And in some areas, you know, they're really people heavy, but money poor.

Other areas are kind of money rich and people poor.

And so whether it's better to focus time or money depends on the cause area.

And then the second is that you mentioned metrics,

and while that's the ideal, and in some areas we do,

we are able to get somewhat quantitative information

about how much impact an area is having.

That's not always true.

For some of the issues, like you mentioned existential risks,

well, we're not able to measure in any sort of precise way

like how much progress we're making.

And so you have to instead fall back on just rigorous argument and evaluation,

even in the absence of data.

So let's first sort of linger on your own story for a second.

How do you yourself practice effective altruism in your own life?

Because I think that's a really interesting place to start.

So I've tried to build effective altruism into at least many components of my life.

So on the donation side, my plan is to give away most of my income

over the course of my life.

I've set a bar I feel happy with and I just donate above that bar.

So at the moment, I donate about 20% of my income.

Then on the career side, I've also shifted kind of what I do,

where I was initially planning to work on very esoteric topics

in the philosophy of logic, philosophy of language,

things that are intellectually extremely interesting,

but the path by which they really make a difference to the world is,

let's just say it's very unclear at best.

And so I switched instead to researching ethics to actually just working

on this question of how we can do as much good as possible.

And then I've also spent a very large chunk of my life over the last 10 years

creating a number of nonprofits who again in different ways

are tackling this question of how we can do the most good

and helping them to grow over time too.

Yeah, we mentioned a few of them with the career selection, 80,000.

80,000 hours.

80,000 hours is a really interesting group.

So maybe also just a quick pause on the origins of effective altruism

because you paint a picture who the key figures are,

including yourself in the effective altruism movement today.

Yeah, there are two main strands that kind of came together

to form the effective altruism movement.

So one was two philosophers, myself and Toby Ord at Oxford,

and we had been very influenced by the work of Peter Singer,

an Australian model philosopher who had argued for many decades

that because one can do so much good at such little cost to oneself,

we have an obligation to give away most of our income

to benefit those in extreme poverty,

just in the same way that we have an obligation to run in

and save a child from drowning in a shallow pond

if it would just ruin your suit that cost a few thousand dollars.

And we set up Giving What We Can in 2009,

which is encouraging people to give at least 10% of their income

to the most effective charities.

And the second main strand was the formation of GiveWell,

which was originally based in New York and started in about 2007.

And that was set up by Holden Carnovsky and Elie Hassenfeld,

who were two hedge fund dudes who were making good money

and thinking, well, where should I donate?

And in the same way as if they wanted to buy a product for themselves,

they would look at Amazon reviews.

They were like, well, what are the best charities?

Found they just weren't really good answers to that question,

certainly not that they were satisfied with.

And so they formed GiveWell in order to try and work out

what are those charities where they can have the biggest impact.

And then from there and some other influences,

kind of community grew and spread.

Can we explore the philosophical and political space

that effective altruism occupies a little bit?

So from the little and distant in my own lifetime

that I've read of Ayn Rand's work, Ayn Rand's philosophy of objectivism,

espouses, and it's interesting to put her philosophy in contrast

with effective altruism.

So it espouses selfishness as the best thing you can do.

But it's not actually against altruism.

It's just you have that choice, but you should be selfish in it, right?

Or not, maybe you can disagree here.

But so it can be viewed as the complete opposite of effective altruism

or it can be viewed as similar because the word effective is really interesting.

Because if you want to do good, then you should be damn good at doing good, right?

I think that would fit within the morality that's defined by objectivism.

So do you see a connection between these two philosophies

and other perhaps in this complicated space of beliefs

that effective altruism is positioned as opposing or aligned with?

I would definitely say that objectivism, Ayn Rand's philosophy,

is a philosophy that's quite fundamentally opposed to effective altruism.

In which way?

Insofar as Ayn Rand's philosophy is about championing egoism

and saying that I'm never quite sure whether the philosophy is meant to say

that just you ought to do whatever will best benefit yourself,

that's ethical egoism, no matter what the consequences are.

Or second, if there's this alternative view, which is, well,

you ought to try and benefit yourself because that's actually the best way

of benefiting society.

Certainly, in Atlas Shalaguchi is presenting her philosophy

as a way that's actually going to bring about a flourishing society.

And if it's the former, then well, effective altruism is all about promoting

the idea of altruism and saying, in fact,

we ought to really be trying to help others as much as possible.

So it's opposed there.

And then on the second side, I would just dispute the empirical premise.

It would seem, given the major problems in the world today,

it would seem like this remarkable coincidence,

quite suspicious, one might say, if benefiting myself was actually

the best way to bring about a better world.

So on that point, and I think that connects also with career selection

that we'll talk about, but let's consider not objectives, but capitalism.

And the idea that you focusing on the thing that you are damn good at,

whatever that is, may be the best thing for the world.

Part of it is also mindset, right?

The thing I love is robots.

So maybe I should focus on building robots

and never even think about the idea of effective altruism,

which is kind of the capitalist notion.

Is there any value in that idea in just finding the thing you're good at

and maximizing your productivity in this world

and thereby sort of lifting all boats and benefiting society as a result?

Yeah, I think there's two things I'd want to say on that.

So one is what your comparative advantages,

what your strengths are when it comes to career.

That's obviously super important because there's lots of career paths

I would be terrible at if I thought being an artist was the best thing one could do.

Well, I'd be doomed, just really quite astonishingly bad.

And so I do think, at least within the realm of things that could plausibly be very high impact,

choose the thing that you think you're going to be able to really be passionate at

and excel at over the long term.

Then on this question of should one just do that in an unrestricted way

and not even think about what the most important problems are.

I do think that in a kind of perfectly designed society, that might well be the case.

That would be a society where we've corrected all market failures,

we've internalized all externalities,

and then we've managed to set up incentives such that people just pursuing their own strengths

is the best way of doing good.

But we're very far from that society.

So if one did that, then it would be very unlikely that you would focus

on improving the lives of nonhuman animals that aren't participating in markets

or ensuring the long run future goes well,

where future people certainly aren't participating in markets

or benefiting the global poor who do participate,

but have so much less kind of power from a starting perspective

that their views aren't accurately kind of represented by market forces too.

Got it.

So yeah, instead of pure definition capitalism,

it just may very well ignore the people that are suffering the most,

the white swath of them.

So if you could allow me this line of thinking here.

So I've listened to a lot of your conversations online.

I find, if I can compliment you, they're very interesting conversations.

Your conversation on Rogan, on Joe Rogan was really interesting,

with Sam Harris and so on, whatever.

There's a lot of stuff that's really good out there.

And yet, when I look at the internet and I look at YouTube,

which has certain mobs, certain swaths of right leaning folks,

whom I dearly love.

I love all people, especially people with ideas.

They seem to not like you very much.

So I don't understand why exactly.

So my own sort of hypothesis is there is a right left divide

that absurdly so caricatured in politics,

at least in the United States.

And maybe you're somehow pigeonholed into one of those sides.

And maybe that's what it is.

Maybe your message is somehow politicized.

Yeah, I mean.

How do you make sense of that?

Because you're extremely interesting.

Like you got the comments I see on Joe Rogan.

There's a bunch of negative stuff.

And yet, if you listen to it, the conversation is fascinating.

I'm not speaking, I'm not some kind of lefty extremist,

but just it's a fascinating conversation.

So why are you getting some small amount of hate?

So I'm actually pretty glad that Effective Altruism has managed

to stay relatively unpoliticized because I think the core message

to just use some of your time and money to do as much good as possible

to fight some of the problems in the world can be appealing

across the political spectrum.

And we do have a diversity of political viewpoints among people

who have engaged in Effective Altruism.

We do, however, do get some criticism from the left and the right.

Oh, interesting.

What's the criticism?

Both would be interesting to hear.

Yeah, so criticism from the left is that we're not focused enough

on dismantling the capitalist system that they see as the root

of most of the problems that we're talking about.

And there I kind of disagree on partly the premise where I don't

think relevant alternative systems would say to the animals or to the

global poor or to the future generations kind of much better.

And then also the tactics where I think there are particular ways

we can change society that would massively benefit, you know,

be massively beneficial on those things that don't go via dismantling

like the entire system, which is perhaps a million times harder to do.

Then criticism on the right, there's definitely like in response

to the Joe Rogan podcast.

There definitely were a number of Ayn Rand fans who weren't keen

on the idea of promoting altruism.

There was a remarkable set of ideas.

Just the idea that Effective Altruism was unmanly, I think, was

driving a lot of criticism.

Okay, so I love fighting.

I've been in street fights my whole life.

I'm as alpha in everything I do as it gets.

And the fact that Joe Rogan said that I thought Scent of a Woman

is a better movie than John Wick put me into this beta category

amongst people who are like basically saying this, yeah, unmanly

or it's not tough.

It's not some principled view of strength that is represented

by a spasmodic.

So actually, so how do you think about this?

Because to me, altruism, especially Effective Altruism, I don't

know what the female version of that is, but on the male side, manly

as fuck, if I may say so.

So how do you think about that kind of criticism?

I think people who would make that criticism are just occupying

a like state of mind that I think is just so different from my

state of mind that I kind of struggle to maybe even understand it

where if something's manly or unmanly or feminine or unfeminine,

I'm like, I don't care.

Like, is it the right thing to do or the wrong thing to do?

So let me put it not in terms of man or woman.

I don't think that's useful, but I think there's a notion of acting

out of fear as opposed to out of principle and strength.

Yeah.

So, okay.

Yeah.

Here's something that I do feel as an intuition and that I think

drives some people who do find Canvaean Land attractive and so on

as a philosophy, which is a kind of taking control of your own

life and having power over how you're steering your life and not

kind of kowtowing to others, you know, really thinking things through.

I find like that set of ideas just very compelling and inspirational.

I actually think of effect of altruism has really, you know, that

side of my personality.

It's like scratch that itch where you are just not taking the kind

of priorities that society is giving you as granted.

Instead, you're choosing to act in accordance with the priorities

that you think are most important in the world.

And often that involves then doing quite unusual things from a

societal perspective, like donating a large chunk of your earnings

or working on these weird issues about AI and so on that other

people might not understand.

Yeah, I think that's a really gutsy thing to do.

That is taking control.

That's at least at this stage.

I mean, that's you taking ownership, not of just yourself, but

your presence in this world that's full of suffering and saying

as opposed to being paralyzed by that notion is taking control

and saying I could do something.

Yeah, I mean, that's really powerful.

But I mean, sort of the one thing I personally hate too about the

left currently that I think those folks to detect is the social

signaling. When you look at yourself, sort of late at night, would

you do everything you're doing in terms of effective altruism if

your name, because you're quite popular, but if your name was

totally unattached to it, so if it was in secret.

Yeah, I mean, I think I would.

To be honest, I think the kind of popularity is like, you know,

it's mixed bag, but there are serious costs.

And I don't particularly, I don't like love it.

Like, it means you get all these people calling you a cuck on

Joe Rogan.

It's like not the most fun thing.

But you also get a lot of sort of brownie points for doing good

for the world.

Yeah, you do.

But I think my ideal life, I would be like in some library solving

logic puzzles all day and I'd like really be like learning maths

and so on.

So you have a like good body of friends and so on.

So your instinct for effective altruism is something deep.

It's not one that is communicating

socially. It's more in your heart.

You want to do good for the world.

Yeah, I mean, so we can look back to early giving what we can.

So, you know, we're setting this up, me and Toby.

And I really thought that doing this would be a big hit to my

academic career because I was now spending, you know, at that time

more than half my time setting up this nonprofit at the crucial

time when you should be like producing your best academic work

and so on.

And it was also the case at the time.

It was kind of like the Toby order club.

You know, he was he was the most popular.

There's this personal interest story about him and his plans

donate and sorry to interrupt but Toby was donating a large

amount. Can you tell just briefly what he was doing?

Yeah, so he made this public commitment to give everything

he earned above 20,000 pounds per year to the most effective

causes. And even as a graduate student, he was still donating

about 15, 20% of his income, which is so quite significant

given that graduate students are not known for being super

wealthy.

That's right. And when we launched Giving What We Can, the

media just loved this as like a personal interest story.

So the story about him and his pledge was the most, yeah, it

was actually the most popular news story of the day.

And we kind of ran the same story a year later and it was

the most popular news story of the day a year later too.

And so it really was kind of several years before then I

was also kind of giving more talks and starting to do more

writing and then especially with, you know, I wrote this book

Doing Good Better that then there started to be kind of attention

and so on. But deep inside your own relationship with effective

altruism was, I mean, it had nothing to do with the publicity.

Did you see yourself?

How did the publicity connect with it?

Yeah, I mean, that's kind of what I'm saying is I think the

publicity came like several years afterwards.

I mean, at the early stage when we set up Giving What We Can,

it was really just every person we get to pledge 10% is, you

know, something like $100,000 over their lifetime.

That's huge.

And so it was just we had started with 23 members, every single

person was just this like kind of huge accomplishment.

And at the time, I just really thought, you know, maybe over

time we'll have a hundred members and that'll be like amazing.

Whereas now we have, you know, over four thousand and one and

a half billion dollars pledged.

That's just unimaginable to me at the time when I was first kind

of getting this, you know, getting the stuff off the ground.

So can we talk about poverty and the biggest problems that you

think in the near term effective altruism can attack in each

one. So poverty obviously is a huge one.

Yeah. How can we help?

Great.

Yeah.

So poverty, absolutely this huge problem.

700 million people in extreme poverty living in less than two

dollars per day where that's what that means is what two dollars

would buy in the US.

So think about that.

It's like some rice, maybe some beans.

It's very, you know, really not much.

And at the same time, we can do an enormous amount to improve

the lives of people in extreme poverty.

So the things that we tend to focus on interventions in global

health and that's for a couple of few reasons.

One is like global health just has this amazing track record

life expectancy globally is up 50% relative to 60 or 70 years

ago. We've eradicated smallpox that's which killed 2 million

lives every year almost eradicated polio.

Second is that we just have great data on what works when it

comes to global health.

So we just know that bed nets protect children from prevent

them from dying from malaria.

And then the third is just that's extremely cost effective.

So it costs $5 to buy one bed net, protects two children for

two years against malaria.

If you spend about $3,000 on bed nets, then statistically

speaking, you're going to save a child's life.

And there are other interventions too.

And so given the people in such suffering and we have this

opportunity to, you know, do such huge good for such low cost.

Well, yeah, why not?

So the individual.

So for me today, if I wanted to look at poverty, how would

I help? And I wanted to say, I think donating 10% of your

income is a very interesting idea or some percentage or some

setting a bar and sort of sticking to it.

How do we then take the step towards the effective part?

So you've conveyed some notions, but who do you give the

money to? Yeah.

So GiveWell, this organization I mentioned, well, it makes

charity recommendations and some of its top recommendations.

So Against Malaria Foundation is this organization that buys

and distributes these insecticide seeded bed nets.

And then it has a total of seven charities that it recommends

very highly. So that recommendation, is it almost like a star

of approval or is there some metrics?

So what are the ways that GiveWell conveys that this is a

great charity organization?

Yeah.

So GiveWell is looking at metrics and it's trying to compare

charities ultimately in the number of lives that you can save

or an equivalent benefit.

So one of the charities it recommends is GiveDirectly, which

simply just transfers cash to the poorest families where poor

family will get a cash transfer of $1,000 and they kind of

regard that as the baseline intervention because it's so simple

and people, you know, they know what to do with how to benefit

themselves. That's quite powerful, by the way.

So before GiveWell, before the Effective Altruism Movement, was

there, I imagine there's a huge amount of corruption, funny

enough, in charity organizations or misuse of money.

Yeah.

So there was nothing like GiveWell before that?

No.

I mean, there were some.

So, I mean, the charity corruption, I mean, obviously

there's some, I don't think it's a huge issue.

They're also just focusing on the long things. Prior to GiveWell,

there were some organizations like Charity Navigator, which

were more aimed at worrying about corruption and so on.

So they weren't saying, these are the charities where you're

going to do the most good. Instead, it was like, how good

are the charities financials?

How good is its health?

Are they transparent? And yeah, so that would be more useful

for weeding out some of those worst charities.

So GiveWell has just taken a step further, sort of in this

21st century of data.

It's actually looking at the effective part.

Yeah. So it's like, you know, if you know the wire cutter for

if you want to buy a pair of headphones, they will just look

at all the headphones and be like, these are the best headphones

you can buy.

That's the idea with GiveWell.

Okay.

So do you think there's a bar of what suffering is?

And do you think one day we can eradicate suffering in our

world? Yeah.

Amongst humans?

Let's talk humans for now. Talk humans.

But in general, yeah, actually.

So there's a colleague of mine calling the term abolitionism

for the idea that we should just be trying to abolish

suffering. And in the long run, I mean, I don't expect to

anytime soon, but I think we can.

I think that would require, you know, quite change, quite

drastic changes to the way society is structured and perhaps

even the, you know, the human, in fact, even changes to human

nature. But I do think that suffering whenever it occurs

is bad and we should want it to not occur.

So there's a line.

There's a gray area between suffering.

Now I'm Russian.

So I romanticize some aspects of suffering.

There's a gray line between struggle, gray area between

struggle and suffering.

So one, do we want to eradicate all struggle in the world?

So there's an idea, you know, that the human condition

inherently has suffering in it and it's a creative force.

It's a struggle of our lives and we somehow grow from that.

How do you think about, how do you think about that?

I agree that's true.

So, you know, often, you know, great artists can be also

suffering from, you know, major health conditions or depression

and so on. They come from abusive parents.

Most great artists, I think, come from abusive parents.

Yeah, that seems to be at least commonly the case, but I

want to distinguish between suffering as being instrumentally

good, you know, it causes people to produce good things and

whether it's intrinsically good and I think intrinsically

it's always bad.

And so if we can produce these, you know, great achievements

via some other means where, you know, if we look at the

scientific enterprise, we've produced incredible things

often from people who aren't suffering, have, you know,

pretty good lives.

They're just, they're driven instead of, you know, being

pushed by a certain sort of anguish.

They're being driven by intellectual curiosity.

If we can instead produce a society where it's all cavet

and no stick, that's better from my perspective.

Yeah, but I'm going to disagree with the notion that that's

possible, but I would say most of the suffering in the world

is not productive.

So I would dream of effective altruism curing that suffering.

Yeah, but then I would say that there is some suffering that

is productive that we want to keep the because but that's

not even the focus of because most of the suffering is just

absurd and needs to be eliminated.

So let's not even romanticize this usual notion I have,

but nevertheless struggle has some kind of inherent value

that to me at least, you're right.

There's some elements of human nature that also have to

be modified in order to cure all suffering.

Yeah, I mean, there's an interesting question of whether

it's possible.

So at the moment, you know, most of the time we're kind

of neutral and then we burn ourselves and that's negative

and that's really good that we get that negative signal

because it means we won't burn ourselves again.

There's a question like could you design agents humans such

that you're not hovering around the zero level you're hovering

it like bliss.

Yeah, and then you touch the flame and you're like, oh no,

you're just slightly worse bliss.

Yeah, but that's really bad compared to the bliss you

were normally in so that you can have like a gradient of

bliss instead of like pain and pleasure on that point.

I think it's a really important point on the experience

of suffering the relative nature of it.

Maybe having grown up in the Soviet Union were quite poor

by any measure and when I when I was in my childhood,

but it didn't feel like you're poor because everybody around

you were poor there's a and then in America, I feel I for

the first time begin to feel poor.

Yeah.

Yeah, because of the road there's different.

There's some cultural aspects to it that really emphasize

that it's good to be rich.

And then there's just the notion that there is a lot of

income inequality and therefore you experience that inequality.

That's where suffering go.

Do you so what do you think about the inequality of suffering

that that we have to think about do you think we have to

think about that as part of effective altruism?

Yeah, I think we're just things vary in terms of whether

you get benefits or costs from them just in relative terms

or in absolute terms.

So a lot of the time yeah, there's this hedonic treadmill

where if you get you know, there's money is useful because

it helps you buy things or good for you because it helps

you buy things, but there's also a status component too

and that status component is kind of zero sum if you were

saying like in Russia, you know, no one else felt poor

because everyone around you is poor.

Whereas now you've got this these other people who are

you know super rich and maybe that makes you feel.

You know less good about yourself.

There are some other things however, which are just

intrinsically good or bad.

So commuting for example, it's just people hate it.

It doesn't really change knowing the other people are

commuting to doesn't make it any any kind of less bad,

but it's sort of to push back on that for a second.

I mean, yes, but also if some people were, you know on

horseback your commute on the train might feel a lot better.

Yeah, you know the there is a relative Nick.

I mean everybody's complaining about society today forgetting

it's forgetting how much better is the better angels of

our nature how the technologies improve fundamentally

improving most of the world's lives.

Yeah, and actually there's some psychological research

on the well being benefits of volunteering where people

who volunteer tend to just feel happier about their lives

and one of the suggested explanations is it because it

extends your reference class.

So no longer you comparing yourself to the Joneses who

have their slightly better car because you realize that

you know people in much worse conditions than you and

so now, you know your life doesn't seem so bad.

That's actually on the psychological level.

One of the fundamental benefits of effective altruism.

Yeah is is I mean, I guess it's the altruism part of

effective altruism is exposing yourself to the suffering

in the world allows you to be more.

Yeah happier and actually allows you in the sort of

meditative introspective way realize that you don't need

most of the wealth you have to to be happy.

Absolutely.

I mean, I think effective options have been this huge

benefit for me and I really don't think that if I had

more money that I was living on that that would change

my level of well being at all.

Whereas engaging in something that I think is meaningful

that I think is stealing humanity in a positive direction.

That's extremely rewarding.

And so yeah, I mean despite my best attempts at sacrifice.

Um, I don't you know, I think I've actually ended up

happier as a result of engaging in effective altruism

than I would have done.

That's such an interesting idea.

Yeah, so let's let's talk about animal welfare.

Sure, easy question. What is consciousness?

Yeah, especially as it has to do with the capacity to

suffer. I think there seems to be a connection between

how conscious something is the amount of consciousness

and stability to suffer and that all comes into play

about us thinking how much suffering there's in the

world with regard to animals.

So how do you think about animal welfare and consciousness?

Okay.

Well consciousness easy question.

Okay.

Um, yeah, I mean, I think we don't have a good understanding

of consciousness.

My best guess is it's got and by consciousness.

I'm meaning what it is feels like to be you the subjective

experience that's seems to be different from everything

else we know about in the world.

Yeah, I think it's clear.

It's very poorly understood at the moment.

I think it has something to do with information processing.

So the fact that the brain is a computer or something

like a computer.

So that would mean that very advanced AI could be conscious

of information processors in general could be conscious

with some suitable complexity, but that also some suitable

complexity.

It's a question whether greater complexity creates some

kind of greater consciousness which relates to animals.

Yeah, right.

Is there if it's an information processing system and it's

smaller and smaller is an ant less conscious than a cow

less conscious than a monkey.

Yeah, and again this super hard question, but I think my

best guess is yes, like if you if I think well consciousness,

it's not some magical thing that appears out of nowhere.

It's not you know, Descartes thought it was just comes in

from this other realm and then enters through the pineal

gland in your brain and that's kind of soul and it's conscious.

So it's got something to do with what's going on in your

brain.

A chicken has one three hundredth of the size of the brain

that you have ants.

I don't know how small it is.

Maybe it's a millionth the size my best guess which I may

well be wrong about because this is so hard is that in some

relevant sense the chicken is experiencing consciousness

to a less degree than the human and the ants significantly

less again.

I don't think it's as little as three hundredth as much.

I think there's everyone who's ever seen a chicken that's

there's evolutionary reasons for thinking that like the

ability to feel pain comes on the scene relatively early

on and we have lots of our brain that's dedicated stuff

that doesn't seem to have to do in anything to do with

consciousness language processing and so on.

So it seems like the easy so there's a lot of complicated

questions there that we can't ask the animals about but

it seems that there is easy questions in terms of suffering

which is things like factory farming that could be addressed.

Yeah, is that is that the lowest hanging fruit?

If I may use crude terms here of animal welfare.

Absolutely.

I think that's the lowest hanging fruit.

So at the moment we kill we raise and kill about 50 billion

animals every year.

So how many 50 billion in?

Yeah, so for every human on the planet several times that

number of being killed and the vast majority of them are

raised in factory farms where basically whatever your view

on animals, I think you should agree even if you think well,

maybe it's not bad to kill an animal.

Maybe if the animal was raised in good conditions, that's

just not the empirical reality.

The empirical reality is that they are kept in incredible

cage confinement.

They are de beaked or detailed without an aesthetic, you

know chickens often peck each other to death other like

otherwise because of them such stress.

It's really, you know, I think when a chicken gets killed

that's the best thing that happened to the chicken in the

course of its life and it's also completely unnecessary.

This is in order to save, you know a few pence for the price

of meat or price of eggs and we have indeed found it's also

just inconsistent with consumer preference as well people

who buy the products if they could they all they when you

do surveys are extremely against suffering in factory farms.

It's just they don't appreciate how bad it is and you know,

just tend to go with easy options.

And so then the best the most effective programs I know of

at the moment are nonprofits that go to companies and work

with companies to get them to take a pledge to cut certain

sorts of animal products like eggs from cage confinement

out of their supply chain.

And it's now the case that the top 50 food retailers and

fast food companies have all made these kind of cage free

pledges and when you do the numbers you get the conclusion

that every dollar you're giving to these nonprofits result

in hundreds of chickens being spared from cage confinement.

And then they're working to other other types of animals

other products too.

So is that the most effective way to do in have a ripple

effect essentially it's supposed to directly having regulation

from on top that says you can't do this.

So I would be more open to the regulation approach, but

at least in the US there's quite intense regulatory capture

from the agricultural industry.

And so attempts that we've seen to try and change regulation

have it's been a real uphill struggle.

There are some examples of ballot initiatives where the

people have been able to vote in a ballot to say we want

to ban eggs from cage conditions and that's been huge.

That's been really good, but beyond that it's much more

limited. So I've been really interested in the idea of

hunting in general and wild animals and seeing nature as

a form of cruelty that I am ethically more okay with.

Okay, just from my perspective and then I read about wild

animal suffering that I'm just I'm just giving you the

kind of yeah notion of how I felt because animal because

animal factory farming is so bad.

Yeah that living in the woods seem good.

Yeah, and yet when you actually start to think about it

all I mean all of the animals in the animal world the

living in like terrible poverty, right?

Yeah.

Yeah, so you have all the medical conditions all of that.

I mean they're living horrible lives.

It could be improved.

That's a really interesting notion that I think may not

even be useful to talk about because factory farming is

such a big thing to focus on.

Yeah, but it's nevertheless an interesting notion to think

of all the animals in the wild as suffering in the same

way that humans in poverty are suffering.

Yeah, I mean and often even worse so many animals we

produce by our selection.

So you have a very large number of children in the expectation

that only a small number survive.

And so for those animals almost all of them just live short

lives where they starve to death.

So yeah, there's huge amounts of suffering in nature that

I don't think we should you know pretend that it's this kind

of wonderful paradise for most animals.

Yeah, their life is filled with hunger and fear and disease.

Yeah, I did agree with you entirely that when it comes

to focusing on animal welfare, we should focus in factory

farming, but we also yeah should be aware to the reality

of what life for most animals is like.

So let's talk about a topic I've talked a lot about and

you've actually quite eloquently talked about which is the

third priority that effective altruism considers is really

important is existential risks.

Yeah, when you think about the existential risks that

are facing our civilization, what's before us?

What concerns you?

What should we be thinking about from in the especially

from an effective altruism perspective?

Great. So the reason I started getting concerned about

this was thinking about future generations where the key

idea is just well future people matter morally.

There are vast numbers of future people.

If we don't cause our own extinction, there's no reason

why civilization might not last a million years.

I mean we last as long as a typical mammalian species

or a billion years is when the Earth is no longer habitable

or if we can take to the stars then perhaps it's trillions

of years beyond that.

So the future could be very big indeed and it seems like

we're potentially very early on in civilization.

Then the second idea is just well, maybe there are things

that are going to really derail that things that actually

could prevent us from having this long wonderful civilization

and instead could cause our own cause our own extinction

or otherwise perhaps like lock ourselves into a very bad

state. And what ways could that happen?

Well causing our own extinction development of nuclear

weapons in the 20th century at least put on the table

that we now had weapons that were powerful enough that

you could very significantly destroy society perhaps

and all that nuclear war would cause a nuclear winter.

Perhaps that would be enough for the human race to go

extinct.

Why do you think we haven't done it? Sorry to interrupt.

Why do you think we haven't done it yet?

Is it surprising to you that having, you know, always

for the past few decades several thousand of active ready

to launch nuclear weapons warheads and yet we have not

launched them ever since the initial launch on Hiroshima

and Nagasaki.

I think it's a mix of luck.

So I think it's definitely not inevitable that we haven't

used them.

So John F. Kennedy, general Cuban Missile Crisis put the

estimate of nuclear exchange between the US and USSR

that somewhere between one and three and even so, you know,

we really did come close.

At the same time, I do think mutually assured destruction

is a reason why people don't go to war.

It would be, you know, why nuclear powers don't go to war.

Do you think that holds if you can linger on that for a

second, like my dad is a physicist amongst other things

and he believes that nuclear weapons are actually just

really hard to build which is one of the really big benefits

of them currently so that you don't have it's very hard

if you're crazy to build to acquire a nuclear weapon.

So the mutually shared destruction really works when you

talk seems to work better when it's nation states, when

it's serious people, even if they're a little bit, you

know, dictatorial and so on.

Do you think this mutually sure destruction idea will

carry how far will it carry us in terms of different kinds

of weapons?

Oh, yeah, I think it's your point that nuclear weapons

are very hard to build and relatively easy to control

because you can control fissile material is a really

important one and future technology that's equally destructive

might not have those properties.

So for example, if in the future people are able to design

viruses, perhaps using a DNA printing kit that's on that,

you know, one can just buy.

In fact, there are companies in the process of creating

home DNA printing kits. Well, then perhaps that's just

totally democratized.

Perhaps the power to reap huge destructive potential is

in the hands of most people in the world or certainly

most people with effort and then yeah, I no longer trust

mutually assured destruction because some for some people

the idea that they would die is just not a disincentive.

There was a Japanese cult, for example.

Ohm Shinrikyo in the 90s that had they what they believed

was that Armageddon was coming if you died before Armageddon,

you would get good karma.

You wouldn't go to hell if you died during Armageddon.

Maybe you would go to hell and they had a biological weapons

program chemical weapons program when they were finally

apprehended.

They hadn't stocks of southern gas that were sufficient to

kill 4 million people engaged in multiple terrorist acts.

If they had had the ability to print a virus at home,

that would have been very scary.

So it's not impossible to imagine groups of people that

hold that kind of belief of death as suicide as a good

thing for passage into the next world and so on and then

connect them with some weapons then ideology and weaponry

may create serious problems for us.

Let me ask you a quick question on what do you think is

the line between killing most humans and killing all humans?

How hard is it to kill everybody?

Yeah, have you thought about this?

I've thought about it a bit.

I think it is very hard to kill everybody.

So in the case of let's say an all out nuclear exchange

and let's say that leads to nuclear winter.

We don't really know but we you know might well happen

that would I think result in billions of deaths would

it kill everybody?

It's quite it's quite hard to see how that how it would

kill everybody for a few reasons.

One is just those are so many people.

Yes, you know seven and a half billion people.

So this bad event has to kill all you know, all almost

all of them.

Secondly live in such a diversity of locations.

So a nuclear exchange or the virus that has to kill people

who live in the coast of New Zealand which is going to

be climatically much more stable than other areas in the

world or people who are on submarines or who have access

to bunkers.

So there's a very like there's just like I'm sure there's

like two guys in Siberia just badass.

There's the just human nature somehow just perseveres.

Yeah, and then the second thing is just if there's some

catastrophic event people really don't want to die.

So there's going to be like, you know, huge amounts of

effort to ensure that it doesn't affect everyone.

Have you thought about what it takes to rebuild a society

with smaller smaller numbers like how big of a setback

these kinds of things are?

Yeah, so then that's something where there's real uncertainty

I think where at some point you just lose genetic sufficient

genetic diversity such that you can't come back.

There's it's unclear how small that population is.

But if you've only got say a thousand people or fewer

than a thousand, then maybe that's small enough.

What about human knowledge and then there's human knowledge.

I mean, it's striking how short on geological timescales

or evolutionary timescales the progress in or how quickly

the progress in human knowledge has been like agriculture.

We only invented in 10,000 BC cities were only, you know,

3000 BC whereas typical mammal species is half a million

years to a million years.

Do you think it's inevitable in some sense agriculture

everything that came the Industrial Revolution cars planes

the internet that level of innovation you think is inevitable.

I think given how quickly it arose.

So in the case of agriculture, I think that was dependent

on climate.

So it was the kind of glacial period was over the earth

warmed up a bit that made it much more likely that humans

would develop agriculture when it comes to the Industrial

Revolution. It's just you know, again only took a few thousand

years from cities to Industrial Revolution if we think okay,

we've gone back to this even let's say agricultural era,

but there's no reason why we wouldn't go extinct in the

coming tens of thousands of years or hundreds of thousands

of years.

It seems just vet.

It would be very surprising if we didn't rebound unless

there's some special reason that makes things different.

Yes.

So perhaps we just have a much greater like disease burden

now so HIV exists.

It didn't exist before and perhaps that's kind of latent

and you know and being suppressed by modern medicine

and sanitation and so on but would be a much bigger problem

for some, you know, utterly destroyed the society that

was trying to rebound or there's just maybe there's something

we don't know about.

So another existential risk comes from the mysterious the

beautiful artificial intelligence.

Yeah.

So what what's the shape of your concerns about AI?

I think there are quite a lot of concerns about AI and

sometimes the different risks don't get distinguished enough.

So the kind of classic worry most is closely associated

with Nick Bostrom and Elias Joukowski is that we at some

point move from having narrow AI systems to artificial

general intelligence.

You get this very fast feedback effect where AGI is able

to build, you know, artificial intelligence helps you to

build greater artificial intelligence.

We have this one system that suddenly very powerful far

more powerful than others than perhaps far more powerful

than, you know, the rest of the world combined and then

secondly, it has goals that are misaligned with human goals.

And so it pursues its own goals.

It realize, hey, there's this competition namely from humans.

It would be better if we eliminated them in just the same

way as homo sapiens eradicated the Neanderthals.

In fact, it in fact killed off most large animals on the

planet that walk the planet. So that's kind of one set of

worries. I think that's not my I think these shouldn't

be dismissed as science fiction.

I think it's something we should be taking very seriously,

but it's not the thing you visualize when you're concerned

about the biggest near term.

Yeah, I think it's I think it's like one possible scenario

that would be astronomically bad.

I think that other scenarios that would also be extremely

bad comparably bad that are more likely to occur.

So one is just we are able to control AI.

So we're able to get it to do what we want it to do.

And perhaps there's not like this fast takeoff of AI capabilities

within a single system.

It's distributed across many systems that do somewhat different

things, but you do get very rapid economic and technological

progress as a result that concentrates power into the hands

of a very small number of individuals, perhaps a single

dictator. And secondly, that single individual is or small

group of individuals or single country is then able to like

lock in their values indefinitely via transmitting those

values to artificial systems that have no reason to die

like, you know, their code is copyable.

Perhaps, you know, Donald Trump or Xi Jinping creates their

kind of AI progeny in their own image. And once you have

a system that's once you have a society that's controlled

by AI, you no longer have one of the main drivers of change

historically, which is the fact that human lifespans are

you know, only a hundred years give or take.

So that's really interesting.

So as opposed to sort of killing off all humans is locking

in creating a hell on earth, basically a set of principles

under which the society operates that's extremely undesirable.

So everybody is suffering indefinitely.

Or it doesn't, I mean, it also doesn't need to be hell on

earth. It could just be the long values.

So we talked at the very beginning about how I want to

see this kind of diversity of different values and exploration

so that we can just work out what is kind of morally like

what is good, what is bad and then pursue the thing that's

bad. So actually, so the idea of wrong values is actually

probably the beautiful thing is there's no such thing as

right and wrong values because we don't know the right

answer. We just kind of have a sense of which value is more

right, which is more wrong.

So any kind of lock in makes a value wrong because it

prevents exploration of this kind.

Yeah, and just, you know, imagine if fascist value, you

know, imagine if there was Hitler's utopia or Stalin's utopia

or Donald Trump's or Xi Jinping's forever.

Yeah, you know, how good or bad would that be compared

to the best possible future we could create? And my suggestion

is it would really suck compared to the best possible

future we could create.

And you're just one individual.

There's some individuals for whom Donald Trump is perhaps

the best possible future.

And so that's the whole point of us individuals exploring

the space together.

Exactly.

Yeah, and what's trying to figure out which is the path

that will make America great again.

Yeah, exactly.

So how can effective altruism help?

I mean, this is a really interesting notion they actually

describing of artificial intelligence being used as extremely

powerful technology in the hands of very few potentially

one person to create some very undesirable effect.

So as opposed to AI and again, the source of the undesirableness

there is the human.

Yeah, AI is just a really powerful tool.

So whether it's that or whether AI's AGI just runs away

from us completely.

How as individuals, as people in the effective altruism

movement, how can we think about something like this?

I understand poverty and animal welfare, but this is a far

out incredibly mysterious and difficult problem.

Great.

Well, I think there's three paths as an individual.

So if you're thinking about, you know, career paths you

can pursue.

So one is going down the line of technical AI safety.

So this is most relevant to the kind of AI winning AI taking

over scenarios where this is just technical work on current

machine learning systems often sometimes going more theoretical

to on how we can ensure that an AI is able to learn human

values and able to act in the way that you want it to act.

And that's a pretty mainstream issue and approach in machine

learning today.

So, you know, we definitely need more people doing that.

Second is on the policy side of things, which I think is

even more important at the moment, which is how should developments

in AI be managed on a political level?

How can you ensure that the benefits of AI are very distributed?

It's not being, power isn't being concentrated in the hands

of a small set of individuals.

How do you ensure that there aren't arms races between different

AI companies that might result in them, you know, cutting corners

with respect to safety.

And so there the input as individuals who can have is this.

We're not talking about money.

We're talking about effort.

We're talking about career choices.

We're talking about career choice.

Yeah, but then it is the case that supposing, you know, you're

like, I've already decided my career.

I'm doing something quite different.

You can contribute with money too, where at the Center for Effective

Altruism, we set up the Long Term Future Fund.

So if you go on to effectivealtruism.org, you can donate where

a group of individuals will then work out what's the highest value

place they can donate to work on existential risk issues with

a particular focus on AI.

What's path number three?

This was path number three.

This is donations with the third option I was thinking of.

Okay.

And then, yeah, there are, you can also donate directly to organizations

working on this, like Center for Human Compatible AI at Berkeley,

Future of Humanity Institute at Oxford, or other organizations too.

Does AI keep you up at night?

This kind of concern?

Yeah, it's kind of a mix where I think it's very likely things are

going to go well. I think we're going to be able to solve these

problems. I think that's by far the most likely outcome, at least

over the next.

By far the most likely.

So if you look at all the trajectories running away from our

current moment in the next hundred years, you see AI creating

destructive consequences as a small subset of those possible

trajectories.

Or at least, yeah, kind of eternal, destructive consequences.

I think that being a small subset.

At the same time, it still freaks me out.

I mean, when we're talking about the entire future of civilization,

then small probabilities, you know, 1% probability, that's terrifying.

What do you think about Elon Musk's strong worry that we should

be really concerned about existential risks of AI?

Yeah, I mean, I think, you know, broadly speaking, I think he's

right.

I think if we talked, we would probably have very different

probabilities on how likely it is that we're doomed.

But again, when it comes to talking about the entire future of

civilization, it doesn't really matter if it's 1% or if it's

50%, we ought to be taking every possible safeguard we can to

ensure that things go well rather than poorly.

Last question, if you yourself could eradicate one problem from

the world, what would that problem be?

That's a great question.

I don't know if I'm cheating in saying this, but I think the

thing I would most want to change is just the fact that people

don't actually care about ensuring the long run future goes well.

People don't really care about future generations.

They don't think about it.

It's not part of their aims.

In some sense, you're not cheating at all because in speaking

the way you do, in writing the things you're writing, you're

doing, you're addressing exactly this aspect.

Exactly.

That is your input into the effective altruism movement.

So for that, Will, thank you so much.

It's an honor to talk to you.

I really enjoyed it.

Thanks so much for having me on.

If that were the case, we'd probably be pretty generous.

Next round's on me, but that's effectively the situation we're

in all the time.

It's like a 99% off sale or buy one get 99 free.

Might be the most amazing deal you'll see in your life.

Thank you for listening and hope to see you next time.