Home | Search

2025-01-19

minor update 2026-02-12

Why have hope about fixing risks from ASI?

Disclaimer

Quick Note

Hope

Among the people I talk to who are convinced there are significant risks from ASI, some of them have given up hope on any attempt at fixing the problem. This post is for them.

I am generally not a fan of salesmanship and using self-fulfilling prophecies and stuff in order to motivate people, so this post won't contain that. Instead I will list reasons based on concrete evidence from the world that give me hope.

Most heads of AI companies and governments don't want to die. If they were convinced the risk to their own life is large, there is still non-trivial probability they might just all stop.
A majority of people in the US or world population don't want to die, and don't want a future with superintelligence either. Even a "benevolent superintelligence future", whatever that means to different people, currently appeals to only a small number of people. If they were aware the risk to their life is large, they might take action towards that.
Just because people are not convinced of the problem now, doesn't give you very strong evidence they will remain unconvinced when we are let's say, a year away from building ASI. When we are that close, there will be a lot more evidence that could cause people to take it seriously - there will both be more advanced AI integrated into the economy, and there will be a lot more other people terrified of ASI, making it less crazy for you to believe it as well.
Mass protests have succeeded many times before. Multiple governments across the world have been successfully toppled over via mass protests. I don't support toppling over the US govt unless absolutely necessary, but I am saying that if even toppling the US govt is within the realm of possibility, then pressuing the US govt to not build ASI is also within the realm of possibility. Govts are known to respond to mass civil disobedience, and to electoral pressures if they are sufficiently large.

Some examples specific to the US govt

After the Hiroshima bombing, many people in the US including John von Neumann, Leslie Groves, etc supported the US govt threatening to nuke everyone else and establishing a permanent nuclear monopoly, making the US govt the defacto world govt. President Truman rejected this plan. I think one factor for why he rejected this plan was that he knew there would be mass protests by the US public (much bigger than the protests already happening), he would lose his votebank, and he would have to make the US even less democratic to pursue this plan.
I think the Vietnam war was stopped sooner than it would have, if there were no mass protests against it.

Meaning

If your meaning-making works anything similar to mine, another thing that is helpful to remember:

The number of people working on plans that make sense to me is very small. (Probably less than 100 full-time people? Unsure of the exact number.) If you manage to do as well as these people or better, and if our species survives as a result, you will hopefully be remembered as one of the great men or women of history who made it happen. I definitely think Yudkowsky's name is going in the history books [1] of 2100, and probably a few more names will go there alongside his.

[1] Well sure, maybe we aren't using books to acquire knowledge then, maybe we will straight upload it via BCIs or something, who knows. But you get my point.

Enter email or phone number to subscribe. You will receive atmost one update per month

Comment