2026-03-04
updates 2026-03-05
Is my interest in fixing ASI risk out of self-interest or altruism?
Disclaimer
- Quick Note
- Trigger warning - murder, suicide
- Contains politically sensitive info
Main
- I have often wondered if my obsession with fixing ASI risk comes from self-interest or altruistic interest.
- I've gone back and forth on this multiple times with no clear conclusion.
- Many life philosophies explicitly demarcate the two, so it seems important to figure out which of these is driving me more. (Assuming a comparison like this is possible.)
- Here's yet another of my hypothesis around this topic:
- Back in college I started gaining some interest in being altruistic, only after my basic needs had been met. For instance, having enough friends and having enough money and having freedom and so on. I am self-interested by default, but once all of those basic needs are satisfied, then I seem more interested in altruism.
- Nowadays, altruism for me increasingly means steering the world in a direction I like, which could be different than the direction you like. I could totally screw people over while still being altruistic. It just means that underlying motivation was vaguely to help some set of people or the other. You cannot help everyone.
- However, now that I am internalising that I personally might be murdered by the people at the AI companies, I am again adopting an increasingly self-interested stance. Saving my own life is primary, saving other people's lives is secondary. I think I have have non-zero amount of care for other people's lives too, but I think saving my own life is now way more important.
- There are some edge cases where I have flirted with the idea of sacrificing my life to prevent ASI risk. I think most of all I want to steer the future in a way I like. I have understood a lot more, how zero sum politics is, and how I will necessarily have to screw over people even if I want good for a lot of people.
2026-03-05
Update
- More thoughts:
- Even before I knew about ASI risk, I have often asked myself thought experiments like:
- Q1: For what dollar value $N am I willing to kill a loved one?
- Q2: How many strangers (M) am I willing to kill to save my own life?
- Q3: How many loved ones am I willing to kill to save my own life? Is there any loved one for whom I would sacrifice my own life?
- For any lesswrong fuckers reading this, I am not an impartial utilitarian. I obviously care about myself and loved ones more than strangers, and this continues to hold true after a lot of reflection.
- To protect privacy of people in my social circle, I have to remove a lot of details around which people in my circle, and what I thought when, and what specific events in my life led to what thinking, and so on.
- Some thoughts on answers:
- A1: At one point I thought the answer to first question (dollar value to kill a loved one) was $10M. I soon realised this number was too low. I would be taking a hit to both my self-esteem and my reputation, it could be irreversible and prevent me from acting as effectively in future, and hence $10M is too low a number.
- A2: I am definitely slaughtering atleast a hundred strangers to save myself. It is only when this number goes into actual billions of people that it becomes more clear to me that I'd rather sacrifice myself. I don't know what the answer is if I was faced with a decision involving a few thousand or a few million people.
- A3: I have a lot of personal experiences here, but, to protect privacy of people around me, I can't talk about any of it. The really short version is that I don't think there's a single person in my life where I can confidently say I would sacrifice my life to save theirs.
- I have increasingly moved away from thought experiments to answer such questions though:
- See [Power buys you distance from the crime by Elizabeth]. It is easy to press a button or order your subordinates to kill people. It's much harder to pick a gun or knife and kill them yourself.
- It is easy to make a quick action overnight. (Just like, say, it is easier to commit suicide on impulse overnight, than to actually think it through and do it.) It is much harder to spend a few years actually systematically planning to kill people. The latter requires a lot more internal moral clarity and conviction. When you run a thought experiment, you're often trying to simulate the former not the latter. So, not only is it difficult to get accurate forecasts when you run this simulation in your mind, but you're also trying to simulate the wrong thing.
- At the very least, try using thought experiments where you have to spend a month planning the murder, and you have to commit the murder with a knife, not by pressing a button.
- Coming back to ASI risk:
- Now that the stakes are literal billions of murders, it's again looking increasingly promising for me to justify instrumental harm to stop this.
- I am definitely willing to take a knife and kill tens of strangers and even some of my loved ones, if I could guarantee saving my own life in the end.
- But, when I am contemplating civilisation-level moves like blowing open everyone's secrets all at once (this could get millions of people killed), I seem a lot more hesitant. Killing these many people doesn't seem worth it, if the only goal is saving my own life. It may be worth it, if the goal is saving other billions of people. I am again forced to take a more impartial detached stance here, when contemplating such big moves.
- I still seem increasingly okay with planning a hundred simultaneous assassintions to save my own life. Especially since I will not actually be assassinating any of my loved ones here. I don't need to be altruistic to endorse something like this. The reason I am not planning a hundred assassinations seems more to do with this plan being too hard (compared to other plans), not because I have some moral principle against it.
- There's also the question of whether I am willing to spend my life in jail, just to succeed at the hundred assassination plan. I think the answer is yes. If the goal was just saving my own life, it's not so clear. But since we will save literal billions of people, then yes, I can sacrifice my life.
- I continue to love the comments in Inside the acquarium, that talk about how the biggest mass murderers in history (like Stalin and Mao) think of themselves as doing a great good. They might treat their loved ones well and even animals around them well. I think there is a deep truth to this. If I end up killing millions of people in real life, it seems a lot more likely that I was motivated by some altruistic greater good logic, than I was motivated by saving my own life.
- Very interesting - does this basically mean I am applying utilitarianism but I give my own life the moral weight of 100 lives, and some of my loved ones get a moral weight of maybe 10 lives?
- I don't know, I don't want to force fit a theory here, before I collect enough raw data I actually trust. As mentioned above, thought experiments are a dog shit method of collecting raw data, only actual life experience is trustworthy, but that is a fucking slow process. Need to move faster to speed run this.
Subscribe
Enter email or phone number to subscribe. You will receive atmost one update per month.
Alternatively, you can subscribe via RSS, or a third-party service like blogtrottr
Comment
Enter comment