Home | Search

2026-03-14

Update

• See also: Amjad Masad on Beware the Metagame
• add the caveat that ASI breaks all these silicon valley billionaire heuristics, and I don't have much confidence any of these people have a plan for ASI either. ASI risk is a bad-feedback-loop problem and maybe going a little bit more meta could make sense.
• I go too meta too often though, so I still need to the nudge of, go less meta, play the actual game, not the metagame.

2026-03-06

How does thinking about my morality cash out into real world actions as of 2026-03-05?

Disclaimer

• Quick Note
• Contains politically sensitive info
• Incomplete - some sections marked think more

Main

• I increasingly seem to get stuck in nerd snipes that are unproductive as opposed to productive, and waste my time. Hence I commit to asking myself once a day, how does this nerd snipe cash out into real world actions?
• For thinking about morality, as of 2026-03-05, here is the answer:
  • Most important
    • I can stay long-term motivated to work on the leaking/hacking cluster of projects only if have clearer moral values. I find myself literally unable to work otherwise.
    • (update 2026-03-06) It is true that if I pursue a tight redaction policy, I will not get many people killed, and then I can sidestep all the moral questions by saying damage of our plan is low. However, leaving internal conflict around other questions unsolved still feels bad, and maybe this internal tension will still cloud my judgment? I don't know, will look at this tmrw.
  • Also
    • The primary way our morality will express itself is in terms of redaction policy, and to a lesser extent, in terms of what attack targets we choose. Morality is connected with anti-ASI political ideology which is connected with redaction policy. Who deserves privacy on what, and who does not? It is better if there's a coherent anti-ASI ideology that determines this, that many people believe in, and we don't have to invent an entire ideology ad-hoc. Just like communism or libertarianism or similar is a coherent political ideology that can answer moral questions in the real world. I have to personally decide a redaction policy I feel okay with, either alone, or with a future cofounder.
      • I currently have a redaction policy I seem okay with, but it is ad-hoc, and I would prefer a coherent political ideology shared with others, over something ad-hoc, if it can be invented quickly.
      • Increasingly feeling like this is a dead end, as in ROI on investing time in inventing new ideology is not there. Just stick to your ad-hoc redaction policy and make sure you can rewire your moral values to become okay with this.
    • I have to personally go meet irl or cold email or make posts online, to get hackers on our team. As the situation gets more grim, we will (hopefully) increasingly become capable of recruiting the best hackers in the world. Doing so requires alignment of my values with theirs, or I have to break these people psychologically and get them to change their values. This again requires some coherent value system and associated redaction policy I believe in.
    • One of the biggest failure modes of this entire plan is that we will get assimilated into Indian or Russian intelligence agencies, and have no freedom of our own. This is very bad. The best hackers in the world are not nationalist, and will not work for any org driven by nationalist ideology. I am also not nationalist, and am quite uninterested in working in any org driven by nationalist ideology. Our redaction policy cannot be decided by nationalism. For instance it would be bad if we just have to hand over all capabilities research that we steal, to Indian or Russian intelligence. (Wait, is that really so bad? These people are too incompetent to run an ASI lab anyway. To do - think more) We don't want to waste time on attack targets or projects that make sense from nationalist pov but not anti-ASI pov. (Like what? Spell out clear examples. To do - think more)
    • Avoiding this failure mode also requires me to have a clear value system that can attract competent hackers, who will even be willing to risk their life to say no to the Indian or Russian intelligence when necessary.
    • Also, I can get favours from friends, allies, etc more easily if our values align. Some people will be too scared regardless because they're afraid of law (and don't wanna work fulltime on ASI risk for whatever stupid reason), but some might help. (I already have real world examples of this, cant share for privacy reasons.)

Subscribe

To subscribe, enter "Subscribe " followed by Signal username, email address or whatsapp phone number. You will receive atmost one update per month. View previous updates here.

Alternatively, you can use RSS, or a third-party service like blogtrottr.

Comment

Enter comment