Home | Search

2026-05-05

Marriage and shared preference orders

Disclaimer

Quick Note
Thinking aloud, may not endorse after a lot of thought

Disclaimer - I think preference orders are stupid

I find the whole cluster of beliefs around "utilitarianism" and "preference orders" and similar, atleast a little bit stupid. Looking back at my really old notes, I seem to have found it stupid since I was 16 years old itself. Sorry to any EA/LW people who are mad about it. (Note to self - why are you running their mind sims here in the first place? clearly their writings on this topic have hurt you atleast a little)

The simple reason for why is that humans are not optimal vnm agents, and there is no reason for humans to try to be optimal vnm agents.

How to exploit a suboptimal decision-theory agent, who doesn't understand vnm theorem

Make them run in circles going from town A to town B to town C, while constantly spending money until they either a) go bankrupt b) do self-reflection, realise they will eventually go bankrupt, and self-modify their values to be more consistent

How to exploit a human

Wait till they suffer. For example, wait till their wife or parent dies, or they get divorced, or similar.
Then shamelessly target them while they are vulnerable, and offer your religion/ideology as a solution to their suffering.
Your ideas might actually help them with their original psychological wound, but they are also likely to mindlessly copy-pasted another 10 belief sets from that ideology, that may not even be related to this original wound.

If you are a human, you should be more concerned about being vulnerable to the second type of exploit, not the first one. There are lots of people who will target you as their next victim for the latter (many spiritual leaders for instance), but not that many people who will use the former.

(Sometimes, people cite casinos as an example of vnm dutchbooks that operate in real world (not thought experiment), but I think casinos exploit human psychological potential for addiction, a lot more than they exploit humans being bad at EV calculations or decision theory. "Addiction" is again an exploit that works well on humans but not on decision-theoretical agents.)

Side Note

If the suffering was due to loss of relationship, offer them a relationship-centred ideology. If the suffering was due to physical violence on them, offer them a violence/nonviolence-centred ideology. If the suffering was due to loss of money, offer them an ideology centred around poverty and similar.

Disclaimer - I think too much theory is stupid, practice is what matters.

I think practice not theory is what matters, revealed not stated preferences are what matter, and most theory gets discarded when it makes contact with reality.

This includes this post or similar posts I'm writing on my blog. I will be completely unsurprised if future Samuel encounters some intense suffering and abandons atleast 50% of the theoretical abstract writing on this blog.

My preference order

Given these disclaimers, I will share my preference order anyway. I am like, 75% consequentialist, not 100%.

My legacy
My survival
My friends' / family members' survival

My legacy is stuff I am literally willing to die for.

I am willing to sacrifice my life to ensure technological progress goes well.
I am willing to sacrifice my life to eradicate a false ideology from the world, and especially so if this false ideology threatens a promising technologically advanced future.
Most urgently, I am willing to sacrifice my life to stop ASI.

Myself is just basics I need for survival.

I am willing to fight for my survival. I need basic food, water, physical safety, etc to survive, and I am willing to fight pretty hard to get these, if I feel highly constrained on them.
When put under life-or-death circumstances, I care about my own survival than I care about the survival of any of my friends or family members. I am only able to care about them and offer them excess money/time/attention/etc, because I am not in an immediate life-or-death situation myself.

My friends / family members

Self-explanatory
I obviously care about friends and family members more than I care about strangers. I think any ideology that causes you to kill a loved one to save two strangers, not in some thought experiment but actually in practice, is really stupid. (Looking at you, utilitarianism.) I don't currently want to waste time criticising this.

Community

I don't care about any community much at the moment. I anyway seem to do well with 1-to-1 friends and poorly with friend "groups". Many of my friends literally don't know each other, and I've met them in completely different circumstances.
Friend groups in general seem to steer away from truth-seeking and towards boring common interests, and that may be a factor for why I prefer 1-to-1.
In theory atleast, this could change in future and I could come to care about some community.

Nationality

I don't care about any particular nation.
I actually wrote a long para here, which I later realised I should delete because there are actual safety risks to me to publish them.
I will put just one line here. If India in future has any leader who actually carries forward Nehru's atheist pro-technology mindset, I might consider myself a part of said group of people or care about the nation's interest. As long as India's core values are either hindu or muslim, I don't consider myself a part of this group, or feel any true desire to prioritise my nation over others.
If I end up prioritising India's interests, it will probably be surface-level instrumental considerations, rather than actually deeply caring about India or Indian values or Indian culture or similar.

Could my future partner change this preference order?

One way this preference order could change is if I end up caring about someone enough that I am willing to sacrifice my life for them. Right now 2 > 3, but this could change for someone, atleast in theory. For instance, my future partner.

However, the following preference order seems unstable?

My legacy
My partner's survival
My survival

What if my partner has her own values that she cares about, beyond mere survival? Where do they fit into this preference order?

My future partner could have her own legacy or footprint she wants to leave on the world, that she is willing to herself die for.
My future partner could have loved ones she cares about more than herself, and is willing to die for.

One solution to this is just to break up

When times are good, I could stay married. I could work to further my legacy, and she could work to fulfill whatever or whoever she cares about, and we could also spend time together.
But, in time of crisis, where I am forced to choose either her preferences or my legacy, or she is forced to choose either her preferences or hers, then we break up.
To be clear, I think this solution is suboptimal but also not like, horrible.
Many people end up with marriages that break apart in 5-10 years, and some of them do still seem happy in life.
My guess is that being forced to make tradeoffs like this is one possible reason for why people divorce. (I need more data here, I don't trust my current limited data to form strong judgments based on it.)

Initial seed versus value transitions

You can control this somewhat by picking the right partner

You could avoid picking a partner who cares more about their family members than themselves.
You could avoid picking a partner who cares more about some legacy than themselves.
Basically, pick a partner who is boring and a little submissive lmao.

I think atleast some politicians/billionaires basically end up picking a wife like this.

See also: Melinda Gates, Mackenzie Scott
- Side Note: Some people think Mackenzie Scott is like some heroic altruistic person for donating Jeff Bezos wealth. And like yes, this is more interesting move in life than just sitting on the wealth for life and doing nothing. But also, I would put a lot of probability mass on the following hypothesis: "Mackenzie Scott wants a boring life, and doesn't want the psychological stress of being a billionaire, or of trying to reshape the world in your image. Donating all the wealth was an easy way for her to go back to her boring middle-class life."

This solution does not solve for value transitions though.

Both your and your partner's values could change, after you have already been married for some years.
This strikes me as a noticeably harder problem than just picking a wife with the right values on day one of your marriage.

Maybe shared legacy can help?

If you are willing to sacrifice yourself for legacy A, and your wife is willing to sacrifice herself for legacy B, and A and B have some overlap, maybe this overlap can help?

It then becomes very important to understand what A and B are, and how those could transition over time.

My current answer: "Stable political systems for technologically advanced future"

This is far too abstract a legacy to base your marriage on, the lack of details will 100% break things when you run this experiment in the real world.
For instance, if I want to be immortal and have 100s of children, and my wife wants a boring life and wants to die in old age, this causes a big difference in our desired legacies. I might end up spending all my effort trying to become immortal, and she might want my effort to be spent in a different direction. Or this could even worse, she could end up actively opposed to the pursuit of immortality and see it as destructive for civilisation, and we could literally end up political enemies and divorce over this.

This is a Wicked Problem

Just staring at this problem fills me with pessimism

It reminds me of what Holden Karnofsky (the pro-ASI mass murderer) calls Wicked Problems
There's no way I can zero-shot solve this problem in isolation using theory alone.
This will require a bunch of iteration over time to solve, lots of people will have to divorce due to their differences in values over artificial superintelligence or human genetic engineering, for the next generation to learn about this and do better. (Assuming the next generation even exists)

On the flip side, maybe this is a strategy to break the pro-ASI people?

If this problem is that Wickedly hard, this means a lot of pro-ASI people also are in basically short-term marriages based on flimsy moral foundations. (They might protest this, but don't trust their words, only their actions.)

If I remind their partners about this, I could actually be able to break apart some of these marriages, using words alone.
Actually, words alone may not be enough. I will have to engineer a crisis in which it becomes apparent to both people in the marriage that they have a difference in values.
TO DO - Think more about what such a crisis looks like?
- For instance, maybe the pro-ASI software developer expects to be uploaded but his wife wants biological children. If you create a lot of advertising to remind the wife that she wants children, then she might eventually divorce the developer over this.
- (There's also more violent strategies here, like, literally poison the wife with food products that reduce her fertility window, and force her to take this decision sooner. Not going into detail here.)
- Maybe the pro-ASI developer's wife has parents who are going to die soon. She wants to spend time with them or spend money on their medical treatments, but the pro-ASI developer thinks the Singularity will solve all this. If you remind both people about this value difference, maybe they will divorce each other. Or in particular, if you remind the wife that the developer doesn't want to spend money on said medical treatments.
- There is a very boring answer that totally could work on non-zero number of people, which is to honeytrap some of the single pro-ASI developers with women who are just going to make their life hell in a bunch of ways. For instance, find them a lover who is addicted to hard drugs or emotionally manipulative or similar.
- (Update - Holy fuck, I can actually imagine stuff like this working, atleast on some people. Maybe I should test it out?) Like, actually write out a post "why you should divorce your pro-ASI partner" and go cold DM it to some of these women. Will think a bit more before doing this.
- See also: Otto van Bismarck's comments on marriage, Allen Dulles had casual sex with hundreds of women

I don't think this sort of strategy is sufficient to stop ASI by itself.

There are enough single people who will happily (or unhappily) stay single and build ASI anyway. Who cares about marriages that last 30 years when the Singularity is 3 years away? Having sex is not a mandatory prerequisite to writing pytorch.
There are also atleast some people building ASI who are self-aware enough to realise their relationships are short-term and won't survive the route to the Singularity. So like, they basically feel single inside the privacy of their mind, but they have someone to have sex with, that's it.
- (This is important. It is entirely possible to feel alone, even if you have people around you, even if you are married.)
I can also totally imagine a few people continuing to build ASI as an act of revenge against me (or anti-ASI people like me), if I manage to successfully break apart their marriages.

Side Note - Maybe I should study people's personal lives full-time?

I explicitly studied personal lives of previous whistleblowers.
I have implicitly searched personal lives of all software developers turned activists, I have atleast one public note on this.
I have atleast one public note on personal lives of SV billionaires.
I have paid atleast a little bit of attention to personal lives of both anti-ASI and pro-ASI people.

Honestly I should do a bunch of GPT-5.5 searches and post those in public at the very minimum, even if I don't have time to properly research all these people myself.

Will use my standard tricks to fragment the task and avoid gpt-5.5's detectors.

Enter email or phone number to subscribe. You will receive atmost one update per month

Comment

Enter comment