What does the offence-defence balance for hyperpersuasion look like against arbitrarily superhuman minds?
Disclaimer
Thinking aloud, may not endorse after a lot of thought. Written at midnight and possibly forgot about it afterwards.
idk if this post is scifi or timepass or precursor to something useful
Main
Examples of situations where hyperpersuasion matters
Deciding if and how to keep a superhuman AI in a box
Deciding if and how to listen to radio signals from alien civilisations
Deciding if and how to send radio signals to alien civilisations
Why is hyperpersuasion so important?
Information travels at the speed of light, and using barely any amount of energy. Hypersuasion is in some ways a pure information problem, like pure math. Especially so if you can accurately scan and simulate the opponent mind inside your own, then finding memetic attacks against is a pure information problem.
In general, you will pick the best weapon against an opponent. And the best weapon is one with 100% success rate that requires zero expensive operations involving matter, one you can develop even if you're stick in an airgapped cluster, or heck, even if you're stuck inside a mind simulation running within a foreign mind.
Internal takeover
The most powerful ideas are like viruses, they take over their hosts. Do people have ideas or do ideas have people?
You can find yourself running a simulation of someone else's mind only to find your own mind become more and more like theirs as you continue along this process. Example - I have become a little bit more like Allen Dulles simply by thinking about him, as opposed to two years ago when I hadn't thought much about him.
Hardware for human brains is terrible, it's possible the takeover is even more rapid in machine substrate.
Like, for instance, the seven bits (supposedly) of working memory I have, and the cache memory (of few KB or MB) sitting most closely to the working memory - this seems important. Krashen input hypothesis trains what is sitting in the cache memory. But some ideas are memetic and make it easy for you to lodge them there and some ideas make it harder. For instance, mtg colour wheel has gone into my cache very rapidly, where mandarin chinese has not.
If I was running on a machine substrate and scannable, maybe you can literally gradient descend against my cache weighting algo and find good memes that will get stuck there more easily.
Hierarchies of minds
Minds may develop internal defences, such that even if you could hypothetically run them in a hostile simulation and gradient descend against them (or otherwise reverse engineer them) and so on, you find it hard to find memetic attacks against them.
You could end up with hierarchies of minds, where more powerful minds are capable of defence against such takeoever by weaker minds.
Basically there are black box attacks and white box attacks. You can end up with a hierarchy based on white box attacks.
See also: valentine and the hostile telepath problem
I'm sure someone can define some super theoretical version of this. For instance, kolmogorov complexity is for shortest possible turing machine that produces same output. You can define that turing machine A successfully "attacks" B, if when A is given B as input, and B is given as input, and A and B can also send turn-wise inputs to each other, then the net outputs of A and B look more like A or more like B, before the two machines were entangled in this way. You also need both A and B to be capable upto some level, to ensure you can't just have a dumb algo.
Like, being extremely dumb is a defence against memetic attack. A quicksort algo can't be taken over via hostile inputs in a way that a human mind can be taken over. It is generally more intelligent minds that can be taken over, maybe being a universal turing machine is required for takeover, I don't know.
(Lol some humans I know are as dumb as rocks, or in this case as dumb as a quick sort algo, this actually gives them some memetic defence. See also: Nerds are nuts, and what nerds and fundamentalist religious people have in common.)
It's funny going back and reading some alt's comment, realising you like it, then realising okay, holy shit, I was the one who wrote this comment.
See also: old scifi stuff I wrote as a teen, be it the inception fanfic or the other time hole thingie. I have clearly been obsessed with the idea of hypersuasion for very long
Subscribe
Enter "Subscribe: " followed by Signal username, email address or whatsapp phone number. You will receive atmost one update per month. View previous updates here.
Alternatively, you can use RSS, or a third-party service like blogtrottr.
Comment
Enter "Comment: " followed by comment. Alternatively, contact me