Embedding search

I'm writing a post on why I'm so excited by embedding search.

It may have massive upsides and downsides. Also the ingredients for it seem mostly ready. So it'll be difficult to un-invent even if the downsides are greater. It seems better to adapt to a world where better embedding search is present.

A lot of distinct problems in life seem like they could benefit from better embedding search.

Finding a life partner
Finding funding or a job
Finding books, articles, etc relevant to your research
Establishing trust. Getting diverse information sources about a person helps build trust faster.

Some dual use implications:

Finding people who are trying to stay hidden because they are afraid of or competing with you or someone else. (This is dual use.)
Finding secrets leaked to the internet (or any private database) by powerful organisations. (This is dual use.)
In general, even things like finding funding or becoming better at research, etc accelerates everyone irrespective of what their goals are.

Attention markets on the internet right now are quite power-law distributed.

If you want to find people of a specific niche, often you have to first get attention of the mass public, and then filter out the subset of this that is interested in your niche. This is especially true if there doesn't exist an already well-established community with organisers and funders.
Attention markets are power law-distributed because there's atleast 1 million people competing for attention, but any random user has not more than 10 or 20 internet sources they regularly follow. (Sure, you might follow 1000 instagram pages, but you're not paying attention to all of them.) If you can't get into someone's top 20 follows, you're at a massive disadvantage when trying to get mass attention.
There aren't clear rules for how to give or receive attention. In practice this leads to an arms race, one side invents ever new techniques to spam all the platforms, and the other side raises the entry barrier to allow anything in.

Embedding search might (??) also help fix the attention market. For example, I could list 5000 people I'm interested in, and the other person could also list 5000 people they're interested in, and if there's overlap then we connect. At no point do either of us need to send a DM without knowing a good probability estimate of whether this is useful or spam for the other person.

Elites use filters

The more you wield a scarce resource such as attention or money, the more people are competing to get in your network or influence your thinking. And the more you need to be deliberate about what filters you use to let people in or out.
A lot of problems in society are causally downstream of principal-agent problems. Elites don't know who to trust and get fooled into doing massively suboptimal things, basically all the time.
Building better filtering mechanisms for elites seems useful.