2026-11-11
Retrospective - building search API for ethereum blobdata
Disclaimer
- Quick Note
- Contains politically sensitive info
Main
- A couple years back in 2024, I spent (or maybe wasted) ~9 months of my life building open source search engines. In particular, I completed embedding generation and search for the full text content of ~700k books entire libgen. I also did some preliminary work on what searching entire CommonCrawl using an open source search engine would look like. (I did not publish all my work, but I published most of it.)
- I ended up concluding that this work is high value to do, but maybe not the literally highest value thing to do to stop ASI risk. Hence I abandoned it.
- More recently in 2026, my risk profile increased (because I am now openly studying hacking, with activist goals). I got banned by twitter, lesswrong and github. I took some steps to improve opsec, such as using grapheneOS, and deleting apple and google accounts. Hence I again got nudged to look more into open source search engines and distributed social media. I backed up my website's content to ethereum blobdata, and looked into what building a search API over ethereum blobdata and ENS would look like.
- I have again ended up conclusing that this work is high value to do, but maybe not the literally highest value work to do to stop ASI risk. Hence I will be abandoning it. This time I would like to not make the mistake of wasting ~9 months of my life, and I would like to abandon this project much faster.
If you are not persuaded by the case for ASI risk but are still looking for something high impact to work on, I still think the above is one option for what you could work on. The two most censorship-resistant places to store content today are eastern european VPS and ethereum blobdata. Search and discovery for both of these places is not very good.
Side note
- Huge shoutout to erigon dev team for building erigon. Erigon supports snapshot sync by default, they support MCP by default, they are more performant. These are such big UX adds that they made the difference between me being able (and motivated) to run any experiments in this space at all, versus not.
Subscribe
Enter "Subscribe: " followed by Signal username, email address or whatsapp phone number. You will receive atmost one update per month. View previous updates here.
Alternatively, you can use RSS, or a third-party service like blogtrottr.
Comment
Enter "Comment: " followed by comment. Alternatively, contact me