October 2024

How hard would it be to make a new search engine? One that isn’t operated by a big tech company.

Well first you would have to build all the software and pay for all the hardware

Then you would need to build up your index

That is where a problem appears, due to AI scrapers many websites now don’t allow any bots other than google bot and maybe bing bot so you would need to convince publishers to allow you on their sites

Actually doing it would cost crazy amounts of money before you even consider the convincing the sites to let you in thing

US10-$30 billion.

Google and Microsoft Bing, it said, are the only search engines that spend hundreds of millions of dollars annually to maintain their indexes.

That’s tens of billions of dollars

There has never been a startup that got that much in a single funding round much less a non corporate non government organisation that held that much money that isn’t the Catholic church.

I simply cannot imagine a successful open source search engine people want to use

Even duckduckgo often borrows results from Bing

Search engines clearly aren’t going away, people aren’t going to deal with link lists and they want to be able to search the whole web at once

What do you think?