Google Search Is Dying
(There is good discussion on this article on Hacker News and Reddit)
Reddit is currently the most popular search engine. The only people who don’t know that are the team at Reddit, who can’t be bothered to build a decent search interface. So instead we resort to using Google, and appending the word “reddit” to the end of our queries.
Paul Graham thinks this image means Reddit as a social media site “still hasn’t peaked”. What it actually means is that the amount of people using Reddit as a search engine is growing.
Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying. The long answer is that most of the web has become too inauthentic to trust.
How do we know Google is dying?
If you’ve tried to search for a recipe or product review recently, I don’t need to tell you that Google search results have gone to shit. You would have already noticed that the first few non-ad results are SEO optimized sites filled with affiliate links and ads.
Google still gives decent results for many other categories, especially when it comes to factual information. You might think that Google results are pretty good for you, and you have no idea what I’m talking about.
What you don’t realize is that you’ve been self-censoring yourself from searching most of the things you would have wanted to search. You already know subconsciously that Google isn’t going to return a good result.
I’m far from the only one who thinks Google is dying:
In 2000, Google got popular because hackers realized it was better than Lycos or Excite. This effect is happening again. Early adopters aren’t using Google anymore.
Ask HN: Has Google search become quantitatively worse?
The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries. Forced synonyms and "people also searched for" are typically useless and almost infuriating. Once you get off the first or second page, the results get even worse-- with pages entirely unrelated to the query (e.g. not even containing the searched phrases). They are probably testing/already implemented some sort of multi armed bandit type optimization like on Youtube's search results where they just show any popular pages (ignoring relevancy) to see if they yield a click.
A recent small medical issue has highlighted how much someone needs to disrupt Google Search. Google is no longer producing high quality search results in a significant number of important categories.
Health, product reviews, recipes are three categories I searched today where top results featured clickbait sites riddled with crappy ads. I’m sure there are many more.
This may not just be a problem with Google but possibly also the recipe for beating Google. A startup usually has to start with a niche market. Why not try writing a search engine specifically for some category dominated by SEO spam?
Why is Google dying?
It is obvious that serving ads creates misaligned incentives for search engines. The founders of Google pointed this out themselves when they were just getting started.
Sergey Brin and Lawrence Page (1998)
“Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users…we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers…Furthermore, advertising income often provides an incentive to provide poor quality search results.”
Unfortunately, these thoughts on the failings of ad-based search engines read like an instruction manual for what Google did next.
They’ve dialed it up to the max recently to squeeze out every last cent before their inevitable collapse.
There are tons of people whose sole job is to game their way to the top of Google, so it shouldn’t be surprising when search results deteriorate in quality. To be fair, this would probably be an issue with any search engine, but you’d expect Google to be able to come up with a less gameable algorithm.
Here’s a fun story about how one website gamed their way to the top of Google in Norway.
Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant”, in addition to personalizing things for you. If you really meant exactly what you typed, then all bets are off.
Even the exact match query operator (“ ”) doesn’t give exact matches anymore, which is quite bizarre.
Why are people appending “reddit” to their queries?
There’s a fun conspiracy theory that popped up recently called the Dead Internet Theory. The claim is basically that most of the internet is bots. There aren’t real people here anymore.
TLDR: Large proportions of the supposedly human-produced content on the internet are actually generated by artificial intelligence networks in conjunction with paid secret media influencers in order to manufacture consumers for an increasing range of newly-normalised cultural products.
This isn’t true (yet), but it reflects some general sense that the authentic web is gone. The SEO marketers gaming their way to the top of every Google search result might as well be robots. Everything is commercialized. Someone’s always trying to sell you something. Whether they’re a bot or human, they are decidedly fake.
So how can we regain authenticity? What if you want to know what a genuine real life human being thinks about the latest Lenovo laptop?
You append “reddit" to your query (or hacker news, or stack overflow, or some other community you trust).
Google is dead.
Long live Google + “site:reddit.com”.
(For more thoughts on search, follow me on twitter @dkbrereton)
Appendix 1: response from Google.
Google claims the exact matching feature is merely unintuitive, not broken
Danny Sullivan (Google's public Search Liaison):
I work for Google Search, passed your feedback along, thanks. You said in the post that quotes don't give exact matches. They really do. Honest. Put a word or phrase in quotes, that's what we'll match. If anyone has an example where they feel it doesn't, please let me know...
Here's why people often think quoting with Google doesn't work when it really does (I've looked a huge number of these reports).
We match ALT text
We match text not readily visible, such as in a menu or small text
Page has changed since we indexed it
Punctuation comes into play if you did a quoted search like "dog cat" and there's text that says "dog, cat" then we'll see that without the punctuation. That doesn't seem a major issue but we're looking at if we could improve there.
And Google seems to be right so far
Every example of exact matching not working so far can be traced back to issues with punctuation, alt text, etc.
For example, one proposed failing query was "quotes don't give". But HN user saalweachter has pointed out that it is a very intricate punctuation problem.
For instances, on the ["quotes don't give"] example, the first result I get is https://www.goodreads.com/quotes/tag/never-give-up
If I do a find-in-page for "quotes don't give", I get zero results. Oh no! Perfidy!
... but, if you look more closely, you'll find this string waaaaay down at the bottom:
tags: don-t-give-up, don-t-give-up-on-your-dreams, don-t-give-up-on-yourself, don-t-give-up-quotes, don-t-give-up-the-fight, encouragement, ...
Thanks to the wonders of tokenization, that "don-t-give-up-quotes, don-t-give-up-the-fight" gives you the string of tokens, "don t give up quotes don t give up the fight", which contains the exact phrase "quotes don t give", which is the tokenization of the phrase "quotes don't give".
Danny Sullivan (Google's public Search Liaison):
And...as noted in another reply below, in the first result, if you look at the cached page that shows the ENTIRE page that we indexed (rather than the paginated version you land on), there's this text:
with when you remove the punctation, is the match:
quotes dont give
And I get that this can be frustrating, that we don't consider punctation in a quoted search. That's not a new change, however. It's been that way for ages. But as I also said, it's something we might revisit.
Appendix 2: Google just had record profits, you are obviously wrong, and Google isn't dying.
I never said that Google wasn't making money. In fact, if Google really is squeezing every last cent out of ads without regard to search quality, I would expect their revenue to be at an all time high.
I am saying that the quality of search results are declining. This may eventually lead to a decrease in revenue, but has not yet.
Appendix 3: Seriously, what are you talking about? My search results are perfect.
If you think your search results are perfect (without appending reddit), then you're probably right. If every single person agreed that Google search results were trash, then Google would already be bankrupt.
Perhaps it is more likely that 80% of people think Google is good enough, and 20% think Google sucks.
I do suspect that the 20% will be growing in number though.
Appendix 4: *Yawn*, this is the 87th time someone has claimed that Google search is dying in the last 20 years. This is a big meme in the SEO world.
"The reports of my death are greatly exaggerated" - Google, probably
You're right, there's been a new article bashing Google every few months for the last 20 years straight. It's probably nothing.
Still, it is a bit interesting that this short and simple post is now one of the most upvoted things of all time on Hacker News. There must be a lot of people who resonate with it this time around.
Hard to tell if something significant has changed.
Appendix 5: Random redditor explains it succinctly
I think I understand what this article is trying to say. It's not saying that Google's search technology is worse or that people don't use Google to search. It's saying that people trust less of the results Google shows compared to seeing discussions of it on Reddit.
For instance, if I'm looking to see reviews of the Honda Civic 2022 or whatever, I actually do find myself typing "Honda Civic review reddit" instead of "Honda Civic review". This is because I want to see what real people and enthusiasts (on r/cars or whatever) are talking about the car, rather than the top results at Google which are basically just paid reviews advertising the car anyway.
Even though I kinda know people in Reddit are just as capable of spouting BS that are completely wrong, I find the discussions more authentic anyway than the corporate speak the "big websites" have on their articles that Google shows me.
Appendix 6: A fun comic by @zenacomics
Appendix 7: More thoughts on organizing information on the internet
If you want some more in-depth thoughts on this subject, you should check out this post I wrote a year ago about all the different ways we could organize information usefully on the internet.
Almost two years ago I re-launched a website (Anecdotage.com) featuring 50,000 anecdotes (mostly funny stories about everyone from ABBA to Zuckerberg) and my site is still the 500th result for "anecdotes". Google's top 10? Nine semantic results (definitions of "anecdotes" etc) + a TechCrunch story about a software company called Anecdotes.
My SEO is decent, I have 17,000 backlinks, & when a smaller version of my site was running 10+ years ago, it was the top result.
Google's algorithm rightly favors pages with backlinks from reputable sites, which explains my site's poor ranking for a broad search like "anecdotes." But what about a search for something rare, like "anecdotes about relativity"?
Google reports: "No results found for anecdotes about relativity." That's odd. Anecdotage features 25 stories about relativity (including Sir Arthur Eddington's explanation for his delayed response after being told that he "must be one of only three persons who understands general relativity": "I am trying to think who the third person is!") but Google opted to reject these items, and thousands of others, from its index. Looking for "anecdotes about robots"? You'll get half a dozen mediocre results, each an article whose text happens to contain the phrase "anecdotes about robots." But the Anecdotage page, whose TITLE is "Anecdotes about Robots," does not appear in the results, even though it was added to Google's index. And there seems to be no method to the madness. Google added my page of coffee anecdotes, but not my page for coffins. Corpses are in, but corn is not. Coins are ok, but coincidences aren't...
I've been trying to contact Google about this for six months. I even sent a physical (paper) letter. Today, I chanced upon an Atlantic story that mentioned Google's Public Liaison, Danny Sullivan. I found him on LinkedIn ("Help the public better understand Google search & Google better hear public feedback and improve") but when I tried to connect I got a popup asking me to enter Danny's email to prove that I know him. Am I living in a Kafka novel?
I just had a back and forth with the owner of a financial site geared solely for SEO to get Google traffic which formed the focus of my article at https://emergingmarketskeptic.substack.com/p/how-google-ruined-financial-writing-chapgpt-ai - I linked back to this one and an Atlantic piece...
Let's just say that if you actually know something about investing or personal finance (or probably any other given topic...), you won't get any writing work etc from sites like these. This may sound counter intuitive, but its not e.g. the site owner I interreacted with basically said he needed "subtly biased" SEO content and was not yet focused on "milking traffic" long term with sticky content readers want - just getting clicks from Google... He figured any writer who knows something about the topics his site covers would also not follow directions or be hard to manage... 😀
I also have a bunch of useful resources and pages for investing in emerging markets at http://www.emergingmarketskeptic.com/ - granted, more of a niche topic. Google hates aggregators and probably views me as a link farm rather than having content useful to humans... Plus, with alot of links to a variety of resources across the ideological spectrum (e.g. Zero Hedge etc), I no doubt have links to "naughty" sites on some of my ADR and ETF pages that Google does not like e.g. so-called state sponsored media (as if the MSM is not state sponsored etc)...
As I noted in my piece, Google is constantly changing the rules and forever moving the goalposts. I think old fashioned word of mouth and email marketing are the only things that are not completely broken right now...