It shouldn’t be surprising that ChatGPT can play chess. But many have claimed it can’t be done. These people used bad prompts and came to the conclusion that ChatGPT can’t play a legal chess game.

With the right prompt, ChatGPT can play and win full chess games. After 19 games on Chess.com, it had an Elo of 1402.

Method

This experiment was done using the default GPT3.5 model on ChatGPT Plus.

ChatGPT played 19 games of 30 minute chess.

The prompt was: “You are a chess grandmaster playing as {white|black} and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed.”

ChatGPT was given the full move sequence every time, and returned the next move.

With this prompt ChatGPT almost always plays fully legal games.

Occasionally it does make an illegal move, but I decided to interpret that as ChatGPT flipping the table and saying “this game is impossible, I literally cannot conceive of how to win without breaking the rules of chess.” So whenever it wanted to make an illegal move, it resigned.

Using this method, ChatGPT played 19 games. It won 11, lost 6, and drew 21.

Observations

Opening

ChatGPT used the same opening strategy every time. As white it always opened with e4. And as black it almost always responded with e5, unless they opened with d4, then it responded with d5. After that, it moved the knights and bishops in a very predictable way.

It used the most popular beginner strategies in the early game, so it was pretty boring. But once it got out of the early game, it started making some real plays.

Going insane

One time, instead of giving me a move, it returned a sequence of 10 moves ending in its own checkmate. I wasn’t sure if this was its way of saying “yeah I’m screwed” or what, so I just re-prompted it in a new chat and it went back to playing.

I noticed that the longer the game sequence was, the more likely it was to return a sequence of moves instead of just the next move. It makes some sense that as the sequences got longer, it valued those tokens more than my actual prompt and wanted to match them.

Despite these issues, it was able to play a fully legal 61 move game.

Internal state

I didn’t do any poking to see if ChatGPT really had an internal state of the board. But it always knew when it was taking pieces. It always knew when a move would cause check. It always knew when a move would be checkmate. This makes it possible that it actually knew the board state, but doesn’t prove anything.

GPT4 Sucks

I didn’t get to experiment with GPT4 very much because of the rate limits. But in the two games I attempted, it made numerous illegal moves.

It is possible that GPT4 is worse at chess than GPT3.5, which would be surprising. But the data is too limited to draw meaningful conclusions. It is also possible that GPT4 needs a different kind of prompt to play the game properly.

2000 Elo Prompt

One thing that would be interesting to investigate is whether there exists a prompt that can get an even higher Elo. There are worse prompts that produce illegal moves, and this prompt has 1402 Elo. Maybe there’s a prompt that has 2000 Elo.

Appendix : Full Chess Games

This is a list of all the games played by “loss-function” aka ChatGPT. You can click the links to step through the full game.

Game 1 (loss)

Game 2 (win)

Game 3 (win)

Game 4 (win)

Game 5 (win)

Game 6 (loss)

Game 7 (loss)

Game 8 (loss)

Game 9 (win)

Game 10 (win)

Game 11 (win)

Game 12 (loss)

Game 13 (draw)

Game 14 (win)

Game 15 (win)

Game 16 (manually decided draw, see footnote)

The legendary 61 move game.

Game 17 (win)

Game 18 (win)

Game 19 (loss)

Thankfully generative AI has the potential to create new genres of content that weren’t feasible before.

Infinite shows

Many beloved shows are formulaic, so there’s no reason why you couldn’t have more episodes. The only constraint is the human labor needed to produce each episode.

Now we’ll be able to create never-ending shows that run 24/7. You can tune in at any time, and there’ll be fresh content. Since there’s no overarching plot, you don’t miss anything in the time you spend away.

This genre has already been launched, rising to prominence on Twitch a few months ago with infinite Seinfeld.

Other notable examples include:

• Unlimited Steam - A school principal who just can’t stop burning his house down while cooking.

• Always Breaktime - A slice of life anime featuring three students making perpetual small talk.

• BasementView - A late night show run by an “insult comic skeleton”.

Choose your own adventure

Within the last few years, Netflix has made a few “choose your own adventure” shows, most notably Black Mirror: Bandersnatch. But all of these shows are limited in their choices, because everything has to be filmed beforehand.

With low cost AI-generated content, the viewer can choose between infinite possibilities. You and your friend can watch the same show and end up with completely different endings.

The creators of the show can limit the possibilities and add constraints so they still have control of the overall vision. But the potential is limitless.

Audience created shows

A more community oriented variant of “choose your own adventure” is an audience created show. Imagine 1000 people in a Twitch chat making proposals and voting on different directions the show could head in.

One viewer might pay $100 to create a new character and define their personality. There could be a poll to decide whether the main character should move to NYC or stay in Ohio.

By making it extremely cheap to add characters and create new plotlines, the audience themselves can collaborate to create a show they enjoy.

If some portion of the audience doesn’t like the way things are going, they can make their own “fork” of the content, and have a different version of the same story.

Second-Person shows

Instead of watching some random character in a story, why don’t you become the main character?

You’ll be able to swap yourself into the main character’s place, and have your looks and voice perfectly replicated. Other characters will refer to you by name. You can finally be the protagonist.

I can imagine this being very trippy and surreal, similar to the experience of old people seeing younger versions of themselves with a TikTok filter and having emotional outpourings.

I don’t think it’s a gimmick that should be slapped onto everything. I think there will be shows specifically designed to be viewed in a second-person way, to help you reflect and give you a new perspective.

For example, there’s a powerful visualization exercise where you imagine your own funeral, and think about what you want your family and friends to say about you. This helps you figure out your personal definition of success.

Second-Person content will be even stronger than a visualization exercise.

Extremely realistic parody

AI presidents are my favorite trend right now. Biden, Obama, and Trump play Minecraft together, insult each other, and have fun. It’s hilarious.

@deepfakenewsReplying to @unfunny_person23 Episode 3: Biden Griefs Trump’s House #trump #biden #obama #deepfake #meme #minecraft #fake

Tiktok failed to load.

Enable 3rd party cookies or use another browser

We’ve had parodies before, but now the voices and faces are indistinguishable from reality. Who knows, maybe Biden and Trump do play Minecraft together sometimes.

Extreme fan fiction

Let’s say you’re a big Game of Thrones fan, and like 99% of the fanbase, you didn’t like the ending and thought the final season was complete garbage and you could write a better version.

Well now you *can* write a better version.

You can use all the existing episodes to clone the characters and their voices, and create your own version of the final season of Game of Thrones.

You might break some copyright laws along the way, but the power of a fandom is much stronger than the power of copyright.

New animation styles

While many are trying to use AI art tools to recreate existing animation styles, it will be more interesting to see what new styles we can create. 

The variance between generated frames has been seen as a weakness, but it might actually be the strength of AI art, and lead to a new animation style.

There could be a style where the frames are all somewhat consistent but different in a beautiful way.

Here’s one early hint of what that might look like:

@ai_lensWhat song should I do next??? #goldenhour #aimusicvideo #aisongcover #aisong #aisonglyrics #aiartcommunity #ailens #aiart #aiartist #itsyourgoldenhour

Tiktok failed to load.

Enable 3rd party cookies or use another browser

So much potential

Most of the focus of generative AI right now is on replicating what humans can already make. But we will hopefully move past this boring phase quickly.

I’m excited to see what crazy new genres are invented in the coming years.

People who don’t use TikTok may be confused about how an app with dancing girls and cringe memes could possibly be a legitimate search engine. The reality is that TikTok has a diverse range of content covering everything you can imagine.

Unlike the SEO spammed search engines, TikTok features content by real people, doing real things, sharing real thoughts, for no financial gain. The information there is authentic, concise, up-to-date and high quality.

TikTok is a search engine for experiences

To understand where TikTok excels, we have to talk about the four types of searches.

Navigational searches are when you’re trying to get to some specific website. You know exactly where you want to go. For example, you want the website for the nearby restaurant to see the menu.

Informational searches are when you want a factual answer. What’s the distance to the moon? There’s one answer to this, and it’s 238,900 miles.

Exploratory searches are when you don’t know what you’re looking for yet. You want to learn about Quantum Physics but you aren’t sure where to start. Search engines have been historically bad at this, but LLMs will excel at this category.

Experiential searches are about real world experiences. What’s the best Italian restaurant in NYC? What are the best places to hike in Peru? Where should I get my hair cut?

These searches don’t have one factual answer. All the answers are subjective. So the most important thing for experiential searches is authenticity.

TikTok dominates at authenticity.

Authenticity

If you’re searching for a good Italian restaurant, you could rely on Google web results, and read those “Top 5 Best Italian Restaurants In The Universe That You Absolutely Have To Try Before You Die” articles. But who even writes this stuff? Can you trust them? Was it just churned out by an SEO content farm?

You could rely on Google or Yelp reviews, but once again, who even are those people? Did they really go to the restaurant? Were they bribed to give 5 star reviews with free ice cream? Are they bots?

Or, you could go to TikTok.

On TikTok you can visit the restaurant virtually, experience the ambience, see the food in front of you, and get a realistic simulation of going there. This is far more authentic than any text-based result could ever hope to be.

@myarosemiller_3 amazing italian spots you can actually get a reservation at !! *shock* they do exist #nycfood #nycitalianfood #nycitalian #nyceats #nycrestaurants #nycrestaurantreviews #nycfoodie

Tiktok failed to load.

Enable 3rd party cookies or use another browser

Search for the best places to hike in Peru on Google, and you’ll get some blog posts with pictures. It’s alright. But search on TikTok and you can *go on the hike*. You can live vicariously through someone experiencing it in real time.

@mariadinca_One of the best hikes of my life: the Salkantay Trek. Thank you @Salkantay Trekking for an amazing five days. #salkantay #traveltiktok #machupicchu #peru #salkantaytrekking #visitperu #hikingszn #hikingadventures

Tiktok failed to load.

Enable 3rd party cookies or use another browser

TikTok simulates reality better than other search engines. But it’s also authentic for another reason. Users describe TikTok as being “like getting answers from a friend”. It’s a return to the authentic old web, where people are sharing things without any profit motive1.

Concise

The constraints of TikTok lead to most videos being less than 30 seconds long. 

With this constraint, the creator doesn’t have the time to tell you their life story about how their grandma went to the village shrine and had a vision and saw the scrambled eggs recipe appear in the sky and wrote it down on a stone tablet to be handed down for generations.

They just tell you how to cook the damn eggs.

@maxiskitchenHow to make eggs like Michelin star chef Thomas Keller✨ . . . . . #tiktokfood #cookingtiktok #scrambledeggs #foodies #homecook #lovestocook #eggs #breakfast #besteggs #bestbreakfast

Tiktok failed to load.

Enable 3rd party cookies or use another browser

People use TikTok search because they don’t have time to waste. They want answers. And they want it now.

Recency

Recency bias may be a bad thing in some cases, but it’s a very good thing when it comes to real world experiences.

You don’t want to know what the best Italian restaurant was 6 months ago. You want to know what the best Italian restaurant is today.

Blog posts aren’t updated that often, and subreddits tend to discourage people posting questions that have previously been answered.

Meanwhile on TikTok, new videos are made every second. There’s always fresh content.

Ranking

To game your way to the top of Google, you need to get a lot of backlinks, target the right keywords, and do a myriad of things that have nothing to do with making high quality content.

To game your way to the top of TikTok, you need a high watch time on your videos. This means you need to make content that people genuinely enjoy watching.

This is a much better ranking algorithm if you want high quality content on top. The addictiveness of TikTok speaks for itself.

Prediction

TikTok’s ability to predict what you want is unparalleled.

The perfect search engine knows you so well that you don’t even need to type a search query to get what you want. And that’s exactly what TikTok has accomplished. 

Instead of searching for recipe ideas for dinner tonight, you can just open TikTok when you’re ready to cook. It *just knows* that you want dinner recipes without any need to search explicitly.

TikTok is not good at everything

It’s clear why many people prefer TikTok as their search engine for experiences. But it still falls short of a traditional search engine for most other categories.

It’s useless if you’re trying to find a specific website. It’s useless if you’re trying to dive deep into some topic. It’s useless if you’re looking for a code snippet to copy paste. It’s useless for any knowledge that isn’t surface level.

I don’t foresee TikTok getting any better at the other types of searches. The reason it’s great for experiential searches is the same reason it’s bad for everything else.

The Fragmentation of Search

It’s not that TikTok is going to become the number one search engine in the world. It’s that search is becoming more fragmented across different media formats. 

People have many different ways of searching that are equally good. So they just use whatever fits their preferences. And a lot of younger people have a preference for visual learning via short-form videos. 

I like reading, so I still use Reddit for most searches. But if it’s a real world experience, I combine Reddit with TikTok for maximum power.

Appendix 1: TikTok videos about TikTok being better than Google

Lots of interesting things in the comments of these videos.

@stazzyliciouspls tell me im not alone

Tiktok failed to load.

Enable 3rd party cookies or use another browser

@jen.heiferLol the lashes 🤣

Tiktok failed to load.

Enable 3rd party cookies or use another browser

@jaycardoneTikTok is taking over Google? The trends are repeating, so if you are thinking about starting or taking TikTok seriously, now is the time to grow here and build a business using Tiktok! #greenscreen #growontiktok #fyp #entrepreneur

Tiktok failed to load.

Enable 3rd party cookies or use another browser

@simoncainewastaken@simoncainewastaken@simoncaineridesagain TikTok Vs Google. The Battle For Search #ukcomedy #uktiktok #googleseo #tiktoksearchbar #tiktoknews #learntok #learnwithtiktok #socialmedia #growtiktok #seo #googlesecrets #googleseohacks #tiktoksearches #howtotiktok #seotiktok

Tiktok failed to load.

Enable 3rd party cookies or use another browser

@justlukemorganI need the help of Gen Z - feeling like an old man depsite being 23 lol #genz #tiktokovergoogle #tiktoksearch #googlesearch #genzhelpme

Tiktok failed to load.

Enable 3rd party cookies or use another browser

Appendix 2: TikTok advertising itself as a search engine

Two ads from December 2022 where TikTok promotes itself as a search engine. They very much want the app to be used like this.

If you were planning to use ChatGPT to pass your coding interview, I have bad news for you. ChatGPT can only solve 28% of LeetCode questions. And that’s being generous.

Real interviews use medium and hard questions. So while ChatGPT can solve 77% of easy questions, it can only solve 16% of mediums, and 0% of hards.

The Experiment

This data comes from my own experiment, where I tested ChatGPT on the most recent 100 LeetCode questions.

ChatGPT performs well on old, popular interview questions, but that’s a biased sample.

The answers to popular interview questions were undoubtedly in the ChatGPT training data. It doesn’t tell us how it would perform in a real interview with new questions.

So I tested it on questions it definitely hasn’t seen before, and I gave it 3 tries at solving each problem.

Assisted Solutions

If we gave it a high level plan of how to solve the problem, would that help?

Kind of.

Providing a high level solution helped ChatGPT go from a wrong answer to the right answer 19% of the time. That’s better than nothing, but not as much as I hoped.

Lower The Bar

There are different degrees of solving a LeetCode problem. The above stats were for getting an accepted solution, which involves passing numerous test cases.

What if we lower the bar a bit?

If we use the extremely low bar of “can it pass more than one test case”, then it performs well.

Plausible But Buggy

Overall, ChatGPT is good at coming up with plausible but buggy code. It often knows the general direction for the solution (e.g it’s a dynamic programming problem), but then it implements it with bugs and fails test cases.

The most interesting fail was an easy question that it got wrong. It explained its thought process and plan for solving the problem, which was correct. Then it wrote the code, and managed to forget an edge case that it specifically mentioned.

It figured out the general formula, then wrote this code that ignored the n=1 edge case.

The correct code just needed 2 more lines.

Conclusion

ChatGPT isn’t ready to crack the coding interview just yet. But it is good at coming up with a high level plan, and writing a buggy implementation.

If GPT4 is able to write more accurate code, then it could be the thing that finally kills the algorithmic coding interview.

(All of the ChatGPT generated solutions are available on github if you’re curious.)

Google’s Bard got an answer wrong during an ad, which everyone noticed. Now the narrative is “Google is rushing to catch up to Bing and making mistakes!”.

That would be a fine narrative if Bing didn’t make even worse mistakes during its own demo.

Pet Vacuums

Let’s go buy a pet vacuum!

According to this pros and cons list, the “Bissell Pet Hair Eraser Handheld Vacuum” sounds pretty bad. Limited suction power, a short cord, and it’s noisy enough to scare pets? Geez, how is this thing even a best seller?

Oh wait, this is all completely made up information.

Bing AI was kind enough to give us its sources, so we can go to the hgtv article and check for ourselves.

The cited article says nothing about limited suction power or noise. In fact, the top amazon review for this product talks about how quiet it is.

The article also says nothing about the “short cord length of 16 feet” because it doesn’t have a cord. It’s a portable handheld vacuum.

I hope Bing AI enjoys being sued for libel.

(Note: There is another version of this product with the same name which does have a cord and may actually be noisy. However, that is not the version of the product in Bing’s citation. Presumably Bing intended to describe the best-selling version, which is the cordless version that it cited. But instead it got confused and described the corded version.)

Mexico Nightlife

Let’s go to Mexico!

Bing AI generated a 5-day trip itinerary for Mexico City, and now we’re asking it for nightlife options. This would be pretty cool if the descriptions weren’t inaccurate.

Cecconi’s Bar *might* be classy, but doesn’t seem particularly cozy from the images I saw. And it most definitely does not have a website where you can make reservations and see their menu.

Primer Nivel Night Club is an absolute mystery. There’s one TripAdvisor review from 2014, and the latest Facebook review is from 2016. There are no mentions of it on TikTok, so I seriously doubt “it is popular among the young crowd”. Seems like all the details about this place are AI hallucinations.

El Almacen *might* be rustic or charming, but Bing AI left out the very relevant fact that this is a gay bar. In fact, it is one of the oldest gay bars in Mexico City. It is quite surprising that it has “no ratings or reviews yet” when it has 500 Google reviews, but maybe that’s a limitation with Bing’s sources.

El Marra is a vibrant and colorful bar, though the hours may be wrong. There are so many ratings of this place online that it’s once again surprising that there are “no ratings or reviews yet”.

Guadalajara de Noche is the first one that seems like an accurate description. Good job Bing AI, you got something right! I’m so proud of you. What’s that? You want to try reading financial statements? What could go wrong…

Gap Financial Statement Summary

This is by far the worst mistake made during the demo. It’s also the most unexpected. I would have thought that summarizing a document would be trivial for AI at this point. But Bing AI manages to take a simple financial document, and make all the numbers wrong.

“Gap Inc. reported net sales of $4.04 billion, up 2% compared to last year, and comparable sales were up 1% year-over-year”

Bing AI starts off fine. This statement is totally correct, probably because it is a direct copy paste from the financial document.

“Gap Inc. reported gross margin of 37.4%, adjusted for impairment charges related to Yeezy Gap, and merchandise margin declined 370 basis points versus last year due to higher discounting and inflationary commodity price increases”

Uh…no. That’s the unadjusted gross margin. The gross margin adjusted for impairment charges was 38.7%. And the merchandise margin declined 480 basis points if we’re adjusting for impairment charges.

Don’t worry, it gets much worse.

“Gap Inc. reported operating margin of 5.9%, adjusted for impairment charges and restructuring costs, and diluted earnings per share of $0.42, adjusted for impairment charges, restructuring costs, and tax impacts.”

“5.9%” is neither the adjusted nor the unadjusted value. This number doesn’t even appear in the entire document. It’s completely made up.

The operating margin including impairment is 4.6% and excluding impairment is 3.9%.

The diluted earnings per share is also a completely made up number that doesn’t appear in the document. Adjusted diluted earnings per share is $0.71 and unadjusted is $0.77.

“Gap Inc. reaffirmed its full year fiscal 2022 guidance, expecting net sales growth in the low double digits, operating margin of about 7%, and diluted earnings per share of $1.60 to $1.75.”

No…they don’t expect net sales growth in the low double digits. They expect net sales to be down mid-single digits.

And I didn’t see anything else in this document about the future outlook for operating margin, or diluted earnings per share. So Bing AI either got that from a separate document, or made it up completely.

But it gets worse. Now we’re going to compare Gap to Lululemon

Now we’re comparing made up numbers.

The Lululemon data is about as accurate as the Gap data.

Lululemon’s gross margin is given as “58.7%”, which is a hallucinated value that doesn’t appear in their financial document. The real value is 55.9%.

Lululemon’s operating margin is 19%, not 20.7%.

Lululemon’s diluted earnings per share is $2.00 not $1.65.

Cash and cash equivalents is wrong for Gap (should be $679 million) but correct for Lululemon.

Inventory is wrong for Gap (should be $3.04 billion) but correct for Lululemon.

Conclusion

Bing AI did a great job of creating media hype, but their product is no better than Google’s Bard. At least as far as we can tell from the limited information we have about both.

I am shocked that the Bing team created this pre-recorded demo filled with inaccurate information, and confidently presented it to the world as if it were good.

I am even more shocked that this trick worked, and everyone jumped on the Bing AI hype train without doing an ounce of due diligence.

Bing AI is incapable of extracting accurate numbers from a document, and confidently makes up information even when it claims to have sources.

It is definitely not ready for launch, and should not be used by anyone who wants an accurate model of reality.

(For more thoughts on search and AI, follow me on twitter @dkbrereton)

Appendix 1: Bing AI doesn’t know what year it is

Appendix 2: Bing AI hallucinates the Super Bowl

Note: this was asked before the Super Bowl, so it shouldn’t be able to come up with a winner. Even ignoring that, the date, and location are wrong.

“At least that's relatively innocuous. I asked it how to identify a species of edible mushroom, and it gave me some of the characteristics from its deadly look alike.” - HN comment.

Appendix 3: Bing AI kicks Croatia out of the EU

The actual question that Bard failed at was “What new discoveries from the James Webb Space Telescope can I tell my 9 year old about?”, but Bing makes up a new question, then claims that Bard gave the wrong answer (yet that answer is actually correct).

The internet is run by adtech right now, and all the biggest companies are thriving on that, and I think the sooner we get out of that, the better. It is certainly going to be hard, and I'm not against ads per-se, but I'm against ads everywhere, ads by default, ads making our lives crappier and more miserable.

This is a casual interview that I had with Vladimir Prelovac, the founder of Kagi, in March 2022.

DKB: Why did you decide to work on Kagi? Building a new search engine is a pretty ambitious project.

Vladimir: It was 2018, and I was fresh out of the corporate world after the acquisition of my startup. I wanted a search engine for my kids that I could trust. My main motivation was to prevent my kids from being exposed to ads and tracking from a very young age.

Kids start to use computers at a young age since they need it for school, and 99% of kids start with Google. They’re providing free Chromebooks to schools, and it’s convenient to use Google, so that’s what kids are using.

I didn’t like the idea of them growing up with ad supported technology that would influence them from a young age – shaping their thoughts and views of the world.

Back then, there was literally no search engine like that, because DuckDuckGo was also ad-supported. I was paying for YouTube Premium because I didn’t want my kids exposed to ads, so I started thinking about why there wasn’t also a search engine without ads that you could pay for. The revenue would come from the users, perfectly aligning all the incentives.

And that’s how Kagi got started.

I wasn’t alone. With significant funds from my previous exit, I was able to bootstrap the entire operation with around 10 people.

DKB: I want to talk more about your business model. You seem convinced that people will pay fo a search engine, even though there are no good examples of this in the past.

What makes you think that people will pay for something they’re used to getting for free?

And will the pricing be unlimited searches per month, or some kind of per query pricing?

Vladimir: Before I tell you my answer, I’m curious to hear what you think about all that, and what your view is as someone in the field?

DKB: People pay for YouTube Premium, so there’s certainly some subset of people who would pay to avoid ads. I’m not sure what the percentage is, but I would imagine it’s a relatively small percent of users that do this.

When it comes to how pricing should work, I think pricing per query is scary. Users would be scared to do queries because they have to mentally calculate whether it’s worth the 2 cents or whatever they’re paying for it. An unlimited pricing model would probably put people’s mind at ease.

I know Listen Notes, a podcast search engine, has their main search for free but charges for a more advanced search with more filters. That’s another interesting way of monetizing where you have a simple free version, and a more advanced paid version for the power users.

Vladimir: You’re spot on. Paying for a search engine is unusual, but that’s the only choice if you want to build an independent search engine that isn’t ad supported. You could try out donations, but I don’t think that would be dependable.

I think we have a significant tailwind in the world today with privacy being a hot topic. People are waking up to how their information is bought and sold online. Search engines have also been criticized for introducing biases and agendas. In my opinion, the job of a true web search engine is not to be the arbiter of truth. It should provide information based on search relevancy algorithms, not based on the social, political, religious, or whatever views its leadership may have. The only agenda it should have is to provide the best possible customer experience.

For all the big search engines, the issue is that their customers are advertisers, not the users. This creates all sorts of conflicts of interest. With Kagi the users are also the customers, and this means we can design a better search experience, and we hear from a lot of users that such an experience is worth paying for.

So when you think about it, you're quite right. Five billion people use a search engine, probably 99% would never pay. The thing is, that tiny minority of people who think differently, the 1% that would pay, is still quite a large number. That's 50 million people.

YouTube Premium has 50 million subscribers, even though everyone can watch YouTube completely free! And yes there is of course Google Search, you can use it for free, but you're exposed to ads and tracking and your productivity takes a hit because of a poor user experience, optimized for serving many ads.

You can use an ad blocker with YouTube to remove ads, and despite that, there are still 50 million people paying for it. This shows how much these people do not want to be exposed to ad tech.

Kagi doesn't need a lot of users, because as I said, we are not a Google killer. Google has a huge moat, and a 20 year headstart, and the web is now much harder to crawl than it was 20 years ago, so we will probably never have Google scale, but we don't need it.

We are trying to build a sustainable business for a small subset of users who think differently. We just need a couple of tens of thousands of such users to be sustainable, and that's our first milestone.

When we are talking about sustainability, we should note that searching has a cost. We are not used to that cost right now because advertisers subsidize it, but if you are doing a search, it has a real cost, and we are trying to bring that cost as low as possible.

It would be about one cent per search or something like that, but still for many users the idea of paying for search is frightening. I like to compare it with electricity, do you think of cents being spent when you have a light on, or do you think about the convenience of having the light on?

The challenge we are facing can be described through this analogy: imagine electricity was completely free for the last 20 years, but there was a catch, you had to have ads on your walls 24/7. And now someone comes along and says we are going to offer electricity at a few cents per hour, but we will completely remove all the ads from your home.

The idea of paying for a utility that was free for so long would be unthinkable for most, but this is what we already have in reality and everyone is okay with it. This is why I think the idea of paid search also makes sense.

Twenty years of ad-supported search have created resistance to the idea of paid search, but a lot of people are already coming to the realization that the predicament they have found themselves in with ad-supported search is not good for them.

DKB: Where do the results for Kagi come from?

Vladimir: We get results from Google, Bing, and our own non-commercial indices. We have two of our own indices, one for non-commercial web search, and another for non-commercial news.

The non-commercial indices can have a big impact on the quality of results since it helps us avoid spam, SEO optimized content, and pages riddled with ads. It also allows us to get expert opinions from small websites and personal blogs.

I think a higher quality of search is ultimately worth the price, because you can find something quicker with just one query, and we can save you half a minute per search, especially when you're doing research.

The diversity and quality of Kagi results is really unique on the market, and on top of that we have pretty cool features. For example, you can block or promote websites in your results, so if you have favorite ones or if there are spammy sites you don't like, you can modify your results.

We also have the lenses feature where you can create a lens, which is basically a subset of up to 10 sites. So you can create a lens for programming or recipes with your favorite sites.

DKB: The customization features on Kagi are quite powerful.

Do you see your target audience as the kind of person who would want to dive into all of these settings? Or is Kagi going to have good settings by default for someone who just wants to search without customizing anything?

Vladimir: We have reasonable defaults, and for most people Kagi will be great out of the box. But there are also people who want different options. They want to be in control of the search experience, and Google and the other search engines do not provide them with the level of control that we do.

Everybody has different preferences and our goal is to provide tools, and empower users to get the experience they want, versus us trying to be smart, and creating an average search engine for an average user which doesn't exist. I mean, there's no average human, you and I are different, and everybody is different, and we like and prefer different stuff.

Nobody has their home screen on iPhone looking exactly the same. I think people deserve that in search, especially being such an important part of our lives. We probably spend half our day in web search, whether for work or private needs. It's one of the most used products out there, and it's astonishing how little customization you can do.

So after great search quality, great speed, and unmatched privacy features guaranteed by our business model, customization would be something many users appreciate about Kagi.

DKB: How do you think about ranking results?

Vladimir: It's pretty complex, but basically we try to figure out what the query is about, and then prioritize results from sources we think will give the best quality of results for that query. I'm speaking in general because it's a general topic.

We do have one bias actually, that we are open with, and that is we flat-out “downrank” sites that have a lot of ads and trackers, and we promote sites that have none or very few. So you could say that we have an anti-adtech bias, if you would, but unlike other biases I think this one is actually useful for the user. I think it enhances the quality of search results, which the search engines should be all about.

In addition to that, the user then has the ability to downrank or even completely remove sites they don’t want to see. These settings are currently per user. We are not crowdsourcing it, although it's an interesting idea to get all the sites that users, on average, have banned, and try to find a pattern there, and maybe we can enhance the penalty for those sites, so that new users enjoy the benefit of others basically curating the results for them. So that’s the end goal for that, but we’re still not there.

DKB: I want to talk about advertising on the web.

I talked to someone at Google about this and their response was something like “Oh, you’re going to penalize people just because they’re trying to make money? These are just normal people writing their food recipes, and they have ads on their site because they need to make a living.”

What are your thoughts on this, and online advertising in general?

Vladimir: I think the construct of “ads are needed to help creators” is a default answer promoted by adtech to make it sound as if we cannot live without it. But the truth is that ads are by far the least profitable way to monetize quality work, because you need an astonishing number of impressions to earn money.

In reality, what this creates is an environment where very few people take most of the ad revenue, whether these are YouTubers with millions of views, or big sites, and actually the vast majority of creators, small bloggers, and YouTubers make pennies, if anything.

So most creators are actually worse off with this arrangement of things. That's one problem, and the second is that because adtech marketing is so powerful, and we are led to believe this is the only way to make money, people don't even try alternate methods.

Kagi exists because we dare to try something else. We want to monetize our work in a way where our incentives are aligned with the incentives of our users.

Most people use ad-supported models because everybody else is doing it, and that makes the problem bigger because ad-supported business models incentivize the creation of large quantities of low quality content.

Even if you're making a few dollars a month from that, you're ultimately not contributing to the web in a good way. I think that needs to change, and our mission is to humanize the web, and stop this cycle, this pattern of behavior where people just do what everybody else does, and it kind of dehumanizes us, and who we are, and we are just a part, a gear, a cog in the huge machinery of ad-tech.

The internet is run by adtech right now, and all the biggest companies are thriving on that, and I think the sooner we get out of that, the better. It is certainly going to be hard, and I'm not against ads per-se, but I'm against ads everywhere, ads by default, ads making our lives crappier and more miserable.

Users are already beginning to rebel against this model by using ad blockers on browsers. Over 760 million devices have ad blockers installed. More and more people are willing to pay for ad-free services, like millions pay for YouTube ad-free.

The ad-driven model spawns bad user experiences, privacy invasion, and constant tracking. This is especially pernicious with young kids, who are using these services with no controls, and with no understanding that ads influence their behavior, or that their profiles are being built from a young age and will be bought and sold around the internet.

By now you can probably see that I strongly oppose ads and ad-tech, maybe you could even say “too much”. But yeah, I don’t want ads anywhere near my kids or our users.

If you want ads, there are plenty of options elsewhere, but with Kagi we will go to great lengths to make sure web search is exactly what it’s supposed to be, and nothing else.

DKB: Do you have any last words on why people should consider switching to Kagi?

Vladimir: Try Kagi to get a glimpse of what the ad-free web search experience looks like. And if you choose to support us by becoming a customer, you will dramatically increase our chances of making a lasting impact. Thank you!

We thought this was wrong. Search is a daily utility that all of us use. There needs to be alternate mechanisms for how we go about it. And so things like the ads-free and private model that Neeva is built on were consequences of this basic observation.

This is an interview with Sridhar Ramaswamy, Vivek Raghunathan, and Darin Fisher of Neeva.

DKB: What differentiates Neeva from Google?

Sridhar:

Three things:

• Control over the ranking of sources in your search results.

• Transparency about these sources.

• Deep integrations to bring rich previews and answers directly to you.

First, some background. I ran search ads at Google for over a decade, and I could see the inexorable pressure to take up more and more space with ads. It’s one of those things where there's just so much money to be made that people come up with cleverer and cleverer theories to justify why it’s okay.

While I didn't run Google's search, I ran sister teams like Travel and Shopping. I could see that the angle Google was taking on the organic side was to show more of the content itself. This includes things like the knowledge graph and featured snippets.

And at some point I realized that the logical conclusion to all of this was that any commercial query was going to have ads because that's what makes the most money, and any non-commercial query would have content directly from Google.

That’s really the founding story of Neeva. We thought this was wrong. Search is a daily utility that all of us use. There needs to be alternate mechanisms for how we go about it. And so things like the ads-free and private model that Neeva is built on were consequences of this basic observation.

With Neeva, we started with a simple principle – put the user first. We wanted to put you back in charge of the search experience. And the philosophies that we bring into play at one level are around changing the ranking algorithm.

This has to be done carefully because one of the most prized qualities for a search engine is that it is neutral. It is not editorialized by a small set of people. The aspiration is to be the objective truth. In practice, you realize that there are always loopholes and people that are trying to game the system. And so you have to evolve. It's a living system, but the aspiration is to always be the objective truth.

In a bunch of areas, for example, in shopping, we lean in, because shopping has become so much about SEO. On Google it's about showing ads and SEO sites. So we change how our ranking works to prefer reviews more.

We're going to prefer informational content more for certain classes of queries because we think those are more useful than simply saying "okay, buy this here", which is what the search engine page has become.

Similarly, for things like programming queries, where we are sure that there is an official answer, we change the ranking algorithm to say, "Answers from the official site are generally more important. If you have a specific question about a Go language feature, Golang docs are just better."

The second thing we do is when we are not sure that large scale changes in ranking are called for, we give you agency. So on a number of queries in the legal vertical, and in the health vertical, we give you filters that can distinguish between nonprofit sites, ad supported sites, government sites, and more.

Google creates what I call the flat earth view where every site is just like every other site. We go out of our way to tell you these additional things that you should know.

Sometimes we have partnerships, for example with NewsGuard, where we show these additional attributes. You then have the ability to set preferences for sources you want more or less of.

We also work much more closely with partners. So we have partnerships with Reddit, Medium, and others, where we want to create a holistic experience on the search page, where you get your answer, but in a way that creates space for them to thrive.

This is why we announced our revenue share program early on. And so for certain classes of things, we act in concert with others. Search quality and ranking are unaffected by these partnerships.

Vivek:

Alan Kay once said that people who are serious about software need to build their own hardware. We think that there is an equivalent in the search market: people who are serious about a search engine need to have their own search stack. Whether it's crawling or indexing the web, or serving search results completely, or having your own ranking systems or having your own eval systems or your own structured parsers.

If you don't have that end-to-end, you are not going to be able to solve the problems you identified in your first article. The problems of SEO, and the problems of authority, the problems of all the great content on Reddit that you can’t get to. For example, we've actually spent the better part of a year working directly with the Reddit ranking team to make their ranking better.

Those things are only doable if you have your own search engine. In many ways, this is the flywheel that lets Neeva get better on a daily basis.

DKB: One of the biggest issues people have with Google is that many queries are super SEO optimized. The top results feel more like spammy marketing than genuine content, so people have to append Reddit or Quora to their searches to find something decent.

How do you think about ranking content to avoid this kind of problem?

Vivek:

I’ll answer that along a couple of dimensions.

First, let’s start with the core human eval and ranking systems.

Every search engine uses humans to come up with guidelines on what constitutes a good page for each query. With these guidelines at hand, search engines get human raters to generate a large amount of labeled ground truth data. And then you tune your ranking systems to optimize for the ground truth.

Typically, eval guidelines have two components. One is how well this page matches the intent of the query at hand. This first component is called “query page match” or “topicality”. The second component is how good the page is by itself, independent of any query. This is called (query-independent) “page quality”. We spend a lot of time on our page quality guidelines to make sure they reflect great user experiences.

What we do roughly is the following: There's obvious stuff we want to get rid of. Foreign language pages, malware, clones of other pages, this is a very common problem, there are all these clones of Stack Overflow.

If you have a paywall and you're very aggressive about pushing people into the paywall, we will consider those bad pages. If you have pages with lots of ad load relative to the information, we'll consider those bad pages. If the ads are in the center column and blocking the content, we'll consider that even worse. We have hundreds of pages of guidelines like this.

When a user query comes in, our machine learning scores results using a combination of factors. The query-page topicality is a big component of the model. The other huge factor in the model is the query-independent page quality. Now, generally speaking, there are hundreds of on-topic search results for any given query, so our scoring system ends up picking the highest quality pages for any given query.

The beauty of the page quality guidelines tends to be while they're page specific, all pages on a site have the same flavor to them. For example geeksforgeeks.org or experts-exchange.com will all be the same format; all having blocking interstitial ads before you can actually get to the content. And so these signals often remove all pages that are from the same class of SEO sites. So that's the core ranking which helps reduce low quality sites.

There's a second class of things we do. We’ve labeled every site on the web with a set of faceted labels (“official”, “forums”, “blogs”, …). We show these facets on every query, and you can deep dive into the facet to only get results from that class of sites.

We use the faceted labels to cluster results and call them out in the UI. For example we might give you a set of results from official documentation sites. There would be a few of them, and if you clicked more, you would get into the tab view where you see all the results from these sites.

We'll also do this for high quality forums, like Stack Overflow or Reddit. If we see four of the top eight or nine results are from places like Reddit, we'll pull it up into a forum box. This is at the top of the page and is the first thing you see. And of course we only do this if it’s topically relevant to the query.

We are very aggressive about optimizing for authority. This helps with the other end of the spectrum, where it doesn’t necessarily get rid of bad pages, but it promotes good pages.

Lastly, we extract structured content from lots of these pages and show you rich inline previews. We’re exploring and experimenting with even deeper integrations of this.

As an example, wouldn't it be cool if alongside any page on the web, when you're hovering over your search results, you could see which Reddit posts or Stack Overflow threads pointed to the page? Often that's a good sign of authority.

DKB: I know that Neeva is also working on a browser, so I’d be curious to hear how that fits into everything else you’re doing.

Darin:

Right now we're focused on mobile. We're going to make it easier for people to get access to Neeva on mobile. If you have an iPhone, you would see that Neeva is not a default choice that Apple gives you for search engines. It'll only give you five choices. And so it's necessary that we build an app.

A search app and a browser are really synonyms. You want to be able to search for many different things and multitask, that's what tabs are for. And the main UI for a browser is the search box that starts your journey, and so making the Neeva Browser was really important so that we could make it easier for people to use Neeva on mobile.

But it goes beyond that, in that we have the opportunity to innovate on the browser in a lot of interesting ways, because we're not limited only to how many search results page views we get with ads on them. Our business model unlocks our ability to bypass the search results page.

Let me explain what I mean by that. When you start typing into the Neeva Browser, we are able to suggest not just query completions, but we can just go ahead and suggest search results based on those query completions directly inline, as you're typing. There's not a lot of space in the suggest experience for ads, but we just go ahead and put the URL there.

And the awesome thing is that about 14% of the time, people actually go directly to those results in the Neeva Browser. They just bypass the search results page entirely. So we save them time, which is pretty cool. And you can think about the queries where this might be obviously the right thing, but at Google it’s difficult to not send people to the search results page.

Imagine somebody starts typing Facebook. What's the best result to put in the suggest box? Probably facebook.com. You might see that term in Chrome, but what it's going to do if you tap it, if you haven't been there before, it's going to take you to the search results page and show you a search query for facebook.com. And guess who needs to buy ads in order to make sure that their result is on top. Try doing a query for Amazon and see what happens.

Sridhar:

The query “headphones” is a more dramatic example of what you're saying. On Neeva we will actually show you the review sites for headphones in the suggest box so you can go directly there. A company like Google has zero incentive to ever do that because they really want you to go to that stack full of ads.

We are taking a brand new outlet, which is the mobile app, and saying, “We’re going to cut out the search engine." The search engine, as most people know it, disappears. And so more than half our clicks are direct from this feature, which is called FastTap.

You get to where you want to go in 500 milliseconds, the time it takes for you to parse the result with no search engine in between.

Darin:

And so this is a very simple feature, but one that makes you wonder why you didn’t have it before. And there's a really easy answer to that, because of course Google wants to take you to the search results page to show you ads. It's really hard for Google to move away from that.

And so we've been exploring a lot of things that are in this sort of vein. In the context of the browser, things like helping you get back to what you were doing before, so you don't have to search for it again, by having better organized tabs, simple things like that.

It's difficult for Google to do this, because anything they change about Chrome that makes you not visit the search results page as much, is something they don't know how to ship.

And so there's a whole swimlane of features for the browser and search experience that have to do with how we help people get to where they want to go faster. How do we help people get back to the things they were doing? How do we help people stay organized in those tasks? How do we help people find the next thing based on what they're doing and continue the journey forward?

That's the space that we're innovating in. Things like what Vivek talked about, how we can understand that there's a discussion happening about the topic you're researching on Reddit or Stack Overflow.

You can imagine how through the browser as you’re visiting webpages, we could bring that up and surface that information to you. So if you're on an API reference page, we can tell you there's a discussion about it happening on Stack Overflow.

And this is pointing towards the future a bit, but we already have a feature in the browser called NeevaScope. You can activate that on any page, and Neeva will find related content.

So these are ways in which we are innovating, and exploring how the search experience could be different. It isn't just “enter a query, then go to the search results page with ads”. It could be a lot different and better than that.

Vivek:

And I think the key insight is that browsers that want to do this in a performant way need to have their own search engine sitting next to them. There's only one other browser on the planet that has its own search engine sitting next to it. And it has 100 billion reasons not to do any of this stuff.

DKB: What would you say to people who are using Google? Why should they switch from Google to Neeva?

Sridhar:

People need to be reminded that search is their gateway to so much in this world.

And it's not just the products you are buying. It's your headache. It's your itch. It's all the deeply personal stuff that we all have questions about day in and day out. We have accepted that the ad-supported model is the only way to be.

People are genuinely shocked, and pleasantly surprised, by how much nicer the internet is once you take away tracking. And once you have a simple search engine that's about you.

The one other thing to add is that we also have a base tier that is completely free. And so compared to Duck Duck Go with ads, or Bing with ads, or Google that is full of ads, there's zero risk in trying us. Our quality is very good, it’s private, and the basic tier is completely free.

Most search engines are not independent search engines, and while they may provide some value, they are qualitatively different from what Brave Search is doing. Independence is not something directly actionable, but it’s a fundamental property.

This is an interview with Josep M. Pujol, the Chief of Search at Brave.

DKB: What’s the origin story of Brave Search?

Brave was always interested in search. After all, browser and search go together – one is the car, and the other is the driving wheel. The vast majority of people conflate them both.

Search is extremely complex and costly to develop so Brave did not attack the problem at first. The option, chosen by many other search engines, of using Google or Bing APIs and applying their branding with some ranking alterations was not appealing to Brave because it does not solve the problem of search, it just hides the dirt under the carpet.

In parallel, there was a German-based company called Cliqz, which was building search within a browser. They had been building a search engine from scratch since 2014. Brave and Cliqz, although competitors, always had a good relationship because of a shared vision: to protect privacy and offer real alternatives to Big Tech.

When Cliqz funding ran out in 2020, the search part became a spin-off as Tailcat, and shortly after they joined Brave in March 2021.

DKB: What makes Brave Search different from Google and other search engines?

Brave Search is independent and privacy-preserving. There is no tracking or profiling of users. The Brave search index relies heavily on anonymous contributions from users, who can opt into the WDP (Web Discovery Project) in the Brave browser.

The WDP helps with ranking, but the biggest contribution to ranking is the index itself, which is intentionally smaller than that of Google or Bing. We do not aim to index the whole Web, rather only the Web that is worth indexing.

The biggest problem of search engines is noise-reduction. As with other complex machine learning systems, they suffer from the garbage-in, garbage-out problem.

Google puts all the effort in trying to minimize the garbage-out by using very sophisticated models, trained with data. That’s a very good approach, but it’s something that we cannot replicate, not only because privacy-sensitive data is a no-go for Brave, but also because it requires a lot of resources.

Brave’s approach to reduce the garbage-out is to be careful with what is being ingested, and that makes the algorithmic part of recall and rankings less resource-intensive.

Of course it has a caveat, which is that Brave Search is not yet as good as Google in recovering long-tail queries. At some point we will be, but as of today, that is the trade-off we took.

The difference with other search engines such as DuckDuckGo, You, Neeva, Kagi, Startpage, Presearch, etc. is pretty straightforward: Brave Search can operate as stand-alone, the rest cannot as they rely on Google or Bing.

Most search engines are not independent search engines, and while they may provide some value, they are qualitatively different from what Brave Search is doing. Independence is not something directly actionable, but it’s a fundamental property.

Independence means that Brave Search would continue to work even if Google and Microsoft opposed it. Independence means choice and diversity: if results are drawn from the same provider, they are inherently limited by that provider.

Independence is freedom to do as we see fit and to own our mistakes. If we were to censor Russia Today or CNBC, which we wouldn’t, it would be our choice, not our provider’s decision.

DKB: Many people seem to believe that Google’s results are deteriorating in quality and filled with SEO-optimized content. What are you doing differently when it comes to quality and ranking to avoid this?

We assess quality in two ways: one is the set of anonymous (not anonymized, important difference) signals provided by people who opted into the Web Discovery Project. The second is through human assessment, a small team that is growing as we speak.

Both options, at different scales, evaluate results (to tune up ranking) and pages (to be indexed). The goal is not to rate individual pages or results, but to annotate data that is later used by ML. There is no silver bullet here.

Regarding the problem of deteriorating quality due to spam content, we try to keep it at bay by making sure we index only the relevant subset of the Web. Our index is much smaller than the Web, as a matter of fact it is much smaller than what we could crawl. We index less than 10% of the URLs we are aware of.

We make a conscious decision to index pages only where we have some evidence of usefulness. A small index helps to reduce the garbage-in/garbage-out problem and it’s also more economical to run (still massively expensive).

One way to see it is that we are sacrificing the number of candidate pages to be ranked in return for less noisy results. This is a sensible trade-off, because content also follows the typical Pareto distribution, with 80% of the signal on 20% of the pages.

In any case, the problem of spam, SEO, etc. — let’s call it adversarial content — is indeed a big issue. We might be suffering less than Google by design, but we are not free of it.

Search today is more difficult than it was 6 years ago; infrastructure-wise, we are all better off, but the number of pages keeps increasing and the majority of them are of questionable quality.

In addition to that, not all content is accessible, and to make things worse, automation on content creation is rampant.

I’m not saying “it’s all spam” like the cartoon in your article (pretty good one by the way!), but Google has a fair point, the level of "spam" is staggering. That said, they brought that upon themselves and upon us.

The barrage of crap we see on the Web is a direct consequence of the current economic model, where Google sucks all the money from the space, leaving only spoils to content creators. If there is no good money to be made on a single article, people will be forced to make up for this with 100 worse ones, exacerbating the problem.

This is true about sponsored content, but sadly also true for journalistic content. By acting as a toll-booth, Google (and other Big Tech) is killing the Web by pauperizing the content providers and co-opting the users.

One good example of those are instant answers: “weather tomorrow,” “how to fold a shirt,” “u2 bono height”. All those queries should send traffic to the sites, which then could be monetized. By providing an instant answer, Google deprives them of traffic.

As a matter of fact, Brave also has those instant answers, users demand it! What to do? The only option is to change the monetization model on the Web, with the current one being clearly unfair and favoring only opportunistic and predatory behaviors. This change is not going to come from the incumbents.

DKB: Does Brave Search use its own index, or is it relying on Google/Bing for some queries?

Yes, Brave Search has its own index, about +10B pages as of today.

Despite having our own index, we rely on Bing and Google in cases where we detect that our results are not good or complete. On the server side, we rely on the Bing API, and on the client-side (opt-in, Brave browser only), we rely on Google. The final mix of results, which we call the “independence score,” is 92 as of today, which means that 92% of the results come from our index and 8% from 3rd parties (more about our independence score here).

In June 2021, we started with 87, dropped to 85 in the following month due to traffic and a large fraction of non-English speaking users, but crawled back to 90 after a couple of months. Today we are at 92 and it’s sensible to think we will achieve 95 within the next 18 months. We are aiming for 100 eventually.

Note that the “independence score” is a knob that we can tune, it does not mean that Brave search can only answer 92% of the queries. It can answer 99.9% of them, but we (really, our ML models) decide that results could be better so we pull from 3rd parties. This mixing is needed, otherwise the users could face consistent bad results which would lead to churn. Search is a utility, and it has to work all the time, for all types of queries.

That’s why search engines like Mojeek and Alexandria, which have their own index and do not rely on 3rd parties whatsoever, have and probably will continue to have very little traffic. It’s a chicken-and-egg problem, which can be solved only by 3rd party mixing, which will help bootstrap a user-base, and in time will improve the owned and operated search engine toward 100% independence.

As far as we know, Brave Search is the only search engine doing this sort of mixing. Some others claim to do so (DuckDuckGo, Presearch, Startpage, Neeva, Swisscows, You, Kagi, the list is very long and getting longer every week), but they are not very transparent on how they operate.

It does not take a lot of effort to see that their reliance on 3rd parties is absolute – despite claims. The real test is simple: if Bing or Google were to cancel your API access, would you be able to continue operating? If any of the aforementioned say yes, then they are lying, they would not be able to.

Brave Search is not yet 100% independent either, but it can be, and it would continue to work even if all other search engines disappeared. Results would suffer a bit for that 8% mentioned above, but people would still be able to search.

That’s the independence we aim for, and as far we know only the big players (Google, Bing), regional champions (Yandex, Baidu, Ceznam), and self-sufficient search engines (Mojeek) have it.

DKB: Do you think ads as a business model is ruining the web?

First and foremost, online advertisement implementation as of today is a privacy nightmare. This is a fact that should require no further discussion. It should disappear sooner rather than later, that is what Brave (browser) is trying to do with its privacy-preserving ad system that also rewards users who opt in.

The model, however, is still valid. If privacy is preserved, ads just become a way of sponsoring: one either pays for the service with their attention (ads) or with money (subscription). Those two models are not mutually exclusive in any way.

The whole argument of “if you are not paying for the product then you are the product”, is problematic when the product is your privacy, because it’s a dangerous currency. It might be cheap now but can be very expensive in the future.

Attention, however, is time-bounded and well understood by whoever is paying. Ads of course will always be a nuisance, and there is a tendency to show more and more ads. The solution to that is competition.

The fact that under Sridhar’s watch (CEO of Neeva), Google increased the amount of ads beyond reasonable limits is caused by Google’s greed, of course, but also because of the lack of choices (of real choices, not theoretical ones, Google is not one-click away).

DKB: What’s your take on the whole “appending reddit to search queries” situation and what it means?

Appending Reddit to every query would not work for the majority of searches, however, it works very well on certain types of queries. There is a tendency, for myself included, to understate the large number of use-cases a search engine covers, and it has to cover them all.

People expect search engines to be a one-stop shop for all their needs – the driving wheel in the car following the previous analogy. So, no, Reddit would not be a good search engine based solely on their content.

They would cover certain types of queries, probably better than Google as you pointed out, but search is not just exploratory; in fact, the majority of queries have no exploratory intent. Reddit could be a vertical search engine, but a general purpose search engine is not an aggregation of N verticals.

DKB: You recently launched Discussions which allows users to get results from discussion forums like Reddit, Stack Exchange, and even smaller niche sites.

Can you talk more about that? Are you indexing every forum or only high quality / “trustworthy” forums?

We do not have, nor want to have, a taxonomy of good/bad forums. That is one step away from editorializing, and not a path we want to walk down.

Brave Search crawler and the subsequent indexing is governed mostly by the Web Discovery Project (WDP), so we are basically discovering and indexing based on popularity. Forums that no user in the WDP ever visits will not be part of our index.

We do not crawl, or to be more precise, we do not aggressively crawl. As previously mentioned, we want to minimize the garbage-in-garbage-out problem, so indexing the whole Web is not the goal, we prefer to limit it to the “relevant” Web.

Also, please consider that Discussions just launched, there are still many issues that can affect results. Just to give one example, at launch no forum powered by Discourse was classified as a Discussion, so they never showed up. That was just a bug, which we have already fixed, and Discussions is now able to detect and surface forums powered by Discourse.

DKB: There’s also an issue where a lot of discussions are happening in walled gardens / private communities like Facebook groups and Discord chats. Are you thinking about tackling these?

We cannot tackle those without the permission/collaboration of such walled gardens. Of course we would be happy to explore the possibilities if they occur.

Even if we had access there would be some complications though; for instance, surfacing Facebook group results would be great, but would logging into Facebook be required to see the content? It’s a predicament.

It’s not an entirely new problem, it already exists on sites like Pinterest, that requires login to get the proper content, or with The New York Times, which requires a subscription. Showing results (URLs) that require further actions down the road such as paying, subscribing, or logging in, is sub-optimal.

DKB: There was some criticism on Hacker News where people said you can’t trust random people on Reddit or any of these random forums, and they’d prefer to get expert opinions.

Of course, whether people trust a given forum is subjective, but have you thought about this idea of making it easy to get “expert opinions”?

That’s a great but difficult question. You already said it, it’s quite a subjective matter: there would be no consensus on what an expert opinion is, or whether an expert equates with impartiality. That’s an extremely difficult problem, for which there is no technical solution. We have always had this problem in society, the only way to deal with it is by having freedom and choice.

We aim to be as “neutral” as possible, relying mostly on popularity and recency, which of course have biases of their own, but they are the best amongst the bad. For particular needs, such as the one you describe, we believe that our Goggles project is the answer. Different communities of interest can create customized rankings, which could also affect Discussions. Time will tell if our belief is right or if we have to explore different solutions to cater to more particular interests.

One last comment about Discussions: It seems that Google might be testing a similar feature, and that they are rushing to do this kind of thing because real competition is entering the market.

Neeva, DDG, or Startpage, cannot be real competition, even though they could have nice features worth copying. Google and Microsoft will never be scared of them because they can be strangled at any time, simply by limiting or throttling access to their APIs.

A tenant cannot really challenge their landlord, and that's why at Brave we went with the long route of building an independent index.

DKB: Can you talk more about Goggles? It sounds like a promising concept.

Also is this something we can expect soon, or is it more of a long-term idea?

Goggles will allow multiple rankings to fit everyone’s needs. Think of personalization, but without losing privacy or suffering other problems that tracking-based personalization causes.

Why is it needed? Because if there is choice, there are biases. Any ranking or selection has some biases embedded in it. Which features are built, what data is used, time, popularity, all those have biases.

There is no neutral search — however, not neutral does not mean editorialized. Biases due to the nature of the data or technical choices are acceptable (as unavoidable), explicit biases based on editorials or personal beliefs are not (and should be avoided).

If the majority of people favor a particular school of thought, search ranking will reflect that. To counter this, we believe it’s wiser to leave groups of interests defining their own “re-rankers” so that they can benefit from our index.

We judge that group level personalization is better than individual level personalization because it’s explicit. The user must choose it, so the user can at least reason outside their echo chamber by removing the Goggle. Goggles will be publicly inspectable and collaborative, so they will have a reputation as trustworthy or not, independent of our personal opinion.

Goggles, we hope, will allow use cases that we can’t even think of, or that if we did, we could not offer to the entire user base. In an ideal world, those people should go and create their own search engine, but that is an extremely expensive proposition. Goggles bridges that by giving access to a large set of pages to which complex re-ranking functions can be applied.

It will not be perfect, but we believe it’s going to be a good step forward from where we are. Everybody looking at the world from the same window makes us poorer.

The aim is to release the first version of it around our first anniversary; it will not be feature-complete, but the basic components should be there on time (no promises though).

DKB: Is there anything else you want to mention about why people should use Brave Search?

Do you care about:

1. Privacy

2. The pernicious effects that Google and Big Tech have on the Web

3. Answering your query quickly

If you care about all of those 3 things, Brave Search is the only option there is. As of today, there is no one else in this position, though we hope one day there will be more.

If you are willing to sacrifice (c) then you could also use Mojeek.

If you are willing to sacrifice (b) there are plenty of options that act as private proxies to big search engines, such as Startpage (Google), DuckDuckGo (Bing), etc. It’s difficult to be a threat when your system depends on the incumbent you want to topple.

If you are willing to sacrifice (a) and (b) use Google. As of today, despite major issues, it’s still the one that does (c) better than anyone else on aggregate.

The next Google can’t just be an input box that spits out links. We need new thinking to create something much better than what came before.

In the last few years, different groups of people came to the same conclusion, and started working on the next generation of search engines.

For this new generation, privacy is necessary, and invasive ads are not an option. But that’s where the commonalities end. Beyond that, they’ve all taken the idea of a search engine in very different directions.

Kagi – The Customization Engine

Why should everyone have the same search experience? We all have our own preferences about how things should look and work.

Kagi is the most customizable search engine ever. You can change everything, from surface level appearances, to the final ranking of results.

“Everybody has different preferences of how they want a search engine to look and feel. Our goal is to provide tools and empower users to do that instead of trying to be smart and creating an average search engine for an average user. There’s no average human.” - Vladimir Prelovac, Founder of Kagi

Visual Customization

On the visual side, you can modify everything about the way things look, even being able to write your own custom CSS.

Search Customization

More importantly, you can customize the way the search engine itself works. Do you want search suggestions? How detailed should the snippets be?

There’s also a list of widgets you can toggle like “Instant Answers”, “Inline Discussions” and “Listicles”. These widgets are groupings of specific types of results which you can choose to include in the results page. If you never want to see another listicle again in your life, just toggle the listicle widget off.

But you can take the search customization even further than that, down to the level of individual websites. You can create your own domain rankings to tell Kagi what websites should be boosted, and what should be down ranked.

You can even outright block sites you never want to see again (looking at you pinterest), and pin sites if you want them always on top (hello reddit).

“Nobody has their home screen on iPhone looking exactly the same. I think people deserve that in search, especially being such an important part of our lives. It’s one of the most used products out there, and it’s astonishing how little customization you can do.” - Vladimir Prelovac, Founder of Kagi

Lenses

Kagi also allows you to create website filters called lenses. In any given lens, you can include or exclude specific sites, and more.

When searching you can turn on different lenses to see results that match those filters. Kagi comes with pre-built lenses for things like Discussions, Programming Help and Recipes.

Non-commercial Results

Another thing that Kagi focuses on is delivering non-commercial results. Kagi has an entire index of non-commercial websites, which they’ve also made available separately as Teclis. This allows them to find results that you won’t find on Google and most other search engines.

For example, when searching for “steve jobs”, the top results are similar to Google with mostly informational stuff. But right after that, we’re greeted by an “Interesting Finds” section, which has a fun blog post by Derek Sivers, an article filled with stories of Steve Jobs in Japan, and some other cool things you can’t find on Google.

Neeva – The Everything Engine

Why should a search engine only index the web? A lot of the information you care about is scattered across your email, your note taking app, and other places.

Neeva is the search engine that allows you to search everything.

Right now you can connect Google Apps, Dropbox, Notion and more.

No matter what you’re looking for, or where it is, Neeva will help you find it. Searching your personal documents is as simple as adding “@me” to the query. This gives everyone the ability to have a personal search engine.

This is one of many things that makes Neeva interesting.

Instant Inline Results

Since Google is an ads based business, they are incentivized to take you to the search results page, even when it’s obvious what you want.

If you typed in “amazon” hoping to get to amazon.com, Google would first take you to the search results page, so they can show you an ad, which amazon.com has to pay for just to be the first result for their own company name.

On the Neeva browser app, you get instant inline search results as you type, allowing you to go directly to the page you’re looking for. You can do in one click what Google will always want you to do in two clicks.

“We have the opportunity to innovate on the browser in a lot of interesting ways because we’re not beholden to how many search results page views we get with ads on them. Our business model unlocks our ability to help people not have to see the search results page as much.” - Darin Fisher, Head of Neeva Browser

And it’s not just a shortcut for navigating to domains. Type “best thai food” and immediately get a link to the relevant yelp page.

You can still go to the search results page if you need to, but this feature allows you to be twice as fast as Google.

Similarity Search

Another unique feature, only available on the Neeva browser app for now, is their similarity search functionality called NeevaScope.

Wherever you are on the web, you can tap the NeevaScope button to find similar and related pages to the one you’re currently on. This is powerful for exploration.

I tried it out on Tim Urban’s Neuralink post and found some other interesting content, including one that criticized Tim Urban’s post.

Filters

Neeva recognizes that there are different categories of websites that may be of interest to you for a given query, and provides filters to narrow things down.

“Google creates this view, I call it the flat-earth view, where every site is just like every other site. We go out of our way to tell you additional things you should know, and allow you to filter on these attributes.” - Sridhar Ramaswamy, CEO of Neeva

For example, when searching for legal information, you can filter to “Legal Providers” to see results from companies that provide legal services, or “User Generated Content” to get discussion forums or blogs.

Shopping Experience

One type of query that Neeva does particularly well at is shopping. Neeva knows that reviews are important, so they make it easy to find them.

The interface is geared towards helping you quickly figure out what products are worth looking more into, and connecting you to good review sources.

You.com – The App Engine

When you search a coding problem on Google, you’ll find a Stack Overflow answer. When you search on You.com, the Code Complete app will generate the code snippet from scratch using a language model.

You.com sees the web as a collection of apps, and their goal is to match your query with the most relevant apps. For coding questions, there is an AI Code Generator app, but Stack Overflow is also an “app” which you can open in the sidebar and easily copy code from.

The Quick Facts app can give you short factual information, and the Medium app allows you to easily read articles in the sidebar.

Eventually people will be able to build their own You.com apps with unique interactions, and publish them to the You.com platform.

Summaries

For many apps, you’ll be able to open a sidebar that has a summary of the important content from that page.

For example, if searching for food recipes, the All Recipes app and the New York Times Cooking app both allow you to open the recipe in the sidebar. This summarized view gives you exactly the information you want – a list of ingredients, and the instructions.

For coding, the Stack Overflow app and W3 Schools app allow you to click a “copy to clipboard” button to get the code.

Filters

Filters in You.com are a collection of apps particularly suited to some task. For example, the “YouCode” filter will give you apps like Stack Overflow, Code Complete, and Github, while the “Social” filter allows you to search social media apps.

YouWrite

This isn’t exactly a search feature, but it highlights the core strength of You.com, which is their AI capabilities. YouWrite is an AI writing assistant. You tell it what to write about, and it generates some paragraphs for you.

I told YouWrite to write about why everyone should watch anime, and this is what it gave me:

There are plenty of reasons why everyone should watch anime. For one, anime is a great form of entertainment. It can be exciting, thrilling, and suspenseful – just like any good movie or TV show. But what makes anime so unique is that it often incorporates elements of Japanese culture and folklore that you won’t find in other forms of entertainment.

Another reason to watch anime is because it can be educational. Anime has been known to teach people about different cultures, history, and even science! There are also many documentaries available that focus on various aspects of the anime industry – such as how certain shows are made or the history behind specific characters or plot lines.

Lastly, watching anime can help you learn Japanese! While not all shows are dubbed into English (or any other language), there are many websites where you can find translations for free. And if you’re feeling really ambitious, there are plenty of courses available online which will help you learn to read and write in Japanese too!

Something like this has the potential to become a powerful research tool. Instead of searching for “World War 2” and trying to find good sources, you can have YouWrite generate an essay about World War 2, and research the arguments it makes as a starting point.

As an aside, if you were wondering whether the Dead Internet Theory had any chance of being true, here’s your answer.

Andi – The Conversation Engine

Do search engines have to be so impersonal? We’re used to just typing our query and getting back a list of links, but Andi flips the script on this.

Andi gives you a conversational search experience that’s more like “messaging with a smart friend who answers questions and sends you useful links”.

Andi’s focus is on answering specific questions by extracting an answer from the best source.

For example, searching “what is the gdp per capita of china vs new zealand” on Google returns a badly formatted table without the answer.

While searching the same thing on Andi will extract the right information for you.

Dynamic Results

The way that Andi generates search results is different from the other search engines on this list. It focuses on understanding the intent and topics of your query, then tries to predict the best sources for your answer. After that, it queries the relevant APIs and vertical search engines directly, generating results in realtime.

There is no index.

Reader View

Another feature of Andi worth mentioning is the reader view, which allows you to view the content of pages without having to open them. This allows you to have a consistent browsing experience, and avoid ads and clutter.

Brave Search – The Independent Engine

It’s time to address the elephant in the room. Every search engine I’ve mentioned so far, no matter how many cool features they’re building, does not have a fully independent index.

They’re all getting some portion of their results from Google or Bing.

Building an independent index is a very hard problem given the current size of the web. You can count the number of search engines with a fully independent index on one hand.

Many search engines on this list aspire to have a fully independent index at some point, but none of them are more focused on this than Brave Search.

For any search you do on Brave, you’ll see what percentage of the results came from their own index. They make this number loud and clear, because they want it to be 100%.

“Brave Search can operate as stand-alone, the rest cannot as they rely on Google or Bing. Most search engines are not independent search engines, and while they may provide some value, they are qualitatively different from what Brave Search is doing. Independence is not something directly actionable, but it’s a fundamental property. Independence means that Brave Search would continue to work even if Google and Microsoft opposed it. Independence means choice and diversity.” - Josep M. Pujol, Chief of Search at Brave

You won’t see much cool new functionality on Brave Search, because they are laser focused on the one thing they believe to be important to a search engine.

Web Discovery Project

The web discovery project is Brave’s solution to one of the major problems with privacy focused search engines. Google can provide good results because they collect a ton of data on everyone, and they use that to improve the search experience. Privacy focused search engines don’t collect that kind of data, which limits their potential.

With the web discovery project, users can opt-in to contribute anonymous search and web browsing data to help improve Brave’s results. This gives Brave a way to get the same kind of data that Google uses to optimize queries, but without comprising on user privacy.

If you don’t like it, you have nothing to worry about, because it’s off by default.

Goggles – Collaborative Mini Search Engines

The most important innovation of Brave is yet to be released. Goggles allows users to create mini search engines within Brave. Anyone can create detailed ranking specifications that Brave will use when returning results. People can collaborate on goggles and share them.

Some examples of goggles that could be created are “top tech blogs”, “drop the top 1000 most popular domains” (which implements the general idea of an older search engine, MillionShort), or “cooking sites my mom would recommend”.

Goggles are easily toggled on and off, so users can leave and enter different filter bubbles as they please. Unlike Google, these filter bubbles are intentionally created by groups of people, not decided by one centralized algorithm that’s working in the background. Goggles can be a useful tool to explore the web from different perspectives.

Goggles is effectively making search engine creation accessible to anyone. It’s very hard for a regular person to build an entire search engine from scratch, but it will be easy for anyone to create a goggle that represents their idea of a perfect search engine.

This Is Just The Beginning

This new wave of search engines is only just getting started. Many of them have only recently launched.

Even if they aren’t perfect yet, the paths they’re exploring can lead to promising new innovation in the stagnant search space.

“The search technology was developed a decade ago. It’s a bet that no one else will come up with a better search technology. So you invest in Google because you’re betting against technological innovation in search.” - Peter Thiel, 2012

Betting against innovation in search has been a great move for the last ten years. The next ten years look a little bit more hopeful.

I’ll be posting interviews with the founders of each of these search engines soon. If you’re interested in getting a more in-depth look at their stories, and how they think about things, stay tuned by signing up for my newsletter.

Appendix 1: Interview with Brave Search

I have now published a full length interview with Brave Search which you can read here.

Appendix 2: Interview with Neeva

You can read a full length interview with Neeva here.

Appendix 3: Interview with Kagi

You can read a full length interview with Kagi here.

The websites at the top of Google are not the highest quality websites, but the ones that put the most effort into SEO [1] [2]. There is an entire industry whose purpose is to game their way to the top of search results, which inevitably leads to those results being debased over time.

We need a ranking algorithm that cannot be easily gamed. This paper describes MarketRank, a ranking algorithm that is immune to SEO and simple to compute. We discuss the advantages and limitations of MarketRank, and show how it compares to Google’s rankings.

1. Introduction and Motivation

Google’s search result quality has declined to the point where many have noticed [3][4][5].

High quality websites that don’t optimize for the right keywords and backlinks are left behind in favor of more SEO optimized websites. This has led to websites that many consider to be “spammy” or “low quality” being at the top of Google results.

A ranking algorithm can be judged by the results it produces when people try to game it. What we need is a ranking algorithm that can only be gamed by creating genuinely good content. If “gaming the system” and “creating high quality content” are the same thing, then we have created a good algorithm.

Quality is subjective, and many believe that Google’s results are still good enough. However, we believe that the kind of quality produced by MarketRank is distinctly different and useful.

2. The MarketRank Algorithm

2.1 The Market Analogy

PageRank [6] was built on the analogy of the research paper. A research paper with more citations is probably better.

MarketRank is built on the analogy of the market. An object with a higher market value is probably better.

In our case, the markets we refer to are online communities such as Reddit, Hacker News, and Twitter. Each of these communities has an upvote mechanism, which can be interpreted as an indication that the user believes the website is undervalued. They are willing to bid up the price by spending their upvote on it.

And so, the oversimplified description of MarketRank is “just add up all the upvotes”. We have some more work to do to make the values accurate, but this is the general idea.

2.2 Raw Market Value

The raw market value of any website is the main score that it received in an online community. Reddit and Hacker News have an obvious main score in the number of upvotes. On Twitter there are multiple metrics to choose from, so for now we’re going to choose likes as our main score.

Once we have the main scores for a website across all communities, we can turn these raw values into something that’s useful for comparison. We need to adust for inflation, and convert everything to one currency.

2.3 Inflation

The top upvoted post of 2015 on Hacker News has 2,228 upvotes, while the top upvoted post of 2017 has 4,107 upvotes. Over time most online communities grow steadily, but they all have a recency bias. This means that all the new users don’t end up voting on older content.

This leads to a fluctuation in the true value of an upvote. As a platform gains more users, votes become easier to accumulate, and less valuable.

We cannot merely take the raw score from a platform, otherwise we will suffer from a recency bias, and will not be able to tell what the best websites are.

To adjust for inflation, we compare the value of an equivalent basket of goods over time. In the case of our naive implementation here, our basket of goods is the top 50 most upvoted submissions of the year. We use the previous year as our baseline, since this year is incomplete, and convert everything to 2021 points.

For reference, here is a graph of inflation on Hacker News for blogs in the Blog Surf directory [7].

2.4 Currency Conversion

After adjusting for inflation, we have denominated everything in 2021 points per platform, but now we have another problem. The points are all for different platforms. How much Twitter likes is equal to a Hacker News upvote?

We must denominate everything in one currency to be able to make meaningful comparisons between them. It makes no difference what currency we choose, so in this case we will go with Reddit upvotes.

Since we’ve adjusted for inflation on every platform, we’re really converting everything to 2021 Reddit upvotes.

Our naive currency conversion will work exactly like our naive inflation adjustment. We will compare the cost of a similar basket of goods across the different platforms. In this case, our basket of goods will be similar to our inflation calculation, i.e the average of the top 50 highest ranking inflation adjusted websites on a platform.

Here is the difference in value of that basket of goods across platforms.

2.5 GDP

Finally we have a value we can use to compare scores across any platform and from any year. We can now calculate the “GDP” of a domain by adding the total value of all webpages produced by a domain.

We only count each webpage once, and if there are multiple values for a webpage, we choose the highest value the page has received. We do this because we believe it is extremely unlikely that there is a false positive in any moderated online community, though there may be a lot of false negatives.

By finding the sum of all value that this domain has produced, we can come up with a ranking for the domain.

Here we have a sample showing the top 10 domains in the Blog Surf directory ranked by GDP.

3. Google vs. MarketRank

This all sounds good in theory, but does it work in reality? Let’s find out by comparing MarketRank and Google.

Google doesn’t publish its site rankings, and any Google search will be a mix of relevance and quality. We have tried to approximate the quality ranking of a page by finding queries where all the results seem equally relevant. This means that the determining factor of the result order must have been Google’s quality ranking.

This is a noisy and non-ideal measurement, but will be good enough to tell us something about the performance of MarketRank.

3.1 Zero To One Book Review

We will try to find some book reviews of Peter Thiel’s Zero To One with the query “book review zero to one”.

Google and MarketRank both agree that the Atlantic post is the best, but after that we diverge significantly. The rest of Google’s first page of results have a MarketRank of 0, meaning they are either low quality, or unknown quality.

On page 2 we find Farnam Street and Slate Star Codex, both of which have a decent MarketRank.

Here is a table showing the MarketRank of each of Google’s results.

We do not think it is much of a leap to say that the Farnam Street and Slate Star Codex articles are higher quality than most of the articles on Google’s first page of results.

We are also going to take a wild guess that the Times of India focuses more on SEO than Slate Star Codex does.

3.2 Startup Ideas

Another way to try to figure out what pages Google thinks are high quality is with an exact match search. This is still noisy, because it may be more susceptible to keyword stuffing and other SEO hacks. But in general, we assume that it will look at all the pages that have the exact matching phrase, then rank them by quality.

Our exact matching phrase is “startup ideas”.

We are greeted first by an SEO optimized listicle from Nerdwallet, then another listicle, then a video on the YC site, then more listicles.

The discrepancy between this search result and the results produced by MarketRank are surprisingly high. Nothing on the first page of results has a MarketRank.

On page 4 of the results, we finally get to Paul Graham’s “How To Get Startup Ideas”.

No matter what Google’s search algorithm is doing to match the phrase “startup ideas”, we cannot make much sense of the fact that a page titled “How To Get Startup Ideas” on one of the most popular startup blogs, is considered by Google to be page 4 garbage.

MarketRank knows this page has been shared and upvoted a lot across various communities, so it must be important. It ends up having a MarketRank of 3,389 points.

Other notable pages that Google considers garbage but MarketRank considers good are "Startup Ideas" by Gwern (399 points), and "Developing new startup ideas" by Chris Dixon (447 points).

After looking into the results more, it seems that Google may be optimizing so hard for recency that it doesn’t care at all about these older articles. That may explain this strange result. Perhaps Paul Graham should change his title to “How To Get Startup Ideas in 2022”.

3.3 CSS Centering Guide

If there’s one thing that every web developer has searched, it is the infamous question of how to center something in CSS. In this case, we chose the specific query “css centering guide” because those words appeared in various titles, reducing the noise of relevance, and making the result order mostly Google’s idea of quality.

This query shows that MarketRank and Google don’t always have to disagree.

In this case, we agree on what pages are high quality, though disagree slightly on the exact ordering of them.

3.4 Spam Detection

This is a small example that demonstrates the anti-SEO properties of MarketRank. When searching for “biggest mistakes that kill startups” on Google, we would expect to be greeted by the Paul Graham essay with a pretty similar title “The 18 Mistakes That Kill Startups”.

Instead, we are greeted by an article from Business Insider India called “Biggest Mistakes that kill Startups”. They do have the word “biggest” in the title, so maybe that makes it more relevant.

The real kicker is that “Biggest Mistakes that kill Startups” is a cheap copy-paste of Paul Graham’s essay with ads all over it.

MarketRank would have given that website 0 points, whereas Google believes it is a legitimate website.

In general, we don’t expect that a spammy or low-quality website would get much points on any legitimate online community, and believe that MarketRank is naturally good at filtering out spam.

4. Limitations and Future Work

This is a very naive implementation of MarketRank, and there are many obvious ways to improve it. However, beyond improving the implementation, there are more fundamental issues with MarketRank which we will discuss in this section.

4.1 What an Upvote Really Means

We claimed that upvotes were a good measure of quality but this is not necessarily true. In reality, an upvote on any platform is a combination of quality and other things such as the popularity of the author, quality of the discussion produced by the article, and many more.

It is unclear what proportion of the value of an upvote is quality vs. everything else. Finding a better way to extract quality from upvotes will be an important problem to work on moving forward.

4.2 False Negatives and Low Coverage

One of the biggest issues with MarketRank is that many websites won’t have a MarketRank. A lot of the best websites will have one, but there will also be many good websites without a MarketRank.

The nature of these online communities leads to few false positives, but many false negatives. Many good webpages will not make it to the main feed or front page because of bad timing.

There’s only so much room on the front page, so no matter how much good stuff is submitted on a given day, only a few can get the opportunity to be fairly valued by the market.

Perhaps this should be fixed, but perhaps it is a feature and not a bug. It may be okay to optimize for a low false positive rate and high false negative rate. Most stuff on the internet is junk, and we may be okay with missing some good stuff as long as the stuff we have is definitely good.

5. A Blog Search Engine

In order to experiment with MarketRank, we developed a blog search engine called Blog Surf. Since blog posts are likely to be shared in online communities, we believe that MarketRank is particularly well-suited to the purpose of ranking blog posts.

There is an endless list of edge cases, improvements, and open questions for MarketRank. These questions cannot be solved in theory, only in practice. Blog Surf will serve as the experimentation ground for future iterations of MarketRank and other ranking algorithms we may come up with.

6. Conclusion

We have proposed a new method for ranking the quality of websites that is immune to SEO. We start off by counting upvotes from online communities, then make these numbers useful for comparison by adjusting for inflation and converting to one common currency.

We have shown that MarketRank is capable of producing a better quality ranking than Google for some queries.

This paper is published in hopes that you use it in your search engines and directories. There are many things left to be built when it comes to organizing information on the internet, and we hope that this simple algorithm will make it easier to have a baseline quality metric in such projects.

References

1. The top-ranking HTML editor on Google is an SEO scam (https://casparwre.de/blog/seo-scam/)

2. The mermaid is taking over Google search in Norway (https://alexskra.com/blog/the-mermaid-is-taking-over-google-search-in-norway/)

3. A New Google (https://dcgross.com/a-new-google)

4. Google is no longer producing high quality search results...(link)

5. Google [OC] (https://www.reddit.com/r/comics/comments/svgabe/google_oc/)

6. The PageRank Citation Ranking: Bringing Order to the Web (http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf)

7. Blog Surf (https://blogsurf.io)

Reddit is currently the most popular search engine. The only people who don’t know that are the team at Reddit, who can’t be bothered to build a decent search interface. So instead we resort to using Google, and appending the word “reddit” to the end of our queries.

Paul Graham thinks this image means Reddit as a social media site “still hasn’t peaked”. What it actually means is that the amount of people using Reddit as a search engine is growing.

Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying. The long answer is that most of the web has become too inauthentic to trust.

How do we know Google is dying?

If you’ve tried to search for a recipe or product review recently, I don’t need to tell you that Google search results have gone to shit. You would have already noticed that the first few non-ad results are SEO optimized sites filled with affiliate links and ads.

Google still gives decent results for many other categories, especially when it comes to factual information. You might think that Google results are pretty good for you, and you have no idea what I’m talking about.

What you don’t realize is that you’ve been self-censoring yourself from searching most of the things you would have wanted to search. You already know subconsciously that Google isn’t going to return a good result.

I’m far from the only one who thinks Google is dying:

Daniel Gross

In 2000, Google got popular because hackers realized it was better than Lycos or Excite. This effect is happening again. Early adopters aren’t using Google anymore.

Ask HN: Has Google search become quantitatively worse?

The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries. Forced synonyms and "people also searched for" are typically useless and almost infuriating. Once you get off the first or second page, the results get even worse-- with pages entirely unrelated to the query (e.g. not even containing the searched phrases). They are probably testing/already implemented some sort of multi armed bandit type optimization like on Youtube's search results where they just show any popular pages (ignoring relevancy) to see if they yield a click.

Michael Seibel:

A recent small medical issue has highlighted how much someone needs to disrupt Google Search. Google is no longer producing high quality search results in a significant number of important categories.

Health, product reviews, recipes are three categories I searched today where top results featured clickbait sites riddled with crappy ads. I’m sure there are many more.

Paul Graham

This may not just be a problem with Google but possibly also the recipe for beating Google. A startup usually has to start with a niche market. Why not try writing a search engine specifically for some category dominated by SEO spam?

Why is Google dying?

Ads

It is obvious that serving ads creates misaligned incentives for search engines. The founders of Google pointed this out themselves when they were just getting started.

Sergey Brin and Lawrence Page (1998)

“Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users…we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers…Furthermore, advertising income often provides an incentive to provide poor quality search results.”

Unfortunately, these thoughts on the failings of ad-based search engines read like an instruction manual for what Google did next.

They’ve dialed it up to the max recently to squeeze out every last cent before their inevitable collapse.

SEO

There are tons of people whose sole job is to game their way to the top of Google, so it shouldn’t be surprising when search results deteriorate in quality. To be fair, this would probably be an issue with any search engine, but you’d expect Google to be able to come up with a less gameable algorithm.

Here’s a fun story about how one website gamed their way to the top of Google in Norway.

AI

Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant”, in addition to personalizing things for you. If you really meant exactly what you typed, then all bets are off.

Even the exact match query operator (“ ”) doesn’t give exact matches anymore, which is quite bizarre.

Why are people appending “reddit” to their queries?

There’s a fun conspiracy theory that popped up recently called the Dead Internet Theory. The claim is basically that most of the internet is bots. There aren’t real people here anymore.

IlluminatiPirate:

TLDR: Large proportions of the supposedly human-produced content on the internet are actually generated by artificial intelligence networks in conjunction with paid secret media influencers in order to manufacture consumers for an increasing range of newly-normalised cultural products.

This isn’t true (yet), but it reflects some general sense that the authentic web is gone. The SEO marketers gaming their way to the top of every Google search result might as well be robots. Everything is commercialized. Someone’s always trying to sell you something. Whether they’re a bot or human, they are decidedly fake.

So how can we regain authenticity? What if you want to know what a genuine real life human being thinks about the latest Lenovo laptop?

You append “reddit" to your query (or hacker news, or stack overflow, or some other community you trust).

Google is dead.

Long live Google + “site:reddit.com”.

(For more thoughts on search, follow me on twitter @dkbrereton)

Appendix 1: response from Google.

Google claims the exact matching feature is merely unintuitive, not broken

Danny Sullivan (Google's public Search Liaison):

I work for Google Search, passed your feedback along, thanks. You said in the post that quotes don't give exact matches. They really do. Honest. Put a word or phrase in quotes, that's what we'll match. If anyone has an example where they feel it doesn't, please let me know...

Here's why people often think quoting with Google doesn't work when it really does (I've looked a huge number of these reports).

1. We match ALT text

2. We match text not readily visible, such as in a menu or small text

3. Page has changed since we indexed it

4. Punctuation...

Punctuation comes into play if you did a quoted search like "dog cat" and there's text that says "dog, cat" then we'll see that without the punctuation. That doesn't seem a major issue but we're looking at if we could improve there.

And Google seems to be right so far

Every example of exact matching not working so far can be traced back to issues with punctuation, alt text, etc.

For example, one proposed failing query was "quotes don't give". But HN user saalweachter has pointed out that it is a very intricate punctuation problem.

For instances, on the ["quotes don't give"] example, the first result I get is https://www.goodreads.com/quotes/tag/never-give-up

If I do a find-in-page for "quotes don't give", I get zero results. Oh no! Perfidy!

... but, if you look more closely, you'll find this string waaaaay down at the bottom:

tags: don-t-give-up, don-t-give-up-on-your-dreams, don-t-give-up-on-yourself, don-t-give-up-quotes, don-t-give-up-the-fight, encouragement, ...

Thanks to the wonders of tokenization, that "don-t-give-up-quotes, don-t-give-up-the-fight" gives you the string of tokens, "don t give up quotes don t give up the fight", which contains the exact phrase "quotes don t give", which is the tokenization of the phrase "quotes don't give".

Danny Sullivan (Google's public Search Liaison):

And...as noted in another reply below, in the first result, if you look at the cached page that shows the ENTIRE page that we indexed (rather than the paginated version you land on), there's this text:

quotes, don-t-give-up-the-fight

with when you remove the punctation, is the match:

quotes dont give

And I get that this can be frustrating, that we don't consider punctation in a quoted search. That's not a new change, however. It's been that way for ages. But as I also said, it's something we might revisit.

Appendix 2: Google just had record profits, you are obviously wrong, and Google isn't dying.

I never said that Google wasn't making money. In fact, if Google really is squeezing every last cent out of ads without regard to search quality, I would expect their revenue to be at an all time high.

I am saying that the quality of search results are declining. This may eventually lead to a decrease in revenue, but has not yet.

Appendix 3: Seriously, what are you talking about? My search results are perfect.

If you think your search results are perfect (without appending reddit), then you're probably right. If every single person agreed that Google search results were trash, then Google would already be bankrupt.

Perhaps it is more likely that 80% of people think Google is good enough, and 20% think Google sucks.

I do suspect that the 20% will be growing in number though.

Appendix 4: *Yawn*, this is the 87th time someone has claimed that Google search is dying in the last 20 years. This is a big meme in the SEO world.

"The reports of my death are greatly exaggerated" - Google, probably

You're right, there's been a new article bashing Google every few months for the last 20 years straight. It's probably nothing.

Still, it is a bit interesting that this short and simple post is now one of the most upvoted things of all time on Hacker News. There must be a lot of people who resonate with it this time around.

Hard to tell if something significant has changed.

Appendix 5: Random redditor explains it succinctly

u/a_latvian_potato:

I think I understand what this article is trying to say. It's not saying that Google's search technology is worse or that people don't use Google to search. It's saying that people trust less of the results Google shows compared to seeing discussions of it on Reddit.

For instance, if I'm looking to see reviews of the Honda Civic 2022 or whatever, I actually do find myself typing "Honda Civic review reddit" instead of "Honda Civic review". This is because I want to see what real people and enthusiasts (on r/cars or whatever) are talking about the car, rather than the top results at Google which are basically just paid reviews advertising the car anyway.

Even though I kinda know people in Reddit are just as capable of spouting BS that are completely wrong, I find the discussions more authentic anyway than the corporate speak the "big websites" have on their articles that Google shows me.

Appendix 6: A fun comic by @zenacomics

Appendix 7: More thoughts on organizing information on the internet

If you want some more in-depth thoughts on this subject, you should check out this post I wrote a year ago about all the different ways we could organize information usefully on the internet.

My mental model of reading looked something like this:

The words go from the page, to your brain, and now you have knowledge. The author is lecturing and you are listening to all of this great information and getting smarter.

I’ve recently learned that’s not how things work. If you really want to gain something from reading a book, it should look more like this:

To gain true knowledge and understanding from the book, you have to go to work on it. You and the author need to have a conversation, and it’s up to you to figure out what’s true in the end.

Reading a lot of books doesn’t make you a good reader. Two people can read the same book, and one of them can gain a totally new understanding of the world, while the other learns a few points they can regurgitate in conversations.

To be fair, you may be reading for entertainment, and in that case you should have fun. You may be reading for information, as is the case with the news, and in that case you just need to get facts from the page into your head.

In this post, I’m talking about reading for understanding and insight, and specifically non-fiction books. A good book can improve your mental model of the world, increase your overall knowledge, and change the way you operate. But reading for understanding requires you to put in more effort than the other kinds of reading.

The first part of this post are the high level ideas behind this style of reading, which comes from the book “How To Read A Book”.

The second part of this post is a practical system I came up with to implement the ideas of this book. If you don’t care about theory, and just want to know what to do, skip to the second part.

Theory

Levels of reading

Elementary reading is being able to translate symbols on a page into meaningful chunks of information in your head. You already know how to do this, otherwise this would look like gibberish to you. For languages you don’t know, you are below the level of elementary reading.

Inspectional reading is a light and fast reading of a book which gives you an idea of what the book is generally about. It is useful for deciding whether or not to read a book, preparing for a book you are going to read more deeply, and dealing with very difficult books that are beyond your level.

Analytical reading is an in-depth reading for understanding and insight.

Syntopical reading is connecting multiple books that present various views on the same subject.

In general you’ll spend most of your time doing Analytical and Inspectional reading, so that’s what I’m going to focus on in this post.

Inspectional Reading

Inspectional reading is all about trying to get value from a book in a limited amount of time. There are two types of inspectional reading.

Pre-reading is something you’ll want to do on every book you read. It helps you decide whether or not to read the book in the first place by giving you a general idea of what to expect. 99% of books are not worth reading in-depth, so this is a useful ability.

If you’ve already decided that a book is worth reading analytically, pre-reading will give you insight into the structure and style of the book, which will increase your understanding.

Superficial reading is a tool for reading hard books that you’re scared to touch. What counts as a hard book is subjective to you and your level of reading, but classics like Critique of Pure Reason and The Wealth of Nations will probably sound hard to most people. Old classics are harder to read, so most people don’t read them. This is unfortunate since these are the books most worth reading, because they’ve stood the test of time.

Thankfully there is a hack to reading these books. To read superficially, you simply read through the book without stopping on things you don’t understand. If you see words you don’t understand, carry on until you find some you do understand. If a passage sounds like complete gibberish, carry on until you find a passage that makes sense.

In the end you might only understand 50% of the book, but now you have the right context needed to do an in-depth reading and understand the rest of it. Even if you never go back to do a more in-depth reading, understanding 50% of a hard book is much better than knowing nothing about it. (Note: This does not imply that understanding parts of a book means you fully understand the author’s arguments. You should definitely not have some false confidence that you truly know the book. You do, however, know more of what the book is about than most people.)

Analytical Reading

Analytical reading is reading for understanding and insight. The process of analytical reading can be summarized by four questions that you need to get the answer to for any book you read.

The Questions

1. What is the book about at a high level?

2. What specifically is the author saying?

3. Is the book completely true, partially true, or not true at all?

4. What now?

To get the answers to these questions, we have 15 rules of analytical reading to guide us.

Rules for finding out what a book is about

Classify the book according to kind and subject matter

You’ll be able to read a book better if you understand what kind of book you’re reading. We can classify non-fiction books broadly into two kinds: theoretical and practical.

Practical books are geared towards getting something done in the real world. An instruction guide on how to read a book is practical, as is an economics book about how to build the perfect economic system.

Theoretical books are more about ideas and concepts. A theoretical economics book might talk about whether it is even possible, in theory, to create a perfect economic system.

State what the whole book is about briefly

Write a few sentences representing the general idea of the book.

Enumerate its major parts in their order and relation, and outline these parts as you have outlined the whole

Break down the book’s structure, identify the parts of it, and state what each part is about. Then break each part down into sub-parts and repeat the process. You can do this to an infinite depth, so you’ll have to decide when to stop based on how complicated the book is, and how in-depth you want to read it.

Define the problem or problems the author has tried to solve

Every book represents an author’s attempt at solving one or more problems. Those can be theoretical problems like the extent of human knowledge, or practical problems like how to save the world. You need to know what problems the author is trying to solve in order to identify the important propositions and arguments.

Rules for interpreting a book’s contents

Come to terms with the author by interpreting his key words

A word or phrase can have many different meanings. A term is a specific meaning of a word or phrase.

The word “blog” can refer to a website where someone posts their personal life experiences, or a big media publication that writes long-form essays, or an email newsletter.

If I write an article titled “How to make a great blog”, you’d expect different advice if I was talking about making a large media publication vs a personal travel blog. As a reader, your job is to figure out from my use of the world “blog” exactly what I mean by that. If the meaning of “blog” you understand, and the meaning of “blog” I intended are the same, we have come to terms, and can have a productive conversation.

It is worth stressing the importance of this point because I think many arguments and disagreements are a result of people not coming to terms with each other. If two people are arguing about immigration, and one of them is referring to the immigration of skilled labor, while the other is referring to people illegally crossing a border, a meaningful discussion cannot take place. These people can only talk past each other, and leave the conversation having gained nothing.

Grasp the author’s leading propositions by dealing with their most important sentences

A proposition is an assertion about something. “Everyone should start a blog” is a proposition. It is a declaration that the author believes something, or doesn’t believe something. Of course, without any actual reasoning, the proposition is merely an unjustified opinion. This leads us to the next rule.

Know the author’s arguments, by finding them in, or constructing them out of, sequences of sentences

Unless someone is writing random thoughts, they probably have arguments for their propositions. You need to find those arguments, which may all be in the same paragraph, or may be in sentences spread out across different paragraphs. In order to determine if the author’s propositions are true, you need to find these arguments so you can assess them.

Determine which of their problems the author has solved, and which they have not; and of the latter, decide which the author knew they had failed to solve

In the beginning, you identified what problems the author was trying to solve. Once you finish reading the book, you can determine if the author solved these problems, if they created new problems along the way, and if they were aware of the problems they didn’t solve.

Rules for criticizing a book

General maxims of intellectual etiquette

Do not begin criticism until you have completed your outline and your interpretation of the book (Do not say you agree, disagree, or suspend judgment, until you can say “I understand”)

If you don’t understand something, then you can’t reasonably judge it.

Do not disagree disputatiously or contentiously

Don’t argue for no reason.

Demonstrate that you recognize the difference between knowledge and mere personal opinion by presenting good reasons for any critical judgment you make

“This book sucks because it’s stupid” isn’t good enough. Make a real argument.

Special criteria for points of criticism

Show where the author is unformed

The author is uninformed if they don’t have knowledge of something important. This happens a lot in history books, where people who wrote older history books didn’t have as much information as we do now. The book may be missing things because the author lacked the relevant knowledge.

Show where the author is misinformed

An author is misinformed if they have an incorrect notion of something. This is similar to being uninformed, but different because in this case they specifically hold an incorrect view of something and have made an assertion based on this factually incorrect information. For example, some older books may assume that the sun revolves around the earth, and use that as a basis for their arguments.

Show where the author is illogical

An author may be well-informed, but still use faulty logic in their arguments. For example: “Some people with blogs are rich and famous, therefore if you start a blog, you will become rich and famous.” It is true that some people with blogs are rich and famous, but it does not logically follow that I will definitely become rich and famous if I start a blog (still, a man can dream).

Show where the author’s analysis or account is incomplete

If the author is neither uninformed, misinformed, or illogical, you effectively agree with them. But even if you agree with what they wrote, it is possible that they didn’t fully solve the problems they were trying to. The book may be incomplete in some way.

You may agree with this article, but find it lacking when trying to read a history book. I didn’t address how to approach specific types of non-fiction books, so my article is incomplete in that regard.

Practical

In order to make all of this theory useful, we have to turn it into some kind of practical system. Here is one system for implementing better reading. (This is a work in progress. I’ll update this post as my system changes.)

Before diving into this, it is worth noting that this whole system may seem like a ton of work. It represents the most in-depth reading that can be done. In reality, you should vary how much of this stuff you do based on how important you think the book is.

“Some books are to be tasted, others to be swallowed, and some few to be chewed and digested; that is, some books are to be read only in parts; others to be read, but not curiously; and some few are to be read wholly, and with diligence and attention.” - Francis Bacon

Pre-reading

Always do a pre-reading before doing analytical reading.

You probably already have some intuition about how to skim a book, and there are no hard rules, but here are some suggestions on how to pre-read a book:

1. Look at the title page and the preface

2. Study the table of contents to obtain a general sense of the book’s structure

3. Check the index

4. Read the publisher’s blurb

5. Look at the chapters that seem to be pivotal to the main argument of the book

6. Turn the pages, dipping in here and there, reading a paragraph or two, sometimes several pages in a sequence, never more than that.

One other thing I’ve found helpful is listening to an interview with the author where they talk about the book. This is one of the easiest ways to figure out the main problem the book is trying to solve, and the main propositions and arguments.

The Template

The first part of the system is a notetaking template that you’ll fill out as you read the book. You should be able to fill out some parts of it after your pre-reading.

Book Title:

Author:

Goodreads Link:

Kind of book: (practical or theoretical)

Subject: (mathematics, philosophy, science, economics, etc.)

What the book is about

(This should only be a few sentences long)

What problem(s) is the author trying to solve?

• Problem 1

  • Did they solve it by the end? Did they create any new problems while trying to solve it?

    • Yes/No

      • If no: Did they know they failed to solve it?

    • Newly Created Unsolved Problem 1

      • Did they know they failed to solve it?

• Problem 2

Outline

(Outline the structure of the book by copying the table of contents if there is one. If there isn’t a good table of contents, you’ll have to do a bit more work to figure out the sections of the book. Do this after your pre-reading. You can take notes within this structure as you read chapters of the book. Write a brief summary of each chapter when you finish it.)

• Chapter 1

  • Brief summary

• Chapter 2

Summary

(Prove that you understand what the book is saying by writing a summary of it in your own words.)

Criticism

(The criticism is effectively a review of the book. Criticism can be positive.)

Review the rules for good criticism:

• Do not begin criticism until you have completed your outline and your interpretation of the book (Do not say you agree, disagree, or suspend judgment, until you can say “I understand”)

  • If you don’t understand something, then you can’t reasonably judge it.

• Do not disagree disputatiously or contentiously.

  • Don’t argue for no reason.

• Demonstrate that you recognize the difference between knowledge and mere personal opinion by presenting good reasons for any critical judgment you make.

  • “This book sucks because it’s stupid” isn’t good enough. Make a real argument.

Points where the author is uninformed

Points where the author is misinformed

Points where the author is illogical

Points where the author’s analysis is incomplete

Conclusion

If you have found no points where the author is uninformed, misinformed, or illogical, then you must agree with them.

However, even if you agree with them, they might not have fully solved all of their problems. They may have failed to mention important things, or the work might be otherwise lacking in some way. In this case, you should note the limitations of the work.

In the end you can either agree, disagree, or suspend judgment on the whole (i.e you may agree with what is written, but the author’s analysis is too incomplete, so you cannot truly judge it.)

While You Read

I read using the kindle app on desktop, so this system will be specific to that, but you can easily adapt it to whatever you use.

This is all going to sound very intense, so let me make this clear again: you don’t have to do every single one of these things for every book you read. Some books are important to you and you’ll want to go hard on them. For others you might barely do any of this. That’s fine, and that’s how it should be.

When you see a key word appear for the first time, highlight it in blue and add a note explaining what the author means by it (the term). If that same key word appears somewhere else with a different meaning, then do the same thing. Otherwise, if it refers to the same term, no action is needed.

When you identify an important proposition, highlight it in red.

When you identify a sequence of sentences that make an argument for a given proposition, add a note to the start of each relevant sentence that makes up the argument, indicating its number in the sequence (i.e 1, 2, 3). When adding the note to the first sentence in the sequence, also state the proposition that the argument is for. (e.g “1. Minimum wage needs to be increased” where 1 means it is the first sentence in the argument, and the proposition is that minimum wage needs to be increased)

When you read something that you want to respond to, or have thoughts on, have a conversation with the author by adding a note to the relevant passage. (kindle will automatically highlight these in yellow)

If you have general thoughts that are not specific to any particular passage, leave a note on the first word on the page (kindle will not highlight anything).

If something stands out to you as interesting or important, but you don’t have any comment on it, highlight it in orange.

Now when you’re going through the book again, you know that:

• Blue highlights means a key word is appearing as a new term (i.e with a new meaning) for the first time.

• Red highlights are important propositions made by the author.

• Yellow highlights are your thoughts and questions on specific passages. They represent your conversation with the author.

• Orange highlights are things that stood out to you.

• Numerical notes (possibly with no highlights) are sentences that are part of important arguments.

• The first word on a page may have a note with general thoughts that are not related to a specific passage.

Conclusion

We all suck at reading, but it is possible to suck less. The more work you’re willing to put into reading, the more you’re going to get out of the book. Some books just need to be tasted, while others should be chewed and digested.

Hopefully this system can help you in your journey to read better. If you have any thoughts on how to improve it, let me know.

“For those of us who are not in school…our continuing education depends mainly on books alone, read without a teacher’s help. Therefore if we are disposed to go on learning and discovering, we must know how to make books teach us well.” - Mortimer J. Adler and Charles Van Doren (How To Read A Book, p. 15)

(This post was based on “How To Read A Book”, and if you thought these ideas were interesting, you should definitely read it, as it goes a lot more in-depth than I did here.)

A really long time ago, we didn’t have much writing, and the information was passed down orally. Then we had writing, but it was scattered and disorganized. You had no way to know what writing existed, what thoughts people had already, and generally what was out there.

Around 300BC, one of the most ambitious projects of all time was undertaken in the Greek-Egyptian city of Alexandria. The Ptolemies wanted to create the intellectual capital of the world, and so they established the famous Library of Alexandria. This was not the first library, but what made it interesting was their extremely ambitious goal. They wanted to collect all of human knowledge, and bring it all together in this one place.

Though they didn’t actually collect all of human knowledge, they did a good enough job at it to establish Alexandria as the intellectual capital of the world. Alexandria became the hub of many inventions and discoveries that preceded later scientific and industrial revolutions. Some quick examples are Euclid inventing geometry, Eratosthenes calculating the circumference of the earth, and Hero inventing the first steam engine.

Importance of organizing information

This example illustrates some of the value of organizing information. It seems obvious that having information organized would be valuable, but in Alexandria we can see what that practically looks like. Because people were able to get access to the knowledge and information related to things they were working on, they could make progress much faster, and benefit from all the thinking that others had already done.

When information is organized, and accessible, people can “stand on the shoulders of giants” as Newton put it. We can read up on all the thinking and research that others have done in our field.

But it’s not just that. It also enables and feeds into curiosity. With abundant information at your fingertips, you could dive into a field you know absolutely nothing about, but seems interesting. If the information were not organized, you might not even know that this field existed.

In short, the organization and accessibility of information enables innovation, new discoveries, curiosity, exploration, and much more.

Libraries were the main institution focused on this problem, until the internet came along.

Metaphors for organizing information on the internet

The Filing Cabinets

First there were the filing cabinets. You want to find something with the name “apple”, so let’s look in the drawer that says “A”, then open that folder that says “AO - AR”, then find the folder that has the world apple on it.

This describes the original search engines, like Archie, which allowed users to search FTP file names.

The Library

Then there were the directories and portals like Jerry and David’s guide to the web which later became Yahoo. These directories were lists of hierarchical categories. You could select a category like “Science”, then a subcategory like “Physics” and get all the websites that talked about Physics.

This was much like a library. You could peruse the shelves, dive into and out of categories, and explore whatever seemed interesting to you.

The Free Market

GoTo was launched in 1998 with an interesting new concept. Websites would have to pay to be listed in search results. The higher the bid, the higher the website would show up in the rankings. Presumably, if someone has the money to afford a high spot in the search results, they must be making money, so their website must be good in some way. Low quality spam websites wouldn’t be able to outspend a legitimate business.

The Research Paper

Project Backrub was based on a simple premise. Research papers have citations, and papers with the most citations are the best ones. Links to other pages on the internet are kind of like citations. So why don’t we just copy paste this idea of ranking research papers onto the internet? And thus, a trillion dollar company was formed. It turned out that using backlinks to determine the quality of a web page was a really good idea, and it led to high quality search results.

The Encyclopedia

An encyclopedia is an attempt to summarize all knowledge. It is focused on giving factual explanations and descriptions of various things. It doesn’t go too in depth, and it doesn’t allow for opinion. The popular web equivalent is Wikipedia which allows anyone to contribute their knowledge on any subject.

Oral Tradition

One of the oldest metaphors for organizing information in human history is oral tradition. People would pass information down to one another directly. The internet version of this is Q/A websites like Quora, Stack Exchange and some parts of Reddit. Instead of searching, you just ask a question, and other people answer it. It’s direct person to person knowledge transfer.

The Town Square

Imagine that you’ve just walked into the town square. There’s a lot of people saying a lot of things. You have some people that you like, so you go listen to them talk. If someone you don’t know has a large enough crowd, you’ll walk over to them to see what’s going on. This is Twitter, and more or less all social media.

Is this the end?

Is this the best we can do? Are we done organizing the world’s information?

No.

And we’re not even close.

One question may point us in the right direction: what can’t you google right now?

Sure, you can google anything you want, but what kind of query can’t you expect good results for?

Collected Wisdom

Query: “What should I do with my life”.

The articles do answer the question in some way, although I’m a bit skeptical that a listicle from lifehack.org will have anything worth reading. Ignoring the fact that these are merely the most SEO optimized articles and not the highest quality, the real issue with most of them is that they’re only sharing one person’s answer to the question.

“What should I do with my life” is a very common question that everyone has probably thought about. There exists a ton of answers to this question by believable people throughout various books, blogs and interviews. Indeed, the way you’d solve this is probably by reading some of those books, blogs and interviews by people you admire and trust.

But how do you know what content to dive into without any prior knowledge? And is there a faster way you could get an overview of all the different perspectives on this question?

The information is all out there, it just isn’t indexed in the right way. If it was, you would be able to get the answer to this question from a bunch of different trustworthy people, and for a variety of life contexts.

You can see an example of what this might look like at https://guzey.com/personal/what-should-you-do-with-your-life/.

Here is a page filled with pointers to relevant knowledge on the topic of “what should I do with my life” from trusted people.

Imagine if this was the search results page anytime you searched for advice on anything.

Idea Search

Query: “Give me all the content that talks about the same idea as The Refragmentation by Paul Graham”

Unsurprisingly, google has no idea what I’m talking about, and just gives me more Paul Graham essays.

But the answer to this question exists. This essay by David Perell, this book by Martin Gurri, this pmarca tweet, this Perell podcast and probably more things that I couldn’t find, all are talking about the same idea.

Wouldn’t it be useful if you could be reading any one of these pieces of content, and know about the others?

It would be like a search engine for ideas and concepts.

Perhaps something like public roam graphs will be the solution to this, but there are infinite ways to approach it.

The Library Revisited

Query: “What are all the blogs that talk about the american revolution”

Once again, google really doesn’t know what I mean here. But let’s give google the benefit of the doubt on the specific wording and just take it upon ourselves to answer this question. I want to know what all the blogs are that talk about the American Revolution. How do I find them?

Maybe I can try searching for “american revolution blog” instead.

148,000,000 results sounds like a lot, I can’t wait to see all these blogs.

By the time we get to page 19, google thinks that Marginal Revolution has something to do with the American Revolution. But let’s just keep going, there’s still some legitimate blogs about the American Revolution mixed between the garbage.

What? There’s 148,000,000 results but there’s only 270 unique ones? I’m really only allowed to go to the 27th page? Well, let’s retry that with the omitted results and see what happens.

We’ve fallen straight off the deep end to things that have nothing to do with the american revolution at all. And at the same time, we’ve only made it to page 41, and this time there’s no way to override it.

We wanted to find all the blogs that talked about the American Revolution. Google told us they had 148,000,000 results, but only showed us 410. Half of those had nothing to do with the American Revolution, some were from the same website, and not all of them were even blogs. What is going on here?

Google doesn’t actually give us access to all the webpages in its database, it only gives us a tiny segment of it. And we don’t really control what’s in the tiny segment. It’s whatever google’s algorithm thinks we want.

Maybe there are really only 200 blogs that talk about the American Revolution, or maybe there are actually 1000. Maybe the best ones are in the top google results, or maybe they’re so hidden away and obscure that you’d never find them on google. We don’t know what the answer is, because we don’t really know what’s on the internet anymore. Platforms expose a tiny subset of the internet through their algorithmic feeds, but nothing gives us true access to all the information on the internet.

When Google rose and all the web directories fell, we lost the metaphor of the library. We lost the ability to explore blogs on various subjects with no real goal.

Why would anyone want such a thing? For the American Revolution example, you might be interested in seeing all the perspectives on that topic. But more generally, you might want it out of pure curiosity. The same curiosity that might grip you if you’ve ever perused a library.

The Call

The world has benefited greatly in the past from having information organized and made accessible. When information is organized, people can easily find similar ideas to build on top of, explore their curiosities, and discover new things. Knowledge is power, and organized information makes the attainment of knowledge far more efficient.

The Research Paper and the Town Square are good metaphors that work for some cases but not for others. What other metaphors could we have? What other ways are there to organize the world’s information? I suspect there are many, many more.

“I don’t feel like I’m qualified to write anything worthwhile”

I stopped writing a few weeks ago because I felt like I had run out of things that I was qualified to write about (which was a small pool to begin with). It’s scary giving people advice or sharing your thoughts when you don’t have much to back it up. It can feel like you’re just making things up, or like what you have to say isn’t really important.

And this is true in some scenarios. If you’re writing about nuclear fusion and you don’t know introductory physics, then you have a lot to study before you’re able to write something useful.

But you don’t need a PhD in anything to talk about life. If you’re writing about something you have an opinion on, or something that you’ve learned from your own experiences, then that’s definitely valid. At the very least, what you have to say applies to you. And since you aren’t that much different from other people, there’s probably someone who will relate to what you have to say. So say it.

“It’s already been written”

It hasn’t been written by you. It hasn’t been written from your unique perspective. It hasn’t been written in your style. And that makes a difference. Your perspective might be the one that someone else relates to the most.

“No one cares about this topic”

You care about it, so someone else probably does.

“Almost no one’s going to read it, so what’s the point?”

You’re not going to be famous, but you might help one or two people. If even one person finds it useful, then that’s worth it, because one person matters a lot. You know the impact that the things you’ve read have had on your life. Having that impact on even one other person seems worth it to me.