The Indicator from Planet Money

Episode•May 26, 2025•10 min

The dawn of search engines

Today on the show, we bring you a special episode from the Understood feed at CBC podcasts. It's an excerpt from a series called Who Broke the Internet hosted by Cory Doctorow. The four part series details his criticisms on the state of the modern internet and what we can do about it. From his conversations with Eric Corly the publisher of 2600, an iconic hacker magazine, best known under his hacker name Emmanuel Goldstein, to Clive Thompson a tech and culture writer to Steven Levy the author of "In the Plex: How Google Thinks, Works, and Shapes our Lives" this excerpt digs into how search engines started. You can listen to more of the podcast (https://link.mgln.ai/xSUskA) here. Related episodes: The hack that almost broke the internet (Apple (https://podcasts.apple.com/us/podcast/planet-money/id290783428?i=1000655961919) / Spotify (https://open.spotify.com/episode/7aYEYiIeS58LN8DfHr2TVl?si=6a7bc407e4a04c62)) For sponsor-free episodes of The Indicator from Planet Money, subscribe to Planet Money+ via Apple Podcasts or at plus.npr.org (http://plus.npr.org/). Music by Drop Electric (https://dropelectric.bandcamp.com/). Find us: TikTok (https://www.tiktok.com/@planetmoney), Instagram (https://www.instagram.com/planetmoney/), Facebook (https://www.facebook.com/planetmoney), Newsletter (https://www.npr.org/newsletter/money). To manage podcast ad preferences, review the links below: See pcm.adswizz.com (https://pcm.adswizz.com) for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences. Learn more about sponsor message choices: podcastchoices.com/adchoices (https://podcastchoices.com/adchoices) NPR Privacy Policy (https://www.npr.org/about-npr/179878450/privacy-policy)

Apple Podcasts Overcast

Transcript

0:01

Npr.

0:11

This is the indicator from Planet Money. I'm Adrian Ma. Today we've got a special guest episode for you. It's an excerpt from a series called who Broke the Internet? From our colleagues up north at the Canadian Broadcasting Corporation. It's from their Understood feed at CBC Podcasts. And the host is Cory Doctorow. You might have heard of him. He's a journalist, blogger, science fiction writer, and Internet commentator. And his series digs into his criticisms on the state of the modern Internet and what we can do about it. So in this excerpt you're going to hear after the break, Cory's going to talk about how search engines got started.

0:55

Eventually, the Internet got too big for people to know where everything was. So waves of clever people created search engines. Early search engines just looked for pages containing the words you typed, giving priority to pages that contained more of those words. This worked okay, but when it failed, boy, did it ever fail badly. If you wanted your page to rate high on the search results for a query like Mexican food, you could keyword stuff it by adding the words Mexican food a thousand times in tiny white on white type to the bottom of the page. The primitive search engines would count these all up and conclude that your page was the most important Mexican food resource in the world. Which is useless because most hungry people aren't looking for a site that just has the words Mexican food a thousand times. To weed these bad hits out of your search results, you'd have to master all kinds of arcane search engine syntax so you could exclude words and phrases, putting minus signs in front of the kind of junk that was typical of spam sites. It took forever, but those search engines were all we had, so we kept using them.

2:09

We were using, you know, really awful search engines that we thought were pretty good. And you saw Google and all of a sudden the film cleared from our eyes. We thought, wow, you could actually find what you were looking for on the Internet right away. Bang.

2:25

This is Steven Levy. He wrote the canonical history of Google in the How Google Thinks, Works, and Shapes Our Lives. Back in the early 90s, Stephen was Newsweek's tech reporter, and he knew right away that Google was a story.

2:43

When I first saw the Google search engine, my mind was blown.

2:47

So he tracked down their PR person.

2:49

I called her up, I said, I gotta meet these guys.

2:52

These guys being Google's founders, two Stanford grad students named Larry Payton and Sergey Brin. So Stephen booked a ticket from New York to San Francisco and flew out to meet them.

3:04

It was near Halloween. Everyone was dressed in Halloween Costume. So Larry was dressed like a Viking and Sergey was dressed like a cow. He had these big plastic udders coming out from his chest. So the Viking and the cow took me into a little room and explained how page rank works.

3:26

PageRank is the algorithm behind the magic. A new way that Google had developed to deliver its search results. It was totally audacious, and I say

3:37

audacious because of the way they were able to locate. The best result for your query was by basically, no, basically literally downloading the whole web, all of it, which some people thought was ridiculous. But Larry Page understood this could be done.

3:57

All Larry and Sergei needed to pull it off was big, powerful servers, which

4:01

they could get because they had access to servers at Stanford and they would literally hijack them as they came in the loading dock, meant for other departments.

4:11

With all this yoinked power. They set up their first server, building the case for it out of Lego. Yeah, Lego, because it was cheap and cool and colorful like the eventual Google logo.

4:25

And then they would do math. This was like Sergey's specialty to analyze all the links that happen between websites and figuring out what important websites link to other sites. So that way you could filter out which were the perfect answers for the queries you put into the search field. And that turned to be just a quantum leap, better than the previous things.

5:00

The early Internet wasn't perfect. Google came into existence because the search landscape sucked. But the point is, Google could come into existence. There was oxygen and sunshine that reached the forest floor so new things could grow. If you could code, if you had some money, a computer or two, a couple of bins of Lego, you could, you were allowed to build something new that made the web better. Clive Thompson Again, they start off by

5:31

having a really good search engine, the best search engine. And the reason why it's so good is that they are in ferocious competition with like 12 different search engines. And they have to be better than everyone else, have to fight to the top. And they do that.

5:42

Just be the best, make the best product and people will switch. And we did. We all switched to Google because it worked and that mattered to Google in their early days.

5:56

Then there's, you know, the very famous motto that Google had was don't be evil, you know, which was coined by an engineer when, you know, someone from HR had a meeting and, you know, so what is our motto? What is our, what is our, our value? And you know, one guy said, what are you talking about? Just like, don't be evil, you know, I mean, that's it.

6:24

So back in 1999, Stephen wrote that first story for Newsweek, and he kept writing about Google.

6:31

Google's world was our world. You know, things didn't exist, but it wasn't on the Web.

6:37

It's like when Google did well, we all did well because we could find the stuff we were looking for.

6:43

Right? And that came into play even more when they came up with their first big successful business product.

6:50

That was ads.

6:51

Yeah, yeah.

6:52

When Google met advertising, everything changed. The introduction of ads to Google marked a major shift. Emanuel Goldstein Again, Google, in their early days, you didn't have ads on the front page. It was relatively fast, and they just seemed like cool people inventing cool things.

7:18

Larry and Sergey actually hated advertising. And the paper they wrote talking about PageRank, they had a little thing in the end saying, boy, we hate advertising. It's terrible.

7:28

Stephen's talking about an article the Google founders published in a journal in 1998. And in it, they explicitly wrote that quote, advertising income often provides an incentive to provide poor quality search results, but

7:46

they had to make money. So then they figured, well, maybe there's a way to do ads that wouldn't be terrible.

7:55

So in 2000, the founders assigned an engineer to figure this out, one of Google's first eight employees, and he came up with something.

8:05

It was called AdWords. So what it would do was when you put a query into the search engine, it would start an auction instantly, and, you know, your search term would be auctioned off to advertisers who wanted to sell a product that was, you know, related.

8:26

If you search for Mexican food on Google back in the early aughts, the results would have been the most linked to websites for Mexican restaurants in your area and reviews from local blogs and newspapers. And now with AdWords, ads would appear in a colored bar atop the normal page ranked results.

8:42

Yeah, you get the same results, but you get a little extra little, you know, set of possible results that actually might even help you if you were looking to buy something related to the search term. And the advertisers would get an audience which, you know, was primed to buy something. And it worked for Google, it worked for advertisers, and most of all, it worked for users.

9:08

That is, until it didn't,

9:14

thanks to our colleagues over at the Canadian Broadcasting Corporation. If you want to hear more of who broke the Internet, you can find a link in our show notes or find it wherever you listen to podcasts. The original version of this episode was written and produced by Matt Muse, A.C. rowe and Corey Doctorow. Roshni Nair is the coordinating producer. Julian Uzieli did the mixing and sound design. Story editing by Veronica Simmons. The executive producer is Nick McCabe. LOCOS the indicator version of this episode was produced by Cooper Katz McKim and engineered by Sina Alafredo. Kate Cannon is our editor and the indicators of production of NPR.

The dawn of search engines

0:00

0:00

Related Episodes

OpenAI's deals are looking a little frothy

OpenAI's deals are looking a little frothy

Oct 16, 202510 min

OpenAINvidiaOracle

Overly Friendly Emails and other marketing pet peeves

Overly Friendly Emails and other marketing pet peeves

Mar 31, 20259 min

J. CrewWendy'sJoe Burns