Authors: Douglas Edwards
The next day Matt reiterated his need for help but augmented it with an offer of free "porn cookies" for everyone who participated. Search for porn, get a cookie. I was one of those who couldn't resist such a come-on. Over the next few hours I learned several new words and new meanings for some I thought I already knew.
Porn is a cutthroat business that often leads in the exploitation of new technology. Working on what would be dubbed SafeSearch, Matt became aware of a new problem for Google. Spam. Spammers attempted to game the system and thereby win a higher ranking in search results than they deserved. Matt came to think of spammers as "black hats" who placed invisible white text on a white background, stuffed their pages full of keywords, and employed a wide range of sophisticated and devious means to deceive Google's search bot.
"Once you start to see spam, the curse is, you'll see it everywhere," Matt told me. He was offended by spammers' unethical behavior and continued thinking about the problem even after he finished his filter and began working on Google's advertising system. Larry and Sergey thought spam was a non-issue, because they were confident Google's PageRank algorithm would sort the wheat from the chaff.
"It took quite a while for Google to wake up to fact that while PageRank was very spam resistant, it wasn't a hundred percent perfect," Matt noted. He took the initiative to sound the alarm. Although it was no longer his area of responsibility and not an area of concern to management, he asked to work on fighting spam. "One thing I learned at Google," he said to me, "is that you make your own cred. If you propose your own initiative, you're much more likely to do it than if you sit around and wait for someone to say, 'What do you want to do with your life?'"
Within days, Matt connected with the group of eight engineers focused on overall search quality, including Jeff, Sanjay, and Amit Singhal. They invited him to join them around the Ping-Pong table and delve into the deepest aspects of Google's core technology. Matt became privy to Google's secret sauce, the weighting factors that determined whether a website was near the top of the first results page or buried somewhere on page thirty.
Matt always struck me as a pillar of moral rectitude, a keeper of the faith in algorithmic integrity, and an adamant protector of Google's purity—our own avuncular Elliot Ness, bringing to account those who parasitically thrived by bootlegging traffic. As a University of Kentucky student, Matt had enrolled in a co-op program through the Department of Defense and ended up spending a few semesters at the National Security Agency. That internship was great fodder for the conspiracy theorists monitoring Google. Matt's "secret" government tie made their tin hats stand at full attention, because they assumed Matt still served as a conduit between big government and big search.
*
In truth, Matt was one of the least "spooky" guys I knew at Google.
Because he understood the innermost details of Google's ranking calculations, Matt was outraged when a webmaster bulletin board speculated in 2001 that Google was manipulating results to increase sales of our advertising. If a business didn't show up near the top of our results, they alleged, it would have to buy ads to have a presence on the first page users saw after a search. The rumor wasn't surprising, given the practices of most search engines at the time. The Federal Trade Commission had called out eight of Google's competitors for blurring the lines between paid placement, advertising, and algorithmically produced results. Matt again took the initiative to address a problem others didn't immediately see. He went to PR manager David Krane.
"So ... while I'm compiling," he asked Krane, "would it be okay if I stopped by this forum and debunked misconceptions?" Krane reported to Cindy, and Cindy had read the
Cluetrain Manifesto
—a guide advocating that companies speak directly and plainly to the public instead of engaging in Velveeta-smooth, committee-processed, content-lite corporate blandishments. Cindy had made sure everyone else in marketing read it as well. Krane gave Matt carte blanche to speak freely on the company's behalf, without running his posts by PR.
"GoogleGuy" was born.
Matt's nom de plume became an authoritative voice in the webmaster community and a trusted source of information from inside Google. GoogleGuy corrected misinformation, killed rumors, and explained why Google did things that seemed off kilter to outsiders.
Initiative crossed with autonomy provided unforeseen benefits in unexpected areas. Unexpected challenges as well. Other engineers also sought to speak in unvarnished language on Google's behalf without formal endorsement by Cindy's minions—or even our awareness.
Ray Sidney comes to mind.
Ray, Google employee number six, embodied the cult of individual authority. His "dude"-infused speech and ribald and unpredictable passions obscured an education earned at Caltech, Harvard, and MIT. Ray was our first line of defense against webmasters who pummeled Google with automated queries. Webmasters and SEOs
*
wanted to make sure their sites showed up near the top of Google results and so used monitoring software to conduct repeated automatic searches for keywords important to them. In periods of high volume, automated queries slowed down Google for everyone, which is why we considered them a violation of our terms of service.
†
Ray took unauthorized automated queries very personally. If he could figure out the spammer's email address, he sent a terse cease-and-desist warning. If he couldn't find an email address, he blocked the spammer's IP (internet protocol) address—the unique number assigned to a computer connected to the Internet—from accessing Google altogether.
No one was immune. When a user left a book on the Enter key and sent the same query to Google thirty-nine thousand times, Ray cut off access for everyone at that address. The query was "This is the CIA," and it came from that agency's headquarters in Langley, Virginia. Another user searched for "net oil importers" over and over and over again. Ray got annoyed and shut off the State Department as well.
If Ray couldn't identify a specific IP address, he contacted the spammer's Internet service provider (ISP) and asked that they track down the offender themselves and sever his access to Google. If the ISP refused to play along, Ray upped the ante—he blocked access to Google from all of the ISP's addresses. That usually got their attention. It was how Ray shut down access to Google for most of France. The French ISP definitely noticed, all the more so because at the time they were negotiating to become one of our larger customers.
Ray didn't hate the French. He did the same thing to the Germans. Also to a major American ISP, though he did post a note to their customers who complained. "The short story here," he wrote, "is that some user at your ISP was abusing Google. We were unfortunately unable to turn off access just for this evil individual. Since your ISP didn't respond to us, we had no choice but to shut off access to Google from a large number of IP addresses."
Cindy was, as she put it, "displeased" when she read Ray's note reprinted in a headline article on CNET describing Google's rude treatment of users. She "suggested" that I take over user communication related to service interruptions and "work with" Ray to smooth out the rougher edges in his correspondence. It was hard for me to keep up, because Ray was all about initiative. He was not part of the company's business-development team according to the org chart, but he never let reporting lines fence him in.
"Well, to be blunt," Ray told a partner who wanted to renew a deal for Google technology, "it's clear to us what you get out of our relationship, but it's far from clear to us that we get anything out of it. Given that, it seems like poor business practice for us to continue with it. So, unless I'm missing some key observation here, please stop performing Google searches immediately."
Many things made Ray wroth. He sent out long notes to all Googlers demanding we clean up the kitchens, the locker rooms, our interviewing techniques, our security practices, our personal habits, and our grammar. He also urged us to recycle our trash at every opportunity. Once a burr got under his saddle, he didn't wait for it to work itself out. "Can we please, please, please finally just end our relationship with these leeches?" he begged of Larry Page when another partner continued to annoy him. "If only to make me happy?"
Impulsive and opinionated, Ray will always personify for me Google's engineering id, a lone cowboy patrolling the electronic frontier in shocking-pink shorts, facing down the black hats and making them blink, then riding off into a sunset that was only half as colorful as he was.
A single engineer holding that kind of power speaks to the assumptions inherent in Google's culture. Individuals were considered capable of weighing the effects of their actions and presumed to have the best interests of the company (and Google's users) at heart. We were encouraged to act on those interests without hesitation. Spend time doing, not deciding.
Of all the elements of "big-company thinking" I had to unlearn, that was one of the hardest. I constantly sought reassurance that I was empowered to move to the next step, only to be asked, "Why haven't you finished that already?" The upside of this philosophy is that Google did things quickly, most of which turned out to be positive. The downside is that individual Googlers sometimes misinterpreted exactly how much power they possessed and when it was okay to use it.
Shari had discovered the downside the hard way. She had reached the breaking point with Larry and Sergey. They weren't supporting her work with the promotion agency, and without outside help, she couldn't move forward. She threw up her hands in frustration, and while they were up, she tossed in the towel.
At her farewell party at a local Mexican restaurant, I said goodbye to the one other person at Google who completely understood the practice of branding for customer acquisition. Google didn't do that kind of marketing. The company rejected any attempt to graft traditional practices onto its new breed of business.
Over salsa and Dos Equis toasts, I resolved that I would remain open to new ideas and new approaches. I would make it work. I would prove to myself and to my ever-adaptable colleagues that this old Doug could still learn new tricks.
Chapter 11Beyond a startup.
Not yet a search behemoth.
Google's awkward phase.
W
HILE I HAD
been trying to figure out what to do next, the engineers had been killing themselves to do the big, hard, complicated things that absolutely needed to be done. Their yearlong effort would come to fruition just about the time I started to get my bearings.
The engineering story began in June 1999—before I had even heard of Google. Jim Reese, the neurosurgeon turned sysadmin, had just been hired. On his first day, he arrived at eight a.m. and worked straight through for fourteen hours. The next day he came in a little later—about ten a.m.—to add backup servers to Google's intranet and to handle networking issues in the Plex. He left the office around four for an early dinner on his way to Exodus, Google's data center, where he stayed until five in the morning. He did the same the next day, and again every day that week, including Saturday and Sunday. His task, assigned by Larry without explanation, was to install two thousand new servers and bring them online.
That many computers wouldn't fit in the cage Google owned at the time, so Jim needed to arrange for additional space at the data center. "I worked as hard as I could," he said, "negotiating with facilities at Exodus. In 1999, cage space was hard to come by and Exodus was pretty full." Partly that was because of companies like eBay, whose cage was near Google's. "They had a cage ten or twenty times our size and they had perhaps eighty computers in it," Jim recalls, "whereas we had eighty computers in one rack." There were nine racks crowded into Google's cage—but, as Jim and his new associate Schwim realized when they looked closely, it wasn't at capacity.
"If we move every cabinet on this side of the cage three inches, we'll have exactly enough room to fit in another rack," Schwim pointed out, turning sideways to squeeze down the aisle. "The only problem is, there's no way to roll a rack through here. We'll have to take off the side cage wall." They called the facilities manager, described what they wanted, and left for lunch.
When they came back, the black chain-link fencing that had protected the side of Google's space had been unbolted and removed. Jim and Schwim slid the rack in, cabled the computers, connected them to the main switch with fiber, and flipped the switch. Everything lit up the way it was supposed to. Jim double-checked the connections at the back of the rack as Schwim stood at the front typing in commands to monitor its progress.
"It's a go!" Jim heard Schwim announce from the front of the rack.
"The next thing I knew," Jim recalls, "I'm sitting on my butt on the floor of the data center."
"We just lost the rack!" Schwim yelled. "What's going on?" He stepped around to the back and found Jim on his back, groggily rubbing the crown of his head, a two-hundred-pound metal crossbeam, smeared with blood, lying beside him where it had fallen from the top of the cage.
"Uh...," said Jim, shakily pointing upward toward a batch of severed cables that had lain in the beam's path, "we're going to need more fiber."