Authors: Douglas Edwards
We didn't care what the press said. We knew it was a major win. The Googlers at the Plex celebrated accordingly. On Monday Charlie and his crew prepared a luau lunch and served it up al fresco. The grass was green and freshly mown, the food hot and plentiful, and the spirits high. Music filled the air and margaritas sloshed in paper cups hoisted in salute as Larry and Sergey, wearing plastic leis, introduced Yahoo co-founder David Filo. Filo eschewed the customary rhetorical pats on the back in favor of a brief speech that boiled down to, "Thank you. We have a lot to do. You should really get back to work." Perhaps his absent partner, Jerry Yang, was the party guy.
Susan Wojcicki handed out t-shirts she had secretly ordered proclaiming "Google and Yahoo got lucky"—Google's first official commemorative garment. If you want to make a killing trading tech stocks, find a friend in the t-shirt business between San Francisco and San Jose and ask to be alerted any time a rush order gets placed. Conventional wisdom in Silicon Valley states, "If it's not on a t-shirt, it didn't really happen."
Saturday, July 1. Google was serving the 1B index to all its own users from a new West Coast data center. All that remained was to load a copy of the index into the new data center in Virginia and the old one at Exodus. The Virginia transmission went smoothly, but when ops tried sending a copy to Exodus, it failed. The connection between the data centers couldn't be established, so the data couldn't be sent. Without a copy of the index, the third data center would be useless and Google would be unprepared to handle Yahoo's queries, which were due to start flooding in within forty-eight hours.
Jim, Schwim, and Zain Kahn piled into Jim's ten-year-old Volvo station wagon and sped off to check it out. The network line between the data centers hadn't gone live yet. Instead of relying on outsiders to activate the cable, they opted for a backup system known among technicians as "sneakerware."
"We just ripped out the eighty machines that had the index," recalls Schwim, who helped load the machines that held Google's future into the Volvo. The techs climbed in with the hard drives and drove them to Exodus, where they piled them on the floor of the already overcrowded cage. "We stacked up eighty machines on the ground, with nothing around them, not even cabinets, and we plugged them into these ridiculous power strips so we could copy the index off. You have to imagine someone working at Inktomi thinking, we have this beautiful cage and there's a pile of ... 'bleep,' and
they
got the contract?" As one of the ops guys remarked to me later, "Never underestimate the bandwidth of a truck full of hard drives."
While Inktomi's cage may have been beautiful, it wasn't completely secure. Google didn't have enough outlets to plug in all their machines, so Zain crawled under the raised floor and snaked out an unused cable from the Inktomi side of the fence. It would have been the ultimate indignity had anyone from Yahoo's jilted partner been around to witness it, but to Google, it was just an opportunity to improvise.
Early on the morning of Sunday, July 2, Howard Gobioff turned his black Honda Nighthawk into the Google parking lot, killed the motor, pulled off his helmet, shook loose his ponytail, and climbed the stairs. Inside, Romanian roller-hockey enforcer Bogdan Cocosel had been up all night as the push propagated the new index to the thousands of servers in all the data centers. Bogdan nursed the system and, when it appeared to hiccup, cursed it with enthusiasm. Howard sat at the terminal to relieve him.
To those inside the Googleplex, it was a glorious new dawn. There was no going back now. Howard watched as the index skated along the ragged edge of disk capacity. The push held throughout the day, and by the next morning the billion-URL index at last stood locked and loaded and ready to serve. Yahoo would initiate the switchover at eight p.m.
Monday, July 3, 7:45 p.m. The team floated in and around Urs Hölzle's office, anticipating the opening of the spillway and the rush of the incoming torrent of queries into Google's query stream.
Eight p.m. came, but the flood did not appear. Not even a trickle came through. There were no queries from Yahoo being passed to Google. Had Yahoo reconsidered? Had Inktomi somehow sabotaged the deal? Urs called Udi. Yes, it was supposed to have happened at eight p.m. Unfortunately, Yahoo was having problems reconfiguring the DNS (domain name server) that would tell the queries to go to Google instead of Inktomi. No one at Yahoo had changed a DNS entry in quite some time and they had forgotten how to do it.
"You should be seeing it now," Udi told Urs.
"Hmmm ... No."
"Now?"
"Still no. Try changing your DNS expiration time."
A pause.
"How about now?"
Yahoo's traffic came sweeping into Google's data centers and Google itself seemed to swell in magnitude, to be lifted on a crest of queries to the upper tier of online search companies. A loud pop was heard and a cheer went up from the assembled Googlers. Someone had uncorked a single bottle of Dom Perignon and was passing around cups with a sip for each of the dozens of people on hand.
Urs was even more succinct than his Yahoo counterpart had been. "To something!" he said, raising his glass.
The engineers downed their champagne in a gulp and dug into the bag of Big Macs Craig Silverstein had brought in from McDonald's. They wiped their greasy fingers on their jeans and then went home to sleep for many hours. The changeover passed flawlessly. Not a single query was lost. Yahoo had licensed only a portion of Google's full data set, a distinction that would probably make no difference to most Yahoo searchers but meant that
Google.com
retained absolute superiority.
The only nagging question had been whether the 1B index would cross the finish line with the incremental index running alongside it. It didn't. The incremental solution would continue to elude the Sisyphean efforts of the engineers for months to come. Google satisfied Yahoo with assurances that the incremental index would be completed quickly and that until it was, the new indexer would enable monthly updates to ensure freshness.
The Yahoo deal ended all my concerns about Google's future. We had momentum on our side and no visible obstacles in sight. If we could take Yahoo from Inktomi, who would stop us? I allowed myself to believe that I just might be living a Silicon Valley success story.
I called my mom and dad to tell them the news, since I wasn't sure the Internet had made it as far as Jacksonville. I hinted I might be able to pay back the money I had borrowed to buy my stock options. Not anytime soon, mind you, but someday.
My part in the Google-Yahoo tango played out weeks prior to the actual announcement. Omid wanted to cozy up to Yahoo by buying advertising from them as a gesture of good faith, so I scheduled the hundred banner ads I had created to run on their site. Sergey insisted I get the best return on our investment, even though he knew the ultimate goal was fostering good will. He directed me to buy untargeted run-of-site ads because they were cheaper than Yahoo's premium-content channels and because they gave us branding exposure even if nobody clicked on them. Did I mention they were also cheaper?
Yahoo, too, wanted to get the most out of our overture of friendship and resisted when I tried to negotiate lower rates for our buy. It was a difficult conversation in which I had to reconcile Sergey's deal-making directive to maximize value with our larger diplomatic goal of making Yahoo happy. I didn't want to push too hard, yet I felt an obligation not to roll over and accept whatever Yahoo felt they could get away with charging us. No matter what I negotiated, I knew Sergey would think we were paying too much. Then I discovered another complicating factor. The Yahoo sales rep assigned our account was married to David Krane, who had just been hired as Google's PR manager.
David was not the only Google executive in a mixed marriage, that is, one with a spouse working at a potential competitor. He wasn't even the only employee in the marketing department who had married outside the faith.
Let me give you an example of how convoluted and semi-incestuous Silicon Valley gets. We used the company eGroups to mass-mail our Google Friends newsletter to users, because Larry's brother, Carl, was one of eGroups' founders. Larry had done the configuration for the original eGroups server himself, and for a while the company's computational heart had lived under his desk. The same week we announced our deal with Yahoo, Yahoo announced they were buying eGroups for $428 million (Yahoo has been very kind to the Page family). With the integration of eGroups into Yahoo Groups, we began experiencing problems with our newsletter, from formatting issues to administrative headaches. Luckily, one of the software engineers absorbed into Yahoo with eGroups also had a connection to Google marketing. He was Cindy's husband. When our situation was dire and normal channels of communication failed, Cindy's "special friend" could usually help us get our problems addressed.
Silicon Valley is a Petri dish filled with amoeba-like corporations absorbing and digesting smaller technology firms, only to find themselves absorbed or growing large enough to split off their own subsidiaries. Employers have a penchant for hiring from the same pool of candidates over and over again, so everyone ends up working with everyone else at some point, or at least working for the same companies. Job-hopping is encouraged—no, expected—since no one place could possibly be interesting and innovative enough for an entire career. That's why the question Sergey asked when he interviewed me for the job was not "Why do you want to leave the
Mercury News?
" but "Why did you wait so long?"
No wonder social networking took root here; we're one big interconnected family whose members are always happy to find out how we're related to one another. "He's a first employer once removed on the Intel side." "She used to be my assistant at Sun, but she left me for some hot new startup over in Cupertino." A surprising number of tech workers have friends and lovers with whom they share intimacy but not the details of their office lives.
Google was no more immune to the lure of fraternization within the building than it was to relationships that crossed competitive lines. There were romances. There were marriages. On occasion, there were affairs. My sense is that the number of these dalliances was not out of line with a normal distribution in a population the size of Google's, especially one as densely populated with energetic young overachievers. It would be indiscreet for me to go into the details of people's private lives beyond what the participants have acknowledged publicly—and it would also be largely irrelevant, since office relationships had little effect on the course of the company. Usually, anyway. I did detect the tidal force of one pairing tugging at my ability to get my job done.
Larry and Sergey's insistance on seeing performance metrics for marketing redoubled with the addition of our ad buy on Yahoo. They began a drumbeat of demands for better measurement of our customer-acquisition techniques. What about the promotional text on our homepage? Which messages converted the most newbies to regular users? Testimonials? Promises? Comparisons? How many ads did they click? How many searches did they do?
The only way to answer these questions was to generate the homepage dynamically—essentially to implement code that would give us the ability to deliver variant versions of the homepage to users who came to our site. That would enable us to show different users different text and then track what they did after they saw it.
Larry gave me the task of writing the text to be displayed in April 2000 and assigned the coding of the dynamic homepage to Marissa. A logs team would generate the report on how many new users came back. While it took me a long time to get signoff on the homepage messages I wrote, dynamic homepage generation proved even more elusive. Soon every conversation I had with Larry turned into an inquisition about the conversion-rate test.
"Doug, when are we going to see those numbers?" he'd ask me. "We're wasting money because we're not effectively using our most powerful promotional medium."
The conversion-rate test was one of my main OKRs (Objectives and Key Results), and each time I had to tell Larry I had no data, a bit more of my credibility crumbled. I used every method at my disposal to jump-start the project, but I simply couldn't get the priority moved high enough. The only one who could move things along was Marissa, and it had been announced at TGIF that she and Larry were now a couple. Finally, in late August, I trudged upstairs to camp outside Larry's office. I waited until he was alone, then entered and closed the door behind me.
"Larry, I've begged, cajoled, and demanded," I said, "but the dynamic homepage code still hasn't been implemented so we can test conversion rates. Can you recommend some other approach?" I was frustrated and nervous and didn't hide it very well. I was admitting I couldn't get something done. At Google, that was not a career-enhancing move. And I felt uncomfortable telling the company president that the obstacle in my way was the engineer he was dating. Larry listened quietly to my concerns.
"Don't worry, Doug," he reassured me with a broad smile. "We'll work something out." Then he put his hand on my shoulder, gave it a gentle shake, and guided me to the door. It was a strange moment for me. Larry's earnestness emanated in waves, as if he wanted to let me know that he understood the unspoken dilemma I faced, that he and I were all right and that the situation would be all right, and that he would take care of things. I went back to my desk unsure what to do next. What if other issues came up with Marissa? I could already see that we had differing perspectives about our brand. Would I have to go to Larry for resolution each time we disagreed?