A strange this happened last week: Google misplaced my blog.
I’ve run all the usual diagnostics, and I can confirm that Google still knows that my blog exists. But it no longer appears in any of the searches – e.g., “natural gas price”, “unemployment”, “budget deficit”, or “brooke boemio” – that used to help new readers find posts on my site.
Things are so bad, in fact, that my blog doesn’t even come up when you search for “donald marron”. I feel an existential crisis coming on.
I presume this is just the result of some obscure algorithm tweak and that, over time, my posts will reappear in the ranks of the Google-worthy. But it’s fun to imagine that Google is mad at me for my posts criticizing the way it reports unemployment data.
I just checked and, no surprise, Google is still reporting the wrong data. If you search for “unemployment rate”, Google will tell you that the U.S. unemployment rate was 9.6% in August, when in fact it was 9.7%. Why the difference? Because Google is reporting an obscure measure of unemployment, not the one used by 99% of the world.
What’s going on? Well, it turns out that Google is directing users to the wrong data series. As I discussed last month, almost everyone who talks about unemployment is using (whether they know it or not) data that have been adjusted to remove known seasonal patterns in hiring and layoffs (e.g., many school teachers become unemployed in June and reemployed in August or September). Adjusting for such seasonal patterns is standard protocol because it makes it easier for data users to extract signals from the noisy movements in data over time.
For unknown reasons, Google has chosen not to direct users to these data. Instead, Google reports data that haven’t been seasonally adjusted and thus do not match what most of the world is using.
This is troubling, since I have high hopes for Google’s vision of bringing the power of search to data sets. The ability of users to find and access data lags far behind their ability to find and access text. I am hopeful that Google will solve part of this problem.
But data search is not about mindlessly pointing users to data series. You need to make sure that users get directed to the right data series. So far, Google is failing on that front, at least with unemployment data.
P.S. As I discussed in a follow-up post last month, Wofram Alpha has an even more ambitious vision for making data — and computation — available through search. I like many of the things Alpha is trying to do, but they are lagging behind Google in several ways. For example, as I write this, they haven’t updated the unemployment data yet to reflect the new July data. (Click here for Alpha results.)
Yesterday’s deal between Microsoft and Yahoo is a big boost for Bing. Microsoft’s new engine will power search on Yahoo, raising its visibility and, perhaps, eating into Google’s market leadership.
If the stock market is any guide, Microsoft is getting the better of the deal. As Techcrunch notes, Yahoo’s stock fell 12% on the day, lopping almost $3 billion off its market cap:
Microsoft , on the other hand, was up about 1.4% — boosting its market cap by about $3 billion.
The real question, of course, is how the deal will affect Google. GOOG was down about 0.8% (around $1 billion in market cap), a bit more than the decline in the Dow or the Nasdaq. That suggests that Google investors respect the MSFT-YHOO deal, but aren’t running scared just yet.
The logic of the deal seems impeccable. Yahoo is an also-ran in the search space, while Microsoft’s Bing is an exciting new entrant. Just how far Yahoo has trailed in search was driven home for me when I reviewed my posts about the search market (here is a list). Google gets the most attention in those posts, of course, but I also discussed competitors Bing, Wolfram Alpha, and Cuil. But it never occurred to me to mention Yahoo. That oversight is vindicated by today’s deal.
Personally, I am looking forward to having Bing on the Yahoo home page. I’ve spent far too much effort avoiding Yahoo’s search engine (e.g., by uninstalling the annoying Yahoo toolbar that various services foist on you when you get new software). Perhaps now I will have reason to let Yahoo take up a bit more valuable screen space.
Disclosure: I don’t own stock in any of these companies.
Early on, the article notes some ironies of the current situation:
More than 15 years ago, federal regulators began making Microsoft the symbol of anticompetitive behavior in the tech industry. Now, a newly activist DOJ may try to do the same thing to Google.
It is an ironic position for the search giant to find itself in. [CEO Eric] Schmidt not only campaigned enthusiastically for the very Obama administration that appointed [DOJ antitrust chief Christine] Varney, but also was one of the most devoted opponents of Microsoft in the mid-’90s, eagerly helping the government build its case against the software firm.
Wolfram Alpha is devoting enormous resources to the problem of data and computation on the web. As described in a fascinating article in Technology Review, Wolfram’s vision is to curate all the world’s data. Not just find and link to it, but have a human think about how best to report it and how to connect it to relevant calculation and visualization techniques. In short:
[Wolfram] Alpha was meant to compute answers rather than list web pages. It would consist of three elements, honed by hand …: a constantly expanding collection of data sets, an elaborate calculator, and a natural-language interface for queries.
Google may eventually solve the problem of finding data on the web. Too bad its first effort reports the wrong numbers for unemployment.
Since leaving public service, I have occasionally pondered whether to start a company / organization to transform the way that data are made available on the web. The data are out there, but they remain a nuisance to find, a nuisance to manipulate, and a nuisance to display. I cringe every time I have to download CSV files, import to Excel, manipulate the data (in a good sense), make a chart, and fix the dumb formatting choices that Excel makes. All those steps should be much, much easier.
There are good solutions to many of these problems if you have a research assistant or are ready to spend $20,000 on an annual subscription. With ongoing technology advances, however, there ought to be a much cheaper (perhaps even free) way of doing this on the net. With some good programming, some servers, and careful design (both graphic and human factors), it should be possible to dis-intermediate research assistants and democratize the ability to access and analyze data. At least, that’s my vision.
Many organizations have attacked various pieces of this problem, and a few have even made some headway (FRED deserves special mention in economics). But when you think about it, this is really a problem that Google ought to solve. It has the servers, software expertise, and business model to make this work at large scale. And with its launch of a search service for public data it has already signaled its interest in this problem.
As a major data consumer, I wish Google every success in this effort. However, I’d also like to use their initial effort, now almost three months old, as a case study in what not to do.
Google’s first offering of economics data is the unemployment rate for the United States (also available for the individual states and various localities). Search for “unemployment rate united states” and Google will give you the following graph:
Your first reaction should be that this is great. With absolutely no muss and no fuss, you have an excellent (albeit sobering) chart of the unemployment rate since 1990. I would add myriad extensions to this – e.g., make it easier to look at shorter time periods, allow users to look at the change in the unemployment rate, rather than the level, etc. – but the basic concept is outstanding.
Unfortunately, there is one major problem: That’s the wrong unemployment rate.
Google will likely face close scrutiny from the Obama administration. Indeed, it is already the subject of at least three separate antitrust reviews. Here are three ways Google will try to defend itself.
As Jeff Horwitz notes in the Washington Post this morning (“Google Says It’s Actually Quite Small“, previously posted on Slate), the search giant will likely face close scrutiny from the Obama administration. Indeed, Google is already the subject of at least three separate antitrust reviews.
How will Google try to defend itself?
As Horwitz reports, Google will undoubtedly employ two classic defenses:
Defense 1. Being a monopolist isn’t illegal. If firms achieve market dominance through “superior skill, foresight, and industry” (as Justice Learned Hand put it decades ago), that’s fine under our system. We want to reward firms that gain market share by being innovative and delivering value to customers.
People usually think of eBay as the master auctioneer of the internet age. As this month’s Wired points out, however, Google is the real master.
People usually think of eBay as the master auctioneer of the Internet age. As this month’s Wired points out, however, Google is the real master.
Google famously uses auctions to decide which ads appear in which positions when you do a search. But Google takes auctions much further:
Google even uses auctions for internal operations, like allocating servers among its various business units. Since moving a product’s storage and computation to a new data center is disruptive, engineers often put it off. “I suggested we run an auction similar to what the airlines do when they oversell a flight. They keep offering bigger vouchers until enough customers give up their seats,” [Google Chief Economist Hal] Varian says. “In our case, we offer more machines in exchange for moving to new servers. One group might do it for 50 new ones, another for 100, and another won’t move unless we give them 300. So we give them to the lowest bidder—they get their extra capacity, and we get computation shifted to the new data center.”