Zanran helps you to find ‘semi-structured’ data on the web. This is the numerical data that people have presented as graphs and tables and charts. For example, the data could be a graph in a PDF report, or a table in an Excel spreadsheet, or a barchart shown as an image in an HTML page. This huge amount of information can be difficult to find using conventional search engines, which are focused primarily on finding text rather than graphs, tables and bar charts.

Put more simply: Zanran is Google for data.

This is a stellar idea. The web holds phenomenal amounts of data that are hard to find buried inside documents. And Zanran offers a fast way to find and scan through documents that may have relevant material. Particularly helpful is the ability to hover your cursor over each document to see the chart Zanran’s thinks you are interested in before you click through to the document.

Zanran is clearly in beta, however, and has some major challenges ahead. Perhaps most important are determining which results should rank high and identifying recent data. If you type “united states GDP” into Zanran, for example, the top results are rather idiosyncratic and there’s nothing on the first few pages that directs you to the latest data from the Bureau of Economic Analysis. Google, in contrast, has the BEA as its third result. And its first result is a graphical display of GDP data via Google’s Public Data project. Too bad, though, it goes up only to 2009. For some reason, both Google and Zanran think the CIA is the best place to get U.S. GDP data. It is a good source for international comparisons, but it falls out of date.

Here’s wishing Zanran good luck in strengthening its search results as its competes with Google, Wolfram Alpha, and others in the data search.

Everyone, that is, except Google.

If you ask Google (by searching for ”unemployment rate United States“), it will tell you the unemployment rate in July was 9.7%.

What’s going on? Well, it turns out that Google is directing users to the wrong data series. As I discussed last month, almost everyone who talks about unemployment is using (whether they know it or not) data that have been adjusted to remove known seasonal patterns in hiring and layoffs (e.g., many school teachers become unemployed in June and reemployed in August or September). Adjusting for such seasonal patterns is standard protocol because it makes it easier for data users to extract signals from the noisy movements in data over time.

For unknown reasons, Google has chosen not to direct users to these data. Instead, Google reports data that haven’t been seasonally adjusted and thus do not match what most of the world is using.

This is troubling, since I have high hopes for Google’s vision of bringing the power of search to data sets. The ability of users to find and access data lags far behind their ability to find and access text. I am hopeful that Google will solve part of this problem.

But data search is not about mindlessly pointing users to data series. You need to make sure that users get directed to the right data series. So far, Google is failing on that front, at least with unemployment data.

 P.S. As I discussed in a follow-up post last month, Wofram Alpha has an even more ambitious vision for making data — and computation — available through search. I like many of the things Alpha is trying to do, but they are lagging behind Google in several ways. For example, as I write this, they haven’t updated the unemployment data yet to reflect the new July data. (Click here for Alpha results.)

Bing isn’t trying yet.

Early on, the article notes some ironies of the current situation:

More than 15 years ago, federal regulators began making Microsoft the symbol of anticompetitive behavior in the tech industry. Now, a newly activist DOJ may try to do the same thing to Google.

It is an ironic position for the search giant to find itself in. [CEO Eric] Schmidt not only campaigned enthusiastically for the very Obama administration that appointed [DOJ antitrust chief Christine] Varney, but also was one of the most devoted opponents of Microsoft in the mid-’90s, eagerly helping the government build its case against the software firm.

A few weeks ago, I described some of the arguments that Google might use to defend itself. The Wired article elaborates on one of these: it’s fine for a company to be a monopoly if, as John Houseman used to say, they earn it. It then points to the other issues that may raise concerns:

In and of itself, Google’s size is not a legal problem. Varney herself has pointed out that, while Google may enjoy a monopoly in the search-ad business, the company acquired it legally by building better search products that competitors were simply unable to match. Lawyers and economists say that things get complicated, though, when Google moves beyond search and into Web services like online spreadsheets and video sites. Because its search and advertising algorithms are secret, there is no way for competitors or partners to know whether Google tweaks results to direct traffic to its own properties over theirs. Enter a street address into Google’s search engine, for instance, and Google Maps tops the results. Type “Britney Spears” and Google News comes up before People magazine or TMZ.com. … If Google is using its search position to promote its other businesses, that could leave it open to charges of illegal bundling and leveraging–the same changes that Microsoft faced for packaging its browser onto the Windows desktop.

The article does have one major lapse, offering up one of the dumber antitrust arguments:

Google can use the surplus cash from its search business to subsidize the development of new, free products and services. That’s a frightening prospect for would-be competitors without such a robust revenue stream–potentially scary enough to discourage them from entering the market. The possible result: less innovation overall.

By that logic, Google could only avoid antitrust scrutiny by either (a) not making profits or (b) not innovating. That’s absurd.

As someone who has given brief thought to competing with Google, I can tell you that the company is daunting. But not because it’s so profitable (otherwise Apple, George Soros, and Alex Rodriguez would be just as concerning). No, the reasons Google frightens competitors are its servers, software, know-how, user base, brand, etc. Any rational person would think twice before trying to compete with a company that’s become a verb.

Wolfram Alpha is devoting enormous resources to the problem of data and computation on the web. As described in a fascinating article in Technology Review, Wolfram’s vision is to curate all the world’s data. Not just find and link to it, but have a human think about how best to report it and how to connect it to relevant calculation and visualization techniques. In short:

[Wolfram] Alpha was meant to compute answers rather than list web pages. It would consist of three elements, honed by hand …: a constantly expanding collection of data sets, an elaborate calculator, and a natural-language interface for queries.

That is certainly a grand vision. Let’s see how it does if I run the same test “unemployment rate United States” I used for Google:

This certainly brings a new perspective to economic data. My first reaction was to laugh at the idea of summarizing the data as “9 out of 125 workers were unemployed in 2008″ — that’s useless.  On second thought, though, the statement that about 1 in 14 workers were unemployed is a useful way of interpreting the 7.2% unemployment rate. Oh, and as a card-carrying econo-geek, I have to give Wolfram credit for noting that this figure is an estimate.

Unfortunately, this response doesn’t answer my question, which is to find out what the unemployment rate is doing now. Wolfram has chosen to default to annual figures, so that’s what we get: an average for 2008. Sadly, the current rate is much higher, but users wouldn’t learn that if they stopped at this step.

It took me a few tries, but I eventually realized that you can get monthly data by adding the word “monthly” to the search.  Here’s what you get:

Jackpot – almost. The good news is that it provides a graph of the monthly data, and, unlike Google, it uses the right series: the seasonally-adjusted unemployment rate. And it provides a way to change the time period (that’s the “All” with arrow). And it provides some useful context.

But the data haven’t been updated to June yet, even though those data have been out for a week. And there’s no way to track these data back to their original sources, something Google does very well. (Further down the page, there is a link to sources, but you can’t tell which one provided these data.)

Bottom line: Maybe Wolfram Alpha will solve the data problem instead of Google. Or, even better, in addition to Google. As a data consumer, I welcome the competition. But Wolfram should update the data quicker and should help users figure out the right syntax (perhaps a link on the original results saying “monthly” data, for example). Oh, and given the promising results for “unemployment rate United States monthly”, the results for “unemployment rate United States annual” are downright mysterious.

How will Google try to defend itself?

As Horwitz reports, Google will undoubtedly employ two classic defenses:

Defense 1.  Being a monopolist isn’t illegal.  If firms achieve market dominance through “superior skill, foresight, and industry” (as Justice Learned Hand put it decades ago), that’s fine under our system.  We want to reward firms that gain market share by being innovative and delivering value to customers.

To overcome that argument, antitrust authorities (or private plaintiffs) will need to identify some specific actions that Google has taken to “restrain trade”, gaining market share by interfering with competition, rather than by demonstrating superior skill etc.

Google will then reply with:

Defense 2.  Furthermore, we aren’t a monopolist.  To identify a monopolist, you first have to identify a market.  That sounds simple, but it actually inspires lots of debate in antitrust cases (and, in the process, enriches a certain cadre of economists who specialize in such issues). As Horwitz notes, commentators usually characterize Google as enjoying 70% or more of the revenue in the search advertising business, which certainly sounds like a big share. But Google thinks that market definition is too narrow.  If you look at all advertising, for example, Google’s share is microscopic, less then 3%.  Thus, Google will argue, it isn’t even a monopolist.  (The truth is somewhere in the middle. Banner ads certainly compete with search ads, and thus may fall in the same market. But billboards?)

What happens if those two arguments aren’t sufficient to deflect further antitrust scrutiny? Well, Google has at least one more arrow in its quiver — an arrow that got stronger with this week’s launch of Bing, the new search engine from Microsoft:

Defense 3. Our business constantly faces new competition.  Even if we are a monopolist right now (which we’re not), there’s no guarantee that will last.  This argument emphasizes the dynamic nature of competition.  If new firms can enter the market — and, in fact, are entering the market — then any monopoly power may be transient.  Faced with new competitors, leading firms must continue to innovate if they want to stay at the top of the market.  In short, pressure from new competitors may prevent a dominant firm from exploiting any temporary market power.

The lawyers at Google probably welcomed Bing’s arrival, because it makes this line of argument much more plausible.  Other new competitors — anyone remember Cuil?  – haven’t posed much of a threat to Google, but Bing may be different.  So may Wolfram Alpha, which provides a really different approach to search.

If antitrust scrutiny heats up, you should expect to hear Google extolling the virtues of both those competitors.