The Deep Web?

By roopakroopak (1227767376|%a, %b %e at %I:%M%p)

Introduction

I naturally have a tendency to be skeptical of web resources that I have never heard of. Before our class on The Deep Web, I had no idea that the Deep Web existed. The idea that these were resources that a normal Google search could not access interested me slightly, but I assumed the information couldn't really be that useful. I decided to go ahead and give the "Deep Web" a try and explore what it had to offer. I thought it might change my perceptions like RSS feeds and news searches had.

Unfortunately, I was not really wowed by this new resource and it didn't alter my perceptions. I went through each of the Deep Web resources given in class to conduct exercises and submitted the query 'oil consumption' (with quotes). This was a relatively simply query that I assumed would produce similar results across the board. My assumption turned out to be horribly wrong. I have included the top 5 results returned by each resource below and evaluated each resource based on my results.

Deep Web resources

Scirus

Environmental Defense - Putting the Brakes on Oil Consumption
Plasmatron could cut oil consumption, emissions - MIT News Office
03-Engine Oil Consumption Measurement Brochure
The Determinants of Sulfur Emissions from Oil Consumption in Swedish Manufacturing Industry
SwRI measures real-time wear and engine oil consumption in motorcycle engines

My first observation with Scirus was that the format of the results was similar to that from a normal web search in Google. Each result had a title and a short description below it. This was very simple and straight-forward.

Taking a look at the results, I noticed that each result was relevant in some way to the search. They talked about either measuring oil consumption or reducing oil consumption. They were large websites with many resources and did not talk about current news. Instead, each result was one webpage that discussed something related to oil consumption. Also notable was the fact that of the top five, one result was from 2006, three were from 2007, and one was from 2008. I would not consider this a particularly useful source for my project. Though it did not unlock any wealth of information about oil consumption, it did at least provide a few interesting pages.

Google Scholar

Consumption of Olive Oil and Specific Food Groups in Relation to Breast Cancer Risk in Greece
Fish oil consumption and decreased risk of cardiovascular disease: a comparison of findings from …
Oil and the Macroeconomy since World War II
Olive-oil consumption and health: the possible role of antioxidants.
The Effect of Olive Oil and Fish Consumption on Rheumatoid Arthritis-A Case Control Study

Once again, the format of the results was similar to that from a normal web search in Google. This was simple to use just like with Scirus above.

Unfortunately, the results returned were terrible! I think that might actually be an understatement. This was easily the worst resource I had found so far in the class! Just looking at the results made me laugh and think that this might be a joke. Olive oil? Fish oil? How could these possibly be the main topics of 4 of the top 5 results? The only result that actually spoke about the type of oil I was looking for spoke about World War II.

Could this possibly get worse? Somehow it actually did. The publication dates for the results in the order they are listed above were: 1995, 1986, 1983, 2000, and 1991. Professor Moore mentioned during our class on the Deep Web that sometimes the publication dates are wrong in Google Scholar. I'm not sure if this is the case with these results, but I am assuming that the majority of them have the correct publication dates. They just happened to be not at all relevant to what I was looking for and from at least 8 years ago. All I have to say about Google Scholar is that it is horrible and I probably will not be relying on it for any help.

BNET

Cyprus Solar Thermal Market and Technology Assessment
Palm oil consumption increasing, says council, BUSINESS TIMES
BP SHOWS AREAS OF MOGAS GROWTH; FLAT GLOBAL OIL CONSUMPTION
BP: WORLD OIL DEMAND DIPS IN 2001; U.S. REFINING CAPACITY CREEPS HIGHER
Too Soon to Re-enter Oil Stocks?

The format of the results from BNET was slightly different from the two resources above and a normal Google search, but it had the same degree of simplicity and straight-forwardness. It also contained tags below the results, which I found rather convenient.

Besides the article on palm oil, these results were relevant. Some of the results linked to excerpts from the papers or articles on the BNET site and some linked to other websites. It was a bit annoying that most of them required registration to view the actual paper or article. However, these articles were very informative and BNET seemed to be a good resource for extensive research on a topic. Just like with the previous two resources, BNET's publication dates for the top 5 results were not very recent. The fifth result was from the day I conducted the search (08/13/2008), but the other results were all from 2002 and 2003.

Turbo10

Energy Information Administration - International …
NationMaster - Oil > Consumption (most recent) by …
Dodge, Chrysler DaimlerChrysler Vehicle Defects
Oil Industry Statistics from Gibson Consulting
Petroleum - Wikipedia, the free encyclopedia

Now this was a good example of bad formatting of results. The default was 5 results and there was no description below the titles. There were also no publication dates listed. The only information that was given besides the title and the domain name was the source that provided to result to Turbo 10 (for 4 of the results this was alltheweb.com and for 1 of the results this was dmoz.org). This was completely ineffective.

The results were not bad in terms of relevance. The first result was an excellent resource with all kinds of statistics about oil. This was the same resource I had gotten most of my data for my project from at this point. The second and fourth results were both very informative resources as well. As far as quality of results goes, this was the best of the Deep Web resources so far. I must note that I found it amusing that the fifth result was simply a link to the Wikipedia entry for 'Petroleum.'

Librarians' Internet Index

AAA Fuel Cost Calculator
Corporate Average Fuel Economy (CAFE)
How Gas Prices Work
Oil Spiel
Fueleconomy.gov

This was the best of all of the resources in formatting of results. Below each result title was a description, the url, the topics (i.e. Automobiles, Oil & Gas), and the date the site was last updated. I could have no complaints about this. It was even better than Google.

The results themselves did not make me quite as happy as the formatting, but they weren't bad either. The first and fifth results were completely irrelevant. The middle three contained interesting information about fuel economy, how consumption is taken into account to determine gas prices, and peak oil. All three of these were potentially useful sources. It is important to note that the first three results were last updated in 2005, the fourth was was updated in 2006, and the fifth was last updated in 2008. Though this was a good resource overall, like with most of the others, the results were not very current.

Conclusion

I did not find any of the Deep Web search engines to be especially useful for me. I can say, however, that I can see each of them being useful except for Google Scholar. I did only submit one query to each of the Deep Web search engines, so naturally my results cannot be taken as conclusive evidence of how useful they will be in general. However, I believe that some of the observations I had can provide good clues as to how useful they will be for research on other topics. If you would like to take a look at the full lists of results returned by the Deep Web search engines for my query ('oil consumption'), here are four of them: Scirus, Google Scholar, BNET, and Librarians' Internet Index. For Turbo 10 you will actually have to type in the query ('oil consumption'), because the url does not change in the address bar when you conduct a search.

050822b_DeepTrawl.gif
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License