On the overlap, the precision and estimated recall of search engines. A case study of the query "Erdos"

In this paper we investigate the retrieval capabilities of six Internet search engines on a simple query. As a case study the query "Erdos" was chosen. Paul Erdos was a world famous Hungarian mathematician, who passed away in September 1996. Existing work on search engine evaluation considers only the first ten or twenty results returned by the search engine, therefore approximation of the recalls of the engines has not been considered so far. In this work we retrieved all 6681 documents that the search engines pointed at and thoroughly examined them. Thus we could calculate the precision of the whole retrieval process, study the overlap between the results of the engines and give an estimate on the recall of the searches. The precision of the engines is high, recall is very low and the overlap is minimal.