When the ability to search through even a small part of the Web's holdings first appeared in 1995, it looked little short of miraculous. Although we take this facility for granted now, it seems likely that it was the constant improvement in public search engines that helped make the Web a mass medium.
In a sense, things began with Yahoo. This is a directory, rather than a search engine, but it opened people's eyes to both the richness of the then-young Web, and the power of being able to search through it.
One of the first true search engines was Lycos, which was named after a running spider - the software robots that follow the links of the Web and catalogue the pages they find there are often called spiders. In March 1995, I marvelled at how Lycos had indexed a "massive"
1.75 million Web pages. Lycos is still very much with us, but other early search engines have faded since those pioneering days.
For example, I wonder whether many people use Webcrawler. It still exists, but as part of Excite its future must be uncertain, given the way that Excite has sold off many of its assets. Another early search engine that now leads only a ghostly twilight existence is the World Wide Web Worm, whose opening page is mirrored in Colombia.
I wrote about the Galaxy service back in October 1994; remarkably, it is still around, though much changed from its early form (there is an interesting history avaialble). One year later I mentioned Open Text's engine; the site exists, but the search facility is long gone.
At least Altavista is still going strong - but perhaps not as strong as it might have done. When I first wrote about it six years ago, Altavista seemed to represent a next-generation service, so powerful were its searches. Alas, that early lead was squandered as Altavista gradually lost its way. Today, it is one of the better sites for searching, but it has definitely been supplanted by a more recent arrival - Google.
As with Altavista, it was Google's underlying technology that clearly placed it ahead of the pack when it first appeared. But what is remarkable is that it has succeeded in maintaining that position despite fierce competition. Google seems to have achieved this through a single-minded devotion to providing the best search engine on the Internet and not getting distracted by trying to turn itself into a broader portal (as Altavista did).
There is a history of the company and a timeline. By June 2000, Google was already claiming to be the Web's biggest search engine, with more than one billion indexed Web pages, a figure that has now risen to three billion.
One of Google's great strengths is the breadth of its offering - it allows you to search for images as well as through its huge archive of Usenet postings (acquired from Deja.com). Also worth noting is the advanced search option and the topic-specific searches. These include searches for the Apple Macintosh world and GNU/Linux.
Google's support for open source is hardly surprising, since it derives its power from the world's largest GNU/Linux cluster - an extraordinary 10,000 machines ( www.google.com/press/highlights.html).
The only worry is that Google might be tempted to lose its focus as it chases after higher revenues. So far, its track record is exemplary, but, given his disastrous time at Novell, the appointment of Eric Schmidt as Google's chief executive does not augur well.