[Info-vax] AltaVista background information

[dsnyder]

On Oct 15, 3:52 pm, MG <marcog... at SPAMxs4all.nl> wrote:
> Recently I thought about AltaVista, the search engine I used in the
> 1990s, without any doubt my favourite one back then.  I remember the
> "[d|i|g|i|t|a|l]" branding well, but unfortunately didn't really look
> beyond that back then.  (I was also quite young.)
>
> I was wondering, are there documents (articles, books and so forth)
> telling a thing or two about how it came into being and outlining
> the technological foundation?  I'm, needless to say, interested if
> VMS was involved.  (Else Tru64 or any other DEC operating system
> and hardware platform of interest.)
>
>   - MG

>From the archives, a bit of info on AltaVista:

The Hardware Behind AltaVista


   AltaVista: AlphaStation 500, 256 MB memory, 6GB disk.
   AlphaStation 500's handle all external traffic to the site.
   They run a custom multi-threaded Web server which sends
   queries to the Web indexer and News indexer.


   Web Indexer: AlphaServer 8400 5/300, 10 processors, 6 GB
   memory, 210 GB RAID disk. This model is the most powerful
   computer built by Digital. These servers run the query
   engine. The Web index is larger than 40 GB, but most
   requests take less than a second.


   Scooter: AlphaServer 4100 5/300, 1.5 GB memory, 30 GB RAID
   disk. The super-spider runs from this machine. It fetches
   pages from the Web and sends them to Vista, our primary web
   indexer.


   Vista: AlphaServer 4100 5/300, 2 processors, 2GB memory,
   180GB RAID disk. This machine indexes Scooter output and
   serves as a central distribution point for new index data.


   News Indexer: AlphaServer 600 5/333, 896MB memory, 13 GB
   disk. This machine keeps an up-to-date index of the news
   spool: since new articles appear and old articles expire all
   the time, it is in fact quite busy, even though the index it
   serves is much smaller than the Web index.


   News Server: AlphaServer 600 5/333, 896MB memory, 24 GB RAID
   disks. It maintains a current news spool for the News
   Indexer. It also serves the articles via http to those of
   you who don't want to know about news servers but want to
   read news.


Those aren't typos, either -- those are machines with many, MANY
megabytes
of RAM, and they're hooked up to some of the fastest random-access
backing
store in the world with capacities in the many, many gigabyte range.
AltaVista is fast because DEC threw some phenomenally capable
resources at
the problem.