[Info-vax] improve performance of /EXCLUDE

[George Cornelius]

In article <ma9ci1$ahh$1 at dont-email.me>, David Froble <davef at tsoft-inc.com> writes:
> mcleanjoh at gmail.com wrote:
> 
>> What difference do larger RMS buffers make to the search time?

RMS buffers are primarily for data blocks and are associated with
open files.

And DIRECTORY may bypass the RMS level and go straight to the
filesystem anyway.
 
>> I once wrote a 'copy' program (with an entry point to make it
>> callable) that saw how much memory I had available (i.e. WSMAX less
>> current allocation) and used a large chunk of that as the copy
>> buffer.  I *think* it was faster than normal copy but it wasn't easy
>> to check this because on the first run the file got cached on the SAN
>> and read access fell quite a bit. A 'search' using a similar system
>> might be very fast.
> 
> I don't think that you'll do better than reading whatever the disk 
> cluster size is.  You're still limited by disk seeks and transfers.

Actually, cluster size has little to do with it unless your file
is heavily fragmented.

Transfer time you cannot eliminate (and caching, unless it involves
clairvoyance, tends to only improve response the _second_ time you
access something).  What you can eliminate is seek time and
latency.  If you can buffer several tracks worth of data and do
the readahead as, basically, one contiguous operation, then
on a real disk you can largely avoid both kinds of access time
slowdown.
 
RAID and other virtualizations, along with caching, make these
things difficult to measure. In the end the underlying media
transfer speed is what matters in the case of a one-time copy
of a large file.  The rest can be considered to be just smoke
and mirrors.

George