[Info-vax] improve performance of /EXCLUDE

[Stephen Hoffman]

On 2015-01-28 04:28:34 +0000, mcleanjoh at gmail.com said:

> On Wednesday, January 28, 2015 at 10:57:32 AM UTC+11, Stephen Hoffman wrote:
>> It seems you might have little experience with a system that offers a  
>> faster search?
> 
> Are you talking of a pre-processing search that's parsed all file and 
> saved the index in some file, much like web search engines?

Your apparent confusion and my clear inability to clearly communicate 
what the plug-ins provide would tend to imply that you haven't used 
something akin to the Spotlight tool[1] before; not in any depth.  
There's nothing wrong with that at all, of course.  More a case of 
figuring out what sort of phrasing and detail I should use here, in 
order to try to communicate the benefits.  If you have used Spotlight 
and its grammar[2] on the other hand, then you'd probably be familiar 
with its benefits.  This is part of why I keep suggesting that folks 
here go try other operating systems and tools, too.  Find out what's 
out there, and how it can solve your problems — solving your problems 
either differently, or quite possibly even solving them better.  But I 
digress...

> I can see that being useful for some things (e.g. searching on a set of 
> specific files) but not so much for wider use where the cost-benefit 
> doesn't weigh up.

A number of years ago, having found and started using the mdfind[3] 
command — Spotlight's command-line interface — the benefits of a faster 
search stack up very quickly, simply due to how much faster the search 
results can be acquired.  The Spotlight search grammar is a little 
baroque when first encountered, but it does work.  Yes, Google's search 
grammar is no less baroque.[4]  But I digress.  Again.

Tools such as Spotlight are particularly effective when there's a 
mechanism to notify the search indexer to re-index recent file system 
changes.  A tool such as locate does pretty well, but it's usually 
based on polling and thus doesn't catch quick changes as efficiently.  
Various web search engines offer pingers, too; a way to allow a web 
site to notify the crawlers of changes.  Ugh.  Digression.   Again.  
Bad Hoff.  No more digressions.

> I just did a search across a whole bunch of .FOR, .C and even .OLB 
> files to see if a specific string of characters existed in any of them. 
>  Would mdfind be able to handle the .OLB object library?

Yes, it would be.   There are standard plug-ins.   If you happen to 
have your own custom format file, then you can write your own custom 
plug-in.  For an environment such as VMS, I'd expect the standard 
plug-ins would be able to extract and cache the contents of common file 
formats including text, help and object libraries, and from zip 
archives and BACKUP savesets, probably also symbol table files and 
shareable image entrypoints, and from some other common resources.  The 
ht:/Dig had search tool had plug-ins, and was extensible through custom 
plug-ins, as well — this extensibility and this sort of a modular 
design isn't at all unusual for these search tools, after all.

With VMS, it'd be interesting to see plug-ins available some of the 
kernel data structures indexed, such as the running processes, logical 
name tables, and the lock resources, too.

But I'd expect there'd be some interesting user extensions and 
plug-ins, too.[5]

########
[1] 
https://developer.apple.com/library/mac/documentation/Carbon/Conceptual/MetadataIntro/Concepts/WhatIsSpotlight.html> 

[2] 
<https://developer.apple.com/library/mac/documentation/Carbon/Conceptual/SpotlightQuery/Concepts/QueryFormat.html> 

[3] 
<http://0xfe.blogspot.com/2006/03/using-spotlight-from-os-x-commandline.html> 

[4] <https://support.google.com/websearch/answer/2466433?hl=en>
[5] <https://github.com/nate-parrott/Flashlight#readme>



-- 
Pure Personal Opinion | HoffmanLabs LLC