[Info-vax] Looking for some text search ideas

[Paul Sture]

On 2014-09-26, David Froble <davef at tsoft-inc.com> wrote:
> Perhaps in place of discussing non-existent malware on the non-existent 
> VMS on x86, I might solicit some ideas.
>
> Our applications are not using a RDBMS.
>
> A request has come up to be able to find any data which contains some 
> specific text.  An example might be any product description that 
> contains the text "gasket".  Using keys won't help, because the key 
> might be "head gasket".
>
> This is similar I believe to the SQL request something like
>
> SELECT * from PRODUCT where DESCRIPTION %like% gasket

The correct syntax there (for the flavours of SQL I know) would be:

SELECT * from PRODUCT where DESCRIPTION like %gasket%

where 'like' is a case blind search and % is the wildcard character. You'd
probably want to list only the fields you are interested in rather than
the *, e.g. SELECT DESCRIPTION, PRICE from...

> My perspective is that on today's systems with gobs of memory that much 
> of a database's information is probably in memory, thus not incurring 
> the overhead of lots of disk seeks.
>
> It's also my perspective that such a search is a sequential pass through 
> the data looking for matches.
>
> And so this is my question.  Does anyone know of a more effective method 
> than a sequential pass through the data of searching a list of data 
> looking for text matches?

Associative arrays spring to mind, probably but not necessarily addressed
by a hash of the key:

<https://en.wikipedia.org/wiki/Associative_array>

Or of course an RMS indexed file could be used.

Either way would involve building an initial key/value store which will
be resource intensive.  Keeping it up to date will depend on how dynamic
the master file is; my guess is that for parts descriptions this will
be pretty static for most of the time with occasional flurries of
activity when new models come onto the market.

> I'm looking at possibilities from global data making the search 
> available to all, to storage inside the one function currently needing 
> this capability.  Some more research into the application needs will 
> determine the answer to this question.

OK.

-- 
A quick recap of Thursday 25th September 2014:
http://pbs.twimg.com/media/ByZfyyXIQAAXTai.jpg
Happy Thursday!