[Info-vax] Large mailboxes

[Arne Vajhøj]

On 12/1/2020 6:04 PM, Jan-Erik Söderholm wrote:
> Den 2020-12-01 kl. 21:02, skrev Michael Moroney:
>> The discussion of mailboxes and MQ kind of reminded me of an old problem:
>> Consider an application which produces a fairly large number of small 
>> (100-200
>> byte) messages. The messages are sent via TCP to another system, and 
>> this can't
>> be changed. (which is why something like MQ wasn't used)
>>
>> We don't want to lose any messages.
>> In order to do that, the code was changed to write the messages to an 
>> indexed
>> file (1 key, the timestamp).  A second process reads records ftom the 
>> file,
>> sends to the remote system and upon acknowledgement, deletes the record.
>> (if no record, it sleeps and tries again)  If the link/remote system 
>> was down,
>> messages would accumulate in the indexed files.  Once the link was 
>> back, the
>> second process would empty it.
>>
>> In other words, an RMS indexed file was just a FIFO. By being a file, 
>> records
>> are not lost. That worked, but despite the file being almost always 
>> empty or at
>> most 1-2 records, the file grew and grew and that part of the system 
>> slowed. A
>> workaround was to create a new version of the file once in a while, 
>> and delete
>> the old one (once known to be empty).  A $ CONVERT/RECLAIM on an empty or
>> nearly empty file would chew on it for a while and would make the file 
>> faster
>> [note this was done only to understand what was going on].  The 
>> slowness was
>> from accumulated RMS cruft from all the deleted records.
>>
>> So in theory, is it possible to implement a VMS file as a FIFO and work
>> efficiently?  That is one process can write records in order, a second
>> process can read and delete records in order, without RMS cruft from
>> accumulating?
> 
> It depends on the definition of a "VMS file". I see no issues with having
> an Rdb table as the buffer. It will reuse deleted space if the process
> doing the delete just disconnect and reconnect now and then (to free the
> deleted space). Could be once a day or so. But on the other hand, there
> is no real performance issue with locked free space, Rdb will just not
> use it.

I have mentioned it already, but if one use a message queue that
can use relational database for message storage, then one can
get the queue functionality out of the box on top of the database
(Rdb or other).

Arne