[Info-vax] Large mailboxes

[Dave Froble]

On 12/1/2020 3:02 PM, Michael Moroney wrote:
> The discussion of mailboxes and MQ kind of reminded me of an old problem:
>
> Consider an application which produces a fairly large number of small (100-200
> byte) messages. The messages are sent via TCP to another system, and this can't
> be changed. (which is why something like MQ wasn't used)
>
> We don't want to lose any messages.
> In order to do that, the code was changed to write the messages to an indexed
> file (1 key, the timestamp).  A second process reads records ftom the file,
> sends to the remote system and upon acknowledgement, deletes the record.
> (if no record, it sleeps and tries again)  If the link/remote system was down,
> messages would accumulate in the indexed files.  Once the link was back, the
> second process would empty it.
>
> In other words, an RMS indexed file was just a FIFO. By being a file, records
> are not lost. That worked, but despite the file being almost always empty or at
> most 1-2 records, the file grew and grew and that part of the system slowed. A
> workaround was to create a new version of the file once in a while, and delete
> the old one (once known to be empty).  A $ CONVERT/RECLAIM on an empty or
> nearly empty file would chew on it for a while and would make the file faster
> [note this was done only to understand what was going on].  The slowness was
> from accumulated RMS cruft from all the deleted records.
>
> So in theory, is it possible to implement a VMS file as a FIFO and work
> efficiently?  That is one process can write records in order, a second
> process can read and delete records in order, without RMS cruft from
> accumulating?
>

Just about any file system, RMS, RDBMS, etc would see a FIFO queue as 
it's most difficult task.  I'd guess the "cruft" would be universal in 
such a solution.

Without thinking much about the task, I'd suggest that a linked list 
might be a decent solution.  No cruft.

-- 
David Froble                       Tel: 724-529-0450
Dave Froble Enterprises, Inc.      E-Mail: davef at tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA  15486