[Info-vax] Large mailboxes

[Hein RMS van den Heuvel]

On Tuesday, December 1, 2020 at 3:02:54 PM UTC-5, Michael Moroney wrote:
> The discussion of mailboxes and MQ kind of reminded me of an old problem: 
:
> In other words, an RMS indexed file was just a FIFO. By being a file, records 
> are not lost. That worked, but despite the file being almost always empty or at 
> most 1-2 records, the file grew and grew and that part of the system slowed.:
:
> So in theory, is it possible to implement a VMS file as a FIFO and work 
> efficiently? That is one process can write records in order, a second 
> process can read and delete records in order, without RMS cruft from 
> accumulating?


No, not easily. This may warrant a dedicated topic.
There are two things causing 'crud'
1) Each Indexed file databucket is assigned a high-key value which is using in the index above it.
With ever growing key values, once a bucket cannot hold a record with the next key value, a new bucket is allocated. 
The old bucket highest key value now is frozen and entered in the index. 
The next bucket gets the new record with a 'high key' (0xffffffff or 'zzzzzzzz' - so to speak - yes this is sloppy wording)
Going forward no new record will ever be targeted to any older bucket even though over time they are empty they still have a fixed key range.
2) each record gets a new 16 bit id within the bucket. So even when records are deleted before the bucket is filled, the ids are consumed and after 64K records a new bucket is allocated and the old one never ever re-used 

You can postpone the growth by creating larger buckets (63 block max) but the price there is disk IO bandwidth on the write and delete (reads are from cache).
You can try to reused bucket better for example by coming up with round robin PK values for example by making them  HHMMSSYYYYMMDD -but now the application needs to know whether the  prior date was truly processed.

CONVERT/RECLAIM can help, but needs exclusive file access, so you might was well convert or just put a new file in place and move in the stragglers.

Nothing pretty out there.

Hein