[Info-vax] Large mailboxes

[Marc Van Dyck]

Arne Vajhøj laid this down on his screen :
>
> I could add that as a rule of thumb I would
> only use VMS mailboxes (or Windows pipes or
> *nix unix sockets) to buffer hundreds or a
> few thousands of messages.
>
> If I needed hundreds of thousands or
> millions I would look for a message queue
> (and if I needed billions I would look at
> Kafka).
>
> Arne

Long story... Do you remember DECps, the polycenter performance
management product ? Well, it is still around, and we are still using
it. CA ported it to Itanium 12 years ago, but didn't do any functional
update since they received it from DEC. There are large parts of it
that can't be used anymore because it has no idea about what current
hardware can do.

So now we think it's time to replace it, for those two reasons :
- it is functionally obsolete (doesn't know anything about TCP/IP and
fibre channel, for example)
- it will probably never be ported to X86

We use it for the following purposes:
- Display system performance in real-time (dashboard)
- Publish performance graphs on a web site
- Deep dive into performance related issues (with the Motif interface)
- Long term capacity planning (export data to Excel via CSV files)

The last 3 uses can be easily taken over by Perfdat, but not the first
one. Perfdat locks the file it uses to store collected data, so there 
is
no way to exploit them in real time. T4 and TDC do the same. So we
decided to roll our own...

Collecting the data we need is not difficult, either from a DCL loop, 
or
from a program issuing $GETRMI calls. To display them, we're going to
use an in-house developped web service that can store and display any
metric that one cares to throw at it. The interface to send the data is
cURL, which under VMS has a tendency to get stuck when used 
intensively.
So we'll decouple the collector and the sender, and put a large mailbox
between the two. If the mailbox is large enough, we can tolerate a 
stall
of the sender for several minutes, and it will catch up with the 
mailbox
contents very quickly once restarted.

This kind of data does not need to survive a reboot (there will always
be perfdat to look at what happened anyway) and a mailbox is a cheap
and easy to use buffer for this kind of purpose. We have plenty of
memory to dedicate to that. Probably not what was in the head of people
who designed that 40 years ago, but I don't see why it would not work.
If there is really a hard limit of 64k entries, we'll just put in
some mailboxes in parallel. My question was basically to determine
whether I need to do that or not.

-- 
Marc Van Dyck