[Info-vax] Recovering messages from a MAIL.MAI file?

[gwilliams at cfa.harvard.edu]

On Monday, January 22, 2018 at 5:43:36 AM UTC, Phillip Helbig (undress to reply) wrote:
> In article <8df65c3e-394c-47bd-8461-49526216bda0 at googlegroups.com>,
> gwilliams at cfa.harvard.edu writes:
> 
> > I've been handed a zip file containing some MAIL*.MAI files
> > from a colleague.  They came from a VMS system that was
> > decommissioned some years back.  He has asked me to recover
> > the messages.  The externally-stored messages are easy, the
> > problem is the short messages stored within MAIL.MAI.
> >
> > The zip file was not created with the -V flag ("save VMS
> > attributes"), so the unzipped MAIL.MAI does not have the correct
> > attributes.
> >
> > By comparing to an active MAIL.MAI, I can use set file/attr
> > to set RFM:VAR. ORG:IDX, MRS:2048, RAT:NONE and BKS:5.  But I
> > see no way to set the prolog definition.
> 
> My guess is not just that the attributes are wrong, but also that some
> data, i.e. the index, is missing.  IIRC, short messages (up to 1 block?
> 4 blocks?) are stored in the MAIL.MAI file, together with the header
> information, while for longer messages only the first block (4 blocks?)
> with a pointer to the external file containing the rest of the message.
> If (and I don't know) this pointer is in the file you have, perhaps you
> could write a program to reconstruct a proper MAIL.MAI file.
> 
> (By the way, nice to see SOME VMS at CFA; say hello to Steve Willner for 
> me!)

We're finally migrating our operation off VMS onto Linux.  I expect our
VMS machines to be EOL'ed by the summer or autumn/fall.

The MAIL.MAI file should be complete, but the indexes are a concern.
In the MAIL.FDL generated from a working file:

IDENT   FDL_VERSION 02 "28-JAN-2018 19:25:12   OpenVMS ANALYZE/RMS_FILE Utility"

SYSTEM
        SOURCE                  OpenVMS

FILE
        ALLOCATION              19584
        BEST_TRY_CONTIGUOUS     yes
        BUCKET_SIZE             5
        CLUSTER_SIZE            32
        CONTIGUOUS              no
        EXTENSION               15
        FILE_MONITORING         no
        NAME                    "D23:[GARETH.MESSAGES]MAIL.MAI;1"
        ORGANIZATION            indexed
        OWNER                   [GARETH]
        PROTECTION              (system:RW, owner:RW, group:, world:)
        GLOBAL_BUFFER_COUNT     0
        GLBUFF_CNT_V83          0
        GLBUFF_FLAGS_V83        none

RECORD
        BLOCK_SPAN              yes
        CARRIAGE_CONTROL        none
        FORMAT                  variable
        SIZE                    2048

AREA 0
        ALLOCATION              19232
        BEST_TRY_CONTIGUOUS     yes
        BUCKET_SIZE             5
        EXTENSION               15

AREA 1
        ALLOCATION              160
        BEST_TRY_CONTIGUOUS     yes
        BUCKET_SIZE             5
        EXTENSION               5

AREA 2
        ALLOCATION              192
        BEST_TRY_CONTIGUOUS     yes
        BUCKET_SIZE             3
        EXTENSION               3

KEY 0
        CHANGES                 no
        DATA_KEY_COMPRESSION    no
        DATA_RECORD_COMPRESSION yes
        DATA_AREA               0
        DATA_FILL               100
        DUPLICATES              no
        INDEX_AREA              1
        INDEX_COMPRESSION       no
        INDEX_FILL              100
        LEVEL1_INDEX_AREA       1
        NAME                    ""
        NULL_KEY                no
        PROLOG                  3
        SEG0_LENGTH             8
        SEG0_POSITION           0
        TYPE                    bin8

KEY 1
        CHANGES                 yes
        DATA_KEY_COMPRESSION    yes
        DATA_AREA               2
        DATA_FILL               100
        DUPLICATES              yes
        INDEX_AREA              2
        INDEX_COMPRESSION       yes
        INDEX_FILL              100
        LEVEL1_INDEX_AREA       2
        NAME                    ""
        NULL_KEY                yes
        NULL_VALUE              0
        SEG0_LENGTH             39
        SEG0_POSITION           9
        TYPE                    string

ANALYSIS_OF_AREA 0
        RECLAIMED_SPACE         0

ANALYSIS_OF_AREA 1
        RECLAIMED_SPACE         0

ANALYSIS_OF_AREA 2
        RECLAIMED_SPACE         0

ANALYSIS_OF_KEY 0
        DATA_FILL               66
        DATA_KEY_COMPRESSION    0
        DATA_RECORD_COMPRESSION 25
        DATA_RECORD_COUNT       23509
        DATA_SPACE_OCCUPIED     18025
        DEPTH                   2
        INDEX_COMPRESSION       0
        INDEX_FILL              51
        INDEX_SPACE_OCCUPIED    140
        LEVEL1_RECORD_COUNT     3605
        MEAN_DATA_LENGTH        327
        MEAN_INDEX_LENGTH       10
        LONGEST_RECORD_LENGTH   1576

ANALYSIS_OF_KEY 1
        DATA_FILL               97
        DATA_KEY_COMPRESSION    74
        DATA_RECORD_COUNT       53
        DATA_SPACE_OCCUPIED     159
        DEPTH                   1
        DUPLICATES_PER_SIDR     294
        INDEX_COMPRESSION       82
        INDEX_FILL              2
        INDEX_SPACE_OCCUPIED    3
        LEVEL1_RECORD_COUNT     2
        MEAN_DATA_LENGTH        1485
        MEAN_INDEX_LENGTH       41

I don't see how the sizes of AREA1 and AREA2 can be determined.

Gareth