[Info-vax] ODS-5 data/file recovery

[MG]

On 16-feb-2013 15:22, Stephen Hoffman wrote:
> 1: Give up on that file, and move on?  This depends on how irreplaceable
> the data in that file really was; what it will cost you to recreate the
> contents.

First of all, thank you very much for thinking along and the, as
always, lengthy post and I appreciate all the tips, pointers and
suggestions.

'Giving up' is certainly something I plan on doing, but I'm keeping it
as my last choice.  (I fortunately have a somewhat recent back-up of
the bulk of the data, but it's a few months old by now.  But, at least
it is something.)


> 1a: Most of us have made and have learned from this and similar cases —
> as this will not be the only time a file goes missing with OpenVMS, or
> any other operating system — and will look at and potentially improve
> the backup strategy.  (Each time I encounter one of these cases, the
> more I value operating systems that provide integrated backup
> mechanisms, and applications with integrated backup.  With OpenVMS,
> there's no template, no automatic operation, and you get to roll your
> own.  Which is a PITA.  But I digress.)

I'm still not even sure what happened and I wouldn't even know which
logs to look into.  I did naturally look into SAMBA$HOME, but no trace
of anything bearing the name of the file that I lost...

That's the big pity, it's just one (although, rather big) file and it
is gone.  It shouldn't have happened, but it did.

To give some additional back story, this is what --- in my recollection
--- happened:

I was transferring files from ODS-5 (hw. RAID 5 slices, physically
connected with Ultra320 SCSI LVD/SE to my primary VMS [V8.4] HP
rx2620 'home server') via CIFS, whilst mounted on a Linux (Mint 14)
AMD64 system over gigabit ethernet.

Most of the file transfers, as I was transferring multiple files, seem
to be going fine (and are still on-going).  I do recall I cancelled the
transfer, of the file that went 'missing' (let's call it "file.zip").

To sum up what happened, event-wise, as far as I remember:

    - [Linux host]: Initiate file transfer (move) "file.zip" from VMS
      ODS-5 CIFS share to a remote NAS device ("dm-raid" RAID 5 array
      XFS volume) over internal GbE LAN.

    - [Linux host]: Cancel transfer.  Reason: Takes too long, decide
      to transfer to a faster (non-RAID) disk, for speed and because
      I will employ the file soon.

    - [Windows host]: Initiate file transfer (move) "file.zip" from
      VMS ODS-5 CIFS share to local storage (NTFS) volume.  This
      seems to be going fine.  Until, at some point, I get the
      message, along the lines of: 'Can't continue, can't read from
      source file.'  (I don't recall the exact error message.)

After that, the first thing I did was look with DFU on the ODS-5
volume, but sadly found nothing...  This isn't supposed to happen.
I have cancelled countless of both 'copy' and 'move' commands,
both through CLI and GUI (e.g. 'file manager'), and it nearly
always went fine and the source would not be touched until the
file was transferred or even checked (I assume it at least looks
at the checksum, if it roughly matches).

Only in Linux, I found some traces (parts of the file, presumably
cached as I looked into the Zip with one of the many GUI archive
explorer type tools in Linux (Mint, as mentioned before).  The
traces were stored in $HOME/.cache/.fr-%%%%%%.


> 1b: shift to a computing platform with the tools you expect and need?
> CIFS has long been "fun" to deal with on OpenVMS.  OpenVMS disk services
> aren't particularly stellar implementations, either; the CIFS/Samba port
> is old, partial, ill-documented, and complex.  Some of my experiences
> with it are at <http://labs.hoffmanlabs.com/node/1350>. This also
> includes (preferably integrated) backup tools and data recovery tools,
> as there are a variety of circumstances where files can end up
> unavailable, regardless of the operating system.  Other platforms can
> have easier and newer and variously better tools.

Tell me all about it...  But, the irony is, CIFS gave me lesser head-
aches than NFS!  So, there's not much choice.  Well, other than FTP and
SFTP (lacking the bulk of the features of FTP, like hiding file version
numbers to non-VMS hosts).  And with FTP, I also ran into trouble,
especially with certain file attributes.


> 2: Read the on-disk structure (ODS) specs, and scrounge up, or roll your
> own tools?   Treat this as an opportunity to learn the innards of the
> ODS-2 and ODS-5 file systems and data structures, and the programming
> languages and tools.  As disk structures go, ODS-2 and ODS-5 are not
> particularly demented, and the ODS-2 portions are documented, and the
> data structure declarations are all available in the libraries.

I will do that; or, actually, I already made a start and began reading
more into this since yesterday.


> 2a: The ODS2.DOC text file is the starting point for this quest, if you
> don't have access to Kirby's book.  Also see
> <http://labs.hoffmanlabs.com/node/209>.  With the exception of an
> identifying constant in the volume header to differentiate ODS-2 from
> ODS-5 and the addition of FI5DEF structures used for filenames that
> don't fit into the existing FI2DEF data structures, ODS-2 and ODS-5 are
> basically identical on-disk.  Subsequent to the ODS2.DOC text file, both
> acquired longer bitmaps at V7.2, and both can have a GPT
> <http://developer.apple.com/library/mac/#technotes/tn2166/_index.html>
> on Itanium which caused some of the headers to be relocated
> <http://labs.hoffmanlabs.com/node/112>.  Dig the starting block
> addresses and the block lengths out of the header for each extent, and
> dig the extents out of any extension headers that might be involved, and
> read them in.

Thanks, I will keep that in mind and look into this!  (I run I64 V8.4
at the moment, as a Hobbyist.)


> 3: Hire somebody to recover the files for you?   This usually won't be
> cheap, so see 1 above.

It depends on exactly how expensive, that I may even consider it.  Any
recommendations for services, organizations/companies, etc. out there,
with great VMS data recovery experience?

  - MG