[Info-vax] Database Journaling, Failover (was: Re: C... the only winning move is not to play...)

[Stephen Hoffman]

On 2014-02-13 01:17:10 +0000, David Froble said:

> 1) Journaling or logging of all data updates, with the capability of 
> rebuilding the updates from a static backup and the journals.

RMS Journaling on VMS would be one approach.  That can be licensed for 
and available to applications using the RMS record or RMS block I/O 
calls.  That's usually all based on RMS journaling with the DECdtm 
transaction management.

> 2) In the event a port away from VMS must be done, a method to have 
> duplicate real time copies of the data, one in the current DAS data 
> files, and another in one of the free / open database products.  This 
> would allow porting in small pieces while the applications continued to 
> be used, and updated.

That's conceptually simple, but tends to get gnarly around failure 
scenarios.   When one write goes through successfully but the parallel 
write fails, you're skewed.  Next step is to figure out what to do, and 
either back out the error, fail out, or recover.  This whether the 
failing write was aimed at the primary or some secondary database, or 
at the journal.  Things get more interesting when a secondary error 
arises in the recovery path.

If you're planning on running write I/O via both the existing VMS path 
and the new database path in parallel, that makes the design much more 
complex.  (That may end up using something like ZeroMQ 
<http://zeromq.org> across hosts, or database high availability or 
replication features 
<http://dev.mysql.com/doc/refman/5.7/en/replication.html> 
<http://www.postgresql.org/docs/current/interactive/high-availability.html>.) 


If you're rolling your own, there are various papers on database 
reliability and recovery, too; tools and techniques and algorithms.

It's usually easier to use one of the existing databases than to write 
one.  Unfortunately, swapping out an existing database within a typical 
large and accreted application environment is seldom a small and 
isolated project; those database tendrils seem to get everywhere.


-- 
Pure Personal Opinion | HoffmanLabs LLC