[Info-vax] DCL crashing bug update

[abrsvc]

On Tuesday, July 25, 2017 at 6:38:04 AM UTC-4, clair... at vmssoftware.com wrote:
> On Monday, July 24, 2017 at 9:50:34 PM UTC-4, Simon Clubley wrote:
> > I finally found some time to look in some detail at the DCL bug I found
> > and have managed to get some firm answers as well as a surprisingly
> > simple reproducer now that I understand the problem better.
> > 
> > There are also some questions for VSI at the end of this post.
> > 
> > On the plus side, I can't think of a way to turn this into a security
> > vulnerability because it isn't what I originally thought it was. On the
> > minus side, it turns out that Unix isn't the only operating system with
> > an in-band null character (ie: 0x00) problem. :-)
> > 
> > Basically, if you craft a specific sequence of command lines with lots
> > of embedded null characters in one of the command lines and inject
> > those command lines directly into DCL, then you will get a crash on
> > VMS Alpha v8.4. I don't know if VAX or IA64 is also affected.
> > 
> > Before going any further, here's a sample command procedure which
> > demonstrates the problem. READ THE NOTES FULLY BEFORE USING.
> > 
> > =======================================================================
> > $ set verify
> > $ !
> > $ ! WARNING: You run this command procedure and use the output from it at
> > $ ! your own risk! Do NOT run this command procedure or the bug triggering
> > $ ! sequence on production systems or on any system which is important to you.
> > $ !
> > $ ! While exploring this bug on VMS Alpha running under the FreeAXP emulator,
> > $ ! I experienced multiple crashes during a reboot. I do not know for sure
> > $ ! if this has anything to do with the triggering of this bug or not so
> > $ ! you have been warned.
> > $ !
> > $ ! ==> To be safe, I recommend you perform a reboot immediately after you
> > $ ! have finished examining this bug even though I am having a hard time
> > $ ! seeing how supervisor mode code could mess up the system to this level
> > $ ! so for now I am assuming this is probably just an emulator artifact.
> > $ !
> > $ ! Run this command procedure to generate simon_test.tmp and then do the
> > $ ! following:
> > $ !
> > $ !	$ recall/erase
> > $ !	$ recall/input=simon_test.tmp
> > $ !	$ recall/all
> > $ !
> > $ ! At this point, DCL crashes on me when using the VMS Alpha v8.4 hobbyist
> > $ ! distribution.
> > $ !
> > $ ! Simon.
> > $ !
> > $ set noverify
> > $	write sys$output "Generating test file"
> > $
> > $	open/write outch simon_test.tmp
> > $	call generate_line "2ab" 1197 0
> > $	call generate_line "" 1500 66
> > $	call generate_line "" 1400 67
> > $
> > $	close outch
> > $	write sys$output "Test file generated"
> > $	exit
> > $
> > $ generate_line:
> > $ subroutine
> > $	chr[0,7] = f$integer(p3)
> > $	oline = p1
> > $	i = 1
> > $
> > $ next_char:
> > $	if i .gt. p2 then $ goto line_complete
> > $	oline = oline + chr
> > $	i = i + 1
> > $	goto next_char
> > $
> > $ line_complete:
> > $	write/symbol outch oline
> > $	exit
> > $
> > $ endsubroutine
> > =======================================================================
> > 
> > and here is the crash I get with the above sequence:
> > 
> > [snip]
> > 
> > $ set noverify
> > Generating test file
> > Test file generated
> > $ recall/erase
> > $ recall/input=simon_test.tmp
> > $ recall/all
> > 
> > [snip first two command lines which are ok]
> > 
> >   3 
> > 
> >   Improperly handled condition, bad stack or no handler specified.
> >     Signal arguments:   Number = 0000000000000005
> >                         Name   = 000000000000000C
> >                                  0000000000000000
> >                                  000000007FFC9CD0
> >                                  000000007AF96F98
> >                                  0000000000000012
> > 
> >     Register dump:
> >     R0  = 000000007FF9CC30  R1  = 000000007FF9FEA3  R2  = 000000000000000A
> >     R3  = 000000007FF9CC30  R4  = 000000007AEF85F0  R5  = 0000203320200000
> >     R6  = 0000000000000001  R7  = 000000007FFA4F28  R8  = 000000007FF9CDE8
> >     R9  = 000000007FFC9CD1  R10 = 000000007FFA4F28  R11 = 000000007FFCDC18
> >     R12 = 000000007FFCDA98  R13 = 00000000000000FB  R14 = 0000000000000000
> >     R15 = 000000000000002F  R16 = 000000007FFCEDE2  R17 = 00000000000011CA
> >     R18 = 000000007FFC8CAF  R19 = 0000000000000001  R20 = 000000007FFC8CB1
> >     R21 = 000000007AEF84B0  R22 = 000000000000612F  R23 = 000000007FF9FEA4
> >     R24 = 000000007FF9FEA6  R25 = 0000000000000001  R26 = 000000007AF97268
> >     R27 = 000000007AEF84B0  R28 = 000000007AF97404  R29 = 000000007FF9CC00
> >     SP  = 000000007FF9CC00  PC  = 000000007AF96F98  PS  = 0000000000000012
> > 
> > My test payloads were generated using HP Pascal programs because it's
> > a lot easier to work with binary records with nulls embedded in them
> > in Pascal than in C. The above command procedure is just a Q&D
> > command procedure so you can easily explore this bug if you wish.
> > 
> > Note that it's only 0x00 which causes the crash. If you generate a
> > file with the same structure but with 0x00 replaced with 0x01 then
> > DCL does not crash (although it doesn't show the command line with
> > the 0x01 characters embedded in it).
> > 
> > The DCL command history buffer is a fixed size wrap-around buffer.
> > Originally, I thought this was a problem with my fuzzing data causing
> > DCL to get confused about where the start of the buffer is[*], but my
> > attempts to embed a valid internal representation of a DCL command
> > within another larger command always resulted in DCL displaying the
> > larger command correctly (and ignoring the embedded command) so it
> > became clear that wasn't the problem.
> > 
> > [*] I quickly found the longword which identifies the next free byte
> > to write to within the buffer (it's immediately prior to the buffer
> > data itself) but I couldn't find anything which identified the offset
> > to the first command within the history buffer area. This is why
> > I originally thought DCL might be scanning for all the 0x00 characters
> > which are immediately prior to the first command when the first command
> > is not at the start of the buffer.
> > 
> > If you want to examine the command history buffer from another process
> > then running SDA, setting the target PID you want to examine and then
> > using the following command will get you started:
> > 
> > SDA> examine CTL$AG_CLIDATA;1400
> > 
> > Also note that if you increase the first record from 1200 to 1300 bytes,
> > I get buffer wrap-around and if the first record is all nulls then the
> > trailing line length field is not removed from the command history when
> > the first record is purged from the history. Also note that this does
> > not occur if 0x01 is used instead of 0x00 (for example).
> > 
> > My questions for VSI are these:
> > 
> > 1) I've been concerned for some time that as new functionality was
> > added to VMS then maybe this new functionality allowed the existing
> > functionality to be manipulated in ways that could cause problems
> > because the new functionality was never considered as part of the
> > original design and security audit process.
> > 
> > DCL clearly has code which assumes it is never going to see a stream
> > of 0x00 characters because you could probably never get those characters
> > into the history buffer via the terminal driver. However, $ recall/input
> > now gives you the ability to directly inject any characters you choose
> > directly into the DCL command history buffer.
> > 
> > When adding new functionality to VMS, how much work is done to evaluate
> > the original design assumptions in light of any new functionality ?
> > 
> > 2) While this specific problem turned out not to be an actual vector
> > for a security exploit, does this same style of buffer management
> > exist elsewhere in VMS and in a form which could cause problems if
> > the utility/command allows direct injection of command history ?
> > 
> > Simon.
> > 
> > -- 
> > Simon Clubley, clubley at remove_me.eisner.decus.org-Earth.UFP
> > Microsoft: Bringing you 1980s technology to a 21st century world
> 
> I may be a little confused here but........I executed your command procedure and then the recall commands and got a process crash but the system is still up and running.  Are you saying you got a system crash?
> 
> BTW: I am on an i2 running 8.4-2L1.
> 
> Clair

I think that the key point here may have been missed in the comments:

"While exploring this bug on VMS Alpha running under the FreeAXP emulator"

I plan on trying this on a real Alpha this week to confirm.  I too would expect a process crash not a system issue.

Dan