[Info-vax] DCL crashing bug update

Tue Jul 25 06:38:01 EDT 2017

On Monday, July 24, 2017 at 9:50:34 PM UTC-4, Simon Clubley wrote:
> I finally found some time to look in some detail at the DCL bug I found
> and have managed to get some firm answers as well as a surprisingly
> simple reproducer now that I understand the problem better.
> 
> There are also some questions for VSI at the end of this post.
> 
> On the plus side, I can't think of a way to turn this into a security
> vulnerability because it isn't what I originally thought it was. On the
> minus side, it turns out that Unix isn't the only operating system with
> an in-band null character (ie: 0x00) problem. :-)
> 
> Basically, if you craft a specific sequence of command lines with lots
> of embedded null characters in one of the command lines and inject
> those command lines directly into DCL, then you will get a crash on
> VMS Alpha v8.4. I don't know if VAX or IA64 is also affected.
> 
> Before going any further, here's a sample command procedure which
> demonstrates the problem. READ THE NOTES FULLY BEFORE USING.
> 
> =======================================================================
> $ set verify
> $ !
> $ ! WARNING: You run this command procedure and use the output from it at
> $ ! your own risk! Do NOT run this command procedure or the bug triggering
> $ ! sequence on production systems or on any system which is important to you.
> $ !
> $ ! While exploring this bug on VMS Alpha running under the FreeAXP emulator,
> $ ! I experienced multiple crashes during a reboot. I do not know for sure
> $ ! if this has anything to do with the triggering of this bug or not so
> $ ! you have been warned.
> $ !
> $ ! ==> To be safe, I recommend you perform a reboot immediately after you
> $ ! have finished examining this bug even though I am having a hard time
> $ ! seeing how supervisor mode code could mess up the system to this level
> $ ! so for now I am assuming this is probably just an emulator artifact.
> $ !
> $ ! Run this command procedure to generate simon_test.tmp and then do the
> $ ! following:
> $ !
> $ !	$ recall/erase
> $ !	$ recall/input=simon_test.tmp
> $ !	$ recall/all
> $ !
> $ ! At this point, DCL crashes on me when using the VMS Alpha v8.4 hobbyist
> $ ! distribution.
> $ !
> $ ! Simon.
> $ !
> $ set noverify
> $	write sys$output "Generating test file"
> $
> $	open/write outch simon_test.tmp
> $	call generate_line "2ab" 1197 0
> $	call generate_line "" 1500 66
> $	call generate_line "" 1400 67
> $
> $	close outch
> $	write sys$output "Test file generated"
> $	exit
> $
> $ generate_line:
> $ subroutine
> $	chr[0,7] = f$integer(p3)
> $	oline = p1
> $	i = 1
> $
> $ next_char:
> $	if i .gt. p2 then $ goto line_complete
> $	oline = oline + chr
> $	i = i + 1
> $	goto next_char
> $
> $ line_complete:
> $	write/symbol outch oline
> $	exit
> $
> $ endsubroutine
> =======================================================================
> 
> and here is the crash I get with the above sequence:
> 
> [snip]
> 
> $ set noverify
> Generating test file
> Test file generated
> $ recall/erase
> $ recall/input=simon_test.tmp
> $ recall/all
> 
> [snip first two command lines which are ok]
> 
>   3 
> 
>   Improperly handled condition, bad stack or no handler specified.
>     Signal arguments:   Number = 0000000000000005
>                         Name   = 000000000000000C
>                                  0000000000000000
>                                  000000007FFC9CD0
>                                  000000007AF96F98
>                                  0000000000000012
> 
>     Register dump:
>     R0  = 000000007FF9CC30  R1  = 000000007FF9FEA3  R2  = 000000000000000A
>     R3  = 000000007FF9CC30  R4  = 000000007AEF85F0  R5  = 0000203320200000
>     R6  = 0000000000000001  R7  = 000000007FFA4F28  R8  = 000000007FF9CDE8
>     R9  = 000000007FFC9CD1  R10 = 000000007FFA4F28  R11 = 000000007FFCDC18
>     R12 = 000000007FFCDA98  R13 = 00000000000000FB  R14 = 0000000000000000
>     R15 = 000000000000002F  R16 = 000000007FFCEDE2  R17 = 00000000000011CA
>     R18 = 000000007FFC8CAF  R19 = 0000000000000001  R20 = 000000007FFC8CB1
>     R21 = 000000007AEF84B0  R22 = 000000000000612F  R23 = 000000007FF9FEA4
>     R24 = 000000007FF9FEA6  R25 = 0000000000000001  R26 = 000000007AF97268
>     R27 = 000000007AEF84B0  R28 = 000000007AF97404  R29 = 000000007FF9CC00
>     SP  = 000000007FF9CC00  PC  = 000000007AF96F98  PS  = 0000000000000012
> 
> My test payloads were generated using HP Pascal programs because it's
> a lot easier to work with binary records with nulls embedded in them
> in Pascal than in C. The above command procedure is just a Q&D
> command procedure so you can easily explore this bug if you wish.
> 
> Note that it's only 0x00 which causes the crash. If you generate a
> file with the same structure but with 0x00 replaced with 0x01 then
> DCL does not crash (although it doesn't show the command line with
> the 0x01 characters embedded in it).
> 
> The DCL command history buffer is a fixed size wrap-around buffer.
> Originally, I thought this was a problem with my fuzzing data causing
> DCL to get confused about where the start of the buffer is[*], but my
> attempts to embed a valid internal representation of a DCL command
> within another larger command always resulted in DCL displaying the
> larger command correctly (and ignoring the embedded command) so it
> became clear that wasn't the problem.
> 
> [*] I quickly found the longword which identifies the next free byte
> to write to within the buffer (it's immediately prior to the buffer
> data itself) but I couldn't find anything which identified the offset
> to the first command within the history buffer area. This is why
> I originally thought DCL might be scanning for all the 0x00 characters
> which are immediately prior to the first command when the first command
> is not at the start of the buffer.
> 
> If you want to examine the command history buffer from another process
> then running SDA, setting the target PID you want to examine and then
> using the following command will get you started:
> 
> SDA> examine CTL$AG_CLIDATA;1400
> 
> Also note that if you increase the first record from 1200 to 1300 bytes,
> I get buffer wrap-around and if the first record is all nulls then the
> trailing line length field is not removed from the command history when
> the first record is purged from the history. Also note that this does
> not occur if 0x01 is used instead of 0x00 (for example).
> 
> My questions for VSI are these:
> 
> 1) I've been concerned for some time that as new functionality was
> added to VMS then maybe this new functionality allowed the existing
> functionality to be manipulated in ways that could cause problems
> because the new functionality was never considered as part of the
> original design and security audit process.
> 
> DCL clearly has code which assumes it is never going to see a stream
> of 0x00 characters because you could probably never get those characters
> into the history buffer via the terminal driver. However, $ recall/input
> now gives you the ability to directly inject any characters you choose
> directly into the DCL command history buffer.
> 
> When adding new functionality to VMS, how much work is done to evaluate
> the original design assumptions in light of any new functionality ?
> 
> 2) While this specific problem turned out not to be an actual vector
> for a security exploit, does this same style of buffer management
> exist elsewhere in VMS and in a form which could cause problems if
> the utility/command allows direct injection of command history ?
> 
> Simon.
> 
> -- 
> Simon Clubley, clubley at remove_me.eisner.decus.org-Earth.UFP
> Microsoft: Bringing you 1980s technology to a 21st century world

I may be a little confused here but........I executed your command procedure and then the recall commands and got a process crash but the system is still up and running.  Are you saying you got a system crash?

BTW: I am on an i2 running 8.4-2L1.

Clair