[Info-vax] DCL crashing bug update

[David Froble]

clair.grant at vmssoftware.com wrote:
> On Monday, July 24, 2017 at 9:50:34 PM UTC-4, Simon Clubley wrote:
>> I finally found some time to look in some detail at the DCL bug I found
>> and have managed to get some firm answers as well as a surprisingly
>> simple reproducer now that I understand the problem better.
>>
>> There are also some questions for VSI at the end of this post.
>>
>> On the plus side, I can't think of a way to turn this into a security
>> vulnerability because it isn't what I originally thought it was. On the
>> minus side, it turns out that Unix isn't the only operating system with
>> an in-band null character (ie: 0x00) problem. :-)
>>
>> Basically, if you craft a specific sequence of command lines with lots
>> of embedded null characters in one of the command lines and inject
>> those command lines directly into DCL, then you will get a crash on
>> VMS Alpha v8.4. I don't know if VAX or IA64 is also affected.
>>
>> Before going any further, here's a sample command procedure which
>> demonstrates the problem. READ THE NOTES FULLY BEFORE USING.
>>
>> =======================================================================
>> $ set verify
>> $ !
>> $ ! WARNING: You run this command procedure and use the output from it at
>> $ ! your own risk! Do NOT run this command procedure or the bug triggering
>> $ ! sequence on production systems or on any system which is important to you.
>> $ !
>> $ ! While exploring this bug on VMS Alpha running under the FreeAXP emulator,
>> $ ! I experienced multiple crashes during a reboot. I do not know for sure
>> $ ! if this has anything to do with the triggering of this bug or not so
>> $ ! you have been warned.
>> $ !
>> $ ! ==> To be safe, I recommend you perform a reboot immediately after you
>> $ ! have finished examining this bug even though I am having a hard time
>> $ ! seeing how supervisor mode code could mess up the system to this level
>> $ ! so for now I am assuming this is probably just an emulator artifact.
>> $ !
>> $ ! Run this command procedure to generate simon_test.tmp and then do the
>> $ ! following:
>> $ !
>> $ !	$ recall/erase
>> $ !	$ recall/input=simon_test.tmp
>> $ !	$ recall/all
>> $ !
>> $ ! At this point, DCL crashes on me when using the VMS Alpha v8.4 hobbyist
>> $ ! distribution.
>> $ !
>> $ ! Simon.
>> $ !
>> $ set noverify
>> $	write sys$output "Generating test file"
>> $
>> $	open/write outch simon_test.tmp
>> $	call generate_line "2ab" 1197 0
>> $	call generate_line "" 1500 66
>> $	call generate_line "" 1400 67
>> $
>> $	close outch
>> $	write sys$output "Test file generated"
>> $	exit
>> $
>> $ generate_line:
>> $ subroutine
>> $	chr[0,7] = f$integer(p3)
>> $	oline = p1
>> $	i = 1
>> $
>> $ next_char:
>> $	if i .gt. p2 then $ goto line_complete
>> $	oline = oline + chr
>> $	i = i + 1
>> $	goto next_char
>> $
>> $ line_complete:
>> $	write/symbol outch oline
>> $	exit
>> $
>> $ endsubroutine
>> =======================================================================
>>
>> and here is the crash I get with the above sequence:
>>
>> [snip]
>>
>> $ set noverify
>> Generating test file
>> Test file generated
>> $ recall/erase
>> $ recall/input=simon_test.tmp
>> $ recall/all
>>
>> [snip first two command lines which are ok]
>>
>>   3 
>>
>>   Improperly handled condition, bad stack or no handler specified.
>>     Signal arguments:   Number = 0000000000000005
>>                         Name   = 000000000000000C
>>                                  0000000000000000
>>                                  000000007FFC9CD0
>>                                  000000007AF96F98
>>                                  0000000000000012
>>
>>     Register dump:
>>     R0  = 000000007FF9CC30  R1  = 000000007FF9FEA3  R2  = 000000000000000A
>>     R3  = 000000007FF9CC30  R4  = 000000007AEF85F0  R5  = 0000203320200000
>>     R6  = 0000000000000001  R7  = 000000007FFA4F28  R8  = 000000007FF9CDE8
>>     R9  = 000000007FFC9CD1  R10 = 000000007FFA4F28  R11 = 000000007FFCDC18
>>     R12 = 000000007FFCDA98  R13 = 00000000000000FB  R14 = 0000000000000000
>>     R15 = 000000000000002F  R16 = 000000007FFCEDE2  R17 = 00000000000011CA
>>     R18 = 000000007FFC8CAF  R19 = 0000000000000001  R20 = 000000007FFC8CB1
>>     R21 = 000000007AEF84B0  R22 = 000000000000612F  R23 = 000000007FF9FEA4
>>     R24 = 000000007FF9FEA6  R25 = 0000000000000001  R26 = 000000007AF97268
>>     R27 = 000000007AEF84B0  R28 = 000000007AF97404  R29 = 000000007FF9CC00
>>     SP  = 000000007FF9CC00  PC  = 000000007AF96F98  PS  = 0000000000000012
>>
>> My test payloads were generated using HP Pascal programs because it's
>> a lot easier to work with binary records with nulls embedded in them
>> in Pascal than in C. The above command procedure is just a Q&D
>> command procedure so you can easily explore this bug if you wish.
>>
>> Note that it's only 0x00 which causes the crash. If you generate a
>> file with the same structure but with 0x00 replaced with 0x01 then
>> DCL does not crash (although it doesn't show the command line with
>> the 0x01 characters embedded in it).
>>
>> The DCL command history buffer is a fixed size wrap-around buffer.
>> Originally, I thought this was a problem with my fuzzing data causing
>> DCL to get confused about where the start of the buffer is[*], but my
>> attempts to embed a valid internal representation of a DCL command
>> within another larger command always resulted in DCL displaying the
>> larger command correctly (and ignoring the embedded command) so it
>> became clear that wasn't the problem.
>>
>> [*] I quickly found the longword which identifies the next free byte
>> to write to within the buffer (it's immediately prior to the buffer
>> data itself) but I couldn't find anything which identified the offset
>> to the first command within the history buffer area. This is why
>> I originally thought DCL might be scanning for all the 0x00 characters
>> which are immediately prior to the first command when the first command
>> is not at the start of the buffer.
>>
>> If you want to examine the command history buffer from another process
>> then running SDA, setting the target PID you want to examine and then
>> using the following command will get you started:
>>
>> SDA> examine CTL$AG_CLIDATA;1400
>>
>> Also note that if you increase the first record from 1200 to 1300 bytes,
>> I get buffer wrap-around and if the first record is all nulls then the
>> trailing line length field is not removed from the command history when
>> the first record is purged from the history. Also note that this does
>> not occur if 0x01 is used instead of 0x00 (for example).
>>
>> My questions for VSI are these:
>>
>> 1) I've been concerned for some time that as new functionality was
>> added to VMS then maybe this new functionality allowed the existing
>> functionality to be manipulated in ways that could cause problems
>> because the new functionality was never considered as part of the
>> original design and security audit process.
>>
>> DCL clearly has code which assumes it is never going to see a stream
>> of 0x00 characters because you could probably never get those characters
>> into the history buffer via the terminal driver. However, $ recall/input
>> now gives you the ability to directly inject any characters you choose
>> directly into the DCL command history buffer.
>>
>> When adding new functionality to VMS, how much work is done to evaluate
>> the original design assumptions in light of any new functionality ?
>>
>> 2) While this specific problem turned out not to be an actual vector
>> for a security exploit, does this same style of buffer management
>> exist elsewhere in VMS and in a form which could cause problems if
>> the utility/command allows direct injection of command history ?
>>
>> Simon.
>>
>> -- 
>> Simon Clubley, clubley at remove_me.eisner.decus.org-Earth.UFP
>> Microsoft: Bringing you 1980s technology to a 21st century world
> 
> I may be a little confused here but........I executed your command procedure and then the recall commands and got a process crash but the system is still up and running.  Are you saying you got a system crash?
> 
> BTW: I am on an i2 running 8.4-2L1.
> 
> Clair

I think Simon wrote:

"About those crashes. Here's one that occurred this weekend during a reboot:"

So, most likely two separate things.