[Info-vax] "bad select 38" (OpenSSL on VMS)

[Stephen Hoffman]

On the way to another SSL1-related bug, I've noticed that the s_client 
bug Neil reported (below) now arises within the SSL1 kit, circa HP 
AXPVMS SSL1 V1.0-2C.

It appears that the fix from the old HP SSL ports has been dropped from 
the SSL1 kit.

I'll be having a look at the work-around mentioned below, and (yes) 
will likely also get the underlying bug reported to HPE and/or VSI.

Since the posting I'm replying to here is a few years old and might 
have expired out of your local newsgroup caches, I'm including the full 
text of the previous discussion...


On 2012-03-21 00:55:30 +0000, Neil Rieck said:

> Gents,
> 
> I finally found a few spare hours to solve the "bad select 38" problem
> as seen after executing DCL command "openssl s_client -connect". I
> have a fix but it is not yet "production quality" and will never never
> be as cool as solutions previously produced by the OpenVMS engineers.
> 
> It is important to note that "bad select 38" is never seen on the HP-
> sanctioned versions of OpenSSL, just the ports done by third party
> programmers (a.k.a. we, the unwashed public).
> 
>    ###
> 
> First off, here's a few basics about "UNIX I/O" and "select()".
> 
> I'm sure most of you already know that the standard UNIX devices
> (stdin, stdout, and stderr) correspond to FD (file-descriptors)
> numbers: 0, 1, and 2. As you open new i/o channels (files or sockets),
> you use up more file-descriptors. In the VMS world these UNIX devices
> map directly to SYS$INPUT, SYS$OUTPUT, and SYS$ERROR.
> 
> In the UNIX world, if you want to stall (block) your program while
> waiting for I/O to complete, you indicate channels of interest by
> setting associated bits in one of two longs (one for read, one for
> write), then pass them on to select(). So if you wanted to only wait
> for "read readiness from stdin" you would set bit-0 in the read bit-
> field then pass it on to select(). If you wanted to wait for any one
> of several devices, you would just set more bits.
> 
> But here is where "our VMS problem" develops. In the UNIX world "you
> can use select() to test all i/o channels" while in the VMS world "you
> may only use select() to test network channels (sockets)". In fact,
> error #38 is defined in file "UCX$INETDEF.H" with the label "ENOTSOCK"
> which is mapped to VMS code SS$_NOTNETDEV (Socket operation on
> nonsocket)
> 
> As is typical of my luck, I was working backwards by starting with the
> code, but had I checked the UNIX portability document here:
> 
> UNIX, Linux to OpenVMS Porting Guidelines
> http://h71000.www7.hp.com/portability/portingguidelines.html
> http://h71000.www7.hp.com/portability/portingguidelines.html#_Toc262045735
> 
> ...then I could have saved myself much time.
> 
> So what to do? There are two ways out of this debacle:
> 1) test for keyboard input without using select()
>     (this is easy to do but you might miss i/o from other channels if
> not careful)
> 2) follow the advice of the portability guide then write some code to
> employ a socket to connect to SYS$INPUT (or SYS$COMMAND).
> 
> If you want to see an example of how one programmer ("I Think" it
> might have been a Compaq employee but have no proof because the
> program changes were not documented) solved this problem, then click
> here:
> http://www3.sympatico.ca/n.rieck/demo_vms_html/openvms_demo_index.html#openssl
> 
> Neil Rieck
> Kitchener / Waterloo / Cambridge,
> Ontario, Canada.
> http://www3.sympatico.ca/n.rieck/docs/openvms_notes_ssl.html#bad-select-38


-- 
Pure Personal Opinion | HoffmanLabs LLC