[Info-vax] C and assembly language, was: Re: WHY IS VSI REQUIRING A HYPERVISOR FOR X86 OPENVMS?

[John Reagan]

On Saturday, December 26, 2020 at 6:09:34 AM UTC-5, Simon Clubley wrote:
> On 2020-12-23, John Reagan <xyzz... at gmail.com> wrote: 
> > 
> > Consider a broken C routine that is declared "int" but one or more paths don't have a valid 'return' statement. On Alpha/Itanium, the value in R0/r8 is often the return value from a prior call to something. GEM doesn't really want to reuse R0/r8 for any kind of temporary since it won't live past the next routine call (it will use them if pushed into a corner). The return without a 'return' just gets you that prior R0/r8 value. If your 'int' routine is really a true/false routine, you might have been working for years without any problems. 
> > 
> > However, given the few registers on x86 (and how some of the instructions work), the return register %rax is used in many places for a short-term temporary. So when the C routine exits without a 'return' statement, the value in %rax can be very strange. 
> >
> Interesting. I don't suppose we would happen to know the C program in 
> question ? :-)

Seen them inside of PCSI and CMS recently.

The C NEEDSRETURN isn't perfect as a frontend can only statically analyze a program to see if edges to the end of a routine call contain a return, but control flow can change that.  And I think /STAND=VAXC influences that too, not sure.  We've been trying to remove /STAND=VAXC as we go.  Some modules are trivial, some are just full of places that need typecasts (or better typedef's and macros).  BLISS programmers that write C /STAND=VAXC, just go used to the "free typecasts" that K&R gives you.