[Info-vax] Raxco VMS Tuning Seminar Notes

[AEF]

On Nov 5, 6:31 am, Neil Rieck <n.ri... at sympatico.ca> wrote:
> On Nov 3, 1:43 pm, jls <notva... at yahoo.com> wrote:
>
> [...snip...].
>
> AFAIAC, the problem with the confusion around VMS tuning is due to the
> fact that many concepts are spread across multiple VMS manuals. So
> yesterday I was reading semi-unofficial VMS docs like "OpenVMS System
> Management Guide" which also gives the impression that AWSA runs
> independently every so often (like WATCHDOG or HITMAN). See topic
> 18.7.3.2 or click the following link:
>
> http://books.google.ca/books?id=yn9RN2e32J4C&pg=PA421&lpg=PA421&dq=aw...
>

But the "time" is actually CPU time, not real time. So if a process is
idle, it never reaches AWSTIME (i.e., since the last time it did!),
and the page-fault rate is not calculated.

> But this official document "VMS System Performance Management" (see
> topic: 3.5.6 titled "Voluntary Decrementing")http://h71000.www7.hp.com/doc/73final/6491/6491pro_003.html
> discusses PFRATL in a context which has nothing to do with swapper
> trimming but everything to do with the process limiting itself.
> itself. So thanks for setting me straight.

A process doesn't "limit" itself. PFRATL is a global parameter. OK, I
guess we're getting into semantics here. I think it's more of the
system saying "you don't really need all that memory, as you're not
page faulting all that much, so I'm taking some away based on my
SYSGEN parameters". The problem with this, of course, is that idle
processes, which have a PF rate of zero, are not recognized.

BTW, aren't all working set sizes contolled by SWAPPER? Swapper
trimming is when the "swapper" (I assume that's the same as SWAPPER,
no?) picks processes based on a complex algorithm to trim down, and
has nothing to do with PFRATL/H per se at least. It does this as
memory runs out, of course.

OK, I read the following to say that the "swapper" is the SWAPPER
process:

3.3.1 What Is the Swapper?

The swapper process schedules physical memory. It keeps track of the
pages in both physical memory and on the disk paging and swapping
files so it can ensure that each process has a steady supply of pages
for each job.

I think the PFRATL strategy would work a lot better if idle processes
were somehow included (with an assumed, and correct, PF rate of zero)
and trimmed first. But because they're idle, they don't reach the end
of their current AWSTIME interval, and don't show up on the radar, so
to speak.

[Yeah, I'm getting redundant here. But there's only so much time.
Sorry.]

>From the online Perf. Manual:

3.6.5 Swapper Trimming Versus Voluntary Decrementing

On most systems, swapper trimming is more beneficial than voluntary
decrementing because:

    * Swapper trimming occurs on an as-needed basis.
    * Voluntary decrementing occurs on a continuous basis and affects
only active, computable processes.
    * Voluntary decrementing can reach a detrimental condition of
oscillation.

The AUTOGEN command procedure, which establishes parameter values when
the system is first installed, provides for swapper trimming but
disables voluntary decrementing.

I believe this is in part why, in V5.4-3, "pro-active" memory
reclaimation was introduced:

>From http://h71000.www7.hp.com/doc/73final/6491/6491pro_003.html#proactive

3.7 Active Memory Reclamation from Idle Processes

The memory management subsystem includes a policy that actively
reclaims memory from inactive processes when a deficit is first
detected but before the memory resource is depleted.

The active memory reclamation policy acts on two types of idle
processes:

    * Long-waiting processes
    * Periodically waking processes

3.7.1 Reclaiming Memory from Long-Waiting Processes

A candidate process for this policy would be in the LEF or HIB state
for longer than number of seconds specified by the system parameter
LONGWAIT.

[...]

This actually does what I just said! (Yeah, 20/20 hindsight and all
that.) There's actually a paragraph or two in older versions of the
Perf. Manual that describes more of the motivation of this memory
management policy.

After reading the rest, is there really any need for manual
adjustments? Well, some, but somewhere above it says that performance
is much less dependent on establishing accruate WSQUOTA values. I
don't see it in this current version online.

Anyway, in my experience this policy works great!

>
> ###
>
> Okay so what would a system look like with with AWSA-
> Contraction(Reduction) disabled?
> Well, if the system becomes short of memory then swapper trimming
> would knock some/all processes who have borrowed memory over quota
> back to quota. If the system is still short of memory then some procs
> would be reduced even further to SWPOUTPGCNT (unless it was manually
> set higher) prior to swapping out.
>
> So I think it is better to have processes reducing their own memory
> requirements (acting like good citizens in the system) provided it
> doesn't invoke too much thrashing (hard or soft).

This is why pro-active memory reclamation was developed.

> How could you reduce AWSA thrashing?
> 1) Increase AWSMIN so you don't surrender below a certain value
> 2) Reduce the size of WSDEC to a value below WSINC (the default SYSGEN
> values are the other way around). This would result in process being
> able to get more memory when it was required but surrendering fewer
> pages when required. Don't worry, if you get this wrong then then you
> can always fall back on swapper trimming
> 3) Make AWSTIME some small multiple of QUANTUM. (don't make it too
> high as this would also affect AWSA-Expansion)
>
> But all of this is moot if you've got plenty of memory like I do on my
> AS-DS20e with 3 GB. It is a different story on my AS-2100 which only
> has 128 MB
>
> NSR

I like the following:

2.7.2 Tuning Suggestions

Tuning is rarely required for OpenVMS systems for the following
reasons:

[Hmmm. This makes a good chunk of the manual moot!]

    * The effort required is often more expensive than a capacity
upgrade.
    * The system includes AUTOGEN, which automatically establishes
initial values for all system-dependent system parameters.
    * The system includes features that in a limited way permit it to
adjust itself dynamically during operation. The system can detect the
need for adjustment in the following areas:
          o Nonpaged dynamic pool
          o Working set size
          o Number of pages on the free- and modified-page lists

As a result, these areas can grow dynamically, as appropriate, during
normal operation. For more information on automatic adjustment
features, see Section 3.5. The most common cause of poor system
performance is insufficient hardware capacity.

Although tuning is rarely required, it is appropriate in response to
two particular situations:

    * If you have adjusted your system for optimal performance with
current resources and then acquire new capacity, you must plan to
compensate for the new configuration. In this situation, the first and
most important action is to execute the AUTOGEN command procedure.
    * If you anticipate a dramatic change in your workload, you should
expect to compensate for the new workload.

    [OK, the above says to tune in the two given cases, but chiefly
via AUTOGEN.]

This is not to say that all there is to do is run AUTOGEN once in a
while. The portion of the manual before this describes such.

Another interesting performance thing I've always wanted to ask about:

2.3 Setting RMS File-Extend Parameters

Because files extend in increments of twice the multiblock count
(default is 16), system defaults now provide file extensions of only
32 blocks. Thus, when files are created or extended, increased I/O can
slow performance. The problem can be overcome by:

    * Specifying larger values for system file-extend parameters
    * Setting the system parameter RMS_EXTEND_SIZE
    * Specifying a larger multiblock count
    * Specifying a larger multibuffer count

Okay, this is the author of the manual claiming that the new default
is no good. Say what? Can't they all get on the same page, so to
speak? [No pun intended!]

It seems to me on modern systems with relatively huge disks,
RMS_EXTEND_SIZE should be much, much larger. Perhaps several thousand,
even. This is certainly necessary for the creation of large on-disk
BACKUP save sets and ZIP files, no? (That is, if you want to see them
finish the same day! OK, I exaggerate.)

AEF