[Info-vax] OS implementation languages

[Johnny Billquist]

On 2023-09-03 12:39, Jan-Erik Söderholm wrote:
> Den 2023-09-02 kl. 23:13, skrev Johnny Billquist:
>> On 2023-09-02 02:51, Bob Gezelter wrote:
>>> On Friday, September 1, 2023 at 5:45:41 PM UTC-4, Johnny Billquist 
>>> wrote:
>>>> On 2023-08-31 21:12, Bob Gezelter wrote:
>>>>
>>>>> The RSX design was obligatory due to the then lack of system pool. 
>>>>> Then, even individual IO Packets were precious. Correctness beats 
>>>>> efficiency. For more than two decades, increases in memory capacity 
>>>>> and non-paged pool expansion has rendered the memory minimization 
>>>>> design goal quaintly anachronistic.
>>>> Well. Like I said above, RSX actually is somewhat eager to do this.
>>>> Especially on 11M+, which do have a bit more resources available. In
>>>> plain 11M, you didn't see this done as much, since resources were more
>>>> scarce.
>>>>> The preceding is all in the reprint. A thorough reading of the 
>>>>> referenced paper is highly recommended. When I can find the time to 
>>>>> finish copy-editing the monograph and get it printed. I will note 
>>>>> the general availability in a posting.
>>>> And if you want more deep diving into RSX, just let me know. I'll
>>>> happily guide you through it. But it will be lots of MACRO-11. ;-)
>>>>
>>>> Johnny
>>> Johnny,
>>>
>>> I almost fell into that trap. There are multiple IO Packets in use 
>>> when one does read-ahead/write-behind. If you look at my tuning 
>>> presentation from the late 1980s, you will find precisely that advise.
>>>
>>> However, there is a serialization bottleneck when an individual IO 
>>> Request covers a logically discontiguous sequence of logical blocks 
>>> on a volume. Once again, it is in the previously referenced IEEE 
>>> conference paper. The serialization in IO Packet mapping is described 
>>> in the OpenVMS Internals and Data Structures manual.
>>
>> Well, yes, one individual I/O request is in itself inherently 
>> serialized. But like I said, at least with RSX, the read ahead/write 
>> behind functionality is not done with just one I/O request, because 
>> then you would not actually get much read ahead benefit, since the I/O 
>> don't complete until all of it is complete.
>>
>> It's done with multiple smaller reads. And they can complete in any 
>> order. But software will get data when the request that directly 
>> access the data requested right now have completed. The other I/O 
>> requests are hopefully finished by the time the software gets that 
>> far. And new ones can be issued while the software is still working on 
>> data it previously received.
>>
>>    Johnny
>>
> 
> Since Rdb was mentioned up-thread...
> 
> This is in a way similar to how Rdb uses "Asynchronous Prefetch"
> and "Asynchronous Batch-Write" to get less stalls while waiting
> for disk reads and writes.
> 
> Reads:
> "When the asynchronous prefetch feature is enabled, Oracle Rdb
> examines each process and attempts to predict the process’ future
> access patterns. When Oracle Rdb can predict a sequential scan
> from a process’ successive page requests, it fetches the pages
> that it anticipates will be included in the sequential scan.
> This prefetching of pages (fetching a page before it has been
> requested) is asynchronous. That is, Oracle Rdb does not wait
> for the prefetched pages to be read into memory from disk; it
> continues with its current processing activities."
> 
> So, Rdb doesn't always "prefetch", only when the access pattern
> indicates that it would be a performance win.
> 
> Writes:
> "Oracle Rdb reads and writes pages while executing transactions.
> By default, it supports asynchronous batch-write operations, which
> reduce the number of stalls experienced by database processes while
> waiting for writes to disk to complete.
> The goal of asynchronous batch-write operations is to increase database
> performance by making it possible for a certain number of buffers in
> each process’ allocate set to have write operations in progress at
> any time without causing the process to stall."
> 
> It is worth noting that the data is safe, the transaction journal
> file (AIJ file) has already been updated for these "dirty" buffers,
> so the update can always be recreated, if there is an issue with
> the later write-ack of the data to the tables.
> 
> And again, RMS is not involved in this at all, at's just QIO's.

QIO is such a wonderful solution to the question of I/O, and makes it 
possible to easyly to some nice tricks to get a lot of performance. 
Which is why I sometimes wonder when people claim I/O is so bad in VMS. 
QIO is way better than most I/O under Unix systems, and memory mapped 
I/O as well as the select()/poll() and signals in Unix is there just 
because something similar to QIO don't really exist.

There is still some issues with ODS that certainly could be improved. 
But then it's usually actually RMS that is the bottleneck. But as Bob 
Gelzer observed and commented, RMS can be tweaked to at least give much 
better performance that it does by default.

But also, of course, if programs don't make use of the capabilities QIO 
gives them, then things will also suffer. And I guess a lot of people 
aren't that comfortable in writing programs that have a rather 
asynchronous behavior. And I can understand that. And it's usually more 
difficult to utilize in high level languages. So improving libraries and 
services like RMS to do this for you is probably the best way to make 
things move faster.

I think I remember from maybe a year ago that RMS under VMS don't allow 
you to us locate mode for writes. Which means that any writing of data 
via RMS involves at least one data copy. That's sad, and it would be 
nice if it was improved. RMS under RSX do allow locate mode for writes.

But I wonder how much code isn't using locate mode for reads as well, 
meaning yet another copying of data there.

   Johnny