[Info-vax] Byte range locking

[Jan-Erik Soderholm]

Den 2016-11-23 kl. 05:34, skrev David Froble:
> Jan-Erik Soderholm wrote:
>> Den 2016-11-22 kl. 21:56, skrev David Froble:
>>> John E. Malmberg wrote:
>>>
>>> Hi John ...
>>>
>>>> On 11/22/2016 10:50 AM, David Froble wrote:
>>>>>
>>>>> Ok, likely I'm missing something, but let me ask, what's so difficult?
>>>>>
>>>>> Get a request for a string of bytes that for example spans disk blocks
>>>>> 2367 to 2714, with the first and last blocks possibly (likely) only
>>>>> partly required.  So what?  Get locks for all the blocks, allocate a
>>>>> buffer, read the range of blocks, possibly make changes, and write the
>>>>> buffer back to disk.  If cache is in use, it should just work.  Yes,
>>>>> that's a lot of locks to successfully acquire.  But no mods to the DLM.
>>>>
>>>> RMS can only take out locks for existing blocks.
>>>
>>> We ain't talking about RMS.
>>>
>>>> Many of the fcntl()/flock() requests are for byte ranges that do not
>>>> exist.
>>>
>>> A lock is on a "resource".  The DLM doesn't know anything about files,
>>> existing or otherwise.
>>>
>>>> Also many of the files are stream or an undefined record format, so no
>>>> record IDs to use.
>>>
>>> Not sure why this should matter?
>>>
>>>> With out using RMS locks, the application needs SYSLCK to use the DLM.
>>>
>>> And I have some UWSS routines, just for that.  One might call them "Wizard
>>> Magic".
>>>
>>>> Now setting up byte rangs locks has a lot of overhead, so some platorms
>>>> have implemented Opportunistic locking.
>>>
>>>> The simple explanation for them is that the lock request first attempts
>>>> to do an exclusive lock on the file.  If it can get that, it just tracks
>>>> the byte-range locks internally, but does not actually take them out.
>>>>
>>>> If no other application ever tries to lock that file, then a lot of
>>>> overhead is skipped.
>>>
>>> Yeah, I never understood that.  Either there is a lock, or there isn't.
>>> But that makes sense.
>>>
>>>
>>>> When a second application tries to take out a byte-range lock, the first
>>>> application is notified and it then takes out the byte-range locks and
>>>> releases the exclusive lock.  This is a lot of overhead as buffers need
>>>> to be flushed and distributed caches may be involved on served files.
>>>
>>> I'd ask, why not just wait until the file lock is released?
>>
>> It might never get released, until the whole application (or system)
>> is shutdown. You just don't know that. You end up waiting forever.
>
> I'm still in exploratory mode, so lots of stuff to think about.  However,
> it was my guess that the "file" lock would be a special lock, reserving the
> file for an update, not the file lock(s) for file open mode.
>
> A file update lock could not be granted if there were any outstanding locks
> for update.  Nor would a file update lock be allowed to remain in place for
> other than the update.
>
> It's my understanding that what's driving this issue is lack of SSIO for
> Postgres, and possibly other databases.  While RDB does not, as far as I
> know, have a database engine that does all the work...

Correct. All file operations (apart from some central logging and recover
operations) are done by the user processes. Coordination and locking is
done thought the DLM. Smallest lockable item is a table row.


> it's my vague
> impression that most relational database products do use the database
> engine concept.  In such a case, only one process (I'm guessing) would be
> doing the locking.
>
> Not saying such a capability should be limited to just a few products.
>
>
>>> Lot less work.
>>>
>>>> I do not know if I can use a user written blocking AST with an RMS
>>>> granted lock.  If I could do that, then it should be possible to write an
>>>> byte range locking simulation what would work completely in user mode
>>>> code.
>>>
>>> See above for UWSS ....
>>>
>>> Let me know if you want some code to look at ....
>>