[Info-vax] DCL's flaws (both scripting and UI)

[David Froble]

Stephen Hoffman wrote:
> On 2015-01-19 01:46:10 +0000, David Froble said:
> 
>> Simon Clubley wrote:
>>> I think it's time once again to build a list of DCL's flaws now that
>>> VSI are around. Don't forget however that DCL is both a scripting
>>> language and a UI. My initial list is below.
>>
>> Do you mind if I'm an ass for a bit?  I promise to be more reasonable 
>> at the end.
>>
>>> Some of DCL's UI flaws:
>>>
>>> 1) You can't edit commands across line boundaries. Even Windows CLIs
>>> can do this.
>>
>> It has been explained in the past that the problem was this stuff was 
>> implemented in the terminal driver.  I'd suggest looking at a 
>> different implementation, such as an application that could do the 
>> command line length and buffer and history, and maybe running multiple 
>> copies.  Yes, it is a PITA.
> 
> Ayup.  The terminal driver, another hunk of legacy VMS code.  Code 
> that's accumulated more than a little technical debt, and which is 
> inherently a very complex piece of code.  Code that's best either 
> overhauled or even re-architected and reworked into a newer and simpler 
> design, but this would break compatibility for the applications that 
> (still?) need the more arcane features of the classic terminal driver.
> 
> Serial port controllers themselves are legacy designs, beyond (maybe) 
> the console, and that's often replaced with a management processor.
> 
> There is a cost to providing compatibility.

I don't know the internals.  Does TelNet and such use the terminal 
driver?  Is this one of the pieces of "cruft code" that you' like to see 
go away or be replaced?

There probably is some people using serial lines somewhere.  But from 
what I see, serial lines are a thing of the past.

There is probably someone somewhere running applications on the console.

>>
>>> 2) You can't save your command history automatically, bash style, and
>>> have it restored automatically at the next session.
>>
>> See above for better implementation, not a patch on the terminal driver.
> 
> DCL does most of the command processing already, it's the current 
> command line processing where DCL depends on the features of the 
> terminal driver, and the driver itself can't manage the long-line wrap. 
>  DCL either reads out or loads in command lines, using the terminal driver.

It sounds as if possibly DCL would be the place to implement long lines 
and such.  But, I know nothing ...

>>
>>> With bash, you can have multiple shells active at the same time and
>>> only the commands entered during a specific session will be added to
>>> the history file when that session exits even though the shell has the
>>> full command history from previous shells available (up to a user
>>> defined limit).
>>
>> I don't know why anyone would want 2 script processors at the same 
>> time.   Isn't that multiple sessions?
>>
>> I also don't understand retaining the history.
> 
> History means your commands are available over separate logins, and over 
> server reboots.   It's really handy to have.   You're not copying and 
> pasting commands into files or into notebooks, for when you get back to 
> work the next day.   It's increasingly common to hack this feature into 
> VMS with RECALL commands, but that's rather clunky.

I know what the history does.  I just don't see it being a major 
stumbling point.

>>
>>> Any implementation needs to think about the multiple shells active at
>>> the same time issue before proposing a solution to this.
>>>
>>> 3) No filename completion. This is _really_ annoying especially since
>>> it even exists (after a fashion) in the command prompt on current
>>> versions of Windows.
> 
> Filename and full-on command completion would be handy to have.
> 
>> I still remember, with a grin, JF's idea of user name and password 
>> completion.
>>
>> :-)
>>
>> :-)
> 
> Yes, a joke.  But there are parts of the login that are pretty crufty.

I'm not so sure it was a joke when he wrote it.

> Certificate-based logins would be handy.

Yes.  This would be a good security enhancement.

>  Not quite the same as password 
> completion, but something that's used very heavily with ssh-based 
> logins.  Other systems use biometric readers (problems and all) or 
> two-factor with tokens or out-of-band authentication.
> 
>>
>>> 4) No elegant incremental search through the command history (bash
>>> Ctrl-R style).
>>>
>>> Some of DCL's scripting limitations:
>>>
>>> 1) No structured programming constructs such as (for example) while
>>> loops.
>>
>> Learn the mantra,
>>
>> "DCL is not a programming language".
>> "DCL is not a programming language".
>> "DCL is not a programming languaige".
>> "DCL is not a programming language".
> 
> 
> Is that like "we've always done it this way" or "nobody needs more than 
> 640K"?

No.  It's "there are better existing methods".

>  DCL is a programming language.  It's just an old and clunky one, 
> and part of dusting off VMS means updates.
>>
>>> 2) No ability to iterate cleanly over a list of items (bash "for i in"
>>> style)
>>
>> "DCL is not a programming language".
>> "DCL is not a programming language".
>> "DCL is not a programming language".
> 
> An iterator would be very handy.  Cleaner code.
> 
>>
>> But, there is GOTO ...
> 
> Computed goto, if you're willing to hack the symbols.  Hard to 
> troubleshoot, though.  Or patching your code on the fly, if you're 
> willing to substitute a GOTO for some other command.
> 
>>
>>> 3) No ability to add site specific lexicals.
>>
>> Write your own library routines, and call them from a reasonable 
>> language, such as Basic.
> 
> Why not from the command language?

Well, if you let DCL call user written routines, that would solve 
multiple issues, right?

>>
>>> 4) The output from the lexicals should be an immutable collection of
>>> objects which you can then iterate over without having to worry about
>>> the state changing during iteration.
>>
>> "DCL is not a programming language".
>> "DCL is not a programming language".
> 
> That's variously referred to as an iterator, and that would be handy.  
> More than a few DCL procedures around are iterating over a pattern match 
> of records within a file, or within a list of files, after all.  (This 
> is also where having a JIT would be handy too, as these sorts of loops 
> can be automatically detected and internally compiled within DCL, and 
> the performance of the DCL procedure would hopefully then improve, and 
> this with minimal or no effort on the part of the DCL programmer.  The 
> JIT hopefully improves some parts of the DCL procedure performance "for 
> free", as far as the DCL programmer can see.  Now figuring out how and 
> then retrofitting a JIT into DCL itself, that won't be easy.  That the 
> DCL commands can change at run-time based on a symbol substitution 
> somewhere else can make something like a JIT slightly more interesting.)
> 
>>> 5) No regex support (this is also a UI issue).
>>
>> "DCL is not a programming language".
>> "DCL is not a programming language".
> 
> DCL does shedloads of pattern-matching — badly — so having regular 
> expressions would be useful.   The wildcards % and * only get you so 
> far, and pretty soon you're rolling your own substring matching — this 
> once you're past the stage where you're looking for an FTP tool with 
> in-built scripting or pattern-matching logic, and have decided to write 
> some DCL to provide for the application requirements.  (If you're not 
> re-designing your way out of using FTP, of course.)
> 
>>> 6) Pathetic limits on the maximum size of symbol contents.
>>
>> "DCL is not a programming language".
>> "DCL is not a programming language".
> 
> Yeah, and the lack of binary data support here, too.  This also ties 
> into having objects, where I can deal with a blob of data without having 
> to specifically manage the blob.
> 
> Adding UTF-8 can get ugly if you don't go "all in" on UTF-8 throughout, 
> as you're then having to identify the data encoding in the strings and 
> file records explicitly, or trying to detect it manually.  UTF-8 also 
> means that character string lengths aren't the same as the number of 
> bytes in the string, too.

Your argument on UTF-8 was justified long ago.  I'm willing to bet my 
"betting dollar", my new one, Bob has dibs on my old one, that VSI will 
agree.

>>> 7) No array or collection of objects support. (In addition to normal
>>> arrays, DCL should also support associative arrays.)
>>
>> "DCL is not a programming language".
>> "DCL is not a programming language".
> 
> Yeah; arrays in DCL using symbol substitution is ugly.  Dictionaries — 
> arrays that work something like indexed files, also called key-value 
> stores, and probably what you're referring to with associative arrays — 
> would be handy, too.   More than a little DCL code around ends up either 
> hacking in arrays, or using an RMS indexed (dictionary) or relative 
> (array) file, or ending up having to create and then connect with some 
> compiled code.
> 
>>> DCL has absolutely no way to group related variables together in the
>>> way you can with structs in C or objects in other languages.
>>
>> "DCL is not a programming language".
>> "DCL is not a programming language".
> 
> Scopes are sort-of kind-of related to symbol visibility, but they're 
> rather murky.  Having something akin to a structure or an object would 
> be useful.
> 
>>
>>> 8) You cannot delete a directory tree in one go.
>>
>> But, you can have a command procedure to do so ...
> 
> As others have commented, with V8.4, you can add a qualifier to get there.
> 
>>
>>> 9) differences is very limited by today's standards. The functionality
>>> in GNU diff, with (for example) it's ability to find differences in
>>> whole directory trees and produce patch files for the differences
>>> in an entire tree, should be the minimum baseline for functionality
>>> these days.
>>
>> We know I don't get out much, but when I have something to compare, I 
>> move it to VMS to use DIFF there.  Guess I'm missing something.
> 
> There are other interesting and useful and different tools around; part 
> of what learning another system can help with, even if you're (staying) 
> on VMS.
> 
>> But yeah, I jump through hoops when I want to compare a bunch of 
>> source programs.  I share that pain.
> 
> A good IDE can help here, too.
> 
> For folks on OS X and going beyond the various BSD and GNU tools that 
> are available, the differences support in Xcode can be handy, as can the 
> third-party Kaleidoscope.app differences application.
> 
>>
>>> I also find the unified diff output to be a _lot_ more readable than
>>> the output from the DCL differences command.
>>>
>>> Simon.
>>>
>>
>> Ok, sorry, got carried away with the "paste" ...
>>
>> :-)
>>
>> DEC used to do this thing, they used to let people vote on desired 
>> enhancements, and then choose projects based upon demand.  This might 
>> be a good practice to continue.  Doing so on the internet, rather than 
>> a couple times a year at DECUS.
> 
> I'm actually somewhat skeptical on that, particularly if you've got 
> something new or very different in mind.  Voting does work, but tends to 
> get incremental improvements.   Having done more than a few customer 
> surveys and related, it's unusual to get significant new features 
> through this path.  Now for incremental changes such as getting zip and 
> unzip into the base distro, that got voted on and unfortunately never 
> happened.

When customers have inadequate "vision", there is the software engineers 
to correct the vision, right?

As for ZIP, you're thinking about the HP years.  Why are you surprised?

>> Through all of what you wrote, there is this "feeling" that you are 
>> writing from the perspective of a workstation, not a "server".  Things 
>> such the command line recall and length just don't come up when you're 
>> running programs.  As a software person, I can appreciate what you're 
>> asking for.  But from a production perspective, I don't see that it's 
>> relavent.
> 
> Much of what Simon is suggesting is directly or indirectly associated 
> with application development on VMS, and with using tools such as 
> scripting language improvements that allows more complex and more 
> maintainable applications to be put together more quickly and easily.  
> That's dead-on deep into server application territory, too.

There are some who recognized the need long ago, and developed their own 
tools.  I realize this method isn't much help for the user community as 
a whole.  I'm so deep into my own methods that I'd probably not be as 
good with any new tools, and, maybe that's why I have some problems 
seeing the need for any new tools.

>> Not saying you should not have all that you listed.  But, I'd like you 
>> to discuss them not from a development and / or management station, 
>> but from a "running production" perspective.
> 
> History is about the only one that isn't directly production-execution 
> related, and keeping track of command history is something increasingly 
> common in production environments.

As our users never see a "$" prompt, I don't see how such would help them.

> All its warts aside, DCL is a programming language, and folks can and do 
> write real programs in it.  But DCL can be improved.  Or DCL can be 
> replaced with something more tightly specified.    Why?  For a number of 
> tasks, applications written in scripting languages are generally faster 
> to develop and to deploy than are applications written in compiled 
> languages.   But then DCL doesn't interface with compiled languages all 
> that well 


> — how many of us are tossing around lib$set_logical and 
> lib$set_symbol and related code

Guilty as charged ...

:-)

But, once written, it does work ...

> and not something like "$ program_result 
> = compiled_program_invocation(arg1, arg2)" syntax  — so going full-on 
> compiled for an application is still pretty common.  That's before 
> discussing the various issues with the compilers and of the development 
> environments, too.
> 
> Or turning around that whole "DCL is not a programming language" that 
> you're used here, ask yourself why that is, what the issues and 
> limitations of DCL are here, and whether those can be addressed in DCL 
> or in a hypothetical replacement.  Look less at what can't be done, and 
> more at what can and should be.

Or, possibly, look at what is now being done ...

> Now as for what VSI plans here, we'll find out.  They've more than a 
> little work ahead of them, and they're going to want to reach 
> sustainable revenues.
> 
>