[Info-vax] To MIME or not to MIME

[Dave Froble]

On 8/1/2018 11:34 AM, Stephen Hoffman wrote:
> On 2018-07-31 09:13:55 +0000, Jan-Erik Sderholm said:
>
>> Den 2018-07-31 kl. 00:37, skrev Stephen Hoffman:
>>> On 2018-07-30 22:19:03 +0000, Jan-Erik Sderholm said:
>>>
>>>> I'm not usure that the miced case filename was the reason that the
>>>> sending end switch to UTF-8 encoding of the filename.
>>>>
>>>> Probaby something else in the mail, could be in the plain text part.
>>>>
>>>> I think that, if there is something in the body of the mail that
>>>> forces it into UTF-8 encoding, that also applies to the subject,
>>>> even if it is not strictly needed for the subject...
>>>
>>>
>>> Doesn't really matter what triggered it.
>>
>> It sure does. *If* you are interested in understading the cause and to
>> find a cure for this in the future. If not, then yes, just don't bother.
>
> You're approaching this as a developer with control over their
> end-users.  That case is becoming less common. Enjoy it while you have
> that environment too, if that case applies to your normal development
> environment.

I always thought that developers served the end users ???

> Though with great power...  Some developers have used that same control
> over end-users to inflict some very bad user interfaces and very bad
> designs on their end-user folks.

Well, yes, this has happened.  A major reason for the proliferation of 
PCs in various environments.

>  Reporting to folks other than those
> using your tools has led to a number of abominations in designs, but
> when you're paying the end users they're usually more willing to cope
> with the problems.  Or work around the problems, sometimes with
> unexpected consequences.  That also adds costs onto the trainers and the
> support folks, though some end-users have turned these messes to their
> own advantage and have used flaws to work around limitations in the user
> interfaces.  Compatibility too inevitably and inevitably leads to bad
> and complex user interface designs.  But I digress.
>
> With details such as mail messages and TLS connections and expected
> patches and other sorts of network inter-operations and distributed
> security details, OpenVMS either plays well with others or it gets
> demerits toward potential or eventual replacement.  If you're not
> working in these sorts of environments, then you can avoid some of these
> sorts of cases.

Well, yes, don't use VMS for mail.  Plenty of other platforms in use by 
those who deal with mail.  VMS doesn't do so.  But in many cases this 
doesn't distract from the usefullness of VMS.

>  And can sometimes shift the onus of working around the
> OpenVMS limitations.   But even internal networks in enterprises are
> moving forward.
>
> Sure, knowing the specific trigger might provide an onus-on-the-end-user
> workaround, too.

Not something I can agree with.  A developer's job is to make things 
better for an end user.

>>> It's valid and correct syntax.
>>
>> And that helped Brian, how?

What surprises me is that there has not been one comment on Brian's 
practice to run things from the SYSTEM (I'm guessing) user account with 
a default directory of SYS$MANAGER:.  I've always read that such is a 
good recipe for disaster.  Guess I've been reading the wrong stuff.

> These messes are things we're all going to have to deal with, at least
> until the tools are updated or local replacements are installed, or — in
> those few cases where it's possible — worked around through added
> end-user requirements.
>
> With MIME-encoded messages, messages arriving with valid syntax are
> something that we're going to have to deal with, as mail clients and
> many other aspects of distributed computing are out of our individual or
> organizational control.

Don't do so on VMS.  It doesn't do UTF and such well, if at all.

> We can try workarounds and similar approach and that'll work in isolated
> cases.  In the general case, not so much.  Users make mistakes, remote
> servers not under our control get upgraded and switch to always using
> the unsupported-but-valid syntax, etc.
>
>>> "=?UTF-8?Q?Jan-Erik_S=c3=b6derholm?="
>>
>> "c3 b6" is the UTF8 code for the letter "ö" in my name.
>> Correctly encoded and probably handled OK by some tools.
>
> Ponder whether you're telling me something I don't already know about,
> given I've specifically pointed it out, and in the same context as a
> similar error in OpenVMS.   The specific case I'm dealing with involves
> issues with quoted printable, which means I'm either going to be writing
> a new client, convincing the vendor to open-source the client, migrating
> to a different client, or dealing with the error.  Which parallels what
> Brian is dealing with.  Hence my comment.
>
>>
>>> Be that as it may, the OpenVMS character support work is not a small
>>> project, and definitely not at the top of the VSI schedule by any
>>> stretch.

As you like to comment, Ayep!  So don't try to put the square pegs in 
the round holes.

>> That have never stopped my from finding solutions and work-arounds
>> when these issues pops up.
>
> Yes, the first-principles approach.

Which basically is the job of many of us.

> It's quite possible to work around this with any of various approaches.
>
> It's also how we get enterprise software installations and a whole pile
> of unmaintainable and baroque code.  This because the actual problems
> aren't or can't be fixed, so they're worked around.  That's an expected
> strategy used at existing OpenVMS sites.  That doesn't work so well for
> attracting new OpenVMS installations and new OpenVMS customers. Because
> MIME-encoded messages and web services are expected to be a part of the
> foundation of the platform, and not something where an intrepid
> developer needs to deal with it.  But I digress.

Are they?  Perhaps in some cases.  But not in others.

> All of which adds expense of using of the platform, and increased costs
> tend to discourage new installations and new users, and takes the
> developer away from working on application-specific processing.
>
> Because first-principles is not free.

What is?

> We all could be doing something better than having to deal with
> MIME-encoded messages and particularly messages that are using correct
> syntax that cause the in-built tools to tip over with
> "enterprise-worthy" error messages.
>

Interesting way to describe things.  If I've learned anything from this, 
it's that the utility is faulty, and the error messages are terrible. 
Maybe not as terrible as OpenSSL error messages, but I doubt anything 
can approach that.  A misleading error message can be worse than no message.

Chris knew enough to make a good suggestion.  That's not good enough. 
The utility is garbage.

>
> ps: Some of the more obvious "enterprise-friendly" ways to fix this mess:
>
>   -pre-process and reformat the MIME input prior to using the MIME tool,
> though that can clean up the export header cruft too...

Should not ave to do this.

>   -patch the image directly and correct the problem...

Valid solution.

>   -shim in a shareable image that intercepts either the system I/O calls
> or the language-specific I/O calls and that detects and corrects the
> error on subsequent reads...
>   -intercept on the network via MITM, as the OpenVMS connections are
> insecure...
>   -use some custom processing rules on a Postfix or other intermediate
> mail server...
>   -use alternate MIME-processing tools either locally written, or that
> have been sourced elsewhere...

And hopefully a better product.

>   -require users to send specific messages from specific servers running
> specific versions and push the mess onto the end-users and the support
> team...
>   -write a stub pseudo storage driver using ALTSTART that detects and
> corrects the error on the pseudo disk...
>   -write a correcting device driver, an old OpenVMS strategy and one
> which used to be fairly common for some device-level tasks...
>   -re-roll storage device firmware or controller firmware to correct the
> error...

I wouldn't do many of this things.

>   -acquire a commercial replacement mail client and/or replacement IP
> stack, such as Multinet and PMDF...

Nothing wrong with this suggestion.

>   -migrate to another platform

If all you're doing is mail, maybe a good solution.  But if it means 
re-writing a $50 million application, maybe a very poor solution.

> There are undoubtedly others.  There are folks around that can intercept
> the I/O activity in flight within the OpenVMS kernel and patch the I/O
> there, too.  And no, a platform migration is not something that would be
> expected here.  But there's always a trigger or a series of triggers
> that cause an app or a platform migration.

Face it, much of the platform migrations came out of the "migrate" word 
from DEC, Compaq, and HP.  I doubt anything else comes close.

Just as the lack of valid capabilities in some areas came out of the 
lack of development by HP, and possibly Compaq and DEC.