[Info-vax] proper file format, attributes for non-binary files served by a web server

[Paul Sture]

On Sat, 21 Jan 2012 15:37:09 +0000, Phillip Helbig---undress to reply
wrote:

> I have several files (.txt, .html, .htmlx) from various sources which
> are served by the OSU server.  At least as far as I can tell, all seem
> to work.  Does that mean that they should work with any
> standard-conforming browser?  What about with a different web server on
> VMS?  Is there such a thing as a preferred/recommended format for .txt
> files or html files on VMS?

I don't have experience of OSU on VMS, but have used CSWS and WASD.

With one release, CSWS moved to wanting line feed terminated files, and 
provided a DCL procedure to convert the files in your whole document tree.

I found robots.txt to be a special case.  I forget whether I was using 
CSWS or WASD at the time, but I had it as a <CR><LF> terminated file, and 
the robots validators I found out there choked on it.  Converting it to 
lf-stream format cured that.  It also explained why Google was going 
places I'd told it not to.  

BTW I also found that with any syntax error in robots.txt Google would 
ignore the file completely and rip through parts of my site I'd told it 
not to.

-- 
Paul Sture