Re: Line Separator and Paragraph Separator

From: John Cowan (
Date: Tue Oct 21 2003 - 10:38:21 CST

Jill Ramonsky scripsit:

> Well, I can't say I've ever found a use for putting either a C0 or a C1
> control into a text file, beyond the usual CR, LF and TAB. My code also
> often considers FF to be whitespace, although I've never actually
> (knowingly) encountered it in a real text file.

All RFCs contain FFs at the end of each page.

> I would have thought that low codepoints would be highly valuable
> commodities. Though some may have exotic uses, my experience is that
> most of them don't seem to be used.

That's why Microsoft felt free to reassign 80-9F to graphic characters in
its various codepages, which means they cannot reliably be sent across
serial transmission lines, which is what most control characters were
intended for.

> I have tended to treat the control characters rather like the
> Private Use Area - a space in which I can do what I want so long as
> don't expect the "outside world" to agree.

Indeed, that's safe enough. But Unicode is all about interchange, so if
it reassigned any ISO controls, it would step on other uses of them.

> I've even invented (and used)
> some 8-bit encodings which leave the whole of Latin-1 unchanged (apart
> from the C1s) and use C1 characters a bit like "surrogate pairs" to
> reach the rest.

There is actually a standards-compliant way to achieve "code extension"
of that type, for up to 4 spaces of 94+96 characters each (you are not
allowed to redefine space or DEL):

o use 0E (shift out) to switch to the 2nd space
o use 0F (shift in) to return to the main space
o use 8E (single shift 2) to mark the next byte as being in the 3rd space
o use 8F (single shift 3) to mark the next byte as being in the 4th space

If you need more than 4 spaces, you then enter the Great Pain, also known
as ISO 2022.

Evolutionary psychology is the theory           John Cowan
that men are nothing but horn-dogs,   
and that women only want them for their money.
        --Susan McCarthy (adapted)    

This archive was generated by hypermail 2.1.5 : Thu Jan 18 2007 - 15:54:24 CST