Re: CVS and unicode

[Christian Hujer]

Hi,

Am Sonntag, 11. September 2005 16:28 schrieb Larry Jones:
> Christian Hujer writes [re. line ending conversions]:
> > No, the windows client will only do this modification if you told the
> > client to do so. You can enable / disable this. The windows cvs client is
> > able to checkout non-"-kb"-files (text files) with LF without converting
> > LF to CR/LF. At least this is true for WinCVS.
>
> It is the CVS client's responsibility to convert between the local
> system's line ending conventions and the CVS client/server protocol's
> line ending conventions.
I definitely disagree here.

Example: Data files from the Daimonin project (an MMORPG). They are ISO-8859-1 
text files. They must explicitely only use LF for line ending. They won't 
work with CR/LF. The file format is specified to be plain text with LF line 
endings only. The server and editor will not accept files with CR/LF.
So even if somebody checks out a CVS sandbox on Windows, the line endings must 
be LF, otherwise the files can't be used.

(C and C++ source code of the same project use ISO-8859-1, Java source code 
uses UTF-8)

> (Note that line ending conventions vary 
> widely: some systems use CR, some LF, some CF/LF, and some don't use any
> character[s] at all but rather store an explicit line length.)
Most systems use LF, yes. The only CR system still in use that I know of is 
Mac OS, and within Mac OS X this all is currently migrating from CR to LF to 
fulfil POSIX compatibility.
The only CR/LF systems still in use that I know of are Windows and TOS.
All other systems that I know use LF.

But regardless, if a cvs client changes a file upon checkin or checkout other 
than keyword replacement or diff patching, it eventually breaks the file, see 
above example.

> For the 
> standard CVS client, that conversion is done by the C run-time library
> because the file is opened in non-binary mode.  MSDOS and Unix line
> ending conventions are similar enough that many people like to pretend
> that they're the same and many tools are flexible enough to go along
> with that delusion, although most people eventually encounter a critical
> tool that isn't and all sorts of havoc ensues.  WinCVS is one of those
> flexible tools that's willing to support the delusion, but it makes you
> explicitly ask for it.  The standard command line client is not.
Really? The standard command line client converts LF to CR/LF?  As you see 
I've never ever used the standard cvs command line client on Windows. I 
always get Cygwin. I never use Windows without a Cygwin (which then is 
installed to use LF, of course).

> > And afair cvs on Cygwin does not perform any CR/LF conversion at all.
>
> That depends on whether you've installed Cygwin correctly (using MSDOS
> line endings for text files) or incorrectly (using Unix line endings for
> text files).  Incorrect installations are quite common (and many people
> would vehemently disagree with my characterizing them as incorrect,
> despite the fact that they flaunt the system's text file conventions).
Oh yes, I definitely disagree with your characterization.
Since there is a standard called POSIX, and (to me) it's the task of Cygwin to 
give me the illusion of using a POSIX system even when being forced to use 
the operating system from "The Evil Empire", I wouldn't call it wrong. Also, 
I wouldn't rely on all programs using CR/LF when having installed Cygwin with 
CR/LF, but I think it can be treated as sure that all Cygwin tools use LF if 
installed with UNIX line endings.

From the text editor point of view, there's always an alternative to use a 
text editor that supports LF on Windows.

From a data file point of view, the text file might in fact be a data file 
which must not be modified regarding its line endings or encoding, despite 
the fact that it has been checked in as text file, regardless of the client 
operating system, because some software might rely upon the line endings and 
encodings.
In that context, converting LF to CR/LF is a Bad Thing and must not be done.


Christian