Re: CVS and unicode

[Christian Hujer]

Hi,

Am Montag, 12. September 2005 07:00 schrieb Larry Jones:
> Christian Hujer writes:
> > Example: Data files from the Daimonin project (an MMORPG). They are
> > ISO-8859-1 text files. They must explicitely only use LF for line ending.
> > They won't work with CR/LF. The file format is specified to be plain text
> > with LF line endings only. The server and editor will not accept files
> > with CR/LF.
>
> Then they're not text files.  ISO 8859-1 files won't look anything like
> text on an IBM mainframe that uses EBCDIC instead of ASCII.
Okay, but I think talking of EBCDIC or other non-ASCII-based charsets / 
encodings is really pointless.

> Remember 
> that the main motivation for CVS was source code control.  Compilers
> nearly always insist that source code be in the system's native text
> file format (or a reasonable facsimile thereof), so CVS follows that
> convention. 
Well, I don't know that many compilers, I know javac, jikes, gcc and those 
from Microsoft. All of these accept LF just as well as CR/LF, regardless of 
the platform.

Also, we're trying to solve a problem with CVS that is not solved where files 
are exchanged in different ways like file shares, ftp, removable storage 
devices (floppy, usb, cd, dvd), archives (like zip, tar, rar etc. - sometimes 
the CR/LF-LF problem is handled there, but not often).

> > Most systems use LF, yes. The only CR system still in use that I know of
> > is Mac OS, and within Mac OS X this all is currently migrating from CR to
> > LF to fulfil POSIX compatibility.
> > The only CR/LF systems still in use that I know of are Windows and TOS.
> > All other systems that I know use LF.
> There are more things in heaven and earth than are dreamed of in your
> philosophy.
I hope you're not trying to count CP/M, Multics etc.. I think there are even 
more Inferno, MorphOS or Plan9 users...

> > But regardless, if a cvs client changes a file upon checkin or checkout
> > other than keyword replacement or diff patching, it eventually breaks the
> > file, see above example.
> For one specific misuse.  Not changing the file would break the intended
> use far more often.
I've never seen a project where text files with LF only would be a problem 
when still being LF only on Windows. But I've of course only seen a small 
number of possible projects...

> > Really? The standard command line client converts LF to CR/LF?  As you
> > see I've never ever used the standard cvs command line client on Windows.
> > I always get Cygwin. I never use Windows without a Cygwin (which then is
> > installed to use LF, of course).
>
> Yes, it does.  And Cygwin includes the standard CVS command line client.
> If you install Cygwin with Unix line endings, you're no longer running a
> Windows system, you're running a Unix-like system where the native text
> file format matches the repostory text file format so no conversion
> occurs.
That's a point.

Now who is responsible for adding the CR if I access the file with Notepad? 
(Imo I, the user is, because the OS is too stupid, and I really know more 
about how my file contents should look like than the OS)

> > From a data file point of view, the text file might in fact be a data
> > file which must not be modified regarding its line endings or encoding,
> > despite the fact that it has been checked in as text file, regardless of
> > the client operating system, because some software might rely upon the
> > line endings and encodings.
> > In that context, converting LF to CR/LF is a Bad Thing and must not be
> > done.
>
> In that context, it's not a text file and must be marked as binary, no
> matter how much it may look like text to you.
I disagree, because that breaks the possibility of diffs with version control.

Christian