Re: checking in links to source control

[Greg A. Woods]

[[ you CC'ed a second copy of this message to me, again.  please do NOT
do that! ]]

[ On Friday, September 14, 2001 at 18:36:53 (-0700), Edward Peschko wrote: ]
> Subject: Re: checking in links to source control
>
> symlinks are something that can't be stored directly in file format.

Look a little closer.  Symlinks are, in unix at least, simply pathnames
stored in a file.  The only thing special about the file is one bit in
its inode header.  Symlinks are created and manipulated using system
calls.  You may make those calls from the programming language of your
choice -- just take care to avoid using normal file I/O calls as those
tend to refer to what the link points to, not the link itself.

> Neither
> are directories.

Dirctories too are simply lists of inode numbers and filenames.

The unix filesystem is flatter as the praries around Regina, Sask.  The
directory hierarchy layered over top of a unix filesystem is simply a
naming convention for the convenience of the users, though of course the
kernel too plays an important role in the traversal of these layers.

> Nor are permissions.

Permissions and ownership are just more bits inside the file's inode
header.  They are set using system calls.  You may make those calls from
the programming language of your choice.

> You serialize them into something 
> different (a file), and then unserialize them to get back what you want.

No, you don't "serialize" them, and you probably don't "marshal" them
either.  You might canonicalise them though.....

> the methodology is quite similar to me...

The easiest way to canonicalise permissions, ownerships, symlinks, and
the like is to write a program of some sort which makes them be what you
want them to be.  If you wish to get fancy then write the program in
such a way so that it is data driven from a nice little flat text-file
in some usefule format of your definition and in which you can specify
the desired state of the attributes for your files and links.

.... or just use 'mtree' -- it already does most of what you want

> Like I said, been here, done that. I could make a shell script in every single
> directory whose job is to seek out the perl lib directory and make a link
> to it if it ain't there. I'd rather do nothing.

Ah, but if you think a wee bit and plan a wee bit you'll hopefully be
able to come up with a scheme which won't require too much effort, and
especially not repeated effort, and which at the same time lends the
scheme itself to being "managed" by CVS (i.e. having changes within it
be tracked by CVS).


Of course you might even be using the wrong tool for the job at the most
immediate level in your environment -- i.e. perhaps symlinks are not
what you should be using.  (Maybe you might try using an environment
variable to specify the location of your perl library?)


> I could make a shell script
> that, when reverting back to an old version of a directory, does a cvs diff
> between the old version's directory and the new, and removes all files that 
> don't exist in the new version. yee-haw. I'd rather do nothing.

What the heck are you talking about!?!?!?  That seems like pure nonsense.


> And so on, and so on, and so on. Its called 'up front effort': the reason
> you bundle things into a tool like this is for ease of use. Its a lot of 
> effort at first, yeah, but after that initial effort you do nothing at all.

Yes, exactly what I'm trying to get at.  If you choose the right tool
for the job up front then you'll expend a bit of effort up front and
from there on won't require any effort at all.


> of course I'm doing serialization. That's the definition of serialization; 
> packing up something into a state where it can be transmitted (and stored) and
> then doing the reverse packing somewhere else. We really are talking apples 
> and apples here - you are doing serialization through Makefiles, (or what 
> have 
> you) I'm just standardizing it through CVS.

It seems you are in dire need of a good dictionary, apples and apples or not!

While you're marshaling your objects around maybe you could take a
moment to think about what a "canonical representation" is.

> And you wouldn't have to write a single line of code if we did this. Now,
> tell me which one is easier.

Well since what I can make out of your proposal so far would have been
entirely unusable in my situation it makes no difference and would have
saved no coding on my part (and may even have required addditional
coding to work around!).

CVS is not a build system.  I don't want it "building" anything
whatsoever, except of course the content of my source files.


> Again, it shows me a deficiency in the tool. Why not CVS, again?

I think you are wayyyyy off base.  I think you have a drastic
misunderstanding of how to choose and use tools appropriately.

CVS is a tool for tracking and managing changes made in text files.
That's all it is.  It is not a build system.  It is not a replacement
for a filesystem.  It is not a scripting language.

If you need a simple build system I would suggest make, jam, or cook.

If you need a filesystem I would suggest BSD FFS for general purpose
use, LFS for some special purposes, maybe XFS for others, etc.

If you need a scripting language I'd suggest POSIX Shell/Awk/Sed since
they are most portable (though if portability were of no concern then I
might suggest Ruby or guile or squeak or maybe even rc or ici or pike).

Note that I didn't suggest CVS for any of the above!

Even the one file attribute that CVS does currently manage by accident
of its RCS heritage is not appropriate for management by the version
control tool.

Note too that the issue of extending the RCS storage format in a way
specific to only CVS is of very questionable value, at least without
also offering support for the same features in bare RCS.  (Once upon a
time it wouldn't have been such an issue -- perhaps CVS should again
ditch the built-in RCS implementation in favour of an external one that
can be replaced, fixed, or upgraded relatively independently of CVS.)


> I could see
> *CVS* and mtree as being a good solution..

There you go.  Problem solved.  That's what I use whenever shell scripts
are too tedious -- it works wonderfully and is very easy to manage.


> I was thinking more along the 'mod_XXX.so' level. Although I'll take a look 
> at the info files though..

Now I know you're waayyyy off base.  Some people seem to see a hammer as
their only solution, and all of their problems as nails.  Are you really
one of those people?  Extensibility of something like CVS via dynamic
loading is so ugly, stupid, evil, inelegant and unnecessary that I can't
even begin to come up with a civilised response against it.

> well yeah, if I read you right, that's caveat emptor. Personally, I'd be happy
> just to see if the file changed from being a link to being a file and then 
> back,
> but if you want to get more fancy, you could say:

I don't "want" to get more fancy -- that's a primary requirement that
must be met by any proposed solution.


> Who says? You say its questionable, you've got a preference for makefiles. 
> I don't. Personally I think that the reduction of a process that took 3800 
> lines
> of script to one that takes exactly 0 would be enough for you to show that it
> would be of value. 

Perhaps you do not understand the problem I solved with those 3800
lines.  They were the entirety of my home-grown system administration
tool....


> And personally I think the added benefit of showing revisioning on these 
> files,
> and having it integrated into one system would be more than enough to show
> that it would be of value.

I have all of that already, no matter what the language I choose to
write my symlink creator in.  Why do I need to make my repository
incompatible with the de facto standard RCS tools just to accomplish the
very same thing over again?

> And of course, not having to learn *yet another 
> scripting language* like cfengine (plus the maintenance && training time that
> implies), is enough to show that it has value.

That's why I've been careful to suggest using an appropriate programming
language of your choice.  Since you are writing in perl, perhaps perl is
the appropriate language in this case.....

> Anyhow, its not *me* who is seriously confused, or who even made the original
> point. Its a user of *cvs* who did. Listen to your customers.

First off they're not "my customers"

Secondly CVS is not a market driven product -- it's a free tool.

> Again, this forces down a person's throat your preconception of what a build
> process is and what a project is. That is *not your job*. That is the job
> of the project's *owner*. 

There is no project "owner" (though I dare say if he were still in
control there'd have been little dispute along these lines -- your
proposal would have been shot down long before you even got a chance to
make it! ;-).  What most people call "CVS" is, now, a community effort.

I'm just telling you how things are and how they got to be that way.

You may ignore history and design goals at your own peril.

> ps - why are you avoiding the whole issue of the patches?

I am not the person who can answer this.  However I can perhaps suggest
that if you look around in the obvious places you'll likely find the
answer staring right back at you.

> pps - I asked for feedback on why this wouldn't work, you have given me some
> (although a lot of it is still philosophical). IMO, I have demonstrated that 
> all of the points that you mention are surmountable from a technical point
> of view. So, keep giving it your best shot.. ;-)

You have continued to ignore my technical points and even the generic
requirements I have suggested you must likely meet.

Perhaps you should start by writing a functional specification and
posting it here.  Don't concern yourself yet with petty details about
the implementation, but do keep in mind that the limitations of the
environment you are entering are requirements that your specification
must eventually meet.

> ppps - btw, I checked out cfengine, and for most projects out there, cfengine
> is *way* overkill.

I didn't say it was for "most projects".  I said it would be _my_ choice
if I were to repeat the particular project I was using as an example.

The de facto (on *BSD at least) standard "mtree" program is what I use
today for "most projects" of my own, at least whenever a set of simple
commands (eg. 'install' commands) in a makefile or shell script is not
sufficient or necessary for portability reasons.

-- 
                                                        Greg A. Woods

+1 416 218-0098      VE3TCP      <address@hidden>     <address@hidden>
Planix, Inc. <address@hidden>;   Secrets of the Weird <address@hidden>