bug#11377: configure.am - fails to remove configure before attempting replacement

[Ronald F. Guilmette]

My apologies for the delay in replying.  I have been tied up due to other
responsibilities.

In message <address@hidden>, you wrote:

>tags 11377 + moreinfo
>severity 11377 minor
>thanks
>
>Hi Ronald, sorry for the delay.
>
>On 04/28/2012 10:55 PM, Ronald F. Guilmette wrote:
>> Under certain very reasonable scenarios, the configure file contained within
>> some source tree may perhaps be marked as read-only.
>>
>Before continuing, may I ask you which these scenarios are exactly?

Well, since you asked...

I have been working on GNU software since 1989... going on 23 years now.  I
was one of the earlier contributors to the GNU software development tools,
in particular the GNU C compiler.

Long long ago I established a personal methodology of working on GNU and
other open source software.  Some people may take issue with my approach,
but it works for me, and I am comfortable with it, so I use it, always.

If I am working towards the goal of making changes in some package, e.g.
fixing bugs or adding enhancements, then I always proceed as follows:

     1)  Unzip & untar the package sources into a "src/" directory.  (For
         example, I am currently working on adding some enhancemente to
         the "gthumb" application, so I have unpacked the sources for
         that into a directory called "src/gthumb-2.14.3".)

     2)  In order to ensure that I have a pristine and original copy of the
         current sources available (relative to which I will later make diffs
         representing my personal patches) I run the command "chmod -R ugo-w"
         on the entire sources tree that I just unpacked.  This step is
         critical to insuring that any subsequent patches file I generate
         using `diff -R' will be properly relative to the current official
         sources for the package in question.  (Making all of the regular
         files read-only prevents me from inadvertantly making changes to
         any of the original source files without doing so quite deliberately.)

     3)  I then make a clone of the src/ tree into a separate and parallel
         "work/" tree where I will make all of my local source code changes.
         On FreeBSD, at least, this can be accomplished via the "cp -R -l"
         command.  So, for example, I would do this:

        mkdir work
        cp -R -l src/gthumb-2.14.3 work/gthumb-2.14.3

        Note that in order to both minimize disk space consumption _and_ to
        dramatically speed up any and all subsequent whole tree recursive
        diffs, I do not actually make physical copies of the regular files
        when constructing my clone "work/" tree.  Rather, the -l option for
        cp causes all regular files to simply be hard-linked into both trees.

    4)  After all of the above, I go to work on my work/ tree, making changes
        as necessary to implement the bug fixes and/or enhancements as I think
        are approproate.  Of course, as I do so I make changes to
        various of the source files that I had previously marked as read-only.
        (Note that all of the regular files of the src/ AND work/ trees have
        been marked as read-only, because they are just hard-linked copies
        of one another.)

        Because the regular files files in the work/ tree are read-only,
        each time I need to make changes to one I have to un-link it from
        its src/ counterpart, making an actual new physical copy as I do,
        and then mark that copy as read/write.  I have developed a trivial
        shell script for doing this in one simple step.  I call it "mw"
        (make writable).  Here is the C-shell source for that:

        =====================================================================
        #!/bin/csh
        
        # mw - make writable
        
        set argv=($*)
        
        foreach pathname ($argv)
                echo Processing $pathname
                if (-d $pathname) then
                        echo $0\: cannot mw on a directory\: $pathname
                else
                        rm -f $pathname-mw$$
                        mv $pathname $pathname-mw$$
                        cp $pathname-mw$$ $pathname
                        touch -a -m -r $pathname-mw$$ $pathname
                        rm -f $pathname-mw$$
                        chmod +w $pathname
                endif
        end
        =====================================================================

        So, for example, if I an working in my work/ tree and decide that I
        will be neededing to make changes to the foobar.c and foobar.h files,
        then I will just simply type:

        mw foobar.c foobar.h

    5)  Lastly, when I am done making and testing my local changes, I can
        easily extract all of my local source code changes via this simple
        command:

                diff -c 2 -R src work

        Of course, I then contribute these back to the package maintaner, you
        know, for the greater glory of GNU.

One could easily argue that what I am doing with all of the above is, in
effect, to implement a sort of "poor man's revision control system".  I
personally would not argue that point, one way or the other.

There may perhaps be better ways of doing what I am trying to do with the
methodology described above.  I also would not argue that point either.
What I do know is that _everyone_ who works on software had their own
idiosyncratic methods and approaches, and I am no different in that regard.
I am comfortable with the above way of working, and it has served me well
for the past 23 years of contributing to GNU software.

The only problem with the approach outlined above is that sometimes (rarely)
some Makefiles and/or other tools used as part of a build process (e.g.
automake) may occasionally make the assumption that every regular file
within the current (build) tree is writable, and that thus, when one of these
tools is about to perform some operation that will attempt to effectively
over-write some such file, the over-write can be performed _without_ first
deleting the file in question.

It is these exact (naive and, thankfully, quite rare) assumptions that can
cause the development methodology I have described above to malfunction
because, for example, Makefiles may be written in a way that assumes that
a given file will be properly overwritten, even in cases where it won't be,
e.g. because (a) it has been maked as read-only and also (b) the Makefile
in question failed to take the simple and altogether reasonable precaution
of `rm'ing the file in question before attempting the over-write.

I want to reemphasize what I just said.  Removing a given regular file before
attempting some operation that is intended to fully replace the contents of
that file is a simple and altogether reasonable precaution... one that all
Makefiles should take in all such cases.  It provably will never do any harm
to add this `rm' step to the relevant Makefiles, and it is altogether
apparent that it can be helpful, in some cases, to perform the `rm' step.

Makefiles that have been written _without_ this level of care and caution
can and do malfunction, and when they do, they can be expected to do so in
ways that are likely to be more difficult than necessary to debug, because
the Makefile will be incorrectly tricked into thinking that a given file
has been successflly re-made, when that has *not* actually occured.

Take this simple example Makefile:

        send-command:   command
                cat command > NORAD

        command:
                -echo "Stand Down\n" > command

Assuming that the file named "command" exists before this Makefile is invoked,
and that it somehow came to be marked (accidentally or otherwise) "read only",
and that its pre-existing content is the single line "Fire all missles!\n"...
well... the outcome in this case is, shall we say, sub-optimal, and is not
what one would have hoped.  Note however that inserting the command:

                -rm -f command

at the top of the set of commands used to re-make the "command" file would
make all the difference, and would result in a more suitable outcome.

(Personally, I am an old-school adherent to the philosophy known as
"defensive programming" and this extends even to my Makefiles.  The basic
tenets of defensive programming are: (1) if anything can go wrong, it
_will_ god wrong[1] and (2) if you are given a choice between using a
belt, using suspenders, or using a belt _and_ suspenders, you should
always choose both belt and suspenders.  In this case, over-writting
a file that needs to be aover-written is the belt.  Rm'ing the file first
is the suspenders.)

Anyway, exactly this sort of situation does in fact arise within one of the
automake-generated Makefiles of gthumb-2.14.3.  There is a failure to remove
one configure file before attempting to replace it, and if (as in my case)
the configure file in question may have been marked as read-only, then all
hell breaks loose and things go very very wrong.  (It was this exact problem
that prompted my change request.)

The botom line is just this:  It is maximally fastidious to always make it a
point to rm all files that are about to be over-written.  There are -zero-
cases in which doing so causes any harm whatsoever, however there are at
least some cases in which it is helpful to exercize exactly this level of
caution.  Thus, on balance, it is better to always rm files before over-
writting than it is to sometimes fail to do so.

>> When and if this occurs,
>> automake will fail to remove the configure file before it attempts to genera
>te
>> it anew.  The following trivial patch corrects this problem.
>> 
>> I would really appreciate it if this small patch could be incorporated into
>> future releases of automake.  It does no harm, and it can provably do some
>> good in certain circumstances.
>> 
>If you convince me that the circumstances you describe are actually reasonable
>,
>we should also enhance your patch to cover the similar cases of aclocal.m4,
>Makefile.in, etc. created read-only (and probably add a test case to ensure
>we don't regress with future changes).


That all sounds quite reasonable.


Regards,
rfg


=====
[1]  See also:  Murphy's Law