Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Karl Fogel]

Richard Stallman <address@hidden> writes:
>     >     Many projects require you to run something like autogen.sh
>     >     before you can build from CVS.
>     > 
>     > Is that a semi-standard convention?  If so, maybe Emacs should follow
>     > the convention, maybe we should add that script.
> 
>     It seems to be at least semi-standard now, yes.
> 
> Can you write such a script for Emacs?
>
>     Okay, I'll look into doing so.  I'm not familiar with the exact
>    mechanism of 'make bootstrap', so will need about a week.

Okay, the patch is at the end of this mail.  I've also read over this
whole thread, and realized that people are probably talking from
different assumptions.  Please bear with me while I try to sort some
things out:

It is normal these days for a CVS working copy of a project to require
one extra step in its build procedure:

   $ ./autogen.sh        <=== this is the extra step
   $ ./configure
   $ ./make
   $ ./make install

In other words, the purpose of autogen.sh is to get your working copy
to the "same" state as a distribution tarball, at least as far as the
build procedure is concerned.

In most projects, the autogen.sh script exists because the project
does not keep generated files (such as configure, Makefile, etc) in
the repository.  Instead, the repository holds only `configure.in',
`Makefile.in', and so on.  When you check out the project, you do not
get the derived files.  That's why you have to run autogen.sh -- to
get `./configure', so you can run that to get `Makefile', so you can
run 'make'.  Thus, at its most basic, autogen.sh looks like this:

   #!/bin/sh
   autoconf

Of course, in real life, most autogen.sh scripts are considerably more
complicated.  (For example, they often check the version of autoconf,
and exit telling you to upgrade if the version is too low.)
   
Anyway, the Emacs repository does not follow the convention of not
storing generated files.  It stores `configure', and what's more, the
bootstrap process is 'make bootstrap', which depends on `Makefile'
having been generated.  (And for a final twist, 'make bootstrap'
actually builds the whole system, obviating the need to run 'make'
afterwards.)

Because of all this, the traditional autogen.sh strategy doesn't
really work for Emacs, at least not without a larger change to the way
Emacs is versioned.  So what I have done is implemented autogen.sh in
roughly the way that RMS suggested:

> I don't think you need to change the mechanism of `make bootstrap' much.
> What the script has to do is cause `make' to run `make bootstrap'
> unconditionally, the next time only.
> 
> Right now there is a target `maybe_bootstrap' that checks whether
> a bootstrap is needed; but it doesn't actually do a bootstrap,
> rather it exits with instructions to do `make bootstrap'.

However, the result is a very unusual autogen.sh.  It will probably
confuse people who thought (based on other projects) that they
understood what autogen.sh scripts were all about.

For a long term goal, I think it would be best if we separated out two
distinct sets of requirements:

   1. The ways in which bootstrapping Emacs from a CVS working copy is
      the same as bootstrapping most other projects (e.g., the need
      to get a `configure' from `configure.in')

   2. The ways in which bootstrapping Emacs from a CVS working copy is
      special (e.g., the need to compile .elc files, and to reload and
      redump Emacs after a new build)

Everything in category (1) should be handled by autogen.sh, and we
would stop versioning the `configure' file.  Category (2) would be
handled by a separate script, or by good advice printed by autogen.sh
at the end of its run.

I'm not volunteering for this; I just want to see whether people think
it's a good direction to go.

As Eli Zaretskii said recently:

   > Please remember that anon CVS access to Emacs development sources
   > is relatively new.

Mmm, yes :-).

Here's the patch, not applying until get some feedback:

2002-11-19  Karl Fogel  <address@hidden>

        * autogen.sh: New file.

        * INSTALL-CVS: Adjust for presence of autogen.sh.

Index: autogen.sh
===================================================================
RCS file: autogen.sh
diff -N autogen.sh
--- /dev/null   1 Jan 1970 00:00:00 -0000
+++ autogen.sh  20 Nov 2002 05:41:37 -0000
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+if make maybe_bootstrap > /dev/null 2>&1; then
+  ./configure
+  echo ""
+  echo "Your working copy is now ready for you to run 'make'."
+  echo "If you want to be extra thorough, then run:"
+  echo ""
+  echo "  make"
+  echo "  cd lisp"
+  echo "  make recompile EMACS=../src/emacs"
+  echo "  cd .."
+  echo "  make"
+  echo "  make install     ### optional ###"
+  echo ""
+else
+  echo "$0: Bootstrapping..."
+  ./configure
+  make bootstrap
+  echo "$0: Done bootstrapping."
+fi
Index: INSTALL-CVS
===================================================================
RCS file: /cvsroot/emacs/emacs/INSTALL-CVS,v
retrieving revision 1.1
diff -u -r1.1 INSTALL-CVS
--- INSTALL-CVS 17 Jun 2002 06:07:24 -0000      1.1
+++ INSTALL-CVS 20 Nov 2002 05:41:37 -0000
@@ -2,42 +2,31 @@
 
 Some of the files that are included in the Emacs tarball, such as
 byte-compiled Lisp files, are not stored in the CVS repository.
-Therefore, to build from CVS you must run "make bootstrap"
-instead of just "make":
+Therefore, to build from a CVS working copy, you should first run
 
-  $ ./configure
-  $ make bootstrap
+   ./autogen.sh
 
-The bootstrap process makes sure all necessary files are rebuilt
-before it builds the final Emacs binary.
+then run the appropriate 'make' commands afterwards, if necessary (the
+output from autogen.sh will give you some guidance).
 
-Normally, it is not necessary to use "make bootstrap" after every CVS
-update.  Unless there are problems, we suggest the following
-procedure:
-
-  $ ./configure
-  $ make
-  $ cd lisp
-  $ make recompile EMACS=../src/emacs
-  $ cd ..
-  $ make
-
-(If you want to install the Emacs binary, type "make install" instead
-of "make" in the last command.)
-
-If the above procedure fails, try "make bootstrap".
-
-Users of non-Posix systems (MS-Windows etc.) should run the
-platform-specific configuration scripts (nt/configure.bat, config.bat,
-etc.) before "make bootstrap" or "make"; the rest of the procedure is
+Users of non-Posix systems (MS-Windows etc.) should follow the
+procedure in autogen.sh, but run the appropriate platform-specific
+configuration scripts (nt/configure.bat, config.bat, etc.) before the
+"make bootstrap" or "make" step; the rest of the procedure is
 applicable to those systems as well.
 
-Note that "make bootstrap" overwrites some files that are under CVS
-control, such as lisp/loaddefs.el.  This could produce CVS conflicts
-next time that you resync with the CVS.  If you see such conflicts,
-overwrite your local copy of the file with the clean version from the
-CVS repository.  For example:
+What to do if you get unexpected CVS conflicts later:
+=====================================================
+
+Bootstrapping overwrites some files that are under CVS control, such
+as lisp/loaddefs.el.  This could produce CVS conflicts next time that
+you do a 'cvs update'.  If you see such conflicts, overwrite your
+local copy of the file with the clean version from the CVS repository.
+For example:
 
     cvs update -C lisp/loaddefs.el
+
+Bugs:
+=====
 
 Please report any bugs in the CVS versions to address@hidden