bug#19206: 25.0.50; CC Mode tracks wrong source files

[Alan Mackenzie]

Hello, Sebastian.

On Tue, Dec 02, 2014 at 12:02:26PM +0000, Alan Mackenzie wrote:
> On Tue, Dec 02, 2014 at 12:03:21PM +0100, Sebastian Wiesner wrote:

> > > Am 30.11.2014 um 19:42 schrieb Alan Mackenzie <acm@muc.de>:

> > > On Fri, Nov 28, 2014 at 10:25:42PM -0000, Alan Mackenzie wrote:
> > >> In article <mailman.14863.1417170074.1147.bug-gnu-emacs@gnu.org> you 
> > >> wrote:
> > >>> CC Mode tracks wrong source files when a CC Mode derived mode is
> > >>> installed non-interactively.

> [ .... ]

> > > OK.  The problem was that CC Mode was using the flag
> > > `load-in-progress' to assume that a CC Mode file was being loaded,
> > > for example by a `require' form inside a compilation of another CC
> > > Mode file.  This assumption breaks down when another file, such as
> > > cc-miscompile.el, while loading, initiates compilation of a CC Mode
> > > derivative.

> [ .... ]

> > > No, that wouldn't work.  What I've implemented is when both loading
> > > and byte-compilation are active at the same time, CC Mode walks down
> > > the lisp stack to discover which one of them is actually active.

> > Well, if you say… I find this “solution” horrifying, but I am probably
> > just unable to appreciate the full complexity of this issue.

> It is horrifying.  I spent quite some time over the weekend searching for
> a better solution, without success.  The issue is quite simple: when
> loading and compiling are nested in some unknown order, how do you
> determine which of them is the "innermost" activity?

> But, then again, maybe I could bind the "compilation flags" to nil around
> each invocation of `load' inside the CC Mode compilation system.  Yes,
> I'll give that a try.

I gave it a try, a hard and lengthy try, but in the end it wouldn't work.
So, back to the previous scheme.

Please disregard, revert, or discard the previous patch I sent.

Please try out the following patch in the real situation and let me know
whether it works properly.  The patch is to the Emacs master branch.
After applying it, please first explicitly DELETE cc-*.elc (this is
important), and then rebuild them, either with the make command, or
something like:

    emacs -Q -batch -f batch-byte-compile ..../lisp/progmodes/cc-*.el

With this patch I was able to generate d-mode.elc using cc-miscompile.el,
then to load and run it in the same emacs version which generated it.



diff --git a/lisp/progmodes/cc-bytecomp.el b/lisp/progmodes/cc-bytecomp.el
index 2db5a10..926fc98 100644
--- a/lisp/progmodes/cc-bytecomp.el
+++ b/lisp/progmodes/cc-bytecomp.el
@@ -89,13 +89,60 @@
   ;;`(message ,@args)
   )
 
+(defun cc-bytecomp-compiling-or-loading ()
+  ;; Determine whether byte-compilation or loading is currently active,
+  ;; returning 'compiling, 'loading or nil.
+  ;; If both are active, the "innermost" activity counts.  Note that
+  ;; compilation can trigger loading (various `require' type forms)
+  ;; and loading can trigger compilation (the package manager does
+  ;; this).  We walk the lisp stack if necessary.
+  (cond
+   ((and load-in-progress
+        (boundp 'byte-compile-dest-file)
+        (stringp byte-compile-dest-file))
+    (let ((n 0) elt)
+      (while (and
+             (setq elt (backtrace-frame n))
+             (not (and (car elt)
+                       (memq (cadr elt)
+                             '(load require
+                               byte-compile-file byte-recompile-directory
+                               batch-byte-compile)))))
+       (setq n (1+ n)))
+      (cond
+       ((memq (cadr elt) '(load require))
+       'loading)
+       ((memq (cadr elt) '(byte-compile-file
+                          byte-recompile-directory
+                          batch-byte-compile))
+       'compiling)
+       (t                              ; Can't happen.
+       (message "cc-bytecomp-compiling-or-loading: System flags spuriously 
set")
+       nil))))
+   (load-in-progress
+    ;; Being loaded.
+    'loading)
+   ((and (boundp 'byte-compile-dest-file)
+        (stringp byte-compile-dest-file))
+    ;; Being compiled.
+    'compiling)
+   (t
+    ;; Being evaluated interactively.
+    nil)))
+
+(defsubst cc-bytecomp-is-compiling ()
+  "Return non-nil if eval'ed during compilation."
+  (eq (cc-bytecomp-compiling-or-loading) 'compiling))
+
+(defsubst cc-bytecomp-is-loading ()
+  "Return non-nil if eval'ed during loading.
+Nil will be returned if we're in a compilation triggered by the loading."
+  (eq (cc-bytecomp-compiling-or-loading) 'loading))
+
 (defun cc-bytecomp-setup-environment ()
   ;; Eval'ed during compilation to setup variables, functions etc
   ;; declared with `cc-bytecomp-defvar' et al.
-  (if (not load-in-progress)
-      ;; Look at `load-in-progress' to tell whether we're called
-      ;; directly in the file being compiled or just from some file
-      ;; being loaded during compilation.
+  (if (not (cc-bytecomp-is-loading))
       (let (p)
        (if cc-bytecomp-environment-set
            (error "Byte compilation environment already set - \
@@ -143,7 +190,7 @@ perhaps a `cc-bytecomp-restore-environment' is forgotten 
somewhere"))
 (defun cc-bytecomp-restore-environment ()
   ;; Eval'ed during compilation to restore variables, functions etc
   ;; declared with `cc-bytecomp-defvar' et al.
-  (if (not load-in-progress)
+  (if (not (cc-bytecomp-is-loading))
       (let (p)
        (setq p cc-bytecomp-unbound-variables)
        (while p
@@ -287,8 +334,7 @@ use within `eval-when-compile'."
   `(eval-when-compile
      (if (and (fboundp 'cc-bytecomp-is-compiling)
              (cc-bytecomp-is-compiling))
-        (if (or (not load-in-progress)
-                (not (featurep ,cc-part)))
+        (if (not (featurep ,cc-part))
             (cc-bytecomp-load (symbol-name ,cc-part)))
        (require ,cc-part))))
 
@@ -301,12 +347,6 @@ afterwards.  Don't use within `eval-when-compile'."
      (require ,feature)
      (eval-when-compile (cc-bytecomp-setup-environment))))
 
-(defun cc-bytecomp-is-compiling ()
-  "Return non-nil if eval'ed during compilation.  Don't use outside
-`eval-when-compile'."
-  (and (boundp 'byte-compile-dest-file)
-       (stringp byte-compile-dest-file)))
-
 (defmacro cc-bytecomp-defvar (var)
   "Binds the symbol as a variable during compilation of the file,
 to silence the byte compiler.  Don't use within `eval-when-compile'."
@@ -320,8 +360,7 @@ to silence the byte compiler.  Don't use within 
`eval-when-compile'."
              "cc-bytecomp-defvar: Saving %s (as unbound)" ',var)
             (setq cc-bytecomp-unbound-variables
                   (cons ',var cc-bytecomp-unbound-variables))))
-       (if (and (cc-bytecomp-is-compiling)
-               (not load-in-progress))
+       (if (cc-bytecomp-is-compiling)
           (progn
             (defvar ,var)
             (set ',var (intern (concat "cc-bytecomp-ignore-var:"
@@ -349,8 +388,7 @@ at compile time, e.g. for macros and inline functions."
             (setq cc-bytecomp-original-functions
                   (cons (list ',fun nil 'unbound)
                         cc-bytecomp-original-functions))))
-       (if (and (cc-bytecomp-is-compiling)
-               (not load-in-progress))
+       (if (cc-bytecomp-is-compiling)
           (progn
             (fset ',fun (intern (concat "cc-bytecomp-ignore-fun:"
                                         (symbol-name ',fun))))
diff --git a/lisp/progmodes/cc-defs.el b/lisp/progmodes/cc-defs.el
index 46cb2f9..f92267e 100644
--- a/lisp/progmodes/cc-defs.el
+++ b/lisp/progmodes/cc-defs.el
@@ -1983,19 +1983,22 @@ system."
 
 (defvar c-lang-const-expansion nil)
 
+;; Ugly hack to pull in the definition of `cc-bytecomp-compiling-or-loading`
+;; from cc-bytecomp to make it available at loadtime.  This is the same
+;; mechanism used in cc-mode.el for `c-populate-syntax-table'.
+(defalias 'cc-bytecomp-compiling-or-loading
+  (cc-eval-when-compile
+    (let ((f (symbol-function 'cc-bytecomp-compiling-or-loading)))
+      (if (byte-code-function-p f) f (byte-compile f)))))
+
 (defsubst c-get-current-file ()
   ;; Return the base name of the current file.
-  (let ((file (cond
-              (load-in-progress
-               ;; Being loaded.
-               load-file-name)
-              ((and (boundp 'byte-compile-dest-file)
-                    (stringp byte-compile-dest-file))
-               ;; Being compiled.
-               byte-compile-dest-file)
-              (t
-               ;; Being evaluated interactively.
-               (buffer-file-name)))))
+  (let* ((c-or-l (cc-bytecomp-compiling-or-loading))
+        (file
+         (cond
+          ((eq c-or-l 'loading) load-file-name)
+          ((eq c-or-l 'compiling) byte-compile-dest-file)
+          ((null c-or-l) (buffer-file-name)))))
     (and file
         (file-name-sans-extension
          (file-name-nondirectory file)))))
@@ -2062,6 +2065,9 @@ constant.  A file is identified by its base name."
         ;; language constant source definitions.)
         (c-lang-const-expansion 'call)
         (c-langs-are-parametric t)
+        (file (intern
+               (or (c-get-current-file)
+                   (error "`c-lang-defconst' can only be used in a file"))))
         bindings
         pre-files)
 
@@ -2121,9 +2127,14 @@ constant.  A file is identified by its base name."
     ;; definitions for this symbol, to make sure the order in the
     ;; `source' property is correct even when files are loaded out of
     ;; order.
-    (setq pre-files (nreverse
-                    ;; Reverse to get the right load order.
-                    (mapcar 'car (get sym 'source))))
+    (setq pre-files (mapcar 'car (get sym 'source)))
+    (if (memq file pre-files)
+       ;; This can happen when the source file (e.g. cc-langs.el) is first
+       ;; loaded as source, setting a 'source property entry, and then itself
+       ;; being compiled.
+       (setq pre-files (cdr (memq file pre-files))))
+    ;; Reverse to get the right load order.
+    (setq pre-files (nreverse pre-files))
 
     `(eval-and-compile
        (c-define-lang-constant ',name ,bindings
@@ -2233,9 +2244,7 @@ quoted."
         (if (or (eq c-lang-const-expansion 'call)
                 (and (not c-lang-const-expansion)
                      (not mode))
-                load-in-progress
-                (not (boundp 'byte-compile-dest-file))
-                (not (stringp byte-compile-dest-file)))
+               (not (cc-bytecomp-is-compiling)))
             ;; Either a straight call is requested in the context, or
             ;; we're in an "uncontrolled" context and got no language,
             ;; or we're not being byte compiled so the compile time
diff --git a/lisp/progmodes/cc-langs.el b/lisp/progmodes/cc-langs.el
index 375725e..61e9239 100644
--- a/lisp/progmodes/cc-langs.el
+++ b/lisp/progmodes/cc-langs.el
@@ -3260,10 +3260,7 @@ function it returns is byte compiled with all the 
evaluated results
 from the language constants.  Use the `c-init-language-vars' macro to
 accomplish that conveniently."
 
-  (if (and (not load-in-progress)
-          (boundp 'byte-compile-dest-file)
-          (stringp byte-compile-dest-file))
-
+  (if (cc-bytecomp-is-compiling)
       ;; No need to byte compile this lambda since the byte compiler is
       ;; smart enough to detect the `funcall' construct in the
       ;; `c-init-language-vars' macro below and compile it all straight


-- 
Alan Mackenzie (Nuremberg, Germany).