bug#18140: [PATCH] macros.c: CHECK_VECTOR_OR_STRING invokes wrong_type_argument for Qnil instead of return 0

[Jan Chaloupka]

Changelog:
        line wrapping to 80 characters

In function Fstart_kbd_macro (macros.c), Vlast_kbd_macro of current_kboard is
Qnil for the first invocation. If NILP (append) is false,
current_kboard->kbd_macro_ptr has random value (in our case
0x5353535353535353), which after CHECK_VECTOR_OR_STRING failure (invocation
of wrong_type_argument) results in garbage collecting.
During gc, marking of objects is processed and mark_kboards (keyboard.c) is
invoked. Following for loop is fired:

for (p = kb->kbd_macro_buffer; p < kb->kbd_macro_ptr; p++)
                  mark_object (*p);

Since kb->kbd_macro_ptr is set to 0x5353535353535353, mark_object (*p) is
trying to mark object on address out of memory space (or memory that
cannot be accessed). Thus resulting in SIGSEGV signal.

Solution is to check for Qnil before calling CHECK_VECTOR_OR_STRING and set
len to 0 if Qnil occurs.

https://bugzilla.redhat.com/show_bug.cgi?id=1104012

Signed-off-by: Jan Chaloupka <jchaloup@redhat.com>
---
 src/macros.c |   20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/src/macros.c b/src/macros.c
index 4730a8b..4fd6cb1 100644
--- a/src/macros.c
+++ b/src/macros.c
@@ -85,7 +85,25 @@ macro before appending to it.  */)
       bool cvt;
 
       /* Check the type of last-kbd-macro in case Lisp code changed it.  */
-      len = CHECK_VECTOR_OR_STRING (KVAR (current_kboard, Vlast_kbd_macro));
+      /* If Vlast_kbd_macro is Qnil, skip the check and set len to 0.
+       * Flength returns 0 for Qnil, CHECK_VECTOR_OR_STRING has to do the same.
+       * Otherwise CHECK_VECTOR_OR_STRING fails and results in garbage
+       * collecting, which results in (keyboard.c, mark_kboards(void)).
+       * Among others, mark_kboards it executes:
+       *
+       * for (p = kb->kbd_macro_buffer; p < kb->kbd_macro_ptr; p++)
+       *         mark_object (*p);
+       *
+       * Here, kb->kbd_macro_ptr is not initialized and can contain address
+       * 0x5353535353535353, which results in SIGSEGV trying to access
+       * the address.
+       *
+       * https://bugzilla.redhat.com/show_bug.cgi?id=1104012
+       */
+      if (!NILP (KVAR (current_kboard, Vlast_kbd_macro) ))
+        len = CHECK_VECTOR_OR_STRING (KVAR (current_kboard, Vlast_kbd_macro));
+      else
+        len = 0;
 
       /* Copy last-kbd-macro into the buffer, in case the Lisp code
         has put another macro there.  */