bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Gerd Möllmann]

Sebastian Miele <iota@whxvd.name> writes:

> Put the following into a file named, e.g., ‘test-script’, and ‘chmod +x’
> it.
>
>   #!/bin/sh
>   : ; exec emacs --script "$0" -- "$@" #; -*- lexical-binding: t; mode: 
> emacs-lisp; -*-
>
>   (defmacro lexical-binding-p ()
>     '(let* ((x t)
>             (f (lambda () x))
>             (x nil))
>        (funcall f)))
>
>   (message "%s %s" lexical-binding (lexical-binding-p))
>
> When the script is run, the output is "nil nil", signifying that lexical
> binding is not enabled.
>
> Then find the file in an interactive Emacs session, and interactively
> evaluate (C-x C-e) the two expressions.  The output now is "t t", i.e.,
> lexical binding is in use.
>
> https://lists.gnu.org/archive/html/emacs-devel/2023-11/msg01041.html
> contains more details, which are repeated here:
>
>> From: Jens Schmidt <jschmidt4gnu@vodafonemail.de>
>> Date: Mon, 2023-11-20 21:10 +0100
>>
>> I tried byte-compiling something similar yesterday, which also
>> indicated that the byte-compiler compiles with lexical bindings.  Only
>> the scripting machinery sees dynamical bindings.
>>
>> […]  It seems that the scripting machinery expects a semicolon in the
>> very first column, without that the lexical-binding line is not
>> recognized.  Even a space before the semicolon breaks the recognition.
>>
>> The problem is in function `lisp_file_lexically_bound_p' from lread.c,
>> which is indeed much more strict in its recognition of the -*- ... -*-
>> stanza than the functions `set-auto-mode-1' and
>> `hack-local-variables-prop-line' from files.el.  The Emacs manual
>> ((emacs) Specifying File Variables) only mentions that the stanza has
>> to be in the first line (or the second one if the first is taken by a
>> she-bang), without any restriction where the comment has to start.
>
> ((emacs) Specifying File Variables) states no restrictions on what must
> (not) precede or follow a "-*- … -*-" on the first or second line of a
> file.
>
> Expected: At least consistent behavior.  Ideally, lexical binding
> should be enabled in all cases.
>
> In GNU Emacs 29.1.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version

Same in master, BTW, and everything you write is correct, I think.

I guess one could recognize the ": ; exec" idiom (do nothing
successfully and then exec emacs) in command-line--load-script, but that
feels really strange and brittle.

Can I ask why that idiom is used?