Re: History of bash's support for self-modifying shell scripts?

[Josh Triplett]

On Mon, Sep 10, 2018 at 04:50:29PM -0400, Chet Ramey wrote:
> On 9/10/18 1:25 AM, Josh Triplett wrote:
> > While digging into the details of how bash reads shell scripts, I found
> > some indications that bash goes out of its way to support self-modifying
> > shell scripts. As far as I can tell, after reading and executing each
> > command, bash will seek backward and re-read the script from the
> > byte after the end of that command, rather than executing out of
> > buffered data previously read from the file. (For the purposes of this
> > logic, compound commands get run as a single unit, and this logic kicks
> > in after running the full compound command.)
> 
> It happens in only a few cases: 1) when forking a child to run a command;
> 2) when a redirection specifies the same file descriptor as bash is using
> to read a script; and 3) when bash is reading a script from stdin and the
> read builtin is used to read from that file descriptor.
> 
> The first case is probably the one you're interested in. It's been there
> even since I wrote the buffered input code in 1992, and it's more about
> making sure parent and child shells have a consistent view of the script
> in case the child expects to read from it. It's about being careful, not
> explicitly allowing self-modifying scripts.

Interesting. I don't *think* the behavior I observed corresponds to one
of those cases; I observed it by just having a shell script that
carefully used `dd conv=notrunc of=$0 ...` to write code into the
current script after the current command.

> Previous versions of the shell (through bash-1.12) used stdio, which has
> behavior that varies across systems, especially across parent-child
> boundaries and changing file descriptors due to redirection (which it can't
> really handle at all).
> 
> POSIX says you have to do that anyway if the shell is reading from stdin:
> 
> "When the shell is using standard input and it invokes a command that also
> uses standard input, the shell shall ensure that the standard input file
> pointer points directly after the command it has read when the command
> begins execution. It shall not read ahead in such a manner that any
> characters intended to be read by the invoked command are consumed by the
> shell (whether interpreted by the shell or not) or that characters that are
> not read by the invoked command are not seen by the shell."

I did find that, but that only applies to stdin, not to shell scripts.

I'd certainly love to *only* do this for stdin.

> But it probably isn't needed in the general case. Why not take the code out
> and see what happens with your testing?

When dealing with something with the history and backward compatibility
of bash, I'm hesitant to take that approach with *anything* without
first checking with the experts who made it that way in the first place.
:)

Thanks for the history and details, I appreciate it!

- Josh Triplett