On Sun, 7 Apr 2024, at 5:17 AM, admin@osrc.rip wrote:
Hello everyone!
I've attached a minimal script which shows the issue, and my
recommended
solution.
Affected for sure:
System1: 64 bit Ubuntu 22.04.4 LTS - Bash: 5.1.16(1)-release -
Hardware:
HP Pavilion 14-ec0013nq (Ryzen 5 5500u, 32GB RAM, Radeon grapics, nvme
SSD.)
System2: 64 bit Ubuntu 20.10 (No longer supported.) - Bash:
5.0.17(1)-release - Hardware: DIY (AMD A10-5800k, 32GB RAM, Radeon
graphics, several SATA drives)
and probably a lot more...
Not sure whether or not this is a know issue, truth be told I
discovered
it years ago (back around 2016) as I was learning bash scripting, and
accidentally appended a command to the running script, which got
executed immediately after the script but back then I didn't find it
important to report since I considered myself a noob. I figured
someone
more experienced will probably find and fix it, or there must be a
reason for it. I forgotű it. Now watching a video about clever use of
shell in XZ stuff I remembered, tested it again and found it still
unpatched. :S So now I'm reporting it and hope it helps!
It is a known pitfall, though perhaps not as widely known as it ought
to be. The reason that your usage of (GNU) sed fails as a
self-modification technique is that sed -i behaves as follows.
1) it creates a temporary file
2) it sends its output to the temporary file
3) it renames the temporary file over the original file from which it
read
The consequence of the third step is that the original file is
unlinked. In its place will be a new hard link, bearing the same name,
but otherwise quite distinct from the original. Such can be easily
demonstrated:
$ touch file
$ stat -c %i file
1548822
$ strace -erename sed -i -e '' file
rename("./sedP2oQ5I", "file") = 0
+++ exited with 0 +++
$ stat -c %i file
1548823
See how the revised file has an entirely new inode number? It proves
that sed does not perform 'in-place' editing at all. For more
information regarding that particular topic, take a look at
https://backreference.org/2011/01/29/in-place-editing-of-files/index.html.
Now, at the point that the original file is unlinked, its contents will
remain available until such time as its reference count drops to 0.
This is a characteristic of unix and unix-like operating systems in
general. Let's assume that the file in question is a bash script, that
bash had the file open and that it was still reading from it. Bash will
not yet 'see' your modifications. However, once bash closes the file
and exits, should you then instruct bash to execute the script again,
it will follow the new hard link and thereby read the new file.
Further, assuming that no other processes also had the original file
open at the time of bash exiting, its reference count will drop to 0,
and the backing filesystem will free its associated data.
From this, we may reason that the pitfall you stumbled upon applies
where the file is modified in such a way that its inode number does not
change e.g. by truncating and re-writing the file. One way to
demonstrate this distinction is to apply your edit with an editor that
behaves in this way, such as nano. Consider the following script.
#!/bin/bash
echo begin
sleep 10
: do nothing
echo end
You can try opening this script with nano before executing it. While
the sleep command is still running, replace ": do nothing" with a
command of your choosing, then instruct nano to save the amended
script. You will find that the replacement command ends up being
executed. Repeat the experiment with vim and you will find that the
outcome is different. That's because the method by which vim amends
files is similar to that of sed -i.
You propose a method by which bash might implicitly work around this
pitfall but it would not suffice. If you perform an in-place edit upon
any portion of a script that bash has not yet read and/or buffered -
while bash is still executing said script - then the behaviour of the
script will be affected. If you consider this to be a genuine nuisance,
a potential defence is to compose your scripts using compound commands.
For example:
#!/bin/bash
{
: various commands here
exit
}
Alternatively, use functions - which are really just compound commands
attached to names:
#!/bin/bash
main() {
: various commands here.
exit
}
main "$@"
Doing so helps somewhat because bash is compelled to read all the way
to the end of a compound command at the point that it encounters one,
prior to its contents being executed.
Ultimately, the best defence against the potentially adverse
consequences of performing an in-place edit is to to refrain entirely
from performing in-place edits.
Example.txt