bash and sshd trap anomaly

[Olof Schonbeck]

Hi

In a small bash script we have a trap to cleanup some files when exiting. You 
run the script by ssh to the machine in question and execute the script. If 
your ssh session dies the trap should trigger and clean up the files as the 
script exit but this doesn't happen. 


I apologize that the script has excessive echo to a log file, but it makes it 
easy to follow.  The script is located as /tmp/quirk-plsuper installed on host 
XYZ

----
#!/bin/bash

chars=(A B C D E F G H I J K L M N O P Q R S T U V W X Y Z)
username=plsuper

while grep -q "^$username:" /tmp/passwd; do
        username=plsuper
        for ((i=0;i<3;i++)); do
                username+=${chars[$((RANDOM%${#chars[*]}))]}
        done
done

cleanup () {
        echo "Trying to cleanup " >> /tmp/trap
        grep -v "^$username:" /tmp/passwd > /tmp/passwd.new
        echo "Trying to cleanup 1" >> /tmp/trap
        grep -v "^$username:" /tmp/shadow > /tmp/shadow.new
        echo "Trying to cleanup 2" >> /tmp/trap
        mv /tmp/passwd.new /tmp/passwd
        echo "Trying to cleanup 3" >> /tmp/trap
        mv /tmp/shadow.new /tmp/shadow
        echo "Trying to finished" >> /tmp/trap
}

trap "cleanup" EXIT

password=$(awk -F: '/^plsuper:/{print $2}' /tmp/shadow)

echo "$username:x:1001:1001::/home/plsuper:/bin/bash" >> /tmp/passwd
echo "$username:$password:::::::" >> /tmp/shadow

cat <<-EOF

        WARNING! You've exposed the customers system for unauthorized
        logins with the account '$username'.

        Press enter to remove '$username'.

EOF

read -e
------------

Now ssh to XYZ and execute the script. In a different terminal kill your ssh 
client session. If you ssh back to host XYZ you would expect the passwd and 
shadow file to be "restored" and the /tmp/trap to contain
Trying to cleanup 
Trying to cleanup 1
Trying to cleanup 2
Trying to cleanup 3
Trying to finished

but all I get is 
Trying to cleanup

and none of files been "restored".

Granted this is on a home grown embedded Linux environment (bash, version 
4.2.50), but when trying the same on my localhost running Ubuntu 14.04 I get a 
similar result.

What I do is ssh localhost (ubuntu) and run the script then in another terminal 
I kill the ssh client session to localhost. I now get 
Trying to cleanup 
Trying to cleanup 1
Trying to cleanup 2

so a bit more of the trap is executed it's me thinks some sort of timing or 
race going on.

More than a few tries and test later I find a workaround if I change this 
section
-----
}

trap "cleanup" EXIT

password=$(awk -F: '/^plsuper:/{print $2}' /tmp/shadow)
-----
to
-----
}

trap "cleanup" EXIT
trap true HUP

password=$(awk -F: '/^plsuper:/{print $2}' /tmp/shadow)
-----

It's all working as it should. Only having 
-----
trap "cleanup" HUP 
-----
Doesn't work. You can have both EXIT and HUP on the same line and it also works 
but the trap gets executed twice which is the expected behaviour. 


If you alter the sshd server adding a small 5s sleep in the 
session_pty_cleanup2 function in session.c just before pty_release(s->tty); 
Then the script/trap is also working just fine without the "trap true HUP" 
workaround.

Me guessing now is that ssh is pulling the tty from under the bash script and 
it doesn't get the time needed to execute the trap but that doesn't make sense 
either since the workaround is working. It's clearly some sort of timing issue 
but I can't pin point it.

Looking at the signals coming in to the script if you strace it then it looks 
like this:

rt_sigaction(SIGINT, {0x4a0d40, [], SA_RESTORER, 0x7effdbc841e0}, {0x442b60, 
[], SA_RESTORER, 0x7effdbc841e0}, 8) = 0
rt_sigaction(SIGHUP, {SIG_DFL, [], SA_RESTORER, 0x7effdbc841e0}, {0x45b240, 
[HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM 
SYS], SA_RESTORER, 0x7effdbc841e0}, 8) = 0
kill(31941, SIGHUP)                     = 0
rt_sigreturn(0x7cc5)                    = 0
--- SIGHUP (Hangup) @ 0 (0) ---
Process 31941 detached

Any thoughts?

Jinx