load small history file after append breaks further appending

[Scott Mcdermott]

I've been implementing something in my shell where I'm
using multiple history files, and I switch between
them.  This changes the shell's "identity," to
basically swap its history file with another.  This is being
used to have per-project histories.

The procedure I am using to "switch" is something like
this:

    history -a          # flush anything left
    history -c          # start anew
    HISTFILE=newhist    # prepare to load this one
    history -r          # and do it (identities changed)

however I have also tried this:

    history -a
    history -cr newhist
    HISTFILE=newhist

At this point we should be able to enter new commands,
and when we decide to, "history -a" (append them to the
`newhist' file).

However, bash fails to do this in the case that the new
history file is either very small, or "new," i.e. zero
bytes.

I believe I have tracked down the reason this is
happening:

`bash' keeps the notion of how many lines it has
inserted into the history since the history file was
read (see `bashhist.c:history_lines_this_session').
This amount is zeroed after a successful append.

If this amount ever becomes greater than or equal to
the size of the new history that has been read (which
is what is returned from the global `history_offset' in
`history.c:where_history()'), bash will never allow the
append to succeed again (see the check on whether
history should be appended or not, which can be found
in `bashhist.c:maybe_append_history()').

BUG: This situation arises if there are a few commands
in between the successful append to the old history,
and the import of the new history, IFF the new history
is smaller than said number.

BUG: This happens also for NEW histories.  Even with a

    "history -cr"

(resulting in `history_lines_this_session' value of `1')
instead of

    "history -c; history -r"

(`history_lines_this_session' value of `2'), the value
is still greater than zero, which is the size of a new
history file just read, so it will NEVER work again and
all subsequent history will NOT be written with a
"history -a", which silently fails to append data when
it should.

Note that on the append which occurs at shell exit,
the data WILL be appended, for two reasons:

(1) the test in `maybe_append_history()':

          history_lines_this_session < where_history()

    differs from that in `maybe_save_shell_history()':

          history_lines_this_session <= where_history()
                                     ^^

(2) the history is appended unconditionally anyways if
    `force_append_history' is set (which corresponds
    to `shopt -s appendhist'), whereas no such
    condition is checked for in `maybe_append_history()'.

I am wondering, why not zero `history_lines_this_session'
upon a successful "history -c" ? As it is, clearing the
history does not change the number at all and I can
keep entering commands and clearing them all day;
`history_lines_this_session' continues to rise.  What
reason is there to keep this variable with a positive
value after all history settings are cleared? There's
nothing to write.

The following demonstrates the problem:

   $ id -un                                 # starting as self
  smcdermott
   $ sudo -u test rm -f ~test/.bash_history # give test account
   $ sudo -u test touch ~test/.bash_history # empty history
   $ sudo su - test                         # become test user
   $ wc -l $HISTFILE                        # was indeed empty
  0 /home/test/.bash_history
   $ shopt -s histappend                    # force append on exit
   $ logout
   $
   $ sudo su - test                         # now go back and load
   $ wc -l $HISTFILE                        # 2 commands appended
  2 /home/test/.bash_history
   $ cp $HISTFILE $HISTFILE.new             # which we stow
   $ history -a                             # note append works
   $ wc -l $HISTFILE
  5 /home/test/.bash_history
   $ history -cr $HISTFILE.new              # clear, load stowed 2
   $ HISTFILE=$HISTFILE.new                 # switch to stow file
   $ test 1                                 # add a few commands
   $ test 2                                 # that should be
   $ test 3                                 # appended
   $ history -a                             # like so
   $ wc -l $HISTFILE                        # but they're not
  2 /home/test/.bash_history.new

Now note when I add ONE extra command, otherwise unchanged:
-----------------------------------------------------------

   $ id -un
  smcdermott
   $ sudo -u test rm -f ~test/.bash_history
   $ sudo -u test touch ~test/.bash_history
   $ sudo su - test
   $ wc -l $HISTFILE
  0 /home/test/.bash_history
   $ shopt -s histappend
   $ test 1                         # <--- here makes 3 to save
   $ logout
   $ sudo su - test
   $ wc -l $HISTFILE
  3 /home/test/.bash_history
   $ cp $HISTFILE $HISTFILE.new
   $ history -a
   $ wc -l $HISTFILE
  6 /home/test/.bash_history
   $ history -cr $HISTFILE.new
   $ HISTFILE=$HISTFILE.new
   $ test 1
   $ test 2
   $ test 3
   $ history -a
   $ wc -l $HISTFILE
  10 /home/test/.bash_history.new   # this time it appended!!

  (Note that I used control-d to logout, that's why it was not
  saving the `logout' in history.)

...

I believe this shows the bug.  I suggest zeroing
`history_lines_after_session' upon "history -c" OR having
`force_append_history' cause it to do just that in
`maybe_append_history()' in addition to `maybe_save_history()'
 as it does already.

Please keep me copied as I am not on the list.  Thanks.

-- 
Scott