[lwip-users] [lwip] tcp_ack query

[Kieran Mansley]

I've been seeing some less than optimal behaviour of ACKs when there is
data going in one direction only, and having had a look at the code I'm a
little unsure about whether or not they're being sent in a sensible way.

The problem I have is with tcp_ack.  Data is sent from machine A to
machine B.  As B receives the data it does one of two things (using
tcp_ack()):
 1) if TF_ACK_NEXT is not set, set it now
    - this prompts tcp_fasttmr to send it a short time later

 2) if TF_ACK_NEXT is set (ie. the first case has happened already) use
tcp_send_ctrl to actually queue an ACK, and unset TF_ACK_NEXT.
    - tcp_output() is not called at this point, so it's not dispatched
across the network yet.  It will hitch a ride with some data later.

So, if you only send 1 ACK before the fast timer goes off, the ACK
is dispatched then (case 1 above).  If you send two ACKS though, case (2)
is invoked, and then when tcp_fasttmr is called TF_ACK_NEXT has been
cleared, so it doesn't dispatch the ACKS.  Instead they wait around for
the slow timer to go off, and are then dispatched.  This seems a bit odd,
and cripples streaming performance!

I understand the reason for having a delay before sending ACKS (ie. to
reduce the bandwidth they use) but having a longer delay if you send two
ACKs instead of one is a bit strange.

Changing tcp_ack() to be defined like this gives much better results:

#define tcp_ack(pcb)     if((pcb)->flags & TF_ACK_NEXT) { \
                         } else { \
                            (pcb)->flags |= TF_ACK_NEXT; \
                         }

(ie. If you are already going to send an ACK when the fast timer goes off
you do nothing, otherwise make sure one is sent when the fast timer next
goes off).

Of course, I could have got it all completely wrong, and you have it like
it is for a reason.

Another related problem is the receipt of ACKs!  If one machine has used
up all the tcp segments available from the memp (ie. there are a few
packets in flight and it's waiting for an ACK before sending some more)
then when the ACK arrives it is dropped because there are no more
available segments.  This means the whole thing grinds to a halt.  I can
easily avoid this by ensuring MEMP_NUM_TCP_SEG is defined to be larger,
but it took me a long time to work out what was going on!  I don't think
there's anything wrong with the code (although I find dropping acks just
because you've got a lot of outgoing traffic a little unsettling - maybe
memory for incoming segments and outgoing segments should be done
separately so one direction doesn't starve the other), just that I'd
chosen duff values in lwipopts.h.  Perhaps what I'm suggesting here is to
have the constraints and relationships between the MEMP_* constants
documented in the same way as TCP_SND_QUEUELEN is.  Just a thought anyway.

Cheers,

Kieran

[This message was sent through the lwip discussion list.]