Re: [lwip-users] Out Of Order Sequence (Segments?)

[Jonathan Larmour]

Bill Auerbach wrote:
> Hello,
> 
> I want to understand Out Of Order Sequences (or is it segments?),

I think sequence, strictly, although a sequence is what goes in a segment,
and it's more efficient to ensure a segment consists of a single packet,
rather than using IP fragmentation; so in practice we may often say packet,
or sequence or segment interchangeably.

> why
> they result, and what lwIP does to limit them?

It's not up to lwIP.

> Can someone briefly tell
> me what it’s about, or point me to a good reference?  I find Google is
> spotty here for a good description except for a reference that it must
> be supported by a TCP/IP stack.

Due to TCP windowing, more than one segment can be transmitted at one time
before requiring an acknowledgement. Segments contain information
indicating what byte range in the stream they cover. Suppose host A is
sending 100 bytes at a time to host B. Suppose host A sends two segments in
a row, bytes 700-800 and bytes 800-900. It's possible that due to the
uncertainties of routing over the internet, that the second arrives before
the first because it went a different route. They would then arrive out of
order.

The more likely occasion we need to be worried about out-of-order segments
is when the first segment is dropped entirely due to congestion or an
error. In that case the second segment will arrive and the first segment
won't arrive until there's a retransmission. When packets arrive
out-of-order an ACK is sent or re-sent immediately from the receiving host,
indicating the last successfully received byte (there are also selective
ACKs, although lwIP does not yet implement these unfortunately).

> I ask because I’ve been chasing a system freeze (which is odd because
> the PowerPC is quite good about its exception handling).  It results
> from running out of PBUFs (RAW) in my Ethernet driver.  It turns out I
> am running out because of pbufs left on the ooseq list.  I didn’t know
> to account for pbufs for this use.

Most likely packet loss, and your remote end (the sender) has not
retransmitted the missing packet yet. It could mean that your receive
window size is too large for the pbuf space you have available. In other
words, you are telling the remote end that it can send more data without
acknowledgement than you actually have room for.

>  If I get to a state where pbuf_alloc
> returns NULL in my driver and I return NULL for what looks to be
> forever, the system freezes.  (The debugger won’t even break in which to
> date has been for a very bad thing J ).

That sounds like a port or driver problem (or a bug I haven't encountered).
lwIP can run out of pbufs, but it usually recovers. Especially because (as
you have noticed) the TCP slow timer will drop out of sequence data by
default after 6 multiples of the re-transmission timeout (a dynamic
connection-specific value affected by latency). So before too long your
pbuf memory should be freed.


> Should there be a #define and a test and for the amount of ooseq’s to be
> allowed to be buffered?  If not, what is a practical limit?

Your receive window (TCP_WND in lwipopts.h).

> What are the disadvantages of setting TCP_QUEUE_OOSEQ to 0?

Out of order data is discarded which means that in the case of congestion
or packet loss, more data will probably need to be retransmitted.

> If a driver doesn’t have pbufs and has to return NULL repeatedly, what
> should occur for it to recover these tied up pbufs?  I see that
> tcp_slow_tmr releases these ooseqs, but I seem to not get to this
> point.  Should returning NULL from low_level_input be a problem?

Nothing special should be required by the driver. The bigger problem is why
it doesn't get to that point.

Jifl
-- 
eCosCentric Limited      http://www.eCosCentric.com/     The eCos experts
 **  Visit us at ESC Silicon Valley <http://www.embedded.com/esc/sv>  **
 **  April 15-17 2008, Booth 3012, San Jose McEnery Convention Center **
Barnwell House, Barnwell Drive, Cambridge, UK.       Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.
------["Si fractum non sit, noli id reficere"]------       Opinions==mine