Re: [lwip-users] performance in sending / receiving

[Kieran Mansley]

On Tue, 2007-02-13 at 23:03 +0100, Robert Goeffringmann wrote:
> Hi,
> 
> I got lwip to run on the ps2's main cpu and achieve send speeds of ~4 
> Mbyte/s.
> Unfortunately the speed for receiving data is stuck at ~1-2Mbyte/s and I 
> can't figure out why...
> 
> In both cases, I run a very simple test app on the PS2 that simply opens a 
> socket and calls send (or recv) in an endless loop over and over again with 
> a 40Kbyte large buffer.
> 
> When receiving data, it always starts with lwip advertising a window of 
> 29-32KB, which is quickly filled by the host pc.
> lwip then sends ACKs, reducing the window sizes until it finally sends a 
> zero window..
> Then, after a couple of millis, it sends a new window update with a size of 
> 29-32KB.
> 
> I already don't understand why it doesn't advertise a bigger window to begin 
> with.

It should advertise a maximum receive window equal to TCP_WND, which
from your settings below is around 57KB.  This is less than the maximum
(without window scaling) of 64KB, so should be fine.

The reason it decreases the window down to zero is that it is unable to
keep up with processing the network traffic at the rate it is arriving,
and so packets queue and the window fills up.  This suggests (rather
simplistically) lack of CPU, or more fairly an inefficient stack.

> >From what I can tell my presets in the lwipopts.h are fairly big:
> 
> #define MEMP_NUM_PBUF          100
> #define MEMP_NUM_TCP_SEG        100
> #define PBUF_POOL_SIZE         200
> #define PBUF_POOL_BUFSIZE       1580
> 
> #define TCP_MSS                 1460
> #define TCP_SND_BUF             (TCP_MSS * 40)
> #define TCP_SND_QUEUELEN  64
> #define TCP_WND                 (TCP_MSS * 40)
> #define TCP_SNDLOWAT  (TCP_SND_BUF/2)

These look at first glance to be pretty sensible. 

> Can't I usually expect to be able to achieve roughly the same rates for 
> sending and receiving?

No, not at all.  The overheads on send and receive are very different,
and I've never worked with a TCP/IP network where the send and receive
bandwidths are the same. 

> And.. did anyone ever manage to max out 100mbps entirely using lwip or is it 
> too focused on being lightweight for that?

You're certainly unlikely to do that using the sockets API.  My use for
lwIP, many years ago, was to port it to a (at the time cutting-edge)
gigabit network and I was able to outperform normal ethernet with linux
kernel network stacks.  However, this required (i) special hardware and
(ii) heavily modified lwIP stack, so I don't think you'll get anything
like that out of the box.

The best performance people have reported has always been with the raw
API.  It is much simpler than the sockets API as implemented, involves
far less overhead, and so should give you better performance.

Kieran