Re: [lwip-users] recv ok.. send is super slow

[Rishi Khan]

> > The original send problem turned out that it would send one packet,
> > wait for the ack, then send the next. This was because the
> > TCP_SND_BUF was only 2048 in the unixsim contrib folder, which I used
> > as a basis.
>
> The values used in the examples are often very conservative defaults,
> and not tuned for performance.  lwIP is in general targeted at low
> memory systems rather than high performance ones, which is why the
> defaults lean in that direction.  Some time ago someone posted a  
> list of
> sensible values to use for performance.  These may have made it  
> onto the
> lwIP wiki, but I'm not sure.  Searching the mailing list archives  
> might
> help if not.

I'll check. Thanks.

> > My test program linux-to-linux gets 117.8MB/s (940Gbps). It gets
> > 113.2MB/s with TCP window scaling off, but this is off topic.
> > Linux_send-lwip_recv gets 923.3MB/s (738Gbps). Linux_send-lwip_recv
> > gets 92.3MB/s (738Gbps). lwip_both gets 92.3MB/s (738Gbps).
>
> I think you've confused Gbps and Mbps, and got a decimal place or  
> two in
> the wrong place in those figures, but I think I can make sense of  
> them.
> It looks like you're getting pretty good performance from lwIP.  I
> wouldn't expect it to beat the native Linux stack.

Sorry if there is any confusion here. For clarification, I mean MB is  
megabytes or 1 million bytes. Gb is Gigabits or 1 billion bits or 125MB.

> > It seems that on the recv side, LWIP sends multiple requests for TCP
> > window_size updates.
>
> I think we have an outstanding bug about window updates.  Might be  
> worth
> checking to see if that matches your observed behaviour.
>
> > The window size drops significantly and
> > fluctuates between 30K and 60K. Also, there seems to be a lot of PSH
> > flags when lwip sends. The packets are truncated, but the data
> > payload is just '0x7a7a7a7 ....'
>
> I'll try and find some time to take a look at your packet captures.
>
> > Finally, somehow linux can send packets bigger than the 1500 MTU. How
> > is this possible? Is it fragmenting the packets? The IP flags say
> > 'don't fragment'.
>
> Most likely Linux is doing segmentation offload to your NIC.  i.e.
> passing a larger than MTU socket buffer to the NIC, which then  
> fragments
> it into MTU sized chunks.  This is a pretty standard feature in modern
> NICs to reduce the overheads on the network stack.

I guess I could do the same?

Thanks.

Rishi