[lwip-users] Re: [lwip] Gigabit Ethernet and LWIP

[Adam Dunkels]

On Tuesday 07 May 2002 14.46, you wrote:
> > > I'm working on something similar.  Not Gigabit Ethernet, but a gigabit
> > > rate network.  lwIP in it's current form isn't targetted at this, and
> > > so is unlikely to give the performance you require.  This will be
> > > especially true if you use the sockets interface as the amount of
> > > thread switching and message passing is just too great an overhead. 
> > > I'm creating a somewhat different version for which the path for normal
> > > data delivery and transmission is in a single thread, and which isn't
> > > so concerned about low memory usage.  This should improve things.
> >
> > I don't fully agree with this - when using the raw API (which is single
> > threaded), the performance of lwIP in terms of CPU cycles is really good
> > because no copying or thread switching is being made.
>
> There's still a thread required to pass the data from the NIC to the stack
> however, and a thread boundary to cross there.  I'm changing it to have
> the stack take the data from the network itself.  This tighter integration
> with the network is rather specialised, but gives much increased
> performance and other benefits - my research is to see how these can be
> applied to TCP.  Also, for me a TCP/IP stack without a sockets library has
> limited use - I need to be able to use it with existing applications.  As
> a result I need to run the sockets code in the same thread too!

I see - but the stack and the network device driver doesn't have to run in 
different threads. In fact, in small systems without support from a 
multitasking operating system, the only way to do it is to have the stack and 
the device driver in the same thread. Polling or interrupt driven operation 
works equally well since there aren't any other threads using the CPU so busy 
waiting can be used for the polling loop.

A socket interface for an application running in the same thread as the stack 
- that's an interesting problem. I would presume that only works well for 
communication-centric applications, which spends most of their time waiting 
in read() or accept(). I'm definately looking forward to hearing about your 
work with this!

(By the way, is the direct application<->NIC communication in any way related 
to Van Jacobson's proposed 4.4BSD TCP/IP stack enhancements? His slides are 
avaliable at ftp://ftp.ee.lbl.gov/talks/vj-nkarch.ps.Z - they are a very 
interesting read!)

> > I would say that the main problem with running lwIP with gigabit speeds
> > isn't CPU performance, but rather throughput issues. With gigabit speeds,
> > the bandwidth*delay product becomes so large that the standard TCP
> > mechanisms that lwIP implements cannot "fill the pipe". The TCP
> > extensions defined in RFC1323, "TCP Extensions for High Performance",
> > should really be implemented if high throughput is desired.
>
> Definitely, and that's on my list as well.  CPU performance is still a big
> issue though.  If you are using a lot of CPU to service the network, that
> means less CPU for running the application.

Yes, but without the extra copying and context switching (i.e., assuming that 
lwIP's socket interface isn't used :-), the processing time is dominated by 
the checksum calculation. Implementing a really fast checksum calculation 
wins a lot.

> > Still, interfacing lwIP with gigabit Ethernet is doable, but it will not
> > be able to take full advantage of the gigabit speeds.
> >
> > Kieran, since you are writing a new API, you might be interested in
> > looking at the new event based API for 0.6, by the way.
>
> Yep, will take a look when it comes out.  Sounds like I'm going to have
> fun merging my code with yours!

For those that are interested, the latest CVS code (avaliable from the 
"Download" section of the lwIP pages at http://www.sics.se/~adam/lwip/) 
implements the new event based API. The new example application in the new 
"minimal" project, proj/minimal/echo.c, shows a little of how it works. The 
function right at the end, called lwip_tcp_event(), is the callback function 
used for all events (and all connections).

/adam
-- 
Adam Dunkels <address@hidden>
http://www.sics.se/~adam
[This message was sent through the lwip discussion list.]