Re: [xougen] GPL-ed X compression scheme -- my gaff

[Gian Filippo Pinzari]

On Wednesday 01 October 2003 11:43, Rich Wareham wrote:
> Upshot: Even with the bandwidth turned up to maximum, NX is /really/
> slow on these machines.

We had a number of users reporting very poor performances running
NX on Windows 95/98 and Windows 2000 machines. WinXP doesn't
seem to be affected. We are trying to undestand where is the problem 
as it doesn't seem to be related to the raw X server's drawing perfor-
mances. As you noted, plain X forwarding through SSH works well.

Here are some test cases executed with NXWin (the NX Cygwin 
based X server) and with the WinAXE X server. Statistics are taken 
from nxproxy on the Windows client:

Legenda:

ms in read is time spent encoding/decoding messages. It includes,
in this case, translation of images from JPEG to plain X bitmaps. 
This time measures the efficiency of nxproxy handling compression 
and decompression.

ms in write is time spent between a write() call and the OS giving back
control to the nxproxy process. In theory it should be 0, as nxproxy 
is not doing anything in the meanwhile except copying data to the
TCP/IP layer. 

---ARISTIPPO (Windows 2000 professional)

X Server=NXWin, LINK=Lan, SERVER=Giulietta, DISPLAY= aristippo:0
================================================================
session  1 - time: 57202 Ms idle, 21199 Ms (2133 Ms in read, 19066 Ms inwrite)
session  2 - time: 48477 Ms idle, 21959 Ms (2094 Ms in read, 19865 Ms inwrite)
session  3 - time: 46202 Ms idle, 20499 Ms (2088 Ms in read, 18411 Ms inwrite)
session  4 - time: 42392 Ms idle, 23766 Ms (2078 Ms in read, 21688 Ms inwrite)
session  5 - time: 46010 Ms idle, 20635 Ms (2015 Ms in read, 18620 Ms inwrite)

X Server=WinAxe, LINK=Lan, SERVER=Giulietta, DISPLAY= aristippo:0
=================================================================
session 1 - time: 62513 Ms idle, 6611 Ms (1853 Ms in read, 4758 Ms in write)
session 2 - time: 59641 Ms idle, 6051 Ms (1851 Ms in read, 4200 Ms in write)
session 3 - time: 52859 Ms idle, 6288 Ms (1791 Ms in read, 4497 Ms in write)
session 4 - time: 57857 Ms idle, 6579 Ms (2005 Ms in read, 4574 Ms in write)
session 5 - time: 54957 Ms idle, 6595 Ms (1737 Ms in read, 4858 Ms in write)


---MARTE (Windows 2000 Server)

X Server= NXWin, LINK= Lan, SERVER= Giulietta, DISPLAY= Marte:0
===============================================================
session 1 - time: 79053 Ms idle,  9819 Ms (1442 Ms in read,  8377 Ms in write)
session 2 - time: 72145 Ms idle, 13079 Ms (1462 Ms in read, 11617 Ms in write)
session 3 - time: 39711 Ms idle, 12529 Ms (1503 Ms in read, 11026 Ms in write)
session 4 - time: 44348 Ms idle, 12496 Ms (1537 Ms in read, 10959 Ms in write)
session 5 - time: 38131 Ms idle, 12087 Ms (1498 Ms in read, 10589 Ms in write)

X Server= WinAxe, LINK= Lan, SERVER= Giulietta, DISPLAY= Marte:0
================================================================
sessione 1 - time: 51519 Ms idle, 4023 Ms (1375 Ms in read, 2648 Ms in write)
sessione 2 - time: 53825 Ms idle, 4155 Ms (1438 Ms in read, 2717 Ms in write)
sessione 3 - time: 46480 Ms idle, 4166 Ms (1396 Ms in read, 2770 Ms in write) 
sessione 4 - time: 44466 Ms idle, 4059 Ms (1410 Ms in read, 2649 Ms in write)
sessione 5 - time: 39741 Ms idle, 4079 Ms (1348 Ms in read, 2731 Ms in write)

As you can see, using the WinAxe X server the time spent in write() 
looks to be 1/4 of the time spent in write() using NXWin. Why? For sure 
NXWin causes a higher load of the machine, so the OS needs more time 
to give back control to nxproxy. Is this enough to explain the difference? 
I suppose not. I suppose that the problem is in the 100% asynchronous 
communication between the X clients, proxies and X server that is causing 
the X client side (NX server side) to provide way more data than the X 
server is able to handle. This makes the proxy's internal buffers to grow 
and causes a huge number of fragmented writes that stress the underlying 
Cygwin TCP/IP layer. This should never happen as proxy should send 
a congestion event at the remote side as soon as the X server is not 
able to accept more data. Clearly we need to report this event earlier. 
We could also try to use native Winsock to see if we get better results. 
For sure even the Cygwin based X server needs work. Unfortunately we 
didn't have time to optimize NX on Windows, so far. Most of our work 
has taken place on Linux and the NX server side. 

Here are the statistics taken on the NX server machine (the side that 
makes most of the compression job) running the previous tests. Tests 
were executed on the NoMachine public TestDrive:

---ARISTIPPO (Windows 2000 professional)

X Server=NXWin, LINK=Lan, SERVER=Giulietta, DISPLAY= aristippo:0
================================================================
session  1 - time: 58104 Ms idle, 21199 Ms (54 Ms in read, 3 Ms inwrite)
session  2 - time: 47299 Ms idle, 21959 Ms (39 Ms in read, 4 Ms inwrite)
session  3 - time: 48044 Ms idle, 20499 Ms (43 Ms in read, 1 Ms inwrite)
session  4 - time: 42001 Ms idle, 23766 Ms (41 Ms in read, 2 Ms inwrite)
session  5 - time: 45000 Ms idle, 20635 Ms (47 Ms in read, 2 Ms inwrite)

X Server=WinAxe, LINK=Lan, SERVER=Giulietta, DISPLAY= aristippo:0
=================================================================
session 1 - time: 63322 Ms idle, 6611 Ms (34 Ms in read, 1 Ms in write)
session 2 - time: 60023 Ms idle, 6051 Ms (29 Ms in read, 3 Ms in write)
session 3 - time: 53433 Ms idle, 6288 Ms (39 Ms in read, 2 Ms in write)
session 4 - time: 57123 Ms idle, 6579 Ms (32 Ms in read, 3 Ms in write)
session 5 - time: 55975 Ms idle, 6595 Ms (28 Ms in read, 2 Ms in write)


---MARTE (Windows 2000 Server)

X Server= NXWin, LINK= Lan, SERVER= Giulietta, DISPLAY= Marte:0
===============================================================
session 1 - time: 80083 Ms idle,  9819 Ms (45 Ms in read,  3 Ms in write)
session 2 - time: 74891 Ms idle, 13079 Ms (47 Ms in read, 4 Ms in write)
session 3 - time: 41028 Ms idle, 12529 Ms (53 Ms in read, 5 Ms in write)
session 4 - time: 47982 Ms idle, 12496 Ms (46 Ms in read, 2 Ms in write)
session 5 - time: 40283 Ms idle, 12087 Ms (47 Ms in read, 5 Ms in write)

X Server= WinAxe, LINK= Lan, SERVER= Giulietta, DISPLAY= Marte:0
================================================================
sessione 1 - time: 52432 Ms idle, 4023 Ms (39 Ms in read, 2 Ms in write)
sessione 2 - time: 55963 Ms idle, 4155 Ms (36 Ms in read, 2 Ms in write)
sessione 3 - time: 48973 Ms idle, 4166 Ms (28 Ms in read, 4 Ms in write) 
sessione 4 - time: 50423 Ms idle, 4059 Ms (27 Ms in read, 7 Ms in write)
sessione 5 - time: 41834 Ms idle, 4079 Ms (31 Ms in read, 3 Ms in write)

The session is constituted by a KDE startup with subsequent opening of
the K menu, Konqueror, Kcontrol, Konqueror in file-manager mode, 
OpenOffice, Mozilla, and Konsole. The test generates nearly 11 MB of
data. The average time needed by the Linux side NX proxy to perform its
job is 40 ms. I don't think we can say NX introduces too much overhead. 
Probably it's just that we need to write better Windows code :-).

> Conclusions: Again, totally unscientific but from a quick poke about
> with Task Manager, NX's ssh and proxy use a fair whack of CPU time to do
> the stream compression and pixmap re-encoding.

I suggest you give us another chance trying NX on a Linux computer :-).
Many users report that remote NX sessions run much, much faster than 
local X sessions on the same Linux hardware.

> Of course NX is a /lot/ better on sensibly-powered machines over low
> bandwidth but how many people run X over dialups anymore?

Today very few. We hope to see lot of people using X over low-bandwith
links in future, given this category includes cellular phones and Internet 
enabled PDAs.

Regards,

/Gian Filippo Pinzari.