Shortly, I will be posting the memo below to the Super User's Group news group. All of you have expressed an interest in the results of our evaluation in one way or another, so I'm sending you the memo in the event you aren't currently a regular reader of that newgroup. I will be happy to discuss the memo's contents or related issues with you. Just give me a call.
Regards,
Michael Kearney
Introduction
------------
As you may recall, in June I posted a query here requesting people's
experiences with various MS-Windows based X Window System servers. Well, we've
completed our evaluation and have decided upon Hummingbird's eXceed/W for
our own use here in the UDC. Thanks to all of you who took the
time to respond. Special thanks go to Robert Hooper and Sue McFarland
who demo'd products for me.
Join us in an order
-------------------
We will be placing an order on Tuesday, Oct 12. If you would like to
participate in a group order, give me a call at 898-1153 before
Friday, Oct. 8. Collectively, we can probably save
about $150 off the SINGLE COPY price. Be aware though, that if you need more
than 5 copies, your savings will be less, as we need to add something to the
price to cover our cost of overhead and distribution. You may be better off
ordering directly from the vendor in this case. Their address is given at the
end of this memo. We'll be happy to handle orders of any (reasonable) size,
however. Call me a call and we can discuss the details.
The exact price will depend on the number
of copies we ultimately order, but for planning purposes figure about
$175 per license, plus a $75 handling charge per order. The handling charge
will include one copy of the documentation and diskettes.
Details of the evaluation
-------------------------
If you'd like to know why we made the choice we did,
read on...
Selecting products for evaluation
---------------------------------
Initially, we started with the following list:
eXcursion for Windows DEC
XVision/eXodus White Pine
eXceed/W Hummingbird Communications (HCL)
MicroXwin Starnet Communications
PC-XView Network Computing Devices (NCD)
Vista-eXceed CDC
Xoftware AGE Logic
One notable absence is DESQVIEW/X from Quarterdeck. Several people
recommended it. However, we decided against it as it requires Quarterdeck's
DESQVIEW window manager. We really wanted a product which runs in
the 'standard' MS-Windows environment.
Combining information provided by this group and several Usenet news groups
(comp.windows.X, comp.protocols.tcp-ip.ibmpc), we eliminated
XVision, eXcursion for Windows, and Vista-eXceed, as no one seemed to have
any opinion about them one way or the other. AGE Logic didn't return our
phone calls and so we eliminated Xoftware too, as this level of responsiveness
didn't bode well for resolving technical problems after the sale.
This left eXceed/W, PC-XView, and MicroX. At almost exactly the same
time we were doing this, NCD withdrew PC-XView from marketing and replaced it
with a product called PC-Xware. This is the product we actually evaluated.
Configuration
-------------
We ordered and installed demo versions of these products on an ethernet-attached
IBM Valuepoint 25MHZ 425SX/D running DOS 5.0 and MS-Windows 3.1.
All three products can use FTP Software's PC/TCP (among others)
for TCP/IP communications. We used release 2.2 of PC/TCP for the tests.
Test programs
-------------
We selected the following X windows client programs for testing. They are not
comprehensive, but are typical of the X based tools we use. A product
which doesn't work with these client programs is seriously suspect,
in our opinion.
x11perf - an X11 performance measurement and server functionality tool
xmosaic - NCSA's World Wide Web browser
xv - John Bradley's GIF file viewer
aixterm - RS/6000 version of xterm terminal emulator
xdvi - TeX dvi file previewer
gs - PostScript file previewer
xfig - graphical editor
Evaluation Summary
------------------
Feature eXceed/W(HCL) Xware(NCD) microX(Starnet)
--------------------------- ------------- ---------- ---------------
Runs with PC/TCP on ethernet yes yes yes
Runs with PC/TCP on SLIP (PennNet) yes not all yes
Runs with special serial yes couldn't get no
working
line protocol over PennNet Xpress Xremote none available
x11perf performance (ethernet) 1: 143/268 2: 116/268 3: 14/268
Reliability (TCP/IP ethernet or SLIP) good fair good
Reliability (special serial) fair n/a n/a
Ease of installation (TCP/IP) good good good
Ease of installation (special serial) fair fair n/a
Additional features good good fair
Price/copy, 10 copies(TCP/IP only) $196 n/a $148
Price/copy, 10 copies(TCP/IP+serial) $296 $273 n/a
Price/copy, 10 copies(serial only) $149 $138 n/a
Comments
--------
1.Runs with FTP Software's PC/TCP on ethernet
All three products worked with all the test programs listed above using
a PennNet ethernet connection and FTP Software's PC/TCP (2.2) for the
TCP/IP protocol stack.
Since PC/TCP is common on campus, we wanted a product which would use
it. In the event you have some other TCP/IP implementation
on your PC, be aware that both eXceed and Xware support a much
larger number of TCP/IP transport interfaces than MicroXwin.
2.Runs with FTP Software's PC/TCP using SLIP over PennNet
SLIP provides one way of getting TCP/IP connectivity over serial
lines. We used PennNet dialup lines for testing. microXwin
worked with SLIP right out of the box. eXceed required that we turn SLIP
vj-compression off (it took a call to Hummingbird tech support to find this
out). Xware had difficulty running the complete set of x11perf tests when
using SLIP.
We used x11perf to measure each product's performance when using SLIP.
By and large, there is little performance difference among these products
WHEN USING SLIP. The reason is that the serial communications
speed is the limiting factor - not how efficiently the X server is
implemented. See item 5 below for a discussion of performance over
ethernet where the performance differences are much more clear cut.
3.Runs with special serial line protocol over PennNet
The X windows protocol is network intensive. Using it over serial lines
is problematic, at best. eXceed/W and Xware try to make the best of the
situation by providing additional software, typically running on a UNIX
workstation, which can compress the X protocol and make better use of the
available bandwidth.
eXceed/W uses Xpress (from Tektronix), Xware uses NCD's own Xremote
protocol. microX has no feature of this sort.
We were not able to get Xware working IN THIS MODE.
The reason is that Xware will not allow
you to make an initial PennNet dialup connection
using 7 data bits, even parity (the PennNet default) and then
change it (on the fly) to an 8 bit, no parity connection which the serial
compression protocols require. We view this
as a major deficiency of the product for the Penn environment, although it's
probably something not too difficult to fix. Unfortunately, this was not
the only problem we ran into with Xware.
eXceed also exhibited this 7/8 bit problem in the version we
initially received. After a considerable amount of discussion with
Hummingbird, ordering and receiving an updated version of the product,
AND pulling two different code updates code off
their bulletin board, we were able to get Xpress working fairly
reliably.
The version we ended up with allowed us to establish a 7 bit,
even parity connection to PennNet, log in to the annex terminal server,
change the parity and bit settings, log into one of our RS/6000's, and
start up the host-based part of the protocol. Fortunately, eXceed has
a scripting language which allowed us to automate the entire process.
This is a definite plus.
Due to the difficulty we had getting eXceed/Xpress
working, we only rated it fair for reliability. At the time of this
writing all but one of the test programs (xfig) appeared to work
correctly. We are working with Hummingbird to resolve this problem.
4.x11perf performance (TCP/IP ethernet)
The x11perf program exercises various aspects of X windows servers and
measures their performance. It is not a benchmark in the strict sense
as it makes no attempt to relate the performance test to a real workload.
The program is useful in two ways. First, since it uses a lot of different
X server functions, some of them rather esoteric, it provides a good test
of a server's functionality. Second, it allows you to look at relative
server performance. The 1.3 version of the code, which we used, has 268
different performance tests. The table summarizes the results. Of the 268
tests, eXceed/W performed best on 143, Xware on 116, and microX on 14.
(Totals include ties, which is why they don't add up to 268).
The conclusion we draw from this is that microX clearly does not perform
as well as the other 2 candidates, and that eXceed/W has a noticeable,
but not overwhelming performance edge.
5.Reliability
For our testing in an TCP/IP environment (ethernet or SLIP)
we rated exceed/W and microX good, Xware fair.
Xware appears to have a number of bugs, most of the annoying variety.
For example, the Exit function often fails, requiring a re-boot if you
want to stop the X server program. Xware has trouble completing the
entire group of x11perf tests when using SLIP.
This is understandable in light of Xware's newness, but from a user
perspective is a definite minus.
exceed/W was not without problems, but the ones we found were exclusively in
the area of serial communications (Xpress and SLIP). We were able to
resolve most (as yet, not all) of
these problems with the assistance of Hummingbird's technical support.
Since we couldn't get Xware's special serial protocol (Xremote) working at
all in the PennNet environment, it's hard to rate its reliability, so we
just put 'n/a'.
6.Ease of installation
All three products were easy to install and configure, using a
'Windows-like' installation procedure. None appeared to have an
edge in this regard. This was for the TCP/IP versions however. The
'compressed serial' versions of Xware and eXceed/W (Xremote and
Xpress) were much more difficult to configure and install - requiring
a lot of hand work to get the executables installed on a UNIX workstation
(RS6000 in our case) and setting the various modem, network, and communication
parameters. For this reason, we rated the compressed serial versions
fair for installation ease.
7.Additional features
microXwin is a basic product. There's a X server and that's about it.
All three products provide XDM support, rexec, and rsh for remote
execution. eXceed is the only product we could get working with a compressed
serial protocol over PennNet.
Both eXceed/W and Xware have telnet implementations, and functions
for system management (Xware is good in the area), as well as font and
keyboard management (eXceed/W gets the nod here).
MicroXwin's documentation is extremely thin. eXceed/W and Xware are
considerably more thorough.
8.Price
If you think product feature comparisons are hard, then wait until
you try pricing! For this report, I picked the price of what would be
a typical order in OUR environment. If yours doesn't match quite
closely, IGNORE THIS SECTION. We list the per copy price for 10 copies
of the software, one manual, one diskette set, educational price, fully
supported for one year. The vendors provide a wide variety of discount
and volume purchase arrangements, many with differing support levels,
for which the price can vary significantly from what we've quoted here.
CAVEAT EMPTOR.
Other Caveats
-------------
We only tested the MS-Windows versions of these products using PC/TCP for
TCP/IP transport (ethernet and SLIP) and
special serial protocols. We did NOT test the DOS or
Windows-NT versions of any of these products, nor did we test them using
any other TCP/IP transports. We did not test any of the XDMP features of the
any of the products, nor the X11R5 font server features.
We chose eXceed/W based on the needs of the UDC for supporting its current
customers. To the extent that the product meets your needs as well, all the
better. However, this memo should not be viewed as an ISC-wide endorsement
of the product, nor as a promise of future ISC support. We simply want to let
you know what we found out, and give you an opportunity to save some money
in a joint order.
Conclusions
-----------
We picked eXceed/W for its combination of features, reliability, performance.
It's price is competitive with the others. While it was not without
problems, we felt that, on balance, it was the best choice of the products
we evaluated.
For eXceed/W, there are two choices if you want to work over serial
communications lines: eXceed/Xpress or eXceed/W over TCP/IP using SLIP.
eXceed/Xpress has the advantage of both speed and cost.
It appears to operate at about 3 times the speed of SLIP.
(the range is 1 to 10 times, depending on the function) and
costs less than the TCP/IP version to start, and doesn't require you
to shell out more money for PC/TCP or some other TCP/IP protocol stack.
However, you do lose the flexibility inherent in having TCP/IP connectivity
directly to your desktop PC when you run eXceed/Xpress.
For example, you lose the ability to do FTP to your PC while running
eXceed/Xpress. At this point, eXceed/Xpress does not appear to operate
as reliability as its TCP/IP cousin as it would not run one of our test
programs (xfig).
For our own use, we'll probably install both Xpress and the TCP/IP
versions on our UDC systems, simply to allow us the flexibility to
respond to unknown contingencies. This is probably not necessary
for general use. Please feel free to call me if you have any questions.
Vendors
-------
eXceed/W:
Hummingbird Communications Ltd.
2900 John Street, Unit 4
Markham, Ontario, Canada L3R 5G3
416-470-1203
Rep: Ken Ristevski
PX/Xware:
Network Computing Devices
PC-X Division
9590 S.W. Gemini Drive
Beavertron, Oregon 97005
503-641-2200
Rep: Bill Leineweber
Local Rep: Jack Kacala
215-940-1636
Starnet Communications
3073 Lawrence Expressway
Santa Clara, CA 95051
408-739-0881
Rep: Dick Montgomery
Subject: bat file to start mwm from x-stations
Date: Thu, 14 Oct 93 16:49:11 EDT
From: James A. Sledd (jsledd@econ.sas.upenn.edu)
Put the following lines in a file in your home directory, or in
$HOME/bin, then "chmod +x filename" to change the permissions so that
the file can be executed. Then to start mwm from gdalx06, type
"filename 6", from gdalx04, "filename 4", etc.
export DISPLAY=gdalx0${1}:0
mwm &