Choosing Middleware: Why performance and scalability do (and dont) - - PowerPoint PPT Presentation
Choosing Middleware: Why performance and scalability do (and dont) matter Michi Henning Chief Scientist, ZeroC, Inc. Middleware is (almost) forever Committing to middleware represents major investment Product life cycle typically
Committing to middleware represents major investment Product life cycle typically several years to a decade long
Once you choose middleware you are stuck with it Standardization does not help
You had better exercise due diligence
Feature 5
Feature 4
Feature 3
Feature 2
Feature 1 Middleware 4 Middleware 3 Middleware 2 Middleware 1
The one with the most features must be best, right?
More features is not necessarily better Unless the middleware is very carefully designed:
Up-front licensing cost is typically 2-3% of total cost over
Development cost usually eclipses licensing cost by an
A 5% difference in programmer productivity is likely to
A 5% difference in defect rate and MTBF is certain to
Biggest fallacy in the history of middleware
How steep is the learning curve? What is the quality of the APIs?
What is the quality of the documentation?
High-quality professional training? How reliable is the software?
Quality of support?
What operating systems and compilers are
What languages are supported? Source code available? How complete is the relevant feature set? What are the licensing conditions?
Performance no longer matters when the
Middleware is no longer noticeable if:
For many applications, even bad middleware
Everyone’s first benchmark:
interface I { void ping(); };
Measure round-trip time over loopback to establish messaging rate
compared to native sockets
2,500 7,500 7,500 10,600 8,000 10,500 18,000 WCF (SOAP) WCF (bin) Ice (.NET) RMI Ice (Java) Ice (C++) Sockets 570 2,200 2,300 2,300 2,300 2,300 2,300 WCF (SOAP) WCF (bin) Ice (.NET) RMI Ice (Java) Ice (C++) Sockets
Loopback Network
Everyone’s second benchmark:
typedef sequence<byte> ByteSeq; interface I { void send(ByteSeq bs); };
Measure over loopback to establish throughput compared to native
sockets (Gbit/sec)
0.15 0.52 0.63 0.83 0.83 1.2 1.3 WCF (SOAP) WCF (bin) Ice (.NET) RMI Ice (Java) Ice (C++) Sockets 0.14 0.43 0.52 0.28 0.66 0.75 0.78 WCF (SOAP) WCF (bin) Ice (.NET) RMI Ice (Java) Ice (C++) Sockets
Loopback Network
Everyone’s forgotten benchmark:
typedef sequence<byte> ByteSeq; interface I { ByteSeq receive(); };
Receive slower than send anything between 20% and 300% The same middleware, compiled from the same source code, shows
20% difference on one OS, and 300% on another
The same middleware, compiled from the same source code, can
be faster over the network than over loopback
Data effects Type of data is important:
struct Fixed { int i; int j; double d; }; sequence<Fixed> FixedSeq; interface Throughput { void send(FixedSeq s); FixedSeq recv(); }; struct Var { string s; double d; }; sequence<Variable> VarSeq; interface Throughput { void send(VarSeq s); VarSeq recv(); };
Performance for these tests differs dramatically!
The values in the data make a difference Wire size for Fixed structures with small values versus
Small Values Ice for .NET WCF (binary) WCF (SOAP) Request size 800,057 bytes 601,538 bytes 2,750,781 bytes Random Values Ice for .NET WCF (binary) WCF (SOAP) Request size 800,057 bytes 1,167,968 bytes 3,976,802 bytes
struct Fixed { int i; int j; double d; }; sequence<Fixed> FixedSeq; interface Throughput { void send(FixedSeq s); FixedSeq recv(); }; struct Fixed { int int1Member; int int2Member; double doubleMember; }; sequence<Fixed> FixedSeq; interface Throughput { void send(FixedSeq s); FixedSeq recv(); };
Ice WCF (binary) WCF (SOAP) Request size (short names) 800,057 bytes 601,538 bytes 2,750,781 bytes Request size (long names) 800,057 bytes 600,488 bytes 3,404,862 bytes
Same middleware (Ice), same source, same test, same
Different OS: Vista versus OS X 145Mbit/s 2,500Mbit/s 740Mbit/s 2,850Mbit/s OS X 620Mbit/s 1,250Mbit/s 900Mbit/s 1,800Mbit/s Vista Java (fixed seq) Java (byte seq) C++ (fixed seq) C++ (byte seq)
Most applications do not have high performance
But: many applications have high scalability
High scalability requires high performance But:
Factors:
Stateful interaction of few servers with many
Examples:
One server, one object, one method that does
Lots of clients connected to server (one
Clients each send one message every four
Message deemed successful if it completes within
How many clients can the server support?
3GB 810MB 50MB Working set 30% 25% 15% CPU usage Ice for C++ Ice for Java RMI Requests/sec 20,000 20,000 7,500 # connections 80,000 80,000 30,000 Virtual memory 2.5GB 1.4GB 37GB
RMI fails catastrophically beyond 30,000 clients due to memory
starvation
Ice for Java continues to scale, but requests take more than 1
second due to GC passes
Ice for C++ scales well beyond 120,000 clients
Ice for .NET WCF (binary) WCF (SOAP) Requests/sec 7,250 4,850 8,000 # connections 29,000 19,500 32,000 CPU usage 33% 90% 40%
WCF binary encoder consumes an inordinate amount of CPU WCF SOAP encoder scales better than expected, but fails on the
client side
Fixed-length seq. Ice for .NET WCF (binary) WCF (SOAP) CPU (send) 1 68 84 CPU (receive) 1 62 236
CPU consumption for Fixed throughput test
CPU and bandwidth requirements make it unlikely that WCF will
scale
Server has 50Mbit/sec link to Internet Clients request 50kB of data 1Mbit/sec works out to 1kB/sec of data ⇒ Server can handle 100 concurrent clients If SOAP adds only a factor of 10 in size, the same server
You will need a 500Mbit/sec link instead ($$$) Single server will probably not handle 500Mbit/sec. You
Benchmarking is hard Creating relevant benchmarks is harder Interpreting the results is harder still Small scale results do not indicate large scale
Chances are that what you measure will be
Performance matters only if you need it Performance benchmarks matter only if they
If you don’t need to scale, any middleware will do If you need to scale in the future, SOAP or RMI
The prime criteria (other than performance):
Infrastructure services
Can you afford to build all of this yourself?
Performance is only one of many criteria But: performance does matter because, without it, things
Middleware is in a sorry state:
Watch this space—the web is only one step of many along