Evaluating Object Transactional Monitors within MEP

Evaluating Object Transactional Monitors within MEP

Ian Gorton, Anna Liu, Paul Greenfield, Phong Tran

Advanced Distributed Software Architectures and Technologies (ADSaT),

CSIRO Mathematical and Information Sciences,

Locked bag 17, North Ryde NSW 1670, Australia

email: {Ian.Gorton, Paul Greenfield, Anna.Liu, Phong.Tran}@cmis.csiro.au

Abstract
As more and more object-oriented transactional processing monitors are being developed, users in the transaction processing industries need systematic and critical evaluations of these products. We describe the Middleware Evaluation Project (MEP) which aims to provide an impartial evaluation based on rigorously derived tests and benchmarks. The evaluation framework is based on TPC’s benchmark C, and OTM products are evaluated against a set of comprehensive evaluation criteria. We have already achieved interesting evaluation results for a wide range of OTM products. These evaluation results can be presented at the High Performance Transaction Systems Symposium later this year.
1. Motivation

Traditionally, transactional processing monitors exist in the realm of mainframe computers running CICS, IMS and other monolithic Transaction Processing (TP) monitors. In the last decade distributed TP monitors have demonstrated their suitability for building transaction-based system using 3-tier client-server architecture. Several distributed TP monitors such as Tuxedo, Encina, and Top-End are widely used, providing reliable transactional processing, and hiding the low-level networking details from programmers.

Recently, distributed object-oriented systems are being constructed. Systems constructed in an object-oriented manner exhibit the following features: modularity, reusability, and extensibility. The emergence of de-facto standards such as the Common Object Request Broker Architecture (CORBA) further promotes the construction of distributed object systems.

As a result, TP monitors with object-oriented support are an important new requirement for industries that require large-scale transactional processing systems, such as banking, financial services and telecommunications.

Several object-oriented transactional processing monitors have been developed. These object-oriented transactional monitors (OTMs) are new and in most cases still not widely deployed. These Object-Oriented Transaction Monitors (OTMs) include:

CORBA-based: OrbixOTM from Iona Technologies, ITS from Inprise, M3 from BEA

DCOM-based: MTS from Microsoft

DCE-based: Encina++

Proprietary technologies: Forte

Systematic evaluation of the various object-oriented transaction monitors is needed to understand the strengths and weaknesses of each technology and the appropriate contexts and tasks where each product is most suitably applied. This understanding and knowledge will have important consequences on the use of existing OTMs and the engineering of scalable and reliable transactional object systems.

The Middleware Evaluation Project (MEP) is one such attempt. The MEP project aims to systematically evaluate the strength and weaknesses of existing OTM products by:

implementing a realistically sized example application using a number of OTMs,

using the same platforms, compilers, databases to reduce variables in the evaluation environment,

comparing and contrasting results in terms of performance, scalability, code/effort for features, etc.

The results of this evaluation project will enable organisations to select OTMs based upon rigorously derived evidence, and hence deploying the most suitable product for applications with different requirements.

2. Evaluation framework

The MEP project references the TPC-C application published by the Transaction Processing Performance Council (TPC) as an evaluation application. TPC-C provides a standard benchmark for TP monitor systems. For the purposes of the MEP project, we have chosen the TPC-C benchmark application as a guideline example application only. We have also made major extensions. These extensions include interfaces to message queuing systems and additional DBMSs and transactions. Further, we have devised our own evaluation criteria, which have been carefully designed to more appropriately stress test object-oriented transaction monitors.

3. Evaluation criteria

A set of detailed evaluation criteria has been developed. The following headings summarise the areas we are investigating.

3.1 End product attributes

Scalability

Reliability

Security

Interoperability

Performance

3.2 Build technologies and operational management

Fault management

Advanced functionality/usability

System management

Support for industry standards

3.3 Run-time measures

Time to carry out individual TPC-C transactions

Overall system throughput

Time for remote invocations

Local and remote database access time

Load balancing effectiveness

Single versus multithreaded processes

Overheads of security mechanisms

Scalability for thousands of clients

Further, in order to test for fault handling capabilities, sample scenarios were created, e.g.:

Server failure, what happens?

time to restart server

time to resolve in-flight transactions

client behaviour

Name server fails?

is failover possible?

time to failover

client behavior

4. Results

We have already achieved results for a wide range of OTM products. These include: Iona Technologies’s OrbixOTM, Forte, Encina++, Microsoft’s DCOM-MTS, BEA’s M3, and Inprise’s ITS. We believe these results will be of great interest to the participants of the Eighth High Performance Transaction Systems Symposium, and will be available to present these results in September.