Evaluating Object Transactional Monitors within MEP

Ian Gorton, Anna Liu, Paul Greenfield, Phong Tran

Advanced Distributed Software Architectures and Technologies (ADSaT),

CSIRO Mathematical and Information Sciences,

Locked bag 17, North Ryde NSW 1670, Australia

email: {Ian.Gorton, Paul Greenfield, Anna.Liu, Phong.Tran}@cmis.csiro.au



As more and more object-oriented transactional processing monitors are being developed, users in the transaction processing industries need systematic and critical evaluations of these products. We describe the Middleware Evaluation Project (MEP) which aims to provide an impartial evaluation based on rigorously derived tests and benchmarks. The evaluation framework is based on TPC’s benchmark C, and OTM products are evaluated against a set of comprehensive evaluation criteria. We have already achieved interesting evaluation results for a wide range of OTM products. These evaluation results can be presented at the High Performance Transaction Systems Symposium later this year.

1. Motivation

Traditionally, transactional processing monitors exist in the realm of mainframe computers running CICS, IMS and other monolithic Transaction Processing (TP) monitors. In the last decade distributed TP monitors have demonstrated their suitability for building transaction-based system using 3-tier client-server architecture. Several distributed TP monitors such as Tuxedo, Encina, and Top-End are widely used, providing reliable transactional processing, and hiding the low-level networking details from programmers.

Recently, distributed object-oriented systems are being constructed. Systems constructed in an object-oriented manner exhibit the following features: modularity, reusability, and extensibility. The emergence of de-facto standards such as the Common Object Request Broker Architecture (CORBA) further promotes the construction of distributed object systems.

As a result, TP monitors with object-oriented support are an important new requirement for industries that require large-scale transactional processing systems, such as banking, financial services and telecommunications.

Several object-oriented transactional processing monitors have been developed. These object-oriented transactional monitors (OTMs) are new and in most cases still not widely deployed. These Object-Oriented Transaction Monitors (OTMs) include:

Systematic evaluation of the various object-oriented transaction monitors is needed to understand the strengths and weaknesses of each technology and the appropriate contexts and tasks where each product is most suitably applied. This understanding and knowledge will have important consequences on the use of existing OTMs and the engineering of scalable and reliable transactional object systems.

The Middleware Evaluation Project (MEP) is one such attempt. The MEP project aims to systematically evaluate the strength and weaknesses of existing OTM products by:

The results of this evaluation project will enable organisations to select OTMs based upon rigorously derived evidence, and hence deploying the most suitable product for applications with different requirements.

2. Evaluation framework

The MEP project references the TPC-C application published by the Transaction Processing Performance Council (TPC) as an evaluation application. TPC-C provides a standard benchmark for TP monitor systems. For the purposes of the MEP project, we have chosen the TPC-C benchmark application as a guideline example application only. We have also made major extensions. These extensions include interfaces to message queuing systems and additional DBMSs and transactions. Further, we have devised our own evaluation criteria, which have been carefully designed to more appropriately stress test object-oriented transaction monitors.

3. Evaluation criteria

A set of detailed evaluation criteria has been developed. The following headings summarise the areas we are investigating.

3.1 End product attributes

3.2 Build technologies and operational management

3.3 Run-time measures

Further, in order to test for fault handling capabilities, sample scenarios were created, e.g.:

4. Results

We have already achieved results for a wide range of OTM products. These include: Iona Technologies’s OrbixOTM, Forte, Encina++, Microsoft’s DCOM-MTS, BEA’s M3, and Inprise’s ITS. We believe these results will be of great interest to the participants of the Eighth High Performance Transaction Systems Symposium, and will be available to present these results in September.