Improving MPJ Express performance by enhancing MPJ Express runtime capabilities

Abstract

Message Passing Interface (MPI) is a speci cation for building message pass- ing libraries. MPJ Express is an implementation of the open source Java message passing library, MPI Java 1.2 API. It is being used globally by thou- sands of developers to code and execute applications running in parallel for multi core processors and cloud/cluster computes. MPJ Express comprises of two main components, the core library and the runtime system. Irrespec- tive of the operating system, MPJ Express runtime provides a cohesive way of initiating Java processes on compute nodes. Multiple enhancements can be made to improve performance and scalability of the existing MPJ Express runtime system. It has a sequential boot process i.e. each node on the cluster boots up one by one and thus results in signi cantly higher boot time for large clusters and is dependent on the Java Service Wrapper Project soft- ware to start the daemons. Moreover, it can execute only one user job at a time and relies on network shared le systems that limits MPJ Express us- age in restricted environments. This research focuses on developing an MPJ Express runtime framework which supports parallel job execution and has a remarkably improved MPJ Express daemon management and provides an alternate way of class loading by deploying class les locally. It also removed Wrapper Project dependency and signi cantly improved the boot process performance of MPJ daemons. Parallel job execution allows multiple user to execute jobs simultaneously and new scenarios of MPJ Express usage have been created in restricted environments.