Abstract:
MPJ Express is a Java messaging system that implements an MPI‐like interface. It is
used for writing parallel Java applications on High Performance Computing (HPC)
hardware including commodity clusters. The software is capable of executing in
multicore and cluster mode. In the cluster mode, it currently supports Ethernet and
Myrinet based interconnects and provide specialized communication devices for these
networks. One recent trend in distributed memory parallel hardware is the emergence
of InfiniBand interconnect, which is a high‐performance proprietary network and
provides low latency and high bandwidth for parallel MPI applications. Currently
there is no direct support available in Java (and hence MPJ Express) to exploit the
performance benefits of InfiniBand networks. The only option to run distributed Java
programs over InfiniBand networks is to rely on TCP/IP emulation layers like IP over
InfiniBand (IPoIB) and Sockets Direct Protocol (SDP), which provide poor
communication performance. To tackle this issue in the context of MPJ Express, this
paper presents a low‐level communication device called ibdev that can be used to
execute parallel Java applications on InfiniBand clusters. MPJ Express is based on a
layered architecture and hence users can opt to use ibdev at runtime on an InfiniBand
equipped commodity cluster. ibdev improves Java application performance with
access to InfiniBand hardware using native verbs API. Our performance evaluation
reveals that MPJ Express achieves much better latency and bandwidth using this new
device, compared to IPoIB and SDP. Improvement in communication performance is
also evident in NAS parallel benchmark results where ibdev helps MPJ Express achieve
better scalability and speedups as compared to IPoIB and SDP. The results show that
it is possible to reduce the performance gap between Java and native languages with
efficient support for low level communication libraries.
