Deep Learning using MPJ Express

Abstract

Deep Learning is the term used for describing algorithms involving Deep Neural Networks which are biologically inspired networks. Although they are easily out-performing other techniques in image recognition, they are not convenient to use because of the huge amount of computation involved. Since their performance is proved to increase with depth of additional layers, hence the name, Deep Neural Networks the computation becomes exponentially larger. Most of the deep learning tasks are being done on GPU’s these days, but GPU have limited memory which can pose as a problem for larger networks until. Our Project aims to build a system for commodity clusters that can parallelize this computation using Message Passing Framework, ‘MPJ Express’ in Java. We have used Model Parallelism to achieve this by splitting the filters and running each filter in isolation on all of the images, on different processors. The task is not embarrassingly parallel because of the fully connected layer that needs all the outputs of the filter convolutions to process into a softmax layer, that gives us the classification results. We have been able to achieve remarkable speed up’s and have consistently maintained the accuracy above 95 percent in digit recognition. Our model is scalable and fast and in the future can be used to process any kind of relevant data without any performance lags.