Distributed Reasoning using Map Reduce

Abstract

Resource Description Framework (RDF) provides a standardized framework for semantic data representation and integration. The simplicity and flexibility offered by RDF has gotten it widely accepted in semantic web community. To support scalability and query efficiency semantic web community has already started using distributed RDF stores. Currently there exist several billion RDF annotated triples covering diverse domains which results in ever growing amount of structured data. Reasoning is vital feature of RDF after querying support. To enable reasoning support for huge data sets it becomes crucial to make use of distributed data processing approaches. Map Reduce is a distributed programming paradigm aimed at providing fair data and processing distribution, high availability and high fault tolerance. Considering RDF graph as an input, materializing the closure of this graph using a Map-Reduce implementation can address and resolve the problem of efficient distributed reasoning. The major problem solved by the approach we have used; that is reasoning over large scale RDF data. Map-Reduce provide an efficient solution of this problem. We have designed Map-Reduce jobs for all RDF-Schema entailment rules which distribute the processing of RDF data on multiple nodes to achieve scalability. Our proposed methodology starts with loading schema triples. RDF graph can be divided into A-Box (assertions or data instances) and T-Box (terminologies or schema). Since schema triples are fewer in numbers, we load them in memory for processing in Map-Reduce jobs. To avoid duplicate generation we have grouped triples on the basis of subject and then we perform join on each group. We also minimized the iterations required to compute schema graph closure by executing RDF-Schema rules in a particular order. Entailed Triples were compared with that of Jena and found to be exhaustive and correct.