Header menu link for other important links
X
An approach to implement hadoop cluster for heterogeneous nodes
, A. Gaikwad, S. Akhtar, N. Choudhary
Published in International Journal of Pharmacy and Technology
2016
Volume: 8
   
Issue: 4
Pages: 25667 - 25674
Abstract
The Map Reduce Framework in Hadoop has become an important distributed processing model for large-scale dataintensive applications. Hadoop is widely used for the projects requiring low response time. Computing nodes in the Hadoop (clusters) seems to be homogeneous throughout nature. As transferring a large amount of data leads to excessive network congestions, which in turn can deteriorate system performance. Without considering the network delays-the performance of Hadoop clusters could be significantly downgraded. This paper proposes new strategies in context to the data locality issues in the heterogeneous environments. We will also deal with the problem of placing unstructured data across nodes in such a way that each node has a balanced data processing load. A shuffle-intensive application running on a Hadoop Map Reduce cluster, our data placement scheme adaptively balances the amount of data stored in each node to achieve improved data-processing performance. After that, it would be experimental results on two real shuffleintensive applications show that our results placement and shuffle strategy can always improve the Map Reduce performance by rebalancing data across the nodes before performing a shuffle-intensive application in a different Hadoop cluster. © 2016, International Journal of Pharmacy and Technology. All rights reserved.
About the journal
JournalInternational Journal of Pharmacy and Technology
PublisherInternational Journal of Pharmacy and Technology
ISSN0975766X