Vellore Institute of Technology (VIT) is a private university located in&nbsp;Tamil Nadu, India. Founded in 1984, as Vellore Engineering College, the institution offers 20 undergraduate, 34 postgraduate, four integrated and four research programs. It has campuses in Vellore, Amravati, Bhopal and Chennai.

VIT is one of the top ranked private universities in India according to NIRF, THE and QS Rankings.&nbsp;Govt. of India has recognized&nbsp;VIT, Vellore as an&nbsp;Institution of Eminence. This has allowed VIT to take independent quality initiatives and move up in world ranking.

&nbsp;

&nbsp;

VIT University

Data is increasing rapidly day by day along with the transactional database. Dividing this data and storing it in a distributed manner is an effective way for storage and retrieval. Mining such distributed data with minimum dependence between sub-problems is a crucial task. Finding frequent itemsets and corresponding association rules is a big challenge while considering the aggregation in a distributed environment. To overcome these challenges, we propose a distributed frequent itemset generation and association rule mining algorithm using MapReduce programming model.The proposed scheme generates frequent itemset and mine association rules using a synthesized distributed technique. The rules are mined in a distributed manner, and then weights are assigned to subsets of data and association rules. A proper mixture of association rules that are generated in distributed manner is done using a weighted approach. This paper presents a novel MapReduce-based synthesis approach, which can work well over a distributed storage of large amount of data. © 2020, Indian Academy of Sciences.

School of Computer Science and Engineering

Chennai Campus

World University root node

Sadhana - Academy Proceedings in Engineering Sciences

Distributed synthesized association mining for big transactional data

Fault prediction problems in network systems are often manifested as very onerous for better network management. One of the effective measures is to constantly monitor and analyze the unceasing generation of network logs that capture the activities of a network. The learning algorithms are quite useful for this purpose. However, due to the dynamic nature of network systems, a frequent drift in the logged data may occur which in turn affects the efficiency of the learning algorithms. In this paper, we present a general purpose algorithmic framework for developing easily parallelizable distributed log mining approach, which uses machine learning and distributed processing to achieve a better quality of network services. Our proposed approach monotonously handles the dynamic nature of network logs by tracking the changes in the distribution of logs and takes adequate actions according to that. The entire problem is illustrated as a distributed learning environment, where the complete set of logs is partitioned into assorted data chunks and a distributed weighted ensemble of the information is generated from these chunks. Furthermore, our method is tested on real dataset and experimental analysis shows that a fair amount of scalability and accuracy can be obtained. © 2007-2012 IEEE.

IEEE Systems Journal

DLME: Distributed Log Mining Using Ensemble Learning for Fault Prediction

Handling vast amount of data found in large transactional datasets is an obvious challenge for the conventional data mining algorithms. Addressing this challenge, our paper proposes a parallel approach for proper decomposition of mining problem into sub-problems in order to find frequent patterns from these datasets. The proposed, Pattern Mining for Large Distributed Dataset (PMLDD) approach, ensures minimum dependencies as well as minimum communications among sub-problems. It establishes a linear aggregation of the intermediate results so that it can be adapted to large-scale programming models like MapReduce. In this context, an algorithmic structure for MapReduce programming model is presented. PMLDD guarantees an efficient load balancing among the sub-problems by a specific selection criterion. Further, it optimizes the number of required iterations over the dataset for mining frequent patterns as compared to the existing approaches. Finally, we believe that our approach is scalable enough to handle larger datasets in terms of performance evaluation, and the result analysis justifies all these mentioned concerns. © 2018 KSII.

KSII Transactions on Internet and Information Systems

Pattern mining for large distributed dataset: A parallel approach (PMLDD)

Continuous use of mechanical systems requires precise maintenance. Automatic monitoring of such systems generates a large amount of data which require intelligent mining methods for processing and information extraction. The problem is to predict the faults generated with ball bearing which severely degrade operating conditions of machinery. We develop a distributed fault prediction model based on big data intelligence that extracts nine essential features from ball bearing dataset through distributed random forest. We also perform a rigorous simulation analysis of the proposed approach and the results ensure the accuracy/correctness of the method. Different types of fault classes are considered for prediction purpose and classification is done in a supervised distributed environment. © Springer Nature Switzerland AG 2018.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Applying big data intelligence for real time machine fault prediction

Operations management is a field of management which emphasizes on managing the day to day operations of business organizations. These organizations possess a huge amount of data which needs to be analysed for proper functioning of business. This large amount of data keeps some useful information hidden inside it, which needs to be uncovered. This information can be retrieved using predictive analytics techniques, which predict the patterns hidden inside the data. This data is heterogeneous, processing of such huge amount of data creates challenges for the existing technologies. MapReduce is very efficient in processing this huge amount of data. In the field of operation management, data needs to be processed efficiently, so it is highly required to process data using parallel computing framework due to its large size. This chapter covers different techniques of predictive analytics based on MapReduce framework which helps in implementing the techniques on a parallel framework. © 2017, IGI Global. All rights reserved.

Decision Management: Concepts, Methodologies, Tools, and Applications

Predictive analytics in operations management

Amount of data generated by IoT devices is increasing day by day, it requires efficient and scalable infrastructure to store and process this data. Distributed process is an idle tool for analyzing this amount of data. Pattern mining of the IoT device events can result in better decision making in IoT eco-system. In this paper we have proposed a integrated environment for distributed data collection using a Mapreduce based technique exploiting distributed processing of the data. The proposed approach uses an Apriori and Mapreduce based algorithm for analyzing the data. Available dataset and synthetic datasets has been used for analysis and scalability test of the proposed approach. The approach shows a good amount of scalabity for mining the frequent patterns in event based sensor data. © 2017 ACM.

ACM International Conference Proceeding Series

Pattern generation from event oriented sensor data using distributed sensor transaction model

Operations management is a field of management which emphasizes on managing the day to day operations of business organizations. These organizations possess a huge amount of data which needs to be analysed for proper functioning of business. This large amount of data keeps some useful information hidden inside it, which needs to be uncovered. This information can be retrieved using predictive analytics techniques, which predict the patterns hidden inside the data. This data is heterogeneous, processing of such huge amount of data creates challenges for the existing technologies. MapReduce is very efficient in processing this huge amount of data. In the field of operation management, data needs to be processed efficiently, so it is highly required to process data using parallel computing framework due to its large size. This chapter covers different techniques of predictive analytics based on MapReduce framework which helps in implementing the techniques on a parallel framework. © 2017 by IGI Global. All rights reserved.

Applied Big Data Analytics in Operations Management

Size of data is increasing, it is creating challenges for its processing and storage. There are cluster based techniques available for storage and processing of this huge amount of data. Map Reduce provides an effective programming framework for developing distributed program for performing tasks which results in terms of key value pair. Collaborative filtering is the process of performing recommendation based on the previous rating of the user for a particular item or service. There are challenges while implementing collaborative filtering techniques using these distributed models. Some techniques are available for implementing collaborative filtering techniques using these models. Cluster based collaborative filtering, map reduce based collaborative filtering are some of these techniques. Chapter addresses these techniques and some basics of collaborative filtering. © 2017, IGI Global.

Collaborative Filtering Using Data Mining and Analysis

Collaborative filtering based data mining for large data

When the amount of data is very large and it cannot be handled by the conventional database management system, then it is called big data. Big data is creating new challenges for the data analyst. There can be three forms of data, structured form, unstructured form and semi structured form. Most of the part of bigdata is in unstructured form. Unstructured data is difficult to handle. The Apache Hadoop project provides better tools and techniques to handle this huge amount of data. A Hadoop distributed file system for storage and the MapReduce techniques for processing this data can be used. In this paper, we presented our experimental work done on big data using the Hadoop distributed file system and the MapReduce. We have analyzed the variable like amount of time spend by the maps and the reduce, different memory usages by the Mappers and the reducers. We have analyzed these variables for storage and processing of the data on a Hadoop cluster. © 2014 IEEE.

1st International Conference on Networks and Soft Computing, ICNSC 2014 - Proceedings

An experimental approach towards big data for analyzing memory utilization on a hadoop cluster using HDFS and MapReduce

Data when it becomes in that much amount that it cannot be managed by the traditional database management system then it is Big data. It is difficult to manage this much amount of the data. Hadoop is a technological answer to the Big Data. Data storage and retrieval of information from the data is done by the Hadoop Distributed File System and the Map Reduce Programming model. MapReduce provides effective bench marks for retrieving the information from the Big Data. In this paper we present our experimental work done on the Hadoop Cluster. We have analyzed the time required by the cluster for processing the data with increasing number of nodes into the cluster. We started with a single node and then increase the node by one each time. We have analyzed three types of time the real time, user time, system time is analyzed. © 2014 IEEE.

Proceedings - 2014 6th International Conference on Computational Intelligence and Communication Networks, CICN 2014

A time based analysis of data processing on hadoop cluster

Big Data is a huge amount of data that cannot be managed by the traditional data management system. Hadoop is a technological answer to Big Data. Hadoop Distributed File System (HDFS) and MapReduce programming model is used for storage and retrieval of the big data. The Tera Bytes size file can be easily stored on the HDFS and can be analyzed with MapReduce. This paper provides introduction to Hadoop HDFS and MapReduce for storing large number of files and retrieve information from these files. In this paper we present our experimental work done on Hadoop by applying a number of files as input to the system and then analyzing the performance of the Hadoop system. We have studied the amount of bytes written and read by the system and by the MapReduce. We have analyzed the behavior of the map method and the reduce method with increasing number of files and the amount of bytes written and read by these tasks. © 2014 IEEE.