A Survey on Dynamic Slot Configuration Techniques for Hadoop Clusters

Abstract: Hadoop is an open-source framework used for processing and handling big data in a distributed environment. MapReduce is a programming model using which we can create applications to process massive amount of data laterally, on large clusters of commodity hardware in a reliable manner. One of the major concerns in Hadoop is to minimize the makespan of a collection of MapReduce jobs. The static slot configuration technique in Hadoop may lead to long makespan and low resource utilization. To improve the performance of system dynamic slot configuration and appropriate scheduling policy must be adapted. This paper is a survey on different dynamic slot configuration techniques.

Keywords: Hadoop, MapReduce, minimum makespan, dynamic slot.