Posts

Showing posts from June, 2017

Yarn Apache Hadoop

Image
Yarn (MapReduce 2.0 or MRv2) is a resource negotiator, which is upgraded version of MapReduce is a cluster management technology. Yarn is set up by splitting the functionalities of Resource Manager and Job Scheduling/Monitoring into different daemons. Resource Manager : - There is a single RM per cluster which manages the resources across the cluster. Node Manager : - Node Manager runs on all the nodes of the cluster. The main task of node manager is to launch and monitor the containers.              RM has two main components mainly named as Scheduler and Application Master (AM) . The main task of RM Scheduler is to allocate the resources to the jobs/applications submitted to the cluster. The RM scheduler is termed as pure scheduler as it just performs allocating the resource task and does not perform any monitoring or tracking of the jobs/applications running on the cluster. The scheduler does not offer any guarantee in restar...