Wednesday 17 March 2021

Setup of Map Reduce Program for WordCount in Windows10

Java Download: https://www.oracle.com/in/java/technologies/javase/javase-jdk8-downloads.html

Maven Download: https://maven.apache.org/download.cgi

Maven installation: https://maven.apache.org/install.html

Eclipse download and installation: https://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/2020-12/R/eclipse-java-2020-12-R-win32-x86_64.zip

Hadoop Download:https://hadoop.apache.org/release/3.2.1.html

winutils: https://github.com/cdarlint/winutils/blob/master/hadoop-3.2.1/bin/winutils.exe. Download and copy to bin folder under hadoop

System Variables setup:

JAVA_HOME: C:\Program Files\Java\jdk1.8.0_281

HADOOP_HOME: C:\Program Files\hadoop\hadoop-3.2.1

Path: %PATH%;%JAVA_HOME%\bin;C:\Program Files\apache-maven-3.6.3\bin;%HADOOP_HOME%\sbin;

Map Reduce Code Setup:

Map Reduce Code: https://github.com/anjuprasannan/MapReduceExample

checkout the code from Git as:

git clone https://github.com/anjuprasannan/MapReduceExample.git

Import the project to Eclipse

Edit the input, output and hadoop home directory locations in "/MapReduceExample/src/com/anjus/mapreduceexample/WordCount.java" [Note that the output location should be a non existant directory.]

Build the project:

Right click project -> Maven Clean

Right click project -> Maven Install

Job Execution:

Right Click on "/MapReduceExample/src/com/anjus/mapreduceexample/WordCount.java" -> Run As -> Java Application



"Don't be afraid to give up the good to go for the great."





No comments:

Post a Comment