Back on Hadoop Again

Finally getting back on Hadoop again after so many months.

This time round, I am going to start afresh with Fedora 11 and Hadoop 0.20.2.

The first step was to set up JAVA.

To configure JAVA_HOME environment variable for all users in Fedora 11, as follows:

For sh/bash/ksh/zsh users, create a new .sh file in /etc/profile.d/:

su - -c 'gedit /etc/profile.d/local.sh'

Append the system-wide environment variables to the file. The variable assignments are of the form:

VARIABLE1=value
VARIABLE2=value
export VARIABLE1 VARIABLE2

Something of particular interest is the passphrase-less ssh in Fedora 11. If you follow the Hadoop quick start guide using ssh-keygen in a X Window environment, you will find that it does not work until you logout of and login again. So, the solution is simply logout and login to get your passphrase-less ssh working.

The rest are as simple as laid out in the Hadoop quick start guide.