Hadoop cluster on Virtual box with Docker

Mac OSx

Prerequisites checklist

• Virtual box
• Boot2docker
• Docker-Compose

Setup:

1. mkdir hadoopUpAndRunning
2. cd hadoopUpAndRunning
3. create docker-compose.yml file
4. copy content from README page of [This docker image](https://registry.hub.docker.com/u/badele/debian-hadoop/)
5. boot2docker init
6. boot2docker up

Init HDFS

docker-compose run namenode hdfs namenode -format

Start hadoop cluster

docker-compose up -d

Check port numbers and ip address

$ docker ps
$ boot2docker ip

Access hadoop web-ui:

http://{boot2docker ip-address}:{port number of namenode}/
sample: http://192.168.59.103:50070

Access hadoop fs:

hadoop fs -ls hdfs://{boot2docker ip-addess}/