Apache Hadoop 2.5.1 on Docker Janos Matyas 15 September 2014

Following the release cycle of Hadoop, today we are releasing a new 2.5.1 version of our Hadoop Docker container. Up until today the container was only CentOS based, but during the last few months we got lots of requests to release a Hadoop container on Ubuntu as well. From now on we will have both released, supported and published to the official Docker repository. Enjoy.

Centos

Build the image

In case you’d like to try directly from the Dockerfile you can build the image as:

1
docker build  -t sequenceiq/hadoop-docker:2.5.1 .

Pull the image

As it is also released as an official Docker image from Docker’s automated build repository – you can always pull or refer the image when launching containers.

1
docker pull sequenceiq/hadoop-docker:2.5.1

Start a container

In order to use the Docker image you have just build or pulled use:

1
docker run -i -t sequenceiq/hadoop-docker:2.5.1 /etc/bootstrap.sh -bash

Ubuntu

Build the image

In case you’d like to try directly from the Dockerfile you can build the image as:

1
docker build  -t sequenceiq/hadoop-ubuntu:2.5.1 .

Pull the image

As it is also released as an official Docker image from Docker’s automated build repository – you can always pull or refer the image when launching containers.

1
docker pull sequenceiq/hadoop-ubuntu:2.5.1

Start a container

In order to use the Docker image you have just build or pulled use:

1
docker run -i -t sequenceiq/hadoop-ubuntu:2.5.1 /etc/bootstrap.sh -bash

Testing

You can run one of the stock examples:

1
2
3
4
5
6
cd $HADOOP_PREFIX
# run the mapreduce
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.1.jar grep input output 'dfs[a-z.]+'

# check the output
bin/hdfs dfs -cat output/*

Hadoop native libraries, build, Bintray, etc

The Hadoop build process is no easy task – requires lots of libraries and their right version, protobuf, etc and takes some time – we have simplified all these, made the build and released a 64b version of Hadoop nativelibs on our Bintray repo. Enjoy.

Should you have any questions let us know through our social channels as LinkedIn, Twitter or Facebook.

Comments

Recent Posts