sudo docker pull lyingbo/hadoop:3.2.0 git clone https://github.com/lyingbo/hadoop-cluster-docker.git cd hadoop-cluster-docker sudo ./start-container.sh output:
start hadoop-master container... start hadoop-slave1 container... start hadoop-slave2 container... root@hadoop-master:/# - start 3 containers with 1 master and 2 slaves
- you will get into the /root directory of hadoop-master container
start-all.sh output
Starting namenodes on [hadoop-master] hadoop-master: Warning: Permanently added 'hadoop-master,172.18.0.2' (ECDSA) to the list of known hosts. Starting datanodes hadoop-slave2: Warning: Permanently added 'hadoop-slave2,172.18.0.4' (ECDSA) to the list of known hosts. hadoop-slave1: Warning: Permanently added 'hadoop-slave1,172.18.0.3' (ECDSA) to the list of known hosts. Starting secondary namenodes [hadoop-master] Starting resourcemanager Starting nodemanagers run-wordcount.sh output
input file1.txt: Hello Docker input file2.txt: Hello Hadoop input file3.txt: Hello MapReduce wordcount output: Docker 1 Hadoop 1 Hello 3 MapReduce 1 do 2 like section A sudo ./resize-cluster.sh 5 - specify parameter > 1: 2, 3..
- this script just rebuild hadoop image with different workers file, which pecifies the name of all nodes
sudo ./start-container.sh 5 - use the same parameter as the step 2
do 4~5 like section A