catalogue
Create a hadoop password and enter it twice (the password is not displayed)
Add administrator privileges for hadoop users
Update apt (installation package management tool)
Install change profile vim (vi enhanced)
Configure SSH password less login
Modify the hadoop-env.sh file as follows:
Insert export Java in the first line_ HOME=/usr/lib/jvm/java-8-openjdk-amd64
Modify profile core-site.xml, which will
Modify to the following configuration
Similarly, modify the configuration file hdfs-site.xml, modify to the following configuration
Then run jps to view the process
These four processes must occur, but there are also accidents
1. Jps does not have a NameNode
2. The Jps does not have a DataNode
Create hadoop user
sudo useradd -m hadoop -s /bin/bash
Create a hadoop password and enter it twice (the password is not displayed)
sudo passwd hadoop
Add administrator privileges for hadoop users
sudo adduser hadoop sudo
Update apt (installation package management tool)
sudo apt-get update
Install change profile vim (vi enhanced)
sudo apt-get install vim
Install SSH server(SSH is a security protocol specially provided for remote login sessions and other network services)
sudo apt-get install openssh-server
Log in to this computer
ssh localhost
Configure SSH password less login
exit # Exit ssh localhost just now cd ~/.ssh/ # If you do not have this directory, first execute ssh localhost ssh-keygen -t rsa # There will be a prompt. Just press enter cat ./id_rsa.pub >> ./authorized_keys # Join authorization
OpenJDK:
sudo apt-get install openjdk-8-jre openjdk-8-jdk #Install jdk without downloading sudo vim ~/.bashrc #Edit configuration #export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 #Insert this paragraph in the first line source ~/.bashrc #Configuration takes effect immediately echo $JAVA_HOME #Verify environment variable values Java -version #View version number
Hadoop 2 passed http://mirror.bit.edu.cn/apache/hadoop/common/ Or http://mirrors.cnnic.cn/apache/hadoop/common/ Format: hadoop-2.x.x.tar.gz
Unzip into / usr/local
sudo tar -zxf ~/(Location of installation package)/hadoop-2.x.x.tar.gz -C /usr/local
cd /usr/local/ sudo mv ./hadoop-2.6.0/ ./hadoop # Change the folder name to hadoop sudo chown -R hadoop ./hadoop # Modify file permissions cd /usr/local/hadoop ./bin/hadoop version #Check hadoop version
Modify the hadoop-env.sh file as follows:
Insert export Java in the first line_ HOME=/usr/lib/jvm/java-8-openjdk-amd64

Then modify the core-site.xml in turn and hdfs-site.xml, directly under the directory in the previous step
Modify profile core-site.xml, which will
<configuration> </configuration>
Modify to the following configuration
<configuration> <property> <name>hadoop.tmp.dir</name> <value>file:/usr/local/hadoop/tmp</value> <description>Abase for other temporary directories.</description> </property> <property> <name>fs.defaultFS</name> <value>hdfs://localhost:9000</value> </property> </configuration>
Similarly, modify the configuration file hdfs-site.xml, modify to the following configuration
<configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:/usr/local/hadoop/tmp/dfs/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:/usr/local/hadoop/tmp/dfs/data</value> </property> </configuration>
After modifying the above configuration, next, start hadoop. It needs to be formatted for the first time, and it won't be needed in the future.
cd /usr/local/hadoop ./bin/hdfs namenode -format #Format HDFS file system in hadoop ./sbin-dfs.sh #Start hadoop dfs
Then run jps to view the process
These four processes must occur, but there are also accidents
Unexpected situation
1. Jps does not have a NameNode
./sbin/stop-dfs.sh #Stop hadoop dfs process rm -r tmp #Delete tmp cd bin/ hadoop namenode -format #Format ./sbin/start-dfs.sh #Restart hadoop
2. The Jps does not have a DataNode
./sbin/stop-dfs.sh #Stop hadoop dfs process rm -r ./tmp/ #Delete tmp ./bin/hdfs namenode -format #Format ./sbin/start-dfs.sh #Restart hadoop
After successful startup, you can access the Web interface http://localhost:50070 View NameNode and Datanode information, and view files in HDFS online.