Article

Installation of JDK, Hadoop, Hbase and GUI on Linux Server (Ubuntu 18.04)

Article Directory 0. Preparations 1. Get Ali Cloud ECS Server 2. Create Hadoop users(viii) 3. Configure SSH passwordles...
0. Preparations
1. Get Ali Cloud ECS Server
2. Create Hadoop users(viii)
3. Configure SSH passwordless login
4. Install JDK
5. Graphical interface
6. Install Hadoop_
7. Install HBase
8. References

Article Directory

0. Preparations

First of all, you need to prepare the following (VNC Viewer is perfect for little novices like me who can't see the GUI and panic):

When choosing JDK, Hadoop, HBase versions, be aware of their compatibility

Official Documents (Search for content in the document: Ctrl + F)

1. Get Ali Cloud ECS Server

System: Ubuntu 18.04

During epidemic prevention and control(viii), available to students A 6-Month Free Aliyun ECS server

Acquire conditions:

  • Account is authenticated by real name & student certification
  • Complete a test of 10 multiple choice questions (more than 6 correct answers)

2. Create Hadoop users(viii)

# Create a hadoop user and use/bin/bash as a shell sudo useradd -m hadoop -s /bin/bash # Set password for hadoop user, enter password twice sudo passwd hadoop # Administrator privileges for hadoop users to facilitate deployment sudo adduser hadoop sudo # Switch current user to hadoop user su - hadoop # Update apt of hadoop users to facilitate subsequent software installation sudo apt-get update

3. Configure SSH passwordless login

# 1. Server-side Installation sudo apt-get install openssh-server # Client Installation # Ubuntu has SSH client installed by default # sudo apt-get install openssh-client ------------------------ # 2. Enter the host directory, generate the local secret key and set up the secret-free login # >> Note that instead of using the root user to generate the secret key, you want to use the user you want to set << < here cd /home/hadoop/ ssh-keygen -t rsa -P "" # 2.1 Append public keys to authorized_keys file cat .ssh/id_rsa.pub >> .ssh/authorized_keys # 2.2 Give authorized_keys file permissions chmod 600 .ssh/authorized_keys ------------------------ # 3. Verify success ssh localhost

4. Install JDK

4.1 Method 1:_Manual installation (recommended)_)

4.1.1 Process
  1. Computer Download Linux_JDK8
  2. Use WinSCP Upload the compressed package to the server
  3. Unzip and install JDK8
4.1.2 Download Linux JDK8 on PC

Get Linux_JDK8 (Download requires login to Oracle account)

4.1.3 Use WinSCP to upload compressed packages to the server

Get WinSCP

  1. Fill in server information
  1. Connect to Server
  1. Upload Compressed Packet
4.1.4 Unzip and install JDK8
# 1. Create a directory sudo mkdir /usr/lib/jvm ------------------------ # 2. Unzip the JDK into the directory # >> Note that the file names of the following commands match their own JDK package names < << sudo tar -zxvf jdk-8u241-linux-x64.tar.gz -C /usr/lib/jvm ------------------------ # 3. Modify environment variables # 3.1 Edit.bashrc # Related actions:'i': Enter edit mode,'esc': Exit edit mode,': wq': Save and exit sudo vi ~/.bashrc # 3.2 Add the following at the end of the file # >> Note that the catalog and JDK are consistent with what you created < < < < < < < < < < < < < < < < < < < < < < < export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_241 export JRE_HOME=$/jre export CLASSPATH=.:$/lib:$/lib export PATH=$/bin:$PATH # 3.3 Make environment variables take effect immediately source ~/.bashrc ------------------------ # 4. The system registers this JDK # >> Note that the JDK is consistent with what you've created < < < < <. sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.8.0_241/bin/java 300 ------------------------ # 5. Check to see if the installation was successful java -version

4.2 Method 2: Command line installation (not recommended!_)

I see many JDK command line installation tutorials written like this, but I did not install JDK in this way until the end...😭😤

# 1. Installation Dependency sudo apt-get install python-software-properties sudo add-apt-repository ppa:webupd8team/java sudo apt-get update # 2. Install jdk1.8 sudo apt-get install oracle-java8-installer
4.2.1 Problems encountered during installation

  1. E: Package 'python-software-properties' has no installation candidate

    # >> Solutions < <* # System >= Ubuntu 12.10 Use'software-properties-common'instead of'python-software-properties' sudo apt-get install software-properties-common

  2. add-apt-repository command not found

    # >> Solutions < <* # Install software-properties-common apt-get install software-properties-common

  3. E: Package 'oracle-java8-installer' has no installation candidate

    # >> Trying any of the following will not work < < < # 1. Add dependencies # 2. Update apt-get # 3. Replace apt-get with aptitude # 4. sudo update-alternatives --config java # This method is used for switching between multiple versions of jdk # >> Solutions < <* # 1. Manual Installation # (See Method 2: Manual Installation for details) # 2. Replace with OpenJDK # 2.1 Update package list sudo apt-get update # 2.2 Install openjdk sudo apt-get install openjdk-8-jdk # 2.3 Check to see if the installation was successful java -version

5. Graphical interface

5.1 Process

  1. Server opens ports used by VNC
  2. Server Installation Ubuntu Desktop and VNC Server
  3. Client Installation VNC Viewer
  4. Connect

5.2 Server opens ports used by VNC

  1. Enter Security Group
  1. Add Security Group Rule

âš Note: If the port is not open, the following prompts will appear when connecting with the VNC Viewer:

5.3 Server Install Ubuntu Desktop and VNC Server

# 1. Update apt sudo apt-get update && sudo apt-get upgrade -y ------------------------ # 2. Install desktop # During installation, enter'y'and return if prompted to change the system file to a new version # The installation process lasts approximately 10 minutes sudo apt-get install ubuntu-desktop gnome-panel gnome-settings-daemon metacity nautilus gnome-terminal ------------------------ # 3. Install VNC Server sudo apt-get install vnc4server ------------------------ # 4. Start VNC Server and create a session # >> Note that there is a space between'vncserver'and':' < < # Next you need to set the password vncserver :1 ------------------------ # 5. Close Session vncserver -kill :1 ------------------------ # 6. Edit ~/.vnc/xstartup # This file will not be created until VNC Sever is first started nano ~/.vnc/xstartup # > nano part operation:'ctrl + o'save,'Enter' confirm,'ctrl + x'exit < < <
# 7. Change the file to: #!/bin/sh # Uncomment the following two lines for normal desktop: # unset SESSION_MANAGER # exec /etc/X11/xinit/xinitrc [ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup [ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources xsetroot -solid grey vncconfig -iconic & x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" & x-window-manager & gnome-panel & gnome-settings-daemon & metacity & nautilus &
# 8. Open VNC Server again vncserver :1

5.4 Client Installation VNC Viewer (omitted)

Get VNC Viewer

5.5 Connection(vii)

  1. Enter Server IP +:1, Enter
  1. Enter the password (password set after vncserver:1 was first entered on the command line) and click OK
  1. _Great success!(viii)
Test the network again~

6. Install Hadoop_

6.1 Process

  1. Computer Download Hadoop 2.7.1 (omitted)
  2. Upload Hadoop compressed packages to a Linux server using WinSCP (same as 4.1.3, omitted)
  3. Unzip and install Hadoop 2.7.1

6.2 Unzip and install Hadoop 2.7.1

# 1. Decompression # 1.1 Unzip to/usr/local directory sudo tar -zxvf hadoop-2.7.1.tar.gz -C /usr/local # 1.2 Enter / usr/local directory cd /usr/local # 1.3 Renamed hadoop sudo mv hadoop-2.7.1 hadoop # 1.4 Modify file permissions sudo chown -R hadoop ./hadoop ------------------------ # 2. Configuring environment variables # 2.1 Edit.bashrc vim ~/.bashrc # 2.2 Add the following # Hadoop Environment export HADOOP_HOME=/usr/local/hadoop export CLASSPATH=$($HADOOP_HOME/bin/hadoop classpath):$CLASSPATH export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin # 2.3 Make environment variables take effect immediately source ~/.bashrc # 2.4 Check to see if the installation was successful hadoop version
# 3. Add jdk path in hadoop-env.sh # 3.1 Enter the / usr/local/hadoop/etc/hadoop directory cd /usr/local/hadoop/etc/hadoop # 3.2 Edit hadoop-env.sh vim hadoop-env.sh # 3.3 Modify the JAVA_HOME value # Before modification: export JAVA_HOME=$ export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_241
# 4. Still in the / usr/local/hadoop/etc/hadoop directory, modify both profiles # 4.1 Modify the core-site.xml file # 4.1.1 Edit core-site.xml vim core-site.xml # 4.1.2 Add the following configuration <configuration> <property> <name>hadoop.tmp.dir</name> <value>file:/usr/local/hadoop/tmp</value> <description>Abase for other temporary directories.</description> </property> <property> <name>fs.defaultFS</name> <value>hdfs://localhost:9000</value> </property> </configuration> ---------------- # 4.2 Modify the hdfs-site.xml file # 4.2.1 Edit core-site.xml vim hdfs-site.xml # 4.2.2 Add the following configuration <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:/usr/local/hadoop/tmp/dfs/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:/usr/local/hadoop/tmp/dfs/data</value> </property> </configuration> ------------------------ # 5. Start hadoop # 5.1 Enter/usr/local/hadoop cd /usr/local/hadoop # 5.2 Perform NameNode Formatting ./bin/hdfs namenode –format # 5.3 Start Namenode and Datanode processes ./sbin/start-dfs.sh # 5.4 View startup results jps

A solution without NameNode process under 6.3 jps_

# 1. Turn off dfs stop-dfs.sh # 2. Format namenode hadoop namenode -format # 3. Restart Services start-all.sh

7. Install HBase

7.1 Process

  1. Computer Download HBase 1.6.0 (omitted)
  2. Uploading HBase compressed packages to a Linux server using WinSCP (same as 4.1.3, omitted)
  3. Unzip and install HBase 1.6.0

7.2 Unzip and install HBase 1.6.0

# 1. Decompression # 1.1 Unzip to/usr/local directory sudo tar -zxf hbase-1.6.0-bin.tar.gz -C/usr/local # 1.2 Enter / usr/local directory cd /usr/local # 1.3 Rename mv ./hbase-1.6.0 ./hbase # 1.4 Modify permissions sudo chown -R hadoop:hadoop ./hbase ------------------------ # 2. Configure command line environment variables # 2.1 Edit/etc/profile vim /etc/profile # 2.2 Add environment variables export HBASE_HOME=/usr/local/hbase export PATH=$HBASE_HOME/bin:$PATH
# 3. Modify hbase's configuration file/conf/hbase-env.sh # 3.1 Edit hbase-env.sh vim /usr/local/hbase/conf/hbase-env.sh # 3.2 Add the following export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_241 export HBASE_MANAGES_ZK=true
# 4. Edit the.Xml configuration file conf/hbase-site.xml # 4.1 Edit hbase-site.xml vim /usr/local/hbase/conf/hbase-site.xml # 4.2 Add the following <configuration> <property> <name>hbase.rootdir</name> <value>hdfs://localhost:9000/hbase</value> <description>The location where the data is stored.</description> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>localhost</value> </property> <property> <name>dfs.replication</name> <value>1</value> <description>The number of copies specified is 1 because of pseudo-distributed.</description> </property> </configuration>
# 5. Start HBase # >> Before starting HBase, make sure Hadoop is functioning properly <<< # >> Command line input jps, showing 4 Java processes <<<<* # >> jps, NameNode, DataNode, SecondaryNameNode <<< respectively start-hbase.sh # After starting HBase, executing jps again adds three new Java processes # HMaster, HRegionServer, HQuorumPeer

The browser accesses localhost:16010 and enters the Web administration page

âš NOTE: Ports prior to HBASE1.0 are 60010

The HBase database on the Ubuntu Linux server is now installed!___

8. References

Linda Evening Fifteen original articles have been published. 6. Visits 388 Private letter follow

17 April 2020, 02:57 | Views: 5696

Add new comment

For adding a comment, please log in
or create account

0 comments