python automated management sshy (ssh, ssh-copy-id, ssh-agent)

Introduction to Python Automation Management sshy

ssh Advantage:

  • Secure file transfer
  • Sign in
  • Batch command execution

For an engineer who is new to Linux system management, the steps he sees in system management may be to log on to the server using SSH, modify the application-related configuration files, execute some Linux commands, restart the process, and exit the server.If you have more servers, repeat the process.

These steps are the basics of Linux system management and the basic work of system management.However, in practice, you will not manually pair
Servers operate, but use programs to automate management.Even if the number of servers is small, it is recommended that you write programs for automation.Automated management has many advantages over manual server management.For example:

1) High efficiency:

Automatic operation is more efficient than manual operation.The efficiency here can be understood from two aspects: on the one hand, the efficiency of program execution is more efficient than manual operation; on the other hand, for engineers, using the program can improve their own work efficiency, reduce unnecessary time waste.Even if there is only one server, manual operations can be done quickly, but they are not as efficient as programs.If you are managing a server cluster, it is clear that manual operations are unrealistic, inefficient, tedious and time-consuming.The advantage of a program is that it is written once and run many times.Although writing a program may take more time than a single manual operation, as long as the program is complete, it can be run repeatedly, saving a lot of time.

2) Not prone to error:

As the saying goes, "No man is perfect", if you have been using manual management to manage your server cluster, errors are inevitable.Engineers may experience mood changes, health problems, etc., but the procedure does not.As long as the program is written and exceptions are taken into account, the program can always perform administrative operations strictly and consistently.

3) Enjoy:

One natural benefit of working in the computer industry is that you don't have to do repetitive work.Any repetitive work can be eliminated by writing a program.Eliminating repetitive work not only saves working time, but also increases the sense of fun and achievement.Taking the management of server clusters as an example, does it feel like pointing at a mountain or scolding at a party when you see the programs you write and commanding hundreds or thousands of servers to operate according to the specified requirements?

This chapter will discuss how to use Python to batch manage servers.First, we will introduce the basics of batch management servers, the SSH protocol; then, we will introduce a batch tool written by Python; then we will describe how to operate remote servers in Python programs; and at the end of this chapter, we will introduce a very powerful system management tool, Fabric, which is the focus and difficulty of this chapter.

1. Accessing remote servers using SSH protocol

SSH (Secure Shell) is a security protocol developed by the Internet Working Group of IETF and created on the basis of application and transport layers to provide a secure transport and use environment for Shells on computers.

1. SSH protocol

In the early days of the Internet, communication was clear text, such as rsh, FTP, POP, and Telnet.Once a communication message is intercepted, the content will undoubtedly leak.In 1995, Finnish scholar Tatu Ylonen designed the SSH protocol.Encrypting all login information is a basic solution for Internet security.This scheme has been rapidly promoted all over the world and has now become the standard companion for Linux systems.

SSH is just a protocol with multiple implementations, both commercial and open source.Currently.OpenSSH, widely used under Linux, is a widely used open source software.The paramiko described in this article is a Python implementation of the SSH protocol.

In addition to providing secure transport and login, SSH can also perform batch command execution, which is very convenient to use.Due to the simplicity and ease of use of SSH, several tools described in this chapter, as well as the ASIBLE that will be introduced later, rely on SSH for remote server management.The benefits of using SSH are obvious, making full use of existing mechanisms and eliminating the need to install agent programs on remote servers.As a result, many automators rely on SSH.

2. OpenSSH implementation

OpenSSH (OpenBSD Secure Shell) is a subproject of OpenBSD and an open source implementation of the SSH protocol.On the server side, OpenSSH starts the sshd daemon, which listens on port 22 by default.Clients connect to the server using a username and password. After successful connection, OpenSSH returns a shell to the user, who can use the shell to execute commands on the remote server.

On Debian systems, OpenSSH's server reads the configuration in/etc/ssh/sshd_config by default.In production environments, to prevent ******, the default port number of the SSH service is generally modified, which is done in/etc/ssh/sshd_config.We can also use this profile to prevent users from using passwords for authentication, only keys for authentication.After modifying the configuration file, restart the daemon of OpenSSH by executing the following command:

For an engineer who is new to Linux system management, the steps he sees in system management may be to log on to the server using SSH, modify the application-related configuration files, execute some Linux commands, restart the process, and exit the server.If you have more servers, repeat the process.

These steps are the basics of Linux system management and the basic work of system management.However, in practice, you will not manually pair
Servers operate, but use programs to automate management.Even if the number of servers is small, it is recommended that you write programs for automation.Automated management has many advantages over manual server management.For example:

/etc/init.d/ssh restart

The client of OpenSSH is an executable named ssh, and we can use the SSH command to connect to the remote server.As follows:

[root@python ~]# ssh xgp@localhost 
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is bf:c7:de:84:e1:28:f4:d4:7e:41:49:9f:54:dc:e9:83.
Are you sure you want to continue connecting (yes/no)? yes
xgp@localhost's password: 
[xgp@python ~]$ 

If the server side is not using the default port 22, you can specify the port number of the connection through the -p parameter of the SSH command in the following format:

SSHUsername@remote_Host-p port number

We can also execute Linux commands directly from a remote server using the ssh command without going into an interactive Shell, as follows:

[root@python ~]# ssh root@192.168.1.80 'date'
root@192.168.79.129's password: 
2020 Friday, 15 May, 2005:55:08 CST
s
[root@python ~]#

3. Configuring ssh:

  • Edit/etc/ssh/ssh_config
  • Edit ~/.ssh/config

SSH reads / etc/ssh/ssh_Configuration in config file.For example, the remote server does not use the default port number of 22, we only need to use / etc/ssh/ssh_A simple configuration of config eliminates the need to specify a port number when connecting to a remote server.Except repair/etc/ssh/ssh_Beyond config files, it is more common to modify the ~/.ssh/config file in the user's home directory.

View ssh_config file

[root@python ~]# find / -name ssh_config          #View Find ssh_config file
/etc/ssh/ssh_config
[root@python ~]# cat /etc/ssh/ssh_config | awk #
Usage: awk [POSIX or GNU style options] -f progfile [--] file ...
Usage: awk [POSIX or GNU style options] [--] 'program' file ...
POSIX options:      GNU long options: (standard)
    -f progfile     --file=progfile
    -F fs           --field-separator=fs
    -v var=val      --assign=var=val
Short options:      GNU long options: (extensions)
    -b          --characters-as-bytes
    -c          --traditional
    -C          --copyright
    -d[file]        --dump-variables[=file]
    -e 'program-text'   --source='program-text'
    -E file         --exec=file
    -g          --gen-pot
    -h          --help
    -L [fatal]      --lint[=fatal]
    -n          --non-decimal-data
    -N          --use-lc-numeric
    -O          --optimize
    -p[file]        --profile[=file]
    -P          --posix
    -r          --re-interval
    -S          --sandbox
    -t          --lint-old
    -V          --version

To report bugs, see node `Bugs' in `gawk.info', which is
section `Reporting Problems and Bugs' in the printed version.

gawk is a pattern scanning and processing language.
By default it reads standard input and writes standard output.

Examples:
    gawk '{ sum += $1 }; END { print sum }' file
    gawk -F: '{ print $1 }' /etc/passwd

For example, we often use a user name, port number to access a remote server.To avoid the burden of remembering the server ip, many engineers write a Shell script that holds the user name, port number, and IP address.Next time you log in, you can eliminate the hassle of typing.As follows:

[root@python ~]# vim /opt/login.sh
#!/usr/bin/bash

ssh test@127.0.0.1 -p 22

[root@python ~]# useradd test
[root@python ~]# passwd test
//Change the password for user test.
//New password:
//Invalid password: password less than 8 characters
//Re-enter the new password:
passwd: All authentication tokens have been successfully updated.
[root@python ~]# sh /opt/login.sh 
The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established.
ECDSA key fingerprint is bf:c7:de:84:e1:28:f4:d4:7e:41:49:9f:54:dc:e9:83.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '127.0.0.1' (ECDSA) to the list of known hosts.
test@127.0.0.1's password: 
[test@python ~]$ 

There are better solutions for this need.The command reads the configuration in the ~/.ssh/config file, so we can configure the information to access the remote server in advance in the ~/.ssh/config file.As follows:

[root@python ~]# cp /etc/ssh/ssh_config -p  ~/.ssh/config
[root@python ~]# vim  ~/.ssh/config 

Host python
  ForwardAgent yes
  StrictHostKeyChecking no
  port 22
  User test
  Controlpath ~/.ssh/ssh-%r@%h:%p.sock

Host *
  StrictHostKeyChecking no
  HostName %h
  Port  22
  User  test
  Controlpath ~/.ssh/ssh-%r@%h:%p.sock

Once configured, ssh host2 can be executed directly from the command line to log in using the user name test and port number 209210.166.224.14Medium.In addition, we use wildcards to define the ssh default user name and port number.Suppose we want to use the user name laoyu, port number 2092 to access10.166.226.153With the previous configuration, you can log in directly from the command line.As follows:

[root@python ~]# ssh python
Warning: Permanently added 'python,fe80::c64e:c937:2ea8:6676%ens33' (ECDSA) to the list of known hosts.
test@python's password: 
Permission denied, please try again.
test@python's password: 
Last login: Fri May 15 14:34:54 2020 from localhost
[test@python ~]$ 

As you can see, we only need a simple configuration in the ~/.ssh/config file to be efficient.

4. Log on to the remote server with the key

  • ssh uses a password
  • ssh does not use a password, uses a key

In the example above, we did not specify how to authenticate, and we used a password to authenticate by default.Password authentication is not commonly used in production environments, on the one hand because password authentication is not secure for key authentication; on the other hand, password authentication requires entering a password each time it logs on, which is cumbersome.Using key authentication eliminates the hassle of entering a password.Therefore, in production environments, keys are typically used for login.

The principle of key login is also simple, where the user's public key is stored on a remote server in advance (~/.ssh/authorized_keys file).When you log in with a key, the remote server sends a random string to the user, and SSH encrypts the string with the user's private key and sends it to the remote server.The remote server decrypts with the previously stored public key and, if successful, proves that the user is trusted, allowing the Shell to log on without requiring a password.

In addition to providing the sshd on the server side and the SSH program on the client side, OpenSSH also provides a number of jobs related to key authentication.ssh-keygen is the tool used to generate key pairs.As follows:

[root@python ~]# ssh-keygen 

After ssh-keygen executes, a private key file named id_ rsa and a public key file named id_rsa.pub will exist in the user's ~/.ssh directory.

The next step is to save the public key to the ~/.ssh/authorized_keys file on the remote server.You can save the public key to the authorized_keys file of the remote server using the following command:

[root@python ~]#  ssh root@192.168.1.80 'mkdir -p .ssh && cat >>.ssh/authorized_keys' <~/.ssh/id_rsa.pub 
The authenticity of host '192.168.1.80 (192.168.1.80)' can't be established.
ECDSA key fingerprint is SHA256:UbtfIBIs2tGQU1m/SIvSZX72VUQ+r1fH/aIMxVbdobg.
ECDSA key fingerprint is MD5:56:10:49:b1:ae:f6:3c:d3:5e:2d:c5:d9:38:2c:45:e7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.80' (ECDSA) to the list of known hosts.
root@192.168.1.80's password: 
[root@python ~]# 

In addition to the above command, which uses Shell Scripting to save public keys to a remote server, OpenSSH specifically provides a tool called ssh-copy-id.We can use this tool to save the public key to a remote server, which is more convenient than the previous shell script.As follows:

[root@python ~]# 
[root@python ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.1.80
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.1.80's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '192.168.1.80'"
and check to make sure that only the key(s) you wanted were added.

[root@python ~]# rm -rf  ~/.ssh/config
[root@python ~]# ssh 192.168.1.80
Last login: Fri May 15 15:16:23 2020 from 192.168.1.80
[root@python ~]# 

After configuring private key authentication, you can log in directly with the private key.The SSH command reads the private key file ~/.ssh/id_rsa by default.If the private key file is saved elsewhere, or with a different name, you can use the -t reference to the address of the specified private key file.As follows:

SSH test@192.168.1.80-p 22-i ~/ Location of the private key file (including file name)

It is important to note when using a private key to log in that both the private key file and the authorized_keys file on the remote server must have permissions of 600. Otherwise, the logon error will occur, which is also the most common error encountered by engineers when using a private key to log in.

Test login once

[root@python ~]# vim ~/.ssh/config 

Host python
  ForwardAgent yes
  StrictHostKeyChecking no
  HostName 192.168.1.80
  Port 22
  User test
  Controlpath ~/.ssh/ssh-%r@%h:%p.sock

[root@python ~]# ssh python
test@192.168.1.80's password: 
Last login: Fri May 15 14:49:17 2020 from fe80::c64e:c937:2ea8:6676%ens33
[test@python ~]$ 

See if public keys are consistent

[root@python ~]# ssh 192.168.1.80
Last login: Fri May 15 15:16:23 2020 from 192.168.1.80

[root@python ~]# cat ~/.ssh/authorized_keys        #Remote
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWLJAYWCpbRPQX8gDbr2mIyXMw/qEKd46u4QhcaDPY7CeGd/buIGsWsuz+DAcnowk095rIwspGGHOdt54s+aeXGXcsRh7Hpuf0Py20Krim+v2LIUQW8vQSJDj1HiQUSnNQNPT3HAm0aqQp8u2EZ0StLYtf/uYSbg6rSzW08mKwhBQkrP0olWb+hD4ak3LxA05OI/WnanGKqtqjLg+4MbgGK96fY53dKvwrdt9NWiuof3pLgTw9fTvPU6CD+cH4LmRg8IVhthlBRhrXPA7oa8gvupTvpMYdNPPUSBsVR2rBcimrUFwdOpzb6T30C7o566noRd3t3nNxkQ/HrKalo9Bn root@python

[root@python ~]# exit
//Logout
Connection to python closed.

[root@python ~]# cat ~/.ssh/id_rsa.pub              #Local
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWLJAYWCpbRPQX8gDbr2mIyXMw/qEKd46u4QhcaDPY7CeGd/buIGsWsuz+DAcnowk095rIwspGGHOdt54s+aeXGXcsRh7Hpuf0Py20Krim+v2LIUQW8vQSJDj1HiQUSnNQNPT3HAm0aqQp8u2EZ0StLYtf/uYSbg6rSzW08mKwhBQkrP0olWb+hD4ak3LxA05OI/WnanGKqtqjLg+4MbgGK96fY53dKvwrdt9NWiuof3pLgTw9fTvPU6CD+cH4LmRg8IVhthlBRhrXPA7oa8gvupTvpMYdNPPUSBsVR2rBcimrUFwdOpzb6T30C7o566noRd3t3nNxkQ/HrKalo9Bn root@python

2. Using ssh-agent to manage private keys

OpenSSH also provides a program called ssh-agent that simplifies the management of SSH private keys.Ssh-agent is a long-running daemon whose sole purpose is to cache private keys.

There are several advantages to using ssh-agent:

(1) If we use an encrypted private key, then we will need to enter a password to use the private key file when using the private key.If we use an encrypted private key and do not use ssh-agent, we will have to enter the password every time we use the private key.If you use ssh-agent to manage the private key, you only need to enter the password at the moment the private key joins the ssh-agent, and you do not need to enter the password of the private key for subsequent use.

(2) If we have multiple remote servers and multiple private key files, after using ssh-agent, we do not need to specify which private key file to use every time we log on to the server using the-i parameter.Ssh-agent will attempt to establish a connection using a different private key file.Until successful;

(3) The private key forwarding function can be implemented by using ssh-agent.Suppose you now have three servers, A, B, C.Among them, A is our control node, we can access B directly on A, but we cannot access C directly.If you want to access C, you can only log on to B first and then log on to C from B.In this case, do you need to save the user's private key in B?In this case, we can use agent forwarding.After using agent forwarding, you can access B and C servers in A without saving the private key to the B server. You only need to save the private key in A and the public key in B and C.In order to use agent forwarding, we must use ssh-agent to manage the private key.

If you use Xshell for SSH access under Windows, it is very easy to start ssh-agent by checking "Xagent with password processing (SSH agent)" in Connection --> SSH.

(1) Delete the original key and generate a new key first

[root@python .ssh]# cd ~/.ssh/
[root@python .ssh]# rm -rf id*
[root@python .ssh]# ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
dc:57:da:10:e5:1a:8b:15:e3:98:61:d4:02:9f:56:8f root@python
The key's randomart image is:
+--[ RSA 2048]----+
|        .o+.=..  |
|         o.B.B   |
|          *.E +  |
|       . o o O   |
|        S o = .  |
|           .     |
|                 |
|                 |
|                 |
+-----------------+
[root@python .ssh]# ssh-copy-id -i id_rsa.pub 192.168.1.80
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.1.80's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '192.168.1.80'"
and check to make sure that only the key(s) you wanted were added.

[root@python .ssh]# ssh 192.168.1.80
Enter passphrase for key '/root/.ssh/id_rsa': 
Last login: Fri May 15 15:28:54 2020 from fe80::c64e:c937:2ea8:6676%ens33
[root@python ~]# exit
//Logout
Connection to 192.168.1.80 closed.

Login now requires a private key

(2) Save keys using ssh-agent

On Linux, simply execute the ssh-agent command to start the ssh-agent.After startup, use the ssh-add command to add the private key to the ssh-agent.As follows:

[root@python ~]# ssh-agent bash
[root@python ~]# ssh-add ~/.ssh/id_rsa
Enter passphrase for /root/.ssh/id_rsa: 
Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)
[root@python ~]# 

After adding the private key, you can execute the ssh_add-L command to see which private keys have been added to the ssh-agent.As follows:

[root@python ~]# ssh-add -L
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDsP4VSOhnnCfidbg7e0OAQFcW5wAgPHld9S8KDTWv3X0/LNilYM3RkXQZ10XQ8Mw34i9rXa3SfqaHk6QYHXjNEUv6PEA/rKWY3kLXH9VUVHry4iwt9kVg9PowfccKLXPi8iWpqS7tk5ZEAnxihBtQattMTC44iz9X6hJDEn1r3r3YplJJGilIR+NaYJrM3ltxUBVuoJ82MfHOomhirc37ihLEwNbqRBMPYC4u1SoXDkagFsh0+HcuE0436yEByFxFw87jPmrjl7bgFsPahQsydrXySXOVdCzQJ8WuzJa1RvKr0xmgCjhZKExUYnMGAN9M79UBAyzZf2vUrwvvpPqWd /root/.ssh/id_rsa

After ssh-agent is started, when we try to establish a connection with the remote server, the SSH client will try to authenticate with the remote server using the private key stored in the ssh-agent.

[root@python ~]# ssh 192.168.1.80
Last login: Fri May 15 15:38:28 2020 from 192.168.1.80
[root@python ~]# exit
//Logout
Connection to 192.168.1.80 closed.

Now that the private key is maintained locally, there is no need to enter it

Tags: Python ssh openssh Linux

Posted on Fri, 15 May 2020 13:28:37 -0400 by jason213123