Source code analysis and detail control of Java executing windows commands (preliminary preparation for developing "Java command executor")

1. Method of executing command

Runtime.exec("cmd",...) method: see Java Runtime class source code analysis (preliminary preparation for developing "Java command executor")_ Jiangnan liquor cooking blog - CSDN blog

ProcessBuilder.command("cmd",...).start(): see Java ProcessBuilder class source code analysis (preliminary preparation for developing "Java command executor")_ Jiangnan liquor cooking blog - CSDN blog

2. Source code analysis of Java executing cmd command (Linux hasn't seen it yet. If there is a big difference, a separate explanation will be given)

First of all, you should understand that the Runtime.exec() method has many rewriting methods, but in the jdk source code, the exec() method is finally:

    public Process exec(String[] cmdarray, String[] envp, File dir)
        throws IOException {
        return new ProcessBuilder(cmdarray)
            .environment(envp)
            .directory(dir)
            .start();
    }

about

exec(String command),

exec(String command, String[] envp),

exec(String command, String[] envp, File dir),

The exec method that directly writes the command into a string will eventually be split into a string array cmdarray by StringTokenizer and then passed into ProcessBuilder (in Java Runtime class source code analysis (preliminary preparation for the development of "Java command executor") Jiangnan liquor cooking blog - CSDN blog There are detailed call diagrams of these exec rewriting methods in, so we won't explain them one by one):

    public Process exec(String command, String[] envp, File dir)
        throws IOException {
        if (command.length() == 0)
            throw new IllegalArgumentException("Empty command");
        //Split command
        StringTokenizer st = new StringTokenizer(command);
        String[] cmdarray = new String[st.countTokens()];
        for (int i = 0; st.hasMoreTokens(); i++)
            cmdarray[i] = st.nextToken();
        return exec(cmdarray, envp, dir);
    }

Continue, ProcessBuilder will convert the mdarray string array containing the commands into the ArrayList collection before actually executing these commands (although it will convert this collection into a string array later, I don't quite understand it here, but I haven't analyzed all the source code of ProcessBuilder class, so it may be useful in other places):

    public ProcessBuilder command(String... command) {
        this.command = new ArrayList<>(command.length);
        for (String arg : command)
            this.command.add(arg);
        return this;
    }

Users can also directly create a new ProcessBuilder object and pass in a command to directly reach this step. Quite a number of constructors and object methods are defined in the ProcessBuilder class to complete the construction of a ProcessBuilder object (see details) Java ProcessBuilder class source code analysis (preliminary preparation for developing "Java command executor") Jiangnan liquor cooking blog - CSDN blog ), the most important method in the ProcessBuilder object is the start() method, which starts to execute commands and create a process. The source code of the start() method is relatively long, which is mainly divided into the following steps:

public Process start() throws IOException {
        // Must convert to array first -- a malicious user-supplied
        // list might try to circumvent the security check.
        //1. Reconvert the string collection that holds the command to an array
        String[] cmdarray = command.toArray(new String[command.size()]);
        cmdarray = cmdarray.clone();

        for (String arg : cmdarray)
            if (arg == null)
                throw new NullPointerException();
        // Throws IndexOutOfBoundsException if command is empty
        //2. Judge whether the first command of the array is empty, because the first string is the program name that specifies what to open. Make sure it is not empty
        String prog = cmdarray[0];
        SecurityManager security = System.getSecurityManager();
        if (security != null)
            security.checkExec(prog);

        String dir = directory == null ? null : directory.toString();

        for (int i = 1; i < cmdarray.length; i++) {
            if (cmdarray[i].indexOf('\u0000') >= 0) {
                throw new IOException("invalid null character in command");
            }
        }

        try {
            //3. return another ProcessImpl.start() method
            return ProcessImpl.start(cmdarray,
                                     environment,
                                     dir,
                                     redirects,
                                     redirectErrorStream);
        } catch (IOException | IllegalArgumentException e) {
            String exceptionInfo = ": " + e.getMessage();
            Throwable cause = e;
            if ((e instanceof IOException) && security != null) {
                // Can not disclose the fail reason for read-protected files.
                try {
                    security.checkRead(prog);
                } catch (SecurityException se) {
                    exceptionInfo = "";
                    cause = se;
                }
            }
            // It's much easier for us to create a high-quality error
            // message than the low-level C code which found the problem.
            throw new IOException(
                "Cannot run program \"" + prog + "\""
                + (dir == null ? "" : " (in directory \"" + dir + "\")")
                + exceptionInfo,
                cause);
        }

The start() of the ProcessBuilder object finally return s a

ProcessImpl.start(cmdarray,environment,dir,redirects,redirectErrorStream). ProcessImpl is the concrete implementation of the Process class. The start() method of ProcessImpl is also very long. The start() method of ProcessImpl first loads the environment, the determined standard information, and the output stream of error information

And the input stream for reading external data (the stream here is relative to the created process, because it is also considered that the input and output streams of the new process may be redirected to the user-defined stream by the user, so it is also relatively complex. If someone wants to see the source code carefully, I can only say that I have experience):

    // System-dependent portion of ProcessBuilder.start()
    static Process start(String cmdarray[],
                         java.util.Map<String,String> environment,
                         String dir,
                         ProcessBuilder.Redirect[] redirects,
                         boolean redirectErrorStream)
        throws IOException
    {
        String envblock = ProcessEnvironment.toEnvironmentBlock(environment);

        FileInputStream  f0 = null;
        FileOutputStream f1 = null;
        FileOutputStream f2 = null;

        try {
            long[] stdHandles;
            if (redirects == null) {
                stdHandles = new long[] { -1L, -1L, -1L };
            } else {
                stdHandles = new long[3];

                if (redirects[0] == Redirect.PIPE)
                    stdHandles[0] = -1L;
                else if (redirects[0] == Redirect.INHERIT)
                    stdHandles[0] = fdAccess.getHandle(FileDescriptor.in);
                else {
                    f0 = new FileInputStream(redirects[0].file());
                    stdHandles[0] = fdAccess.getHandle(f0.getFD());
                }

                if (redirects[1] == Redirect.PIPE)
                    stdHandles[1] = -1L;
                else if (redirects[1] == Redirect.INHERIT)
                    stdHandles[1] = fdAccess.getHandle(FileDescriptor.out);
                else {
                    f1 = newFileOutputStream(redirects[1].file(),
                                             redirects[1].append());
                    stdHandles[1] = fdAccess.getHandle(f1.getFD());
                }

                if (redirects[2] == Redirect.PIPE)
                    stdHandles[2] = -1L;
                else if (redirects[2] == Redirect.INHERIT)
                    stdHandles[2] = fdAccess.getHandle(FileDescriptor.err);
                else {
                    f2 = newFileOutputStream(redirects[2].file(),
                                             redirects[2].append());
                    stdHandles[2] = fdAccess.getHandle(f2.getFD());
                }
            }

            return new ProcessImpl(cmdarray, envblock, dir,
                                   stdHandles, redirectErrorStream);
        } finally {
            // In theory, close() can throw IOException
            // (although it is rather unlikely to happen here)
            try { if (f0 != null) f0.close(); }
            finally {
                try { if (f1 != null) f1.close(); }
                finally { if (f2 != null) f2.close(); }
            }
        }

    }

Finally, the start() method of ProcessImpl returns new ProcessImpl (cmdarray, envblock, dir, stdhandles, redirecterrorstream) to create a Process object. In the ProcessImpl constructor:

private ProcessImpl(String cmd[],
                        final String envblock,
                        final String path,
                        final long[] stdHandles,
                        final boolean redirectErrorStream)
        throws IOException
    {
        String cmdstr;
        SecurityManager security = System.getSecurityManager();
        boolean allowAmbiguousCommands = false;
        if (security == null) {
            allowAmbiguousCommands = true;
            String value = System.getProperty("jdk.lang.Process.allowAmbiguousCommands");
            if (value != null)
                allowAmbiguousCommands = !"false".equalsIgnoreCase(value);
        }
        if (allowAmbiguousCommands) {
            // Legacy mode.

            // Normalize path if possible.
            String executablePath = new File(cmd[0]).getPath();

            // No worry about internal, unpaired ["], and redirection/piping.
            if (needsEscaping(VERIFICATION_LEGACY, executablePath) )
                executablePath = quoteString(executablePath);

            cmdstr = createCommandLine(
                //legacy mode doesn't worry about extended verification
                VERIFICATION_LEGACY,
                executablePath,
                cmd);
        } else {
            String executablePath;
            try {
                executablePath = getExecutablePath(cmd[0]);
            } catch (IllegalArgumentException e) {
                // Workaround for the calls like
                // Runtime.getRuntime().exec("\"C:\\Program Files\\foo\" bar")

                // No chance to avoid CMD/BAT injection, except to do the work
                // right from the beginning. Otherwise we have too many corner
                // cases from
                //    Runtime.getRuntime().exec(String[] cmd [, ...])
                // calls with internal ["] and escape sequences.

                // Restore original command line.
                StringBuilder join = new StringBuilder();
                // terminal space in command line is ok
                for (String s : cmd)
                    join.append(s).append(' ');

                // Parse the command line again.
                cmd = getTokensFromCommand(join.toString());
                executablePath = getExecutablePath(cmd[0]);

                // Check new executable name once more
                if (security != null)
                    security.checkExec(executablePath);
            }

            // Quotation protects from interpretation of the [path] argument as
            // start of longer path with spaces. Quotation has no influence to
            // [.exe] extension heuristic.
            cmdstr = createCommandLine(
                    // We need the extended verification procedure for CMD files.
                    isShellFile(executablePath)
                        ? VERIFICATION_CMD_BAT
                        : VERIFICATION_WIN32,
                    quoteString(executablePath),
                    cmd);
        }

        handle = create(cmdstr, envblock, path,
                        stdHandles, redirectErrorStream);

        java.security.AccessController.doPrivileged(
        new java.security.PrivilegedAction<Void>() {
        public Void run() {
            if (stdHandles[0] == -1L)
                stdin_stream = ProcessBuilder.NullOutputStream.INSTANCE;
            else {
                FileDescriptor stdin_fd = new FileDescriptor();
                fdAccess.setHandle(stdin_fd, stdHandles[0]);
                stdin_stream = new BufferedOutputStream(
                    new FileOutputStream(stdin_fd));
            }

            if (stdHandles[1] == -1L)
                stdout_stream = ProcessBuilder.NullInputStream.INSTANCE;
            else {
                FileDescriptor stdout_fd = new FileDescriptor();
                fdAccess.setHandle(stdout_fd, stdHandles[1]);
                stdout_stream = new BufferedInputStream(
                    new FileInputStream(stdout_fd));
            }

            if (stdHandles[2] == -1L)
                stderr_stream = ProcessBuilder.NullInputStream.INSTANCE;
            else {
                FileDescriptor stderr_fd = new FileDescriptor();
                fdAccess.setHandle(stderr_fd, stdHandles[2]);
                stderr_stream = new FileInputStream(stderr_fd);
            }

            return null; }});
    }

The first is to obtain the executable path of the program,

executablePath = getExecutablePath(cmd[0]);

Then re splice the command and parse it again (eliminate some hidden problems caused by spaces and re splice spaces)

// Restore original command line.
StringBuilder join = new StringBuilder();
// terminal space in command line is ok
for (String s : cmd)
    join.append(s).append(' ');

// Parse the command line again.
cmd = getTokensFromCommand(join.toString());

secondly

cmdstr = createCommandLine(
        // We need the extended verification procedure for CMD files.
        isShellFile(executablePath)
            ? VERIFICATION_CMD_BAT
            : VERIFICATION_WIN32,
        quoteString(executablePath),
        cmd);

Check whether the executable file ends in. CMD or. BAT to adjust the path. Whether double quotes and slashes need to be added. Finally

handle = create(cmdstr, envblock, path,
                stdHandles, redirectErrorStream);

The create() method is a native method. It uses the win32 function CreateProcess to create a process. Whether a process can be created depends on this method. This is the end!

3. Detail control

  1. The string array representing the command is {executable file name, parameter, parameter,...},
    For example, cmd /k dir is converted to {"cmd", "/k", "dir"},
    You can also write directly into an entire string, but if you use a string array, you must split it one by one.
  2. "cmd /k dir" and "cmd.exe"  / k dir "is not bad. (Linux system should also pay attention to these problems, and I'll check it.)
  3. cmd /c dir: close the command window after executing the dir command;
    cmd /k dir: do not close the command window after executing the dir command.
    (the effect of Java executing commands on windows system is the same as that of "running" program.)

  4. adopt
    Process process = Runtime.getRuntime().exec("cmd /c dir")
    perhaps
    Process process = new ProcessBuilder().command("cmd", "/k","dir").start()
    First of all, the created process object, whether / c or / k, will not pop up (in windows system), but if "cmd /c start dir" is executed The command will pop up a window. The start command here is to reopen a new process to execute the command, so the stream obtained through the process object is still the input and output stream of the original process, while the original process only executes a start command. The process that really executes the available command is the process generated by the start command, which will cause us to be unable to obtain the input and output stream of the process that actually executes the command Input and output content.

  5. Although / c or / k will not pop up a window, / c will directly terminate the process after executing the specified command, and / k will continue to maintain the process after executing the command. You can continue to pass a new command through process.getOutputStream()

    OutputStream outputStream = process.getOutputStream();
    outputStream.write("dir \n".getBytes(new GBK()));
    //Refresh the stream and force any buffered characters to be written out, otherwise the command may not be executed in time
    outputStream.flush();
    You can also continuously obtain the result information of the executed command through process.getInputStream():
                InputStream inputStream = process.getInputStream();
                byte[] bytes = new byte[1024];
                int len;
                StringBuilder result= new StringBuilder();
                //Just an endless loop, this code is not desirable
                while ((len=inputStream.read())>0)
                    result.append(new String(bytes, 0, len, "GBK"));
                System.out.println(result);

    But the ideal is beautiful and the reality is cruel. First, process.getInputStream().read() will not return - 1 because it executes a command and reads all the data in the buffer. It will only block because the process has not been terminated (although there is no data in the temporary buffer, which is a bit similar to socket) Secondly, even if we don't mind this, the data generated by executing a command will not be buffered completely at one time, so the read () method can only be placed in while, which will not only make us unable to know when the execution result is complete, but also block our current java process. There is a redirectOutput() in the ProcessBuilder object Method can redirect streams (both input and output streams can be redirected):

                Process process = new ProcessBuilder()
                        .command("cmd","dir")
                        .redirectOutput(new File("log"))
                        .redirectErrorStream(true)
                        .start();

    Unfortunately, this method can only direct the stream to the file in the end, and we can't guarantee when the data saved in the file will be the complete command execution result (so there is no solution!).

  6. Then 5 continue. Although the / k command can continue input and output, it has obvious disadvantages (the created process output is intermittent, so it is impossible to judge whether the command is completely executed, and the read() method is inconvenient to block). What about / c/ c is to directly terminate the process after executing a command, so we can read the execution results after the process is terminated to obtain the complete execution results. However, with / c, only one command can be executed, even if cmd command can execute multiple commands at one time:

    CMD Execute multiple commands
     It can be separated by these three methods & && ||
    
    use&Separated. The usage is before and after the command. It will run whether it can be run or not. 1 command&2 Command is to run 1 command and 2 command.
    
    use&&Separated. The usage is to run the following command only after the previous command runs successfully. 1 command&2 Command is to run 2 command after running 1 command without error and successfully.
    
    use||Separated. The usage is to run the following command only after the previous command runs successfully. 1 command&2 Command, that is, the 2 command is run only when the 1 command fails to run successfully.

    However, the one-time results of multiple commands will make us unable to distinguish and can not be used for analysis. If the / c command is executed one by one, each / c command creates a new process, which is also extremely resource consuming. In addition, whether the / c command is executed depends entirely on whether the process is terminated. Therefore, we also need to judge whether the process represented by the process object is terminated. If not, we need to wait for its termination:

                try {
                    cmd.waitFor();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }

    If you don't terminate all the time, it will also lead to java process blocking. If you get data directly through the stream without judgment, it may also lead to read() method blocking. In short, / c and / k can't have both fish and bear's paw. They can choose by themselves according to business needs.

  7. Whether executing / c or / k, the read() method may read - 1 in advance and terminate, resulting in the obtained data is not the complete output result (in fact, there is still data in the stream that has not been read). This is because the process may output an EOF, so it is necessary and inevitable to call read() in a loop to some extent.

    EOF(End of file)Is knowledge C/C++Inside the macro definition, the specific definition formula is#define EOF -1 indicates the end flag of the file. The value is equal to - 1. It is generally used in the functions read from the file, such as fscanf fgetc fgets. Once the file is read, the EOF flag is returned and the function call is ended.

4. Whether the execution process of java executing commands in Linux system is the same as that in Windows system and what problems will it encounter? The author will write a separate chapter.  

Tags: Java

Posted on Sun, 21 Nov 2021 23:16:48 -0500 by idire