Python standard library 06 child process of subprocess package details

The content here is based on the Linux process foundation and the Linux text stream. The main function of the subprocess package is to execute external commands and programs. For example, I need to download files using wget. I call the wget program in Python. In this sense, subprocess functions similarly to shell.

subprocess and the usual encapsulation functions

When we run python, we are creating and running a process. As we described in the Linux process foundation, one process can fork1 subprocess and let this subprocess exec another program. In Python, we use the fork1 subprocess from the subprocess package in the standard library and run an external program (see fork, exec, Linux process foundation).

The subprocess package defines several functions that create child processes, each of which creates child processes in a different way, so we can use one of them as needed. In addition, subprocess provides tools for managing standard streams (standard stream) and pipes (pipe) to use text communication between processes.

When creating child processes using the functions in the subprocess package, note that:

1) after the child process is created, whether the parent process is suspended and waits for the child process to run.

2) what does the function return

3) how does the parent process handle when returncode is not 0?

subprocess.call()

The parent waits for the child to complete

Return exit information (returncode, equivalent to exit code, see Linux process foundation)

subprocess.check_call()

The parent waits for the child to complete

Return 0

Check the exit information. If returncode is not 0, subprocess.CalledProcessError is cited. This object contains the returncode property. except... To check (see Python error handling).

subprocess.check_output()

The parent waits for the child to complete

Returns the output of the child process to standard output

Check the exit information. If returncode is not 0, subprocess.CalledProcessError is cited. This object contains the returncode property and output property. except... To check.

These three functions are used in a similar way, which is illustrated by subprocess.call () :

import subprocess
rc = subprocess.call(["ls","-l"])

We will put the program name (ls) and the parameter (-l)1 into a table and pass them to subprocess.call ().

The entire string can be interpreted by 1 shell:

import subprocess
out = subprocess.call("ls -l", shell=True)
out = subprocess.call("cd ..", shell=True)

We used the parameter shell=True. At this point, we are running the child using 1 whole string instead of 1 table. Python will run 1 shell first, and then use this shell to interpret the entire string.

Some of the commands in the shell command are built-in to shell. These commands must be run through shell, $cd. shell=True allows us to run some of these commands.

Popen()

In fact, all three of our functions above are based on the encapsulation of Popen() (wrapper). The purpose of these encapsulations is to make it easy for us to use child processes. When we want to personalize our requirements, we turn to the Popen class, which generates objects that represent child processes.

Unlike the above encapsulation, after the Popen object is created, the main program does not automatically wait for the child process to complete. We must call the wait() method of the object before the parent process will wait (i.e., block block) :

import subprocess
child = subprocess.Popen(["ping","-c","5","www.google.com"])
print("parent process")

As you can see from the results of the run, the parent process does not wait for child to finish after it starts the child process, but runs print directly.

Compare the waiting situation:

import subprocess
child = subprocess.Popen(["ping","-c","5","www.google.com"])
child.wait()
print("parent process")

In addition, you can perform other operations on the child process in the parent process, such as the child object in our example above:

child.poll()      #  Check child process status 

child.kill()      #  Terminate child process 

child.send_signal()  #  Sends a signal to the child process 

child.terminate()   #  Terminate child process 

The child process PID is stored in child.pid

Text flow control for child processes

The standard input, standard output, and standard error of the child process can also be represented by the following properties:

child.stdin child.stdout child.stderr

We can change the standard input, standard output, and standard error when Popen() sets up a subprocess, and we can use subprocess.PIPE connects the input and output of multiple subprocesses in one place to form a pipeline (pipe):

import subprocess
child1 = subprocess.Popen(["ls","-l"], stdout=subprocess.PIPE)
child2 = subprocess.Popen(["wc"], stdin=child1.stdout,stdout=subprocess.PIPE)
out = child2.communicate()
print(out)

subprocess.PIPE actually provides a cache for text streams. stdout of child1 outputs the text to the cache, and stdin of child2 reads the text from that PIPE. The output text of child2 is also stored in PIPE until the communicate() method reads out the text in PIPE from PIPE.

Note that communicate() is a method of the Popen object that blocks the parent process until the child process completes.

We can also use the communicate() method to input PIPE to the child process:

import subprocess
child = subprocess.Popen(["cat"], stdin=subprocess.PIPE)
child.communicate("vamei")

After we start the subprocess, cat waits for input until we type "vamei" with communicate().

By using the subprocess package, we can run external programs. This greatly extends the functionality of Python. If you already know something about the application of the operating system, you can call the application directly from Python (rather than relying on Python), output the result of the application to Python, and let Python continue processing. The functionality of shell, such as connecting applications with text streams, can be implemented in Python.

conclusion

subprocess.call, subprocess.check_call(), subprocess.check_output()

subprocess.Popen(), subprocess.PIPE

Popen.wait(), Popen.communicate()