How to Get Real-Time Output from Subprocesses in Python?

Real-Time Output from Subprocesses

In Python, the subprocess module provides a convenient way to execute external programs. However, by default, it captures all the program's output and returns it once the process has completed. This can be problematic for long-running processes.

Introducing Iterative Output Polling

To address this, you can leverage the iter() function to retrieve output line by line as the command generates it. Here's a modified version of your execute() function that implements this approach:

<code class="python">def execute(cmd):
    popen = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
    for stdout_line in iter(popen.stdout.readline, ""):
        yield stdout_line
    popen.stdout.close()
    return_code = popen.wait()
    if return_code:
        raise subprocess.CalledProcessError(return_code, cmd)</code>

Usage Example

You can now iterate over the output of the process as it becomes available:

<code class="python">for path in execute(["locate", "a"]):
    print(path, end="")</code>

This example will continuously print the output of the locate a command as it searches for files.

Polling Interval

While this method provides real-time output, the polling interval can impact performance. The universal_newlines parameter ensures that you get lines of text rather than bytes, which incurs an additional overhead. To optimize polling, you can reduce the frequency with which the process's output is checked.

The above is the detailed content of How to Get Real-Time Output from Subprocesses in Python?. For more information, please follow other related articles on the PHP Chinese website!