How to Achieve Real-Time Output from Subprocess in Python?

Realtime Output with Subprocess

In attempting to create a wrapper script with a progress indicator, you encountered a challenge where the output from the wrapped program was buffered despite using subprocess.Popen with stdout=PIPE. As you discovered, neither setting bufsize to 1 or 0 nor using an output loop resolved the issue.

Solution: Using readline

Fortuitously, we stumbled upon a workaround: utilizing the readline method of the stdout pipe. Unlike the for loop method you initially employed, which buffers aggressively, the while True loop enhanced with readline exhibits the desired behavior.

<br>while True:<br>  line = p.stdout.readline()<br>  if not line: break<br>  ...<br>

Underlying Issue

This issue originates from a known bug in Python, tracked as issue 3907. The specific cause of the anomalous buffering is not well understood.

Conclusion

By leveraging the readline method, you can achieve real-time output from programs launched via subprocess without resorting to outdated or potentially insecure techniques like exec*. This workaround remains effective in Python versions beyond 3.x, ensuring forward compatibility.

The above is the detailed content of How to Achieve Real-Time Output from Subprocess in Python?. For more information, please follow other related articles on the PHP Chinese website!