python - Is it a bug to use subprocess.run() in a multithreaded script? - Stack Overflow

admin管理员组

文章数量:1201640

I have a long build script which is a mix of "real" python code and lengthy subprocess.run() calls to things like debootstrap, wget, apt-get or build.sh.

Everything is parallelized. One thread does debootstrap followed by apt-get, while another does build.sh, then both are joined and we start more threads for combining the results and so on. Each thread logs to a dedicated file.

While looking for a generic way to log the output of subprocess.run() started by one of these threads, I came upon the following answer:

Is it a bug to call subprocess.run() in a python script with more than one thread?

I have a long build script which is a mix of "real" python code and lengthy subprocess.run() calls to things like debootstrap, wget, apt-get or build.sh.

Everything is parallelized. One thread does debootstrap followed by apt-get, while another does build.sh, then both are joined and we start more threads for combining the results and so on. Each thread logs to a dedicated file.

While looking for a generic way to log the output of subprocess.run() started by one of these threads, I came upon the following answer: https://stackoverflow.com/a/31868783/876832

Is it a bug to call subprocess.run() in a python script with more than one thread?

• python
• multithreading

Share Improve this question Follow edited Jan 22 at 13:04 Ahmed AEK 17.5k33 gold badges1414 silver badges3939 bronze badges asked Jan 22 at 12:19 FadewayFadeway 58866 silver badges1919 bronze badges 0

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 5

Is it a bug to call subprocess.run() in a python script with more than one thread?

No. I do that all the time.

I think the answer you linked is misguided. After all, you don't even know whether subprocess.Popen() and its facades like subprocess.run() use the fork syscall (especially on Windows, they certainly don't, since that call doesn't exist there).

Sure, if you were to manually os.fork(), I'd exercise caution in a threaded program (and indeed, as the docs say, since 3.12, if Python detects you're doing that, it'll warn you).

Under the hood, though, on POSIX systems, subprocess.Popen() uses os.posix_spawn when possible, and otherwise a very careful fork_exec implementation.

Oh, and as an aside, the question you linked actually uses the Python 2.7-era low-level thread module, these days aptly called _thread, the low-level threading API. That's a different low-level beast than threading.Threads.

本文标签： pythonIs it a bug to use subprocessrun() in a multithreaded scriptStack Overflow