Catch Ctrl+C / SIGINT and exit multiprocesses gracefully in python(捕捉 Ctrl+C/SIGINT 并在 python 中优雅地退出多进程)
问题描述
如何在多进程 python 程序中捕获 Ctrl+C 并优雅地退出所有进程,我需要该解决方案在 unix 和 windows 上都可以工作.我尝试了以下方法:
How do I catch a Ctrl+C in multiprocess python program and exit all processes gracefully, I need the solution to work both on unix and windows. I've tried the following:
import multiprocessing
import time
import signal
import sys
jobs = []
def worker():
signal.signal(signal.SIGINT, signal_handler)
while(True):
time.sleep(1.1234)
print "Working..."
def signal_handler(signal, frame):
print 'You pressed Ctrl+C!'
# for p in jobs:
# p.terminate()
sys.exit(0)
if __name__ == "__main__":
for i in range(50):
p = multiprocessing.Process(target=worker)
jobs.append(p)
p.start()
它有点工作,但我认为这不是正确的解决方案.
And it's kind of working, but I don't think it's the right solution.
推荐答案
先前接受的解决方案 有竞争条件,但它没有使用 map
和 async
函数.
The previously accepted solution has race conditions and it does not work with map
and async
functions.
multiprocessing.Pool
处理Ctrl+C/SIGINT
的正确方法是:
- 在创建进程
Pool
之前让进程忽略SIGINT
.这种方式创建的子进程继承SIGINT
处理程序. - 在创建
Pool
后,在父进程中恢复原始SIGINT
处理程序. - 使用
map_async
和apply_async
而不是阻塞map
和apply
. - 使用超时等待结果,因为默认阻塞等待忽略所有信号.这是 Python 错误 https://bugs.python.org/issue8296.
- Make the process ignore
SIGINT
before a processPool
is created. This way created child processes inheritSIGINT
handler. - Restore the original
SIGINT
handler in the parent process after aPool
has been created. - Use
map_async
andapply_async
instead of blockingmap
andapply
. - Wait on the results with timeout because the default blocking waits to ignore all signals. This is Python bug https://bugs.python.org/issue8296.
<小时>
把它放在一起:
Putting it together:
#!/bin/env python
from __future__ import print_function
import multiprocessing
import os
import signal
import time
def run_worker(delay):
print("In a worker process", os.getpid())
time.sleep(delay)
def main():
print("Initializng 2 workers")
original_sigint_handler = signal.signal(signal.SIGINT, signal.SIG_IGN)
pool = multiprocessing.Pool(2)
signal.signal(signal.SIGINT, original_sigint_handler)
try:
print("Starting 2 jobs of 5 seconds each")
res = pool.map_async(run_worker, [5, 5])
print("Waiting for results")
res.get(60) # Without the timeout this blocking call ignores all signals.
except KeyboardInterrupt:
print("Caught KeyboardInterrupt, terminating workers")
pool.terminate()
else:
print("Normal termination")
pool.close()
pool.join()
if __name__ == "__main__":
main()
正如@YakovShklarov 所指出的,在父进程中忽略信号和取消忽略信号之间有一个时间窗口,在此期间信号可能会丢失.使用 pthread_sigmask
代替在父进程中临时阻止信号的传递可以防止信号丢失,但是在 Python-2 中不可用.
As @YakovShklarov noted, there is a window of time between ignoring the signal and unignoring it in the parent process, during which the signal can be lost. Using pthread_sigmask
instead to temporarily block the delivery of the signal in the parent process would prevent the signal from being lost, however, it is not available in Python-2.
这篇关于捕捉 Ctrl+C/SIGINT 并在 python 中优雅地退出多进程的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持编程学习网!
本文标题为:捕捉 Ctrl+C/SIGINT 并在 python 中优雅地退出多进程
基础教程推荐
- 如何在海运重新绘制中自定义标题和y标签 2022-01-01
- Dask.array.套用_沿_轴:由于额外的元素([1]),使用dask.array的每一行作为另一个函数的输入失败 2022-01-01
- 用于分类数据的跳跃记号标签 2022-01-01
- Python kivy 入口点 inflateRest2 无法定位 libpng16-16.dll 2022-01-01
- 何时使用 os.name、sys.platform 或 platform.system? 2022-01-01
- 线程时出现 msgbox 错误,GUI 块 2022-01-01
- 筛选NumPy数组 2022-01-01
- 在 Python 中,如果我在一个“with"中返回.块,文件还会关闭吗? 2022-01-01
- 使用PyInstaller后在Windows中打开可执行文件时出错 2022-01-01
- 如何让 python 脚本监听来自另一个脚本的输入 2022-01-01