python - tornado怎么处理非常耗时的定时任务？

Question

代码大致如下： {代码...} 其中update()是一个特别耗时的任务目前的做法是加大add_timeout的时间间隔，以避免两个进程全都去做update了 因为对异步编程并不熟，所以想请教一下有没有更合适的做法

PHP中文网 · Answer

It sounds like this task will occupy the main thread for a long time and the specific time is not sure, so using coroutines is obviously not appropriate. It is recommended to try ThreadPoolExecutor, and then use yield threadPool.submit(fn)

ringa_lee · Answer

tornado comes with its own gen module, and its official website documentation has clear examples

It can be coroutineized. (With callbacks, the logic is complicated and it is tiring to read the code; for threads/processes, scheduling issues need to be considered)

伊谢尔伦 · Answer

It’s useless. . If you don't consider anything else, if you have a single thread and spends too long in one place, it is really blocked.
Refer to the code below:

class TestHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    @tornado.gen.coroutine
    def get(self):
        yield gen.Task(self.sleep)
        self.write('ok')
        self.finish()

    def sleep(self, callback):
        for i in range(100000000):
            if i % 100000 == 0:
                pass
        print('ooo')
        callback()

大家讲道理 · Answer

It depends on whether your task is network IO intensive or local IO intensive/CPU intensive. If it is the former, just use PeriodicCallback. If it is the latter, it is recommended to implement a queue and then write a worker to run the task alone