…or why atexit.register() and signal.signal() are evil
UPDATE (2016-02-13): this recipe no longer handles SIGINT, SIGQUIT and SIGABRT as aliases for “application exit” because it was a bad idea. It only handles SIGTERM. Also it no longer support Windows because signal.signal() implementation is too different than POSIX.
Many people erroneously think that any function registered via atexit module is guaranteed to always be executed when the program terminates. You may have noticed this is not the case when, for example, you daemonize your app in production then try to stop it or restart it: the cleanup functions will not be executed. This is because functions registered wth atexit module are not called when the program is killed by a signal:
import atexit, os, signal @atexit.register def cleanup(): print("on exit") # XXX this never gets printed os.kill(os.getpid(), signal.SIGTERM)
It must be noted that the same thing would happen if instead of atexit.register() we would use a “finally” clause. It turns out the correct way to make sure the exit function is always called in case a signal is received is to
register it via signal.signal().
That has a drawback though: in case a third-party module has already
registered a function for that signal (SIGTERM or whatever), your new function will overwrite the
old one:
import os, signal def old(*args): print("old") # XXX this never gets printed def new(*args): print("new") signal.signal(signal.SIGTERM, old) signal.signal(signal.SIGTERM, new) os.kill(os.getpid(), signal.SIGTERM)
Also, we would still have to use atexit.register() so that the function is called also on “clean” interpreter exit and take into account other signals other than SIGTERM which would cause the process to terminate. This recipe attempts to address all these issues so that:
- the exit function is always executed
for all exit signals (SIGTERM,on SIGTERM and on “clean” interpreter exit.
SIGINT, SIGQUIT, SIGABRT) - any exit function(s) previously registered via atexit.register() or
signal.signal() will be executed as well (after the new one). - It must be noted that the exit function will never be executed in case of SIGKILL, SIGSTOP or os._exit().
The code
import atexit import os import signal import sys if os.name != 'posix': raise ImportError("POSIX only") _registered_exit_funs = set() _executed_exit_funs = set() _exit_signals = frozenset([signal.SIGTERM]) def register_exit_fun(fun, signals=_exit_signals): """Register a function which will be executed on "normal" interpreter exit or in case one of the `signals` is received by this process (differently from atexit.register()). Also, it makes sure to execute any previously registered via signal.signal().If any, it will be executed after `fun`. Functions which were already registered or executed via this function will be skipped. Exit function will not be executed on SIGKILL, SIGSTOP or os._exit(0). """ def fun_wrapper(): if fun not in _executed_exit_funs: try: fun() finally: _executed_exit_funs.add(fun) def signal_wrapper(signum=None, frame=None): if signum is not None: pass # smap = dict([(getattr(signal, x), x) for x in dir(signal) # if x.startswith('SIG')]) # print("signal {} received by process with PID {}".format( # smap.get(signum, signum), os.getpid())) fun_wrapper() # Only return the original signal this process was hit with # in case fun returns with no errors, otherwise process will # return with sig 1. if signum is not None: if signum == signal.SIGINT: raise KeyboardInterrupt # XXX - should we do the same for SIGTERM / SystemExit? sys.exit(signum) if not callable(fun): raise TypeError("{!r} is not callable".format(fun)) set([fun]) # raise exc if obj is not hash-able for sig in signals: # Register function for this signal and pop() the previously # registered one (if any). This can either be a callable, # SIG_IGN (ignore signal) or SIG_DFL (perform default action # for signal). old_handler = signal.signal(sig, signal_wrapper) if old_handler not in (signal.SIG_DFL, signal.SIG_IGN): # ...just for extra safety. if not callable(old_handler): continue # This is needed otherwise we'll get a KeyboardInterrupt # strace on interpreter exit, even if the process exited # with sig 0. if (sig == signal.SIGINT and old_handler is signal.default_int_handler): continue # There was a function which was already registered for this # signal. Register it again so it will get executed (after our # new fun). if old_handler not in _registered_exit_funs: atexit.register(old_handler) _registered_exit_funs.add(old_handler) # This further registration will be executed in case of clean # interpreter exit (no signals received). if fun not in _registered_exit_funs or not signals: atexit.register(fun_wrapper) _registered_exit_funs.add(fun)
Usage
As a function:
def cleanup(): print("cleanup") register_exit_fun(cleanup)
As a decorator:
@register_exit_fun def cleanup(): print("cleanup")
Unit tests
This recipe is currently provided as a gist with a full set of unittests. It works with Python 2 and 3.
Notes about Windows
Because of how different signal.signal() behaves on Windows, this code is UNIX only: http://bugs.python.org/issue26350
Proposal for stdlib inclusion
The fact that atexit module does not handle signals and that signal.signal() overwrites previously registered handlers is unfortunate. It is also confusing because it is not immediately clear which one you are supposed to use (and it turns out you’re supposed to use both). Most of the times you have no idea (or don’t care) that you’re overwriting another exit function. As a user, I would just want to execute an exit function, no matter what, possibly without messing with whatever a module I’ve previously imported has done with signal.signal(). To me this suggests there could be space for something like “atexit.register_w_signals“.