TL;DR
Use faulthandler.enable();
Python will print the stack trace to stderr before fully crashing.
Backstory
Today I tried to daemonize a Python script that calls requests.get on macOS,
and it was insta-dying for no apparent reason.
I tested without daemonization, but the problem only popped up when I daemonize it.
I was doing something like this:
from daemon import DaemonContext
from daemon.pidfile import PIDLockFile
import requests
import time
pid = PIDLockFile(path.join(home_dir, 'pidfile_watcher'))
fp = open("out.log", "a+")
fpe = open("err.log", "a+")
with DaemonContext(
pidfile=pid,
stdout=fp,
stderr=fpe
):
while True:
response = requests.get("https://example.com");
print(response.text)
time.sleep(60*10)
fp.close()
fpe.close()
Many researches, print()s and one final check on Console.app (the macOS standard logger) later,
I found that it was segfaulting at requests.get().
Why did it take so long?
Because daemonized script don’t properly log its death even when you pass a file pointer for stderr.
Also, crash report found in Console.app does not provide much information
unless you write most of your program (note the word ‘program’ - this includes not only your script,
but also Python packages and/or even Python itself)!
So the only realistic bet may be to spam print(), which is already tedious
because you need to properly tell DaemonContext to redirect stdout to a file
(I didn’t give any at first, which caused even more headaches).
The solution
It was faulthandler.enable() that I needed to call.
https://docs.python.org/3/library/faulthandler.html
faulthandler — Dump the Python traceback
This causes Python to print stack trace to stderr when signal is received (in macOS, segmentation fault causes a signal SIGSEGV; probably all *nix systems also does.) instead of straight-up dying on the spot. This method enabled me to pinpoint exactly which call was causing my script to crash.
What was the problem, by the way?
Turns out, this is a known, (probably) unavoidable issue when using request package in macOS.
https://github.com/python/cpython/issues/74570
Segfault on OSX with 3.6.1 · Issue #74570 · python/cpython
To avoid this issue, you must drop proxy support as segfault occurs at process related to polling OS’s proxy configuration, which is “not fork safe”.
import os
os.environ['no_proxy'] = "*"