How do I dump an entire Python process for later debugging inspection?
I have a Python application in a strange state. I don't want to do live debugging of the process. Can I dump it to a file and examine its state later? I know I've restored corefiles of C programs in gdb later, but I don't know how to examine a Python application in a useful way from gdb.
(This is a variation on my question about debugging memleaks in a production system.)
Asked by: Sawyer613 | Posted: 27-01-2022
Answer 1
There is no builtin way other than aborting (with os.abort(), causing the coredump if resource limits allow it) -- although you can certainly build your own 'dump' function that dumps relevant information about the data you care about. There are no ready-made tools for it.
As for handling the corefile of a Python process, the Python source has a gdbinit file that contains useful macros. It's still a lot more painful than somehow getting into the process itself (with pdb or the interactive interpreter) but it makes life a little easier.
Answered by: Anna109 | Posted: 28-02-2022Answer 2
If you only care about storing the traceback object (which is all you need to start a debugging session), you can use debuglater (a fork of pydump). It works with recent versions of Python and has a IPython/Jupyter integration.
If you want to store the entire session, look at dill. It has a dump_session, and load_session functions.
Here are two other relevant projects:
If you're looking for a language agnostic solution, you want to create a core dump file. Here's an example with Python.
Answered by: Rafael757 | Posted: 28-02-2022Answer 3
Someone above said that there is no builtin way to perform this, but that's not entirely true. For an example, you could take a look at the pylons debugging tools. Whene there is an exception, the exception handler saves the stack trace and prints a URL on the console that can be used to retrieve the debugging session over HTTP.
While they're probably keeping these sessions in memory, they're just python objects, so there's nothing to stop you from pickling a stack dump and restoring it later for inspection. It would mean some changes to the app, but it should be possible...
After some research, it turns out the relevant code is actually coming from Paste's EvalException module. You should be able to look there to figure out what you need.
Answered by: Miller996 | Posted: 28-02-2022Answer 4
It's also possible to write something that would dump all the data from the process, e.g.
- Pickler that ignores the objects it can't pickle (replacing them with something else) (e.g. Python: Pickling a dict with some unpicklable items)
- Method that recursively converts everything into serializable stuff (e.g. this, except it needs a check for infinitely recursing objects and do something with those; also it could try
dir()andgetattr()to process some of the unknown objects, e.g. extension classes).
But leaving a running process with manhole or pylons or something like that certainly seems more convenient when possible.
(also, I wonder if something more convenient was written since this question was first asked)
Answered by: Alberta338 | Posted: 28-02-2022Answer 5
This answer suggests making your program core dump and then continuing execution on another sufficiently similar box.
Answered by: Emily631 | Posted: 28-02-2022Similar questions
python - Is there a way to do runtime inspection for django apps? (Any IDE that does this?)
I want to do runtime inspection for django apps, meaning, I want to run an app and be able to break/step through/inspect runtime variables.
Is there any IDE that support this or anyway to commandline run django to do this?
I know that the Django shell exists, however, that just sets up an environment and doesn't provide inspection of running code.
Thanks you
python: how to get the class of a calling method through inspection?
For my exception class I'd like to find out whether the function which instantiated the exception object is a method and if so, show the class name.
So in the init method of my exception class I get the name of the calling function:
frame, module, line, function, context, index = inspect.stack()[1]
But is there any way the get the class name (if any) of the calling...
Meaning of <..> in Python inspection
In an IPython session, I have the following:
> my_array?
Type: ndarray
String Form:
[[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.] <...> [ 1.]
[ 1.]
[ 1.]
[ 1.]
[ 1.]
...
python - how to get class name of a class static method through inspection
I found same similar question, but my question is diffent
def trace ():
?
class A():
@staticmethod
def Aha():
trace ()
...
I want the trace output
A.Aha() was called
I have known how to get function name through inspection, and get class name of instance.method, such this:
self_argument = frame.f_co...
python - PyCharm Code Inspection doesn't include PEP 8
How do I get PyCharm 2.7 Code Inspection (under Code > Inspect Code) to include PEP 8 messages?
In the Inspection settings I have "PEP 8 coding style violation" checked and the correct profile selected, but running the inspection does not include any PEP 8 violations.
What am I doing wrong?
python - How to avoid Pycharm inspection error in multiline SQL literals
I recently discovered that PyCharm complains about " expected, unexpected end of file" when I write a SQL Query, that spans multiple lines (using parantheses):
It looks like it detects that this is a SQL statement, but only parses the first line.
Any idea how to correctly write this without disabling inspections or writing very long li...
python - removing py files and retaining pyc files breaks inspection code
The function below works just fine. But if I remove all py files (and leave the pycs intact) then I get an error:
To explain what I mean by 'intact' here is more or less what I did:
1. write a bunch of py files and stick them in a friendly directory structure
2. test code code. It works
3. compile all py files to get pyc files
4. delete py files
5. test code. It fails
The function...
python - Use PyCharm inspection from command line script
I want to incorporate PyCharm CE's code-inspection functionality into my continuous-integration server setup.
For that purpose I would need to call the IDE from the command-line, let it load the project, analyse and have it dump the warnings to stdout or to a file. I imagine something similar to how pylint already works.
Is something like this possible, and if so, how?
Python Inspection Error with custom type with Operator Overloads
Issue: I cannot seem to resolve some Python code inspection errors with type checking.
Background: In my particular application there is a "Galaxy" grid, each Sector of which is another grid of smaller positions. In other words, you can view the map as either an entire global grid (say 100x100), or as a 10x10 grid of "Sectors" that make up the Galaxy where each Sector is itself a 10x10 grid of points.
I am...
python - "This inspection detects instance attribute definition outside __init__ method" PyCharm
I'm using the following class to connect and create a cursor within a firebase database:
class Firebird:
username= "..."
password= "..."
def __init__(self, archive):
self.archive = archive
def connect(self):
try:
self.connection = connect(dsn=self.archive, user=self.username, password=self.password)
except Error, e:
print "Failed to connect to da...
Still can't find your answer? Check out these communities...
PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python