This week at work we ran into a problem where one of our Python processes was consuming close to 3 GB of memory because it’s not properly cleaning up a list. Because of other bugs this process could not be easily restarted without triggering other problems, so our core team asked for some suggestions and I told them “Why don’t you try cleaning up the Python list using GDB and the Python C API ?” I had a vague recollection of someone on our team doing something like this a few years ago.
I also asked them to blog about it, because there aren’t that many resources readily findable on the subject.
So here is Andoni’s take on the problem.
If any Pythonista can suggest how he could have avoided the segfault during garbage collection, please let us know!