Skip to content

Commit 66a1efe

Browse files
paulrosspaulross
committed
Adds example of using Py_AtExit()
Minor clean up and corrections of documentation. Fix error in compiler_flags section. Rename cExcep.c as cExceptions.c
1 parent ec1cf8d commit 66a1efe

File tree

10 files changed

+774
-350
lines changed

10 files changed

+774
-350
lines changed

‎PythonExtensionPatterns.bbprojectd/paulross.bbprojectsettings

+335-239
Large diffs are not rendered by default.

‎doc/sphinx/source/compiler_flags.rst

+2-1
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,8 @@ The sysconfig module allows you to create a generif setup.py script for Python C
122122
_DEBUG_LEVEL = 0
123123
124124
# Common flags for both release and debug builds.
125-
extra_compile_args=[sysconfig.get_config_var('CFLAGS'), "-std=c++11", "-Wall", "-Wextra"]
125+
extra_compile_args = sysconfig.get_config_var('CFLAGS').split()
126+
extra_compile_args += ["-std=c++11", "-Wall", "-Wextra"]
126127
if _DEBUG:
127128
extra_compile_args += ["-g3", "-O0", "-DDEBUG=%s" % _DEBUG_LEVEL, "-UNDEBUG"]
128129
else:

‎doc/sphinx/source/debugging/debug.rst

+1
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,4 @@ This is very much work in progress. I will add to it/correct it as I develop new
2020
debug_tactics
2121
gcov
2222
debug_in_ide
23+
pyatexit

‎doc/sphinx/source/debugging/gcov.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ In the Terminal navigate to the 'Build' directory:
4646

4747
.. code-block:: bash
4848
49-
$ cd cd /Users/$USER/Library/Developer/Xcode/DerivedData/<project name and hash>/Build/
49+
$ cd /Users/$USER/Library/Developer/Xcode/DerivedData/<project name and hash>/Build/
5050
5151
Now navigate to the Intermediates/ directory and in there you should find the code coverage data in a path such as this:
5252

‎doc/sphinx/source/debugging/pyatexit.rst

+287
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
.. highlight:: python
2+
:linenothreshold: 10
3+
4+
.. highlight:: c
5+
:linenothreshold: 10
6+
7+
.. toctree::
8+
:maxdepth: 3
9+
10+
=======================================================
11+
Instrumenting the Python Process for Your Structures
12+
=======================================================
13+
14+
Some debugging problems can be solved by instrumenting your C extensions for the duration of the Python process and reporting what happened when the process terminates. The data could be: the number of times classes were instantiated, functions called, memory allocations/deallocations or anything else that you wish.
15+
16+
To take a simple case, suppose we have a class that implements a up/down counter and we want to count how often each ``inc()`` and ``dec()`` function is called during the entirety of the Python process. We will create a C extension that has a class that has a single member (an interger) and two functions that increment or decrement that number. If it was in Python it would look like this:
17+
18+
.. code-block:: python
19+
20+
class Counter:
21+
def __init__(self, count=0):
22+
self.count = count
23+
24+
def inc(self):
25+
self.count += 1
26+
27+
def dec(self):
28+
self.count -= 1
29+
30+
What we would like to do is to count how many times ``inc()`` and ``dec()`` are called on *all* instances of these objects and summarise them when the Python process exits [#f1]_.
31+
32+
There is an interpreter hook ``Py_AtExit()`` that allows you to register C functions that will be executed as the Python interpreter exits. This allows you to dump information that you have gathered about your code execution.
33+
34+
-------------------------------------------
35+
An Implementation of a Counter
36+
-------------------------------------------
37+
38+
First here is the module ``pyatexit`` with the class ``pyatexit.Counter`` with no intrumentation (it is equivelent to the Python code above). We will add the instrumentation later:
39+
40+
.. code-block:: c
41+
42+
#include <Python.h>
43+
#include "structmember.h"
44+
45+
#include <stdio.h>
46+
47+
typedef struct {
48+
PyObject_HEAD int number;
49+
} Py_Counter;
50+
51+
static void Py_Counter_dealloc(Py_Counter* self) {
52+
Py_TYPE(self)->tp_free((PyObject*)self);
53+
}
54+
55+
static PyObject* Py_Counter_new(PyTypeObject* type, PyObject* args,
56+
PyObject* kwds) {
57+
Py_Counter* self;
58+
self = (Py_Counter*)type->tp_alloc(type, 0);
59+
if (self != NULL) {
60+
self->number = 0;
61+
}
62+
return (PyObject*)self;
63+
}
64+
65+
static int Py_Counter_init(Py_Counter* self, PyObject* args, PyObject* kwds) {
66+
static char* kwlist[] = { "number", NULL };
67+
if (!PyArg_ParseTupleAndKeywords(args, kwds, "|i", kwlist, &self->number)) {
68+
return -1;
69+
}
70+
return 0;
71+
}
72+
73+
static PyMemberDef Py_Counter_members[] = {
74+
{ "count", T_INT, offsetof(Py_Counter, number), 0, "count value" },
75+
{ NULL, 0, 0, 0, NULL } /* Sentinel */
76+
};
77+
78+
static PyObject* Py_Counter_inc(Py_Counter* self) {
79+
self->number++;
80+
Py_RETURN_NONE;
81+
}
82+
83+
static PyObject* Py_Counter_dec(Py_Counter* self) {
84+
self->number--;
85+
Py_RETURN_NONE;
86+
}
87+
88+
static PyMethodDef Py_Counter_methods[] = {
89+
{ "inc", (PyCFunction)Py_Counter_inc, METH_NOARGS, "Increments the counter" },
90+
{ "dec", (PyCFunction)Py_Counter_dec, METH_NOARGS, "Decrements the counter" },
91+
{ NULL, NULL, 0, NULL } /* Sentinel */
92+
};
93+
94+
static PyTypeObject Py_CounterType = {
95+
PyVarObject_HEAD_INIT(NULL, 0) "pyatexit.Counter", /* tp_name */
96+
sizeof(Py_Counter), /* tp_basicsize */
97+
0, /* tp_itemsize */
98+
(destructor)Py_Counter_dealloc, /* tp_dealloc */
99+
0, /* tp_print */
100+
0, /* tp_getattr */
101+
0, /* tp_setattr */
102+
0, /* tp_reserved */
103+
0, /* tp_repr */
104+
0, /* tp_as_number */
105+
0, /* tp_as_sequence */
106+
0, /* tp_as_mapping */
107+
0, /* tp_hash */
108+
0, /* tp_call */
109+
0, /* tp_str */
110+
0, /* tp_getattro */
111+
0, /* tp_setattro */
112+
0, /* tp_as_buffer */
113+
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */
114+
"Py_Counter objects", /* tp_doc */
115+
0, /* tp_traverse */
116+
0, /* tp_clear */
117+
0, /* tp_richcompare */
118+
0, /* tp_weaklistoffset */
119+
0, /* tp_iter */
120+
0, /* tp_iternext */
121+
Py_Counter_methods, /* tp_methods */
122+
Py_Counter_members, /* tp_members */
123+
0, /* tp_getset */
124+
0, /* tp_base */
125+
0, /* tp_dict */
126+
0, /* tp_descr_get */
127+
0, /* tp_descr_set */
128+
0, /* tp_dictoffset */
129+
(initproc)Py_Counter_init, /* tp_init */
130+
0, /* tp_alloc */
131+
Py_Counter_new, /* tp_new */
132+
0, /* tp_free */
133+
};
134+
135+
static PyModuleDef pyexitmodule = {
136+
PyModuleDef_HEAD_INIT, "pyatexit",
137+
"Extension that demonstrates the use of Py_AtExit().",
138+
-1, NULL, NULL, NULL, NULL,
139+
NULL
140+
};
141+
142+
PyMODINIT_FUNC PyInit_pyatexit(void) {
143+
PyObject* m;
144+
145+
if (PyType_Ready(&Py_CounterType) < 0) {
146+
return NULL;
147+
}
148+
m = PyModule_Create(&pyexitmodule);
149+
if (m == NULL) {
150+
return NULL;
151+
}
152+
Py_INCREF(&Py_CounterType);
153+
PyModule_AddObject(m, "Counter", (PyObject*)&Py_CounterType);
154+
return m;
155+
}
156+
157+
If this was a file ``Py_AtExitDemo.c`` then a Python ``setup.py`` file might look like this:
158+
159+
.. code-block:: python
160+
161+
from distutils.core import setup, Extension
162+
setup(
163+
ext_modules=[
164+
Extension("pyatexit", sources=['Py_AtExitDemo.c']),
165+
]
166+
)
167+
168+
Building this with ``python3 setup.py build_ext --inplace`` we can check everything works as expected:
169+
170+
.. code-block:: python
171+
172+
>>> import pyatexit
173+
>>> c = pyatexit.Counter(8)
174+
>>> c.inc()
175+
>>> c.inc()
176+
>>> c.dec()
177+
>>> c.count
178+
9
179+
>>> d = pyatexit.Counter()
180+
>>> d.dec()
181+
>>> d.dec()
182+
>>> d.count
183+
-2
184+
>>> ^D
185+
186+
-------------------------------------------
187+
Instrumenting the Counter
188+
-------------------------------------------
189+
190+
To add the instrumentation we will declare a macro ``COUNT_ALL_DEC_INC`` to control whether the compilation includes instrumentation.
191+
192+
.. code-block:: c
193+
194+
#define COUNT_ALL_DEC_INC
195+
196+
In the global area of the file declare some global counters and a function to write them out on exit. This must be a ``void`` function taking no arguments:
197+
198+
.. code-block:: c
199+
200+
#ifdef COUNT_ALL_DEC_INC
201+
/* Counters for operations and a function to dump them at Python process end. */
202+
static size_t count_inc = 0;
203+
static size_t count_dec = 0;
204+
205+
static void dump_inc_dec_count(void) {
206+
fprintf(stdout, "==== dump_inc_dec_count() ====\n");
207+
fprintf(stdout, "Increments: %" PY_FORMAT_SIZE_T "d\n", count_inc);
208+
fprintf(stdout, "Decrements: %" PY_FORMAT_SIZE_T "d\n", count_dec);
209+
fprintf(stdout, "== dump_inc_dec_count() END ==\n");
210+
}
211+
#endif
212+
213+
In the ``Py_Counter_new`` function we add some code to register this function. This must be only done once so we use the static ``has_registered_exit_function`` to guard this:
214+
215+
.. code-block:: c
216+
217+
static PyObject* Py_Counter_new(PyTypeObject* type, PyObject* args,
218+
PyObject* kwds) {
219+
Py_Counter* self;
220+
#ifdef COUNT_ALL_DEC_INC
221+
static int has_registered_exit_function = 0;
222+
if (! has_registered_exit_function) {
223+
if (Py_AtExit(dump_inc_dec_count)) {
224+
return NULL;
225+
}
226+
has_registered_exit_function = 1;
227+
}
228+
#endif
229+
self = (Py_Counter*)type->tp_alloc(type, 0);
230+
if (self != NULL) {
231+
self->number = 0;
232+
}
233+
return (PyObject*)self;
234+
}
235+
236+
.. note::
237+
``Py_AtExit`` can take, at most, 32 functions. If the function can not be registered then ``Py_AtExit`` will return -1.
238+
239+
.. warning::
240+
Since Python’s internal finalization will have completed before the cleanup function, no Python APIs should be called by any registered function.
241+
242+
243+
Now we modify the ``inc()`` and ``dec()`` functions thus:
244+
245+
.. code-block:: c
246+
247+
static PyObject* Py_Counter_inc(Py_Counter* self) {
248+
self->number++;
249+
#ifdef COUNT_ALL_DEC_INC
250+
count_inc++;
251+
#endif
252+
Py_RETURN_NONE;
253+
}
254+
255+
static PyObject* Py_Counter_dec(Py_Counter* self) {
256+
self->number--;
257+
#ifdef COUNT_ALL_DEC_INC
258+
count_dec++;
259+
#endif
260+
Py_RETURN_NONE;
261+
}
262+
263+
Now when we build this extension and run it we see the following:
264+
265+
.. code-block:: python
266+
267+
>>> import pyatexit
268+
>>> c = pyatexit.Counter(8)
269+
>>> c.inc()
270+
>>> c.inc()
271+
>>> c.dec()
272+
>>> c.count
273+
9
274+
>>> d = pyatexit.Counter()
275+
>>> d.dec()
276+
>>> d.dec()
277+
>>> d.count
278+
-2
279+
>>> ^D
280+
==== dump_inc_dec_count() ====
281+
Increments: 2
282+
Decrements: 3
283+
== dump_inc_dec_count() END ==
284+
285+
.. rubric:: Footnotes
286+
287+
.. [#f1] The ``atexit`` module in Python can be used to similar effect however registered functions are called at a different stage of interpreted teardown than ``Py_AtExit``.

‎src/cCanonical.c

-6
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,6 @@
88

99
#include "Python.h"
1010

11-
12-
#define ASSERT_EXCEPTION_RETURN_VALUE(ret) assert ret && ! PyErr_Occurred() \
13-
|| !ret && PyErr_Occurred()
14-
15-
16-
1711
/* So the canonical form is: */
1812
static PyObject *_func(PyObject *arg1) {
1913
/* Create any local Python *objects as NULL. */

0 commit comments

Comments
 (0)