Skip to content

Commit 610a044

Browse files
paulrosspaulross
committed
Review edits of context_managers.rst. Build all documentation with one cross-reference error.
1 parent cf3cbab commit 610a044

File tree

5 files changed

+248
-55
lines changed

5 files changed

+248
-55
lines changed

‎doc/sphinx/source/context_manager.rst

Lines changed: 229 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@
44
.. toctree::
55
:maxdepth: 3
66

7+
.. _context manger: https://docs.python.org/3/glossary.html#term-context-manager>
8+
.. _\__enter__(): https://docs.python.org/3/library/stdtypes.html#contextmanager.__enter__
9+
.. _\__exit__(): https://docs.python.org/3/library/stdtypes.html#contextmanager.__exit__
10+
711
.. _chapter_context_manager:
812

913
.. index::
@@ -14,7 +18,6 @@ Context Managers
1418
***************************
1519

1620
This chapter describes how to write
17-
`context mangers <https://docs.python.org/3/glossary.html#term-context-manager>`_
1821
for your C objects.
1922

2023
.. index::
@@ -24,8 +27,7 @@ for your C objects.
2427
C Functions
2528
===========================
2629

27-
This is a summary of what is required for the C functions implementing a context
28-
manager.
30+
This is a summary of what is required for the C functions implementing a `context manger`_.
2931
The is no specific ``tp_...`` slot for the context manager functions ``__enter__`` and ``__exit__``, instead they are added
3032
to the object as named, looked up, Python methods.
3133

@@ -36,19 +38,6 @@ to the object as named, looked up, Python methods.
3638
``__enter__``
3739
--------------------------------------
3840

39-
Note that ``__enter__`` is declared with ``METH_NOARGS``.
40-
Link to the Python documentation for
41-
`__enter__ <https://docs.python.org/3/library/stdtypes.html#contextmanager.__enter__>`_.
42-
43-
.. code-block:: c
44-
45-
static PyMethodDef ContextManager_methods[] = {
46-
{"__enter__", (PyCFunction) ContextManager_enter, METH_NOARGS,
47-
PyDoc_STR("__enter__() -> ContextManager")},
48-
/* ... */
49-
{NULL, NULL, 0, NULL} /* sentinel */
50-
};
51-
5241
The C function must, at least, increment the reference count of ``self`` and
5342
return ``self``:
5443

@@ -68,20 +57,8 @@ return ``self``:
6857
``__exit__``
6958
--------------------------------------
7059

71-
The ``__exit__`` function is declared thus.
60+
The `__exit__()`_ function is declared thus.
7261
It takes three arguments so ``METH_VARARGS`` is used.
73-
Link to the Python documentation for
74-
`__exit__ <https://docs.python.org/3/library/stdtypes.html#contextmanager.__exit__>`_.
75-
76-
.. code-block:: c
77-
78-
static PyMethodDef ContextManager_methods[] = {
79-
{"__exit__", (PyCFunction) ContextManager_exit, METH_VARARGS,
80-
PyDoc_STR("__exit__(exc_type, exc_value, exc_tb) -> bool")},
81-
/* ... */
82-
{NULL, NULL, 0, NULL} /* sentinel */
83-
};
84-
8562
The three arguments are each ``None`` if no exception has been raised within
8663
the ``with`` block.
8764
If an exception *has* been raised within the ``with`` block then the
@@ -109,6 +86,24 @@ CPython interpreter:
10986
Py_RETURN_FALSE;
11087
}
11188
89+
--------------------------------------
90+
Method declarations
91+
--------------------------------------
92+
93+
Note that `__enter__()`_ is declared with ``METH_NOARGS`` and `__exit__()`_ is declared with ``METH_VARARGS``:
94+
95+
.. code-block:: c
96+
97+
static PyMethodDef ContextManager_methods[] = {
98+
/* ... */
99+
{"__enter__", (PyCFunction) ContextManager_enter, METH_NOARGS,
100+
PyDoc_STR("__enter__() -> ContextManager")},
101+
{"__exit__", (PyCFunction) ContextManager_exit, METH_VARARGS,
102+
PyDoc_STR("__exit__(exc_type, exc_value, exc_tb) -> bool")},
103+
/* ... */
104+
{NULL, NULL, 0, NULL} /* sentinel */
105+
};
106+
112107
=================================
113108
Understanding the Context Manager
114109
=================================
@@ -153,14 +148,14 @@ The sequence of reference count changes are as follows:
153148
(to 2) and then call ``__enter__`` that is implemented in our C function
154149
``ContextManager_enter``.
155150
#. Our ``ContextManager_enter`` function increments the reference count, so it is now 3.
156-
#. As the context manager exists the scope of the ``with`` statement the CPython interpreter
151+
#. As the context manager ends the ``with`` statement the CPython interpreter
157152
decrements the reference count *twice* to the value 1.
158153
The logic is:
159154

160155
#. Decrement the reference count once as we are exiting the ``with`` statement. The reference count is now 2.
161156
#. Did the ``with`` statement have a target? If not, as in this case, then decrement the reference count once more. The reference count is now 1.
162157

163-
#. The CPython interpreter then calls ``__exit__`` which is implemented in our function
158+
#. after the ``pass`` statement the CPython interpreter then calls ``__exit__`` which is implemented in our function
164159
``ContextManager_exit``.
165160
This does not change the reference count which remains at 1.
166161
#. As the context manager goes out of scope the CPython interpreter decrements the reference
@@ -211,9 +206,9 @@ The sequence of reference count changes are now as follows:
211206
#. As above, the reference count becomes 1.
212207
#. As above, the reference count becomes 2.
213208
#. As above, the reference count becomes 3.
214-
#. As the context manager exists the scope of the ``with`` statement the CPython interpreter
215-
decrements the reference count just *once* to the value 2 as there *is* a target.
216-
#. The CPython interpreter then calls ``__exit__`` which is implemented in our function
209+
#. As the context manager ends the ``with`` statement the CPython interpreter
210+
decrements the reference count just *once* to the value 2 as there *is* a target, called ``context`` in this case.
211+
#. After the ``pass`` statement the CPython interpreter then calls ``__exit__`` which is implemented in our function
217212
``ContextManager_exit``.
218213
This does not change the reference count which remains at 2.
219214
#. As the context manager goes out of scope the CPython interpreter decrements the reference
@@ -256,6 +251,10 @@ First the object declaration, allocation and de-allocation functions:
256251
PyObject_Del(self);
257252
}
258253
254+
-----------------------------------------
255+
``__enter__`` and ``__exit__`` Methods
256+
-----------------------------------------
257+
259258
The ``__enter__`` and ``__exit__`` methods:
260259

261260
.. code-block:: c
@@ -279,6 +278,11 @@ The ``__enter__`` and ``__exit__`` methods:
279278
{NULL, NULL, 0, NULL} /* sentinel */
280279
};
281280
281+
282+
-----------------------------------------
283+
Type Declaration
284+
-----------------------------------------
285+
282286
The type declaration:
283287

284288
.. code-block:: c
@@ -293,6 +297,9 @@ The type declaration:
293297
.tp_new = (newfunc) ContextManager_new,
294298
};
295299
300+
-----------------------------------------
301+
Module
302+
-----------------------------------------
296303

297304
Finally the module:
298305

@@ -326,11 +333,9 @@ Finally the module:
326333
return NULL;
327334
}
328335
329-
.. note::
330-
331-
The actual code in ``src/cpy/CtxMgr/cCtxMgr.c`` contains extra trace
332-
reporting that confirms the reference counts and (no) memory leakage.
333-
336+
-----------------------------------------
337+
Setup
338+
-----------------------------------------
334339

335340
This code is added to the ``setup.py`` file:
336341

@@ -351,3 +356,188 @@ And can be used thus:
351356
352357
with cCtxMgr.ContextManager():
353358
pass
359+
360+
-----------------------------------------
361+
Testing
362+
-----------------------------------------
363+
364+
The actual code in ``src/cpy/CtxMgr/cCtxMgr.c`` contains extra trace reporting that confirms the reference counts and
365+
(no) memory leakage.
366+
367+
This can be run with:
368+
369+
.. code-block:: bash
370+
371+
$ pytest tests/unit/test_c_ctxmgr.py -vs
372+
373+
This test:
374+
375+
.. code-block:: python
376+
377+
def test_very_simple():
378+
print()
379+
with cCtxMgr.ContextManager():
380+
pass
381+
382+
Gives this output:
383+
384+
.. code-block:: text
385+
386+
tests/unit/test_c_ctxmgr.py::test_very_simple
387+
ContextManager_new DONE REFCNT = 1
388+
ContextManager_enter STRT REFCNT = 2
389+
ContextManager_enter DONE REFCNT = 3
390+
ContextManager_exit STRT REFCNT = 1
391+
ContextManager_exit DONE REFCNT = 1
392+
ContextManager_dealloc STRT REFCNT = 0
393+
ContextManager_dealloc DONE REFCNT = 4546088432
394+
395+
This test:
396+
397+
.. code-block:: python
398+
399+
def test_simple():
400+
print()
401+
with cCtxMgr.ContextManager() as context:
402+
assert sys.getrefcount(context) == 3
403+
assert context.len_buffer_lifetime() == cCtxMgr.BUFFER_LENGTH
404+
assert context.len_buffer_context() == cCtxMgr.BUFFER_LENGTH
405+
assert sys.getrefcount(context) == 2
406+
assert context.len_buffer_lifetime() == cCtxMgr.BUFFER_LENGTH
407+
assert context.len_buffer_context() == 0
408+
del context
409+
410+
Gives this output:
411+
412+
.. code-block:: text
413+
414+
tests/unit/test_c_ctxmgr.py::test_simple
415+
ContextManager_new DONE REFCNT = 1
416+
ContextManager_enter STRT REFCNT = 2
417+
ContextManager_enter DONE REFCNT = 3
418+
ContextManager_exit STRT REFCNT = 2
419+
ContextManager_exit DONE REFCNT = 2
420+
ContextManager_dealloc STRT REFCNT = 0
421+
ContextManager_dealloc DONE REFCNT = 4546096048
422+
423+
424+
This test:
425+
426+
.. code-block:: python
427+
428+
def test_memory():
429+
proc = psutil.Process()
430+
print()
431+
print(f'RSS START: {proc.memory_info().rss:12,d}')
432+
for i in range(8):
433+
print(f'RSS START {i:5d}: {proc.memory_info().rss:12,d}')
434+
with cCtxMgr.ContextManager() as context:
435+
print(f'RSS START CTX: {proc.memory_info().rss:12,d}')
436+
# Does not work in the debugger due to introspection.
437+
# assert sys.getrefcount(context) == 3
438+
assert context.len_buffer_lifetime() == cCtxMgr.BUFFER_LENGTH
439+
assert context.len_buffer_context() == cCtxMgr.BUFFER_LENGTH
440+
print(f'RSS END CTX: {proc.memory_info().rss:12,d}')
441+
# Does not work in the debugger due to introspection.
442+
# assert sys.getrefcount(context) == 2
443+
assert context.len_buffer_lifetime() == cCtxMgr.BUFFER_LENGTH
444+
assert context.len_buffer_context() == 0
445+
del context
446+
print(f'RSS END {i:5d}: {proc.memory_info().rss:12,d}')
447+
print(f'RSS END: {proc.memory_info().rss:12,d}')
448+
449+
Gives this output:
450+
451+
.. code-block:: text
452+
453+
tests/unit/test_c_ctxmgr.py::test_memory
454+
RSS START: 300,032,000
455+
RSS START 0: 300,048,384
456+
ContextManager_new DONE REFCNT = 1
457+
ContextManager_enter STRT REFCNT = 2
458+
ContextManager_enter DONE REFCNT = 3
459+
RSS START CTX: 300,048,384
460+
RSS END CTX: 300,048,384
461+
ContextManager_exit STRT REFCNT = 2
462+
ContextManager_exit DONE REFCNT = 2
463+
ContextManager_dealloc STRT REFCNT = 0
464+
ContextManager_dealloc DONE REFCNT = 4546096048
465+
RSS END 0: 300,048,384
466+
RSS START 1: 300,048,384
467+
ContextManager_new DONE REFCNT = 1
468+
ContextManager_enter STRT REFCNT = 2
469+
ContextManager_enter DONE REFCNT = 3
470+
RSS START CTX: 300,048,384
471+
RSS END CTX: 300,048,384
472+
ContextManager_exit STRT REFCNT = 2
473+
ContextManager_exit DONE REFCNT = 2
474+
ContextManager_dealloc STRT REFCNT = 0
475+
ContextManager_dealloc DONE REFCNT = 4546096048
476+
RSS END 1: 300,048,384
477+
RSS START 2: 300,048,384
478+
ContextManager_new DONE REFCNT = 1
479+
ContextManager_enter STRT REFCNT = 2
480+
ContextManager_enter DONE REFCNT = 3
481+
RSS START CTX: 300,048,384
482+
RSS END CTX: 300,048,384
483+
ContextManager_exit STRT REFCNT = 2
484+
ContextManager_exit DONE REFCNT = 2
485+
ContextManager_dealloc STRT REFCNT = 0
486+
ContextManager_dealloc DONE REFCNT = 4546096048
487+
RSS END 2: 300,048,384
488+
RSS START 3: 300,048,384
489+
ContextManager_new DONE REFCNT = 1
490+
ContextManager_enter STRT REFCNT = 2
491+
ContextManager_enter DONE REFCNT = 3
492+
RSS START CTX: 300,048,384
493+
RSS END CTX: 300,048,384
494+
ContextManager_exit STRT REFCNT = 2
495+
ContextManager_exit DONE REFCNT = 2
496+
ContextManager_dealloc STRT REFCNT = 0
497+
ContextManager_dealloc DONE REFCNT = 4546096048
498+
RSS END 3: 300,048,384
499+
RSS START 4: 300,048,384
500+
ContextManager_new DONE REFCNT = 1
501+
ContextManager_enter STRT REFCNT = 2
502+
ContextManager_enter DONE REFCNT = 3
503+
RSS START CTX: 300,048,384
504+
RSS END CTX: 300,048,384
505+
ContextManager_exit STRT REFCNT = 2
506+
ContextManager_exit DONE REFCNT = 2
507+
ContextManager_dealloc STRT REFCNT = 0
508+
ContextManager_dealloc DONE REFCNT = 4546096048
509+
RSS END 4: 300,048,384
510+
RSS START 5: 300,048,384
511+
ContextManager_new DONE REFCNT = 1
512+
ContextManager_enter STRT REFCNT = 2
513+
ContextManager_enter DONE REFCNT = 3
514+
RSS START CTX: 300,048,384
515+
RSS END CTX: 300,048,384
516+
ContextManager_exit STRT REFCNT = 2
517+
ContextManager_exit DONE REFCNT = 2
518+
ContextManager_dealloc STRT REFCNT = 0
519+
ContextManager_dealloc DONE REFCNT = 4546096048
520+
RSS END 5: 300,048,384
521+
RSS START 6: 300,048,384
522+
ContextManager_new DONE REFCNT = 1
523+
ContextManager_enter STRT REFCNT = 2
524+
ContextManager_enter DONE REFCNT = 3
525+
RSS START CTX: 300,048,384
526+
RSS END CTX: 300,048,384
527+
ContextManager_exit STRT REFCNT = 2
528+
ContextManager_exit DONE REFCNT = 2
529+
ContextManager_dealloc STRT REFCNT = 0
530+
ContextManager_dealloc DONE REFCNT = 4546096048
531+
RSS END 6: 300,048,384
532+
RSS START 7: 300,048,384
533+
ContextManager_new DONE REFCNT = 1
534+
ContextManager_enter STRT REFCNT = 2
535+
ContextManager_enter DONE REFCNT = 3
536+
RSS START CTX: 300,048,384
537+
RSS END CTX: 300,048,384
538+
ContextManager_exit STRT REFCNT = 2
539+
ContextManager_exit DONE REFCNT = 2
540+
ContextManager_dealloc STRT REFCNT = 0
541+
ContextManager_dealloc DONE REFCNT = 4546096048
542+
RSS END 7: 300,048,384
543+
RSS END: 300,048,384

0 commit comments

Comments
 (0)