Skip to content

Abort in free-threaded build due to mutation of ChainMap of a Counter in threads #126366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
devdanzin opened this issue Nov 3, 2024 · 6 comments
Closed

Abort in free-threaded build due to mutation of ChainMap of a Counter in threads #126366

devdanzin opened this issue Nov 3, 2024 · 6 comments
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@devdanzin
Copy link
Contributor

devdanzin commented Nov 3, 2024 •
edited by bedevere-app bot
Loading

What happened?

The code below, in a no-gil debug build with PYTHON_GIL=0, results in the following abort:

python: ./Include/internal/pycore_stackref.h:99: _PyStackRef_FromPyObjectSteal: Assertion `obj != NULL' failed.
Aborted

from threading import Thread
from collections import ChainMap, Counter

counter = Counter(range(100))
chainmap = ChainMap(counter)
chainmap2 = ChainMap(counter)

def mutate_removing():
    for x in range(10):
        chainmap.pop(list(chainmap.keys()).pop())
        chainmap2.pop(list(chainmap2.keys()).pop())

def mutate_adding():
    for x in range(50):
        chainmap[max(chainmap) + x + 1] = x
        chainmap2[max(chainmap2) + x + 1] = x

alive = []

for x in range(55):
    alive.append(Thread(target=mutate_removing, args=()))
    alive.append(Thread(target=mutate_adding, args=()))

for obj in alive:
    try:
        print('START', obj)
        obj.start()
    except Exception:
        pass

Found using fusil by @vstinner.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a1+ experimental free-threading build (heads/main:556dc9b8a7, Nov 3 2024, 10:09:47) [GCC 11.4.0]

Linked PRs
@devdanzin devdanzin added the type-crash A hard crash of the interpreter, possibly with a core dump label Nov 3, 2024
@ZeroIntensity
Copy link
Member

It looks like this is some sort of 3.14 regression. On 3.13, it doesn't crash; instead, it just spams RuntimeErrors (which might also be a bug, but let's focus on this first).

This is probably a data race, but that scares me because nearly everything here is pure Python. Valgrind is pointing at native generators being the culprit, I'm looking into it.

@ZeroIntensity ZeroIntensity added the 3.14 new features, bugs and security fixes label Nov 3, 2024
@ZeroIntensity
Copy link
Member

ZeroIntensity commented Nov 3, 2024 •
edited
Loading

Yeah, there's a thread safety issue with native generators (or possibly with list). Here's a smaller reproducer:

from threading import Thread

def my_generator():
    for i in range(100000):
        yield i


def main(gen):
    list(gen)


gener = my_generator()

ts = [Thread(target=main, args=(gener,)) for _ in range(1000)]
for t in ts:
    t.start()

for t in ts:
    t.join()

@ZeroIntensity ZeroIntensity added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Nov 3, 2024
@ZeroIntensity ZeroIntensity mentioned this issue Nov 3, 2024
Merged
@bedevere-app bedevere-app bot mentioned this issue Nov 3, 2024
Draft
@ZeroIntensity
Copy link
Member

I had to split the fix up into two PRs because of backporting: gh-126369 and gh-126371

kumaraditya303 pushed a commit that referenced this issue Nov 5, 2024
@ZeroIntensity
gh-126366: Fix crash if __iter__ raises an exception during `yield …
1371295 
…from` (#126369)
@vstinner
Copy link
Member

vstinner commented Nov 6, 2024

See also #120321 issue and #120327 fix.

@ZeroIntensity
Copy link
Member

Yeah, Sam mentioned that in my (closed) PR. The main concern with merging gh-120327 is that it will have a heavy performance hit on generators. IIRC, the relation with generators in this issue is that collections.abc.Iterable (which ChainMap inherits from) uses a generator in its __iter__.

@picnixz picnixz added the 3.13 bugs and security fixes label Nov 16, 2024
@github-actions github-actions bot mentioned this issue Dec 1, 2024
Closed
picnixz pushed a commit to picnixz/cpython that referenced this issue Dec 8, 2024
@ZeroIntensity @picnixz
pythongh-126366: Fix crash if __iter__ raises an exception during `…
5feed1f 
…yield from` (python#126369)
ebonnal pushed a commit to ebonnal/cpython that referenced this issue Jan 12, 2025
@ZeroIntensity @ebonnal
pythongh-126366: Fix crash if __iter__ raises an exception during `…
7b8f108 
…yield from` (python#126369)
@kumaraditya303
Copy link
Contributor

kumaraditya303 commented Apr 18, 2025 •
edited
Loading

On current main branch it doesn't crash but there is the following tsan warning:

==================
WARNING: ThreadSanitizer: data race (pid=23964)
  Atomic read of size 8 at 0x7fa28a160018 by thread T84:
    #0 _Py_atomic_load_ssize_relaxed /home/realkumaraditya/cpython/./Include/cpython/pyatomic_gcc.h:383:10 (python+0x288082) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #1 dictiter_iternext_threadsafe /home/realkumaraditya/cpython/Objects/dictobject.c:5523:24 (python+0x288082)
    #2 dictiter_iternextkey /home/realkumaraditya/cpython/Objects/dictobject.c:5206:9 (python+0x27d541) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #3 iternext /home/realkumaraditya/cpython/Objects/abstract.c:2870:18 (python+0x1c2444) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #4 PyIter_Next /home/realkumaraditya/cpython/Objects/abstract.c:2920:11 (python+0x1c2444)
    #5 _PyDict_FromKeys /home/realkumaraditya/cpython/Objects/dictobject.c:3217:23 (python+0x276328) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #6 dict_fromkeys_impl /home/realkumaraditya/cpython/Objects/dictobject.c:3616:12 (python+0x287a13) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #7 dict_fromkeys /home/realkumaraditya/cpython/Objects/clinic/dictobject.c.h:36:20 (python+0x287a13)
    #8 cfunction_vectorcall_FASTCALL /home/realkumaraditya/cpython/Objects/methodobject.c:454:24 (python+0x29b180) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #9 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x3ed135) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #10 map_next /home/realkumaraditya/cpython/Python/bltinmodule.c:1482:14 (python+0x3ed135)
    #11 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:5507:36 (python+0x40aa8d) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #12 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f8700) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #13 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1917:12 (python+0x3f8700)
    #14 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1b8f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #15 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f19b2) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #16 PyObject_CallOneArg /home/realkumaraditya/cpython/Objects/call.c:395:12 (python+0x1f19b2)
    #17 call_unbound_noarg /home/realkumaraditya/cpython/Objects/typeobject.c:2892:16 (python+0x2fa002) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #18 maybe_call_special_no_args /home/realkumaraditya/cpython/Objects/typeobject.c:3005:15 (python+0x2fa002)
    #19 slot_tp_iter /home/realkumaraditya/cpython/Objects/typeobject.c:10333:21 (python+0x31a0c5) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #20 PyObject_GetIter /home/realkumaraditya/cpython/Objects/abstract.c:2818:25 (python+0x1c2292) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #21 min_max /home/realkumaraditya/cpython/Python/bltinmodule.c:1927:14 (python+0x3f575d) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #22 builtin_max /home/realkumaraditya/cpython/Python/bltinmodule.c:2036:12 (python+0x3f2bfa) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #23 cfunction_vectorcall_FASTCALL_KEYWORDS /home/realkumaraditya/cpython/Objects/methodobject.c:470:24 (python+0x29b278) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #24 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f152a) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #25 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f152a)
    #26 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1451:35 (python+0x3fcf1c) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #27 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f8700) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #28 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1917:12 (python+0x3f8700)
    #29 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1b8f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #30 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f64af) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #31 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f64af)
    #32 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x450647) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #33 context_run /home/realkumaraditya/cpython/Python/context.c:728:29 (python+0x450647)
    #34 method_vectorcall_FASTCALL_KEYWORDS /home/realkumaraditya/cpython/Objects/descrobject.c:421:24 (python+0x209249) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #35 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f152a) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #36 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f152a)
    #37 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1451:35 (python+0x3fcf1c) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #38 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f8700) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #39 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1917:12 (python+0x3f8700)
    #40 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1b8f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #41 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f64af) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #42 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f64af)
    #43 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f181f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #44 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f181f)
    #45 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f1885) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #46 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x59f532) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #47 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f8e17) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)

  Previous write of size 8 at 0x7fa28a160018 by thread T42:
    #0 new_keys_object /home/realkumaraditya/cpython/Objects/dictobject.c:803:21 (python+0x27f4a4) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #1 dictresize /home/realkumaraditya/cpython/Objects/dictobject.c:2020:15 (python+0x284ffb) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #2 insertion_resize /home/realkumaraditya/cpython/Objects/dictobject.c:1719:12 (python+0x284ce8) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #3 insert_combined_dict /home/realkumaraditya/cpython/Objects/dictobject.c:1728:13 (python+0x284ce8)
    #4 insertdict /home/realkumaraditya/cpython/Objects/dictobject.c:1854:13 (python+0x274354) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #5 setitem_take2_lock_held /home/realkumaraditya/cpython/Objects/dictobject.c:2605:12 (python+0x2738df) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #6 _PyDict_SetItem_Take2 /home/realkumaraditya/cpython/Objects/dictobject.c:2613:11 (python+0x273b0f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #7 PyDict_SetItem /home/realkumaraditya/cpython/Objects/dictobject.c:2633:12 (python+0x273b0f)
    #8 dict_ass_sub /home/realkumaraditya/cpython/Objects/dictobject.c:3432:16 (python+0x28685a) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #9 wrap_objobjargproc /home/realkumaraditya/cpython/Objects/typeobject.c:9346:11 (python+0x31f86f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #10 wrapperdescr_raw_call /home/realkumaraditya/cpython/Objects/descrobject.c:532:12 (python+0x208831) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #11 wrapperdescr_call /home/realkumaraditya/cpython/Objects/descrobject.c:570:14 (python+0x208831)
    #12 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1f09c8) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #13 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x30a1dc) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #14 vectorcall_unbound /home/realkumaraditya/cpython/Objects/typeobject.c:2885:12 (python+0x30a1dc)
    #15 vectorcall_method /home/realkumaraditya/cpython/Objects/typeobject.c:2956:18 (python+0x30a1dc)
    #16 slot_mp_ass_subscript /home/realkumaraditya/cpython/Objects/typeobject.c (python+0x31f750) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #17 PyObject_SetItem /home/realkumaraditya/cpython/Objects/abstract.c:235:19 (python+0x1b9698) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #18 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:11232:27 (python+0x41baf9) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #19 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f8700) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #20 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1917:12 (python+0x3f8700)
    #21 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1b8f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #22 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x30a1a9) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #23 vectorcall_unbound /home/realkumaraditya/cpython/Objects/typeobject.c:2885:12 (python+0x30a1a9)
    #24 vectorcall_method /home/realkumaraditya/cpython/Objects/typeobject.c:2956:18 (python+0x30a1a9)
    #25 slot_mp_ass_subscript /home/realkumaraditya/cpython/Objects/typeobject.c (python+0x31f750) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #26 PyObject_SetItem /home/realkumaraditya/cpython/Objects/abstract.c:235:19 (python+0x1b9698) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #27 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:11232:27 (python+0x41baf9) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #28 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f8700) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #29 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1917:12 (python+0x3f8700)
    #30 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1b8f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #31 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f64af) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #32 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f64af)
    #33 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x450647) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #34 context_run /home/realkumaraditya/cpython/Python/context.c:728:29 (python+0x450647)
    #35 method_vectorcall_FASTCALL_KEYWORDS /home/realkumaraditya/cpython/Objects/descrobject.c:421:24 (python+0x209249) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #36 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f152a) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #37 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f152a)
    #38 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1451:35 (python+0x3fcf1c) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #39 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f8700) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #40 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1917:12 (python+0x3f8700)
    #41 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f1b8f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #42 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f64af) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #43 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f64af)
    #44 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f181f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #45 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f181f)
    #46 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f1885) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #47 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x59f532) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #48 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f8e17) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)

  Thread T84 'Thread-84 (muta' (tid=24055, running) created by main thread at:
    #0 pthread_create <null> (python+0xe11df) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #1 do_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:289:14 (python+0x4f7ca8) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #2 PyThread_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:331:9 (python+0x4f7aca) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #3 ThreadHandle_start /home/realkumaraditya/cpython/./Modules/_threadmodule.c:439:9 (python+0x59f0c7) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #4 do_start_new_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1839:9 (python+0x59f0c7)
    #5 thread_PyThread_start_joinable_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1954:14 (python+0x59de91) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #6 cfunction_call /home/realkumaraditya/cpython/Objects/methodobject.c:569:18 (python+0x29bf67) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #7 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1f09c8) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #8 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f15e8) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #9 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f15e8)
    #10 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:3048:35 (python+0x403190) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #11 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f823f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #12 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1917:12 (python+0x3f823f)
    #13 PyEval_EvalCode /home/realkumaraditya/cpython/Python/ceval.c:829:21 (python+0x3f823f)
    #14 run_eval_code_obj /home/realkumaraditya/cpython/Python/pythonrun.c:1365:12 (python+0x4d59b1) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #15 run_mod /home/realkumaraditya/cpython/Python/pythonrun.c:1436:19 (python+0x4d59b1)
    #16 pyrun_file /home/realkumaraditya/cpython/Python/pythonrun.c:1293:15 (python+0x4d0f40) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #17 _PyRun_SimpleFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:521:13 (python+0x4d0f40)
    #18 _PyRun_AnyFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:81:15 (python+0x4d0698) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #19 pymain_run_file_obj /home/realkumaraditya/cpython/Modules/main.c:402:15 (python+0x51586f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #20 pymain_run_file /home/realkumaraditya/cpython/Modules/main.c:421:15 (python+0x51586f)
    #21 pymain_run_python /home/realkumaraditya/cpython/Modules/main.c:686:21 (python+0x514b9f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #22 Py_RunMain /home/realkumaraditya/cpython/Modules/main.c:767:5 (python+0x514b9f)
    #23 pymain_main /home/realkumaraditya/cpython/Modules/main.c:797:12 (python+0x515108) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #24 Py_BytesMain /home/realkumaraditya/cpython/Modules/main.c:821:12 (python+0x51518b) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #25 main /home/realkumaraditya/cpython/./Programs/python.c:15:12 (python+0x15f7eb) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)

  Thread T42 'Thread-42 (muta' (tid=24013, running) created by main thread at:
    #0 pthread_create <null> (python+0xe11df) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #1 do_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:289:14 (python+0x4f7ca8) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #2 PyThread_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:331:9 (python+0x4f7aca) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #3 ThreadHandle_start /home/realkumaraditya/cpython/./Modules/_threadmodule.c:439:9 (python+0x59f0c7) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #4 do_start_new_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1839:9 (python+0x59f0c7)
    #5 thread_PyThread_start_joinable_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1954:14 (python+0x59de91) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #6 cfunction_call /home/realkumaraditya/cpython/Objects/methodobject.c:569:18 (python+0x29bf67) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #7 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1f09c8) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #8 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f15e8) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #9 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f15e8)
    #10 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:3048:35 (python+0x403190) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #11 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f823f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #12 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1917:12 (python+0x3f823f)
    #13 PyEval_EvalCode /home/realkumaraditya/cpython/Python/ceval.c:829:21 (python+0x3f823f)
    #14 run_eval_code_obj /home/realkumaraditya/cpython/Python/pythonrun.c:1365:12 (python+0x4d59b1) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #15 run_mod /home/realkumaraditya/cpython/Python/pythonrun.c:1436:19 (python+0x4d59b1)
    #16 pyrun_file /home/realkumaraditya/cpython/Python/pythonrun.c:1293:15 (python+0x4d0f40) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #17 _PyRun_SimpleFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:521:13 (python+0x4d0f40)
    #18 _PyRun_AnyFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:81:15 (python+0x4d0698) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #19 pymain_run_file_obj /home/realkumaraditya/cpython/Modules/main.c:402:15 (python+0x51586f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #20 pymain_run_file /home/realkumaraditya/cpython/Modules/main.c:421:15 (python+0x51586f)
    #21 pymain_run_python /home/realkumaraditya/cpython/Modules/main.c:686:21 (python+0x514b9f) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #22 Py_RunMain /home/realkumaraditya/cpython/Modules/main.c:767:5 (python+0x514b9f)
    #23 pymain_main /home/realkumaraditya/cpython/Modules/main.c:797:12 (python+0x515108) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #24 Py_BytesMain /home/realkumaraditya/cpython/Modules/main.c:821:12 (python+0x51518b) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)
    #25 main /home/realkumaraditya/cpython/./Programs/python.c:15:12 (python+0x15f7eb) (BuildId: 4b67e6835168c931af837b36f025fc641b3d86d6)

SUMMARY: ThreadSanitizer: data race /home/realkumaraditya/cpython/./Include/cpython/pyatomic_gcc.h:383:10 in _Py_atomic_load_ssize_relaxed
==================

I think this is because in dictiter_iternext_threadsafe load of dict keys is done with relaxed ordering but it should be done with acquire ordering because keys are stored using release ordering when resizing. Changing the load from relaxed to acquire memory ordering fixes the race.

@bedevere-app bedevere-app bot mentioned this issue Apr 18, 2025
Merged
kumaraditya303 added a commit that referenced this issue Apr 18, 2025
@kumaraditya303
gh-126366: fix thread safety of dict key iterator (#132693)
4c3d187
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-crash A hard crash of the interpreter, possibly with a core dump
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@vstinner @JelleZijlstra @picnixz @ZeroIntensity @kumaraditya303 @devdanzin