Skip to content

gh-159449 - Fix asyncio task unregister race 3.14 - #159450

Closed
jblespiau wants to merge 3451 commits into
python:mainfrom
jblespiau:fix-asyncio-task-unregister-race-3.14
Closed

jblespiau wants to merge 3451 commits into
python:mainfrom
jblespiau:fix-asyncio-task-unregister-race-3.14

Conversation

@jblespiau

@jblespiau jblespiau commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

See #159449

When a thread exits with pending asyncio Tasks, PyThreadState_Clear() moves them onto the shared interp->asyncio_tasks_head list, but each Task still records the _Py_ThreadId() of the thread that created it. unregister_task() skips the stop-the-world pause whenever that id matches the current thread. That also happens for a later thread that reuses the exited thread's pthread_t, and for destructors that run later in PyThreadState_Clear(). The resulting unlocked llist_remove() calls on the shared list race with each other and can leave an already-removed node (next == NULL) linked into it. asyncio.all_tasks(), which asyncio.run() calls on shutdown, then crashes with SIGSEGV in _asyncio_all_tasks(). Record the unique id (PyThreadState.id) of the creating thread state instead, and only take the lock-free path while that thread state is not being cleared. Also drop the assertion in add_tasks_llist() that every linked Task is pending. A Task's state is set before unregister_task() unlinks it, and that removal can be waiting for the stop-the-world pause during which the list is walked. add_one_task() already filters out Tasks that are done.

I don't know if the bug is also present in 3.15, it's worth checking. I am hoping that given we have a reprocuder in the bug, + a bug-fix, it should be doable for the maintainers to port it.

serhiy-storchaka and others added 30 commits September 3, 2026 00:09
…umentation (pythonGH-156822) (pythonGH-156853)

Corrected, in the documentation and in the docstrings:

* GetInputContext() returns bytes, and its result extends to the end of the
  buffered input;
* Parse() accepts a bytes-like object as well as a string, and ignores the
  encoding declaration for a string;
* ParseFile() only supports binary files;
* XML_ERROR_XML_DECL was described as XML_ERROR_NO_ELEMENTS;
* UnparsedEntityDeclHandler is not restricted to Expat 1.2;
* GetSpecifiedAttributeCount() needs ordered_attributes.

Documented the intern parameter and attribute, namespace_prefixes,
SkippedEntityHandler, XML_CTYPE_MIXED and XML_CTYPE_NAME, and EXPAT_VERSION,
version_info and features.

Removed obsolete notes about Expat 1.2 and 1.95.0, and :noindex: from the
content model constants, which are not documented elsewhere; completed the
list of attribute types in AttlistDeclHandler.
(cherry picked from commit d16a691)
…XMLGenerator (pythonGH-156799) (pythonGH-156857)

The namespace declarations were the only attribute values written without
quoteattr(), so a URI containing "&", "<" or a quote character produced a
document which cannot be parsed.
(cherry picked from commit 5056ac5)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
…ion (pythonGH-156354) (pythonGH-156873)

window.encoding does not encode the string arguments on a build with
wide-character support: the curses library converts the characters itself.

In the HOWTO: getch() returns -1, not curses.ERR, when there is no input,
and half-delay mode does the same as no-delay mode.  getkey() returns the
key name only for a special key.  leaveok() is not a synonym for curs_set().
getstr() returns a bytes object, interprets the erase and kill characters,
and limits bytes.  The ACS_* constants are not all larger than 255.
Also document get_wch() before getch(), and read whole characters in the
example.

(cherry picked from commit 161054c)
…ElementTree (pythonGH-156779) (pythonGH-156884)

It was read as deprecating xml.etree.ElementTree itself, because the page
does not mention cElementTree anywhere else.
(cherry picked from commit 50ed1ec)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
For the `atexit` module:

- Move common info from `register` to the module level (a lot of this
  was duplicated -- less maintainable and harder to read)
- Use *interpreter shutdown* consistently, introducing it as a more
  general term for the old docs' *program termination*.
- Use the term *exit handler* consistently
- Move warning for a mitigated footgun to a change entry
- Add a new warning about keeping things usable

Similarly clarify docs for the `atexit` attribute in `weakref`.
(cherry picked from commit 1d28836)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
pythonGH-94987) (python#156881)

Document when these functions started checking code flags for "duck-typed" function-like objects.
(cherry picked from commit 1059e80)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
…tandard (pythonGH-156003) (pythonGH-156362) (pythonGH-156737)

[3.15] pythongh-156002: Bound zipfile decompression for bzip2/LZMA/Zstandard (pythonGH-156003) (pythonGH-156362)

Patch by @tonghuaroot.

zipfile.ZipExtFile._read1() bounds the output of each decompress() call
for DEFLATE members by passing a max_length to zlib, but for bzip2, LZMA,
and Zstandard members it called decompress() with no bound. A whole
compressed chunk was therefore expanded into a single allocation before
the data[:self._left] clip ran, so a consumer that deliberately reads in
small chunks to limit memory (for example zf.open(name).read(8192)) was
silently unprotected for non-DEFLATE members. A small, spec-conformant
archive member declaring a large uncompressed size could drive multi-GB
peak memory.

_read1() now passes a per-call bound to the non-DEFLATE decompress()
(mirroring the DEFLATE branch) and drains the decompressor's internal
buffer across calls by checking needs_input before reading more
compressed input. zipfile's LZMADecompressor wrapper forwards max_length
and exposes needs_input so the bound also holds for LZMA members.

(cherry picked from commit f897dbf)
(cherry picked from commit 1b424c0)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
…t-in types (pythonGH-154363) (python#156337)

* Add a page detailing the time complexity of operations on built-in types (pythonGH-154363)
(cherry picked from commit c3f7c33)

Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: Ned Batchelder <ned@nedbatchelder.com>
Co-authored-by: dgpb <3577712+dg-pb@users.noreply.github.com>

* Remove 3.15-only frozendict mentions

---------

Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: Ned Batchelder <ned@nedbatchelder.com>
Co-authored-by: dgpb <3577712+dg-pb@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
…nt (pythonGH-155640) (pythonGH-156899)

Both handlers look up the header name in lowercase, and the basic
one needs get_all(), so a plain dict does not work.
(cherry picked from commit 1414d2a)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…oncurrent.futures.process (pythonGH-135537) (python#156906)

pythongh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurrent.futures.process (pythonGH-135537)
(cherry picked from commit c8ea867)

Co-authored-by: An Long <aisk@users.noreply.github.com>
…ension methodname processing (pythonGH-156912)

(cherry picked from commit 0625799)

Co-authored-by: John <johnzhou721@gmail.com>
…onGH-146141) (pythonGH-156904)

(cherry picked from commit ced6460)

Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com>
…utside RFC 3454 (pythonGH-155293) (pythonGH-156020)

Due to a bug, some Unicode codepoint attributes were considered
for characters not yet defined in Unicode 3.2.0 or attributes
which changed in later Unicode versions. RFC 3454 (StringPrep)
requires using Unicode 3.2.0 strictly.

(cherry picked from commit 7e109d0)
The cherry-pick needed reworking as pythonGH-144815 wasn't backported to 3.14
and below, so unassigned characters don't have bidi values.

Co-authored-by: Seth Larson <seth@python.org>
Co-authored-by: Stan Ulbrych <89152624+stanfromireland@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
… a range (pythonGH-156901) (pythonGH-156927)

The callers of _PyTokenizer_syntaxerror_known_range() pass columns in bytes,
but SyntaxError.offset and end_offset are columns in characters.

(cherry picked from commit 59a6913)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…b.run import (pythonGH-156915) (python#156959)

pythongh-156896: Stop scrubbing tkinter submodules upon idlelib.run import (python#156915)

Instead, invoke the scrubbing within run.main, which is called after
the import when starting the IDLE user process.

To manually test that scrub_tkinter_submodules
is called when proper, start (or restart) Shell, enter
`import tkinter; dir(tkinter), and check that that font,
messagebox, ttk, and the dialog modules are missing.

It is obvious from the code that the function is not otherwise
called.  To test anyway, continue with `import tkinter.ttk;
import idlelib.run; tkinter.ttk` and check for proper output.

(cherry picked from commit e50d233)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
…ConnectionResetError (pythonGH-156726) (python#156977)

pythongh-127057: reschedule proactor datagram read loop after ConnectionResetError  (pythonGH-156726)
(cherry picked from commit 0ea2971)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
…-156971) (python#156982)

(cherry picked from commit e620377)

Co-authored-by: Ivy Xu <fakeshadow1337@gmail.com>
Co-authored-by: Stan Ulbrych <stan@python.org>
…om a non-zero default (pythonGH-156908) (python#156998)

pythongh-127296: Document that argparse count action starts from a non-zero default (pythonGH-156908)
(cherry picked from commit 7a91841)

Co-authored-by: Owen Carey <37121709+owenthcarey@users.noreply.github.com>
Co-authored-by: Savannah Ostrowski <savannah@python.org>
…il (pythonGH-156974) (pythonGH-157022)

Move wheel_event there from idlelib.tree and add x11_buttons(widget) and
bind_wheel(widget, func), used by the editor, the tree and test_sidebar.
wheel_event now reads the direction from the event, not the platform.

Move its test from test_tree, and test the fix_ functions of util too.
(cherry picked from commit 57594aa)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…the window manager (pythonGH-154370) (pythonGH-157064)

The window manager can take the focus from the application, ignore
lift() and resize a toplevel on its own.

* Hide the root window in the dialog tests, so that it does not compete
  for the focus.
* Take the focus right before generating a key event.
* Tolerate additional focus events.
* Do not check focus_get() and focus_displayof() without the focus.
* Resize the toplevel to fit its content in wait_until_mapped().

(cherry picked from commit 3b56438)
python#157130)

pythongh-156939: Fix struct.pack('0p', bytes) (pythonGH-157071)

If the Pascal string is empty (size=0), do not write the size prefix.
Previously, a NUL byte was written outsize the buffer (buffer
overflow). In practice, the write remains into allocated memory
and is silently ignored: no memory is corrupted.
(cherry picked from commit 23525c9)

Co-authored-by: Victor Stinner <vstinner@python.org>
…-jp (pythonGH-151120) (python#152952)

Encode the payload with output_charset instead of input_charset.
(cherry picked from commit 0777a58)
… modules (pythonGH-157143) (python#157149)

pythongh-157142: Fix linking _freeze_module with built-in test modules (pythonGH-157143)

Programs/_freeze_module is linked with Modules/getpath_noop.o rather than
Modules/getpath.o, and getpath_noop.c only defined _PyConfig_InitPathConfig().
With MODULE_BUILDTYPE=static (the default on wasm), _testinternalcapi.o is
part of LIBRARY_OBJS_OMIT_FROZEN and its get_getpath_codeobject() left
_Py_Get_Getpath_CodeObject() as an undefined reference. Define a stub for it.
(cherry picked from commit c8da735)

Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
…d.start_joinable_thread` on `ThreadHandle.ident` (pythonGH-155085) (python#155124)

pythongh-154937: Fix `_thread._shutdown()` racing with `_thread.start_joinable_thread` on `ThreadHandle.ident` (pythonGH-155085)
(cherry picked from commit a4095ee)

Co-authored-by: Edward Xu <xuxiangad@gmail.com>
…nge during substitution (pythonGH-155761) (python#155771)

pythongh-155752: Do not crash when GenericAlias parameters change during substitution (pythonGH-155761)

An alias argument can gain __typing_subst__ after __parameters__ has been cached, including during a preparation or substitution callback. Check that the argument is present before indexing the substitution arguments.
(cherry picked from commit c0006fa)

Co-authored-by: Darius Houle <dariushoule@gmail.com>
…ompletion_without_do_shell (pythonGH-156623) (python#157156)

pythongh-150922: Fix test_cmd libedit assertion in test_bang_completion_without_do_shell (pythonGH-156623)

GNU readline retypes the full completed word after tab completion, but
libedit redraws only the untyped suffix following a cursor-forward
escape sequence. Assert on the common suffix ('ello') instead of the
literal 'hello' bytes so the test passes on both backends.
(cherry picked from commit 0b4c7da)

Co-authored-by: Shardul Deshpande <iamsharduld@users.noreply.github.com>
…157160) (python#157164)

pythongh-157159: improve the tabnanny usage message (pythonGH-157160)

The usage message embedded sys.argv[0], which under `python -m tabnanny`
is the absolute path of tabnanny.py, and a meaningless virtual path when
the standard library is in a zip archive or embedded in an executable.
Print f"Usage: {sys.executable} -m tabnanny [-v] file_or_directory ..."
instead, and make the test expect that fixed form.
(cherry picked from commit 5ddd59f)

Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
@jblespiau

Copy link
Copy Markdown
Contributor Author

Not done against the correct branch.

@jblespiau jblespiau closed this Oct 15, 2026
@StanFromIreland
StanFromIreland removed request for a team October 15, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.