Repository navigation
Off-by-one error in _winapi.WaitForMultipleObjects #94242
Description
Bug report
_winapi_WaitForMultipleObjects_impl in _winapi.c creates an array of MAXIMUM_WAIT_OBJECTS handles but then arbitrarily enforces a cap of MAXIMUM_WAIT_OBJECTS - 1 handles. This is incorrect. WaitForMultipleObjects is clearly documented as being able to handle MAXIMUM_WAIT_OBJECTS (64) handles. This simple C++ sample code demonstrates this:
#include <stdio.h>
#include <windows.h>
#include
int main(int argc, char* argv[]) {
for (DWORD count = 60; count < 66; ++count) {
printf("About to wait on %d handles.\n", count);
std::vector handles(count);
for (int i = 0; i < count; ++i)
handles[i] = CreateEventW(nullptr, FALSE, TRUE, nullptr);
DWORD wait_result = WaitForMultipleObjects(count, handles.data(), TRUE, INFINITE);
if (wait_result == WAIT_OBJECT_0)
printf(" Wait satisfied by object 0.\n");
else if (wait_result == WAIT_FAILED)
printf(" Wait failed.\n");
else
printf(" Result was %lu.\n", wait_result);
for (int i = 0; i < count; ++i)
CloseHandle(handles[i]);
}
}
This error also shows up here:
3988986#diff-eb5cd13d65d2f215d8dc0d95eb1a1341ff69b226ac56ec6099e39582edecc8cfR219
and here:
3988986#diff-84a955ea642d41d9f164e9f94499d8eb33c357ec7f4362a341951f6f67ab21a1R117
Note that this is in addition to the off-by-one error that was tracked by this issue:
#84444
That is, there used to be an off-by-two error (maximum of 60 processes could be waited on) and now there is an off-by-one error (maximum of 61 processes can be waited on). In Python 3.8.10 the limit is still 60, but in Python 3.9 the limit is (as far as I can tell) 61. It should be 62.
Your environment
Python 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Windows 10
Linked PRs
- gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurrent.futures.process #135537
- [3.15] gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurrent.futures.process (GH-135537) #156905
- [3.14] gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurrent.futures.process (GH-135537) #156906
- [3.13] gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurrent.futures.process (GH-135537) #156907
Activity
eryksun commentedon Jun 24, 2022
If the wait is for any kernel object to be signaled, instead of waiting for all objects to be signaled, and it's the main thread, then the limit must be
MAXIMUM_WAIT_OBJECTS - 1(63) objects because the last index is reserved for_PyOS_SigintEvent(), the kernel event object that gets set by Python'sSIGINThandler:I assume the same limit is applied for all threads for the sake of simplicity. Technically, it could be extended to 64 handles if the condition
!wait_flag && _PyOS_IsMainThread()is false.randomascii commentedon Jun 24, 2022
Gotcha. The comments are incorrect still since they say that the Windows limit is 63 and that Python uses two handles, but in fact the Windows limit is 64 and Python uses three handles.
I think having the limit as 61 instead of 62 is fine. I don't know whether the misleading comments matter or not.
eryksun commentedon Jun 26, 2022
Did you want to submit a PR to update the comment in Lib/concurrent/futures/process.py? Otherwise this can be closed.
Correct calculation of _MAX_WINDOWS_WORKERS
randomascii commentedon Jun 26, 2022
Sure. In progress.
gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurrent.fut…
[3.14] gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurr…
[3.13] gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurr…
[3.15] gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurr…
pythongh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurre…