cpython/Lib/multiprocessing
Gregory P. Smith 9d08423b6e
gh-117378: Fix multiprocessing forkserver preload sys.path inheritance. (GH-126538)
gh-117378: Fix multiprocessing forkserver preload sys.path inheritance.

`sys.path` was not properly being sent from the parent process when launching
the multiprocessing forkserver process to preload imports.  This bug has been
there since the forkserver start method was introduced in Python 3.4.  It was
always _supposed_ to inherit `sys.path` the same way the spawn method does.

Observable behavior change: A `''` value in `sys.path` will now be replaced in
the forkserver's `sys.path` with an absolute pathname
`os.path.abspath(os.getcwd())` saved at the time that `multiprocessing` was
imported in the parent process as it already was when using the spawn start
method. **This will only be observable during forkserver preload imports**.

The code invoked before calling things in another process already correctly sets `sys.path`.
Which is likely why this went unnoticed for so long as a mere performance issue in
some configurations.

A workaround for the bug on impacted Pythons is to set PYTHONPATH in the
environment before multiprocessing's forkserver process was started. Not perfect
as that is then inherited by other children, etc, but likely good enough for many
people's purposes.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-11-09 23:01:32 +00:00
..
dummy bpo-37116: Use PEP 570 syntax for positional-only parameters. (GH-13700) 2019-06-01 11:00:15 +03:00
__init__.py Simplify __all__ in multiprocessing (GH-6856) 2018-07-11 19:22:28 +09:00
connection.py gh-121313: multiprocessing: simplify by increasing the connection buffer size to 64KiB (GH-123559) 2024-09-02 20:32:38 -07:00
context.py gh-84559: Change the multiprocessing start method default to forkserver (GH-101556) 2024-09-26 16:57:19 -07:00
forkserver.py gh-117378: Fix multiprocessing forkserver preload sys.path inheritance. (GH-126538) 2024-11-09 23:01:32 +00:00
heap.py bpo-32759: Free unused arenas in multiprocessing.heap (GH-5827) 2018-04-09 17:37:55 +02:00
managers.py gh-126417: Register multiprocessing proxy types to an appropriate collections.abc class (#126419) 2024-11-05 15:35:45 +05:30
pool.py gh-109649: Use os.process_cpu_count() (#110165) 2023-10-01 03:14:57 +02:00
popen_fork.py gh-83856: Honor atexit for all multiprocessing start methods (GH-114279) 2024-05-03 11:45:46 -07:00
popen_forkserver.py bpo-36888: Add multiprocessing.parent_process() (GH-13247) 2019-05-20 21:37:05 +02:00
popen_spawn_posix.py bpo-36888: Add multiprocessing.parent_process() (GH-13247) 2019-05-20 21:37:05 +02:00
popen_spawn_win32.py gh-118293: Suppress mouse cursor feedback when launching Windows processes with multiprocessing (GH-118315) 2024-04-28 21:10:44 +00:00
process.py gh-83856: Honor atexit for all multiprocessing start methods (GH-114279) 2024-05-03 11:45:46 -07:00
queues.py gh-71052: Enable test_concurrent_futures on platforms that lack multiprocessing (gh-115917) 2024-02-25 11:38:18 -08:00
reduction.py bpo-34521: Fix tests in test_multiprocessing_spawn to use correctly CMSG_SPACE (GH-9613) 2018-09-28 02:51:05 -07:00
resource_sharer.py gh-101225: Increase the socket backlog when creating a multiprocessing.connection.Listener (#113567) 2024-01-13 10:48:33 +01:00
resource_tracker.py gh-104090: Add exit code to multiprocessing ResourceTracker (GH-115410) 2024-02-21 13:54:57 +01:00
shared_memory.py gh-121288: Make error message for index() methods consistent (GH-121395) 2024-07-05 10:50:45 -07:00
sharedctypes.py bpo-32157: Removed explicit quotes around %r and {!r}. (#4582) 2017-11-28 22:54:42 +02:00
spawn.py More actionable error message when spawn is incorrectly used. (#102203) 2023-08-15 18:03:45 -07:00
synchronize.py gh-125679: multiprocessing Lock and RLock - fix invalid representation string on MacOSX. (#125680) 2024-11-07 08:10:57 +00:00
util.py gh-125620: Remove unnecessary import of subprocess in spawnv_passfds (#125624) 2024-10-16 22:42:29 +00:00