mirrors/cpython
0
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2024-12-03 23:06:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

gh-89792: Limit test_tools freeze test build parallelism based on the number of cores (#101841)

Browse Source 
unhardcode freeze test build parallelism. base it on the number of cpus, don't use more than max(2, os.cpu_count()/3).
This commit is contained in:
Gregory P. Smith 2023-02-11 22:07:52 -08:00 committed by GitHub
parent da2fb92643
commit dfc2e065a2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 16 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
Misc/NEWS.d/next/Tests/2023-02-11-20-28-08.gh-issue-89792.S-Y5BZ.rst
View File

@ -1,3 +1,4 @@
``test_tools`` now copies up to 10x less source data to a temporary
directory during the ``freeze`` test by ignoring git metadata and other
artifacts.
``test_tools`` now copies up to 10x less source data to a temporary directory
during the ``freeze`` test by ignoring git metadata and other artifacts. It
also limits its python build parallelism based on os.cpu_count instead of hard
coding it as 8 cores.

15
Tools/freeze/test/freeze.py
View File

@ -163,16 +163,25 @@ def prepare(script=None, outdir=None):
if not MAKE:
raise UnsupportedError('make')
cores = os.cpu_count()
if cores and cores >= 3:
# this test is most often run as part of the whole suite with a lot
# of other tests running in parallel, from 1-2 vCPU systems up to
# people's NNN core beasts. Don't attempt to use it all.
parallel = f'-j{cores*2//3}'
else:
parallel = '-j2'
# Build python.
print(f'building python in {builddir}...')
print(f'building python {parallel=} in {builddir}...')
if os.path.exists(os.path.join(srcdir, 'Makefile')):
# Out-of-tree builds require a clean srcdir.
_run_quiet([MAKE, '-C', srcdir, 'clean'])
_run_quiet([MAKE, '-C', builddir, '-j8'])
_run_quiet([MAKE, '-C', builddir, parallel])
# Install the build.
print(f'installing python into {prefix}...')
_run_quiet([MAKE, '-C', builddir, '-j8', 'install'])
_run_quiet([MAKE, '-C', builddir, 'install'])
python = os.path.join(prefix, 'bin', 'python3')
return outdir, scriptfile, python