cpython/Tools/build
Petr Viktorin 6f1d448bc1
gh-113993: Allow interned strings to be mortal, and fix related issues (GH-120520)
* Add an InternalDocs file describing how interning should work and how to use it.

* Add internal functions to *explicitly* request what kind of interning is done:
  - `_PyUnicode_InternMortal`
  - `_PyUnicode_InternImmortal`
  - `_PyUnicode_InternStatic`

* Switch uses of `PyUnicode_InternInPlace` to those.

* Disallow using `_Py_SetImmortal` on strings directly.
  You should use `_PyUnicode_InternImmortal` instead:
  - Strings should be interned before immortalization, otherwise you're possibly
    interning a immortalizing copy.
  - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
    `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
    backports, as they are now part of public API and version-specific ABI.

* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.

* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
  - `_Py_ID`
  - `_Py_STR` (including the empty string)
  - one-character latin-1 singletons

  Now, when you intern a singleton, that exact singleton will be interned.

* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).

* Intern `_Py_STR` singletons at startup.

* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.

* Beef up the tests. Cover internal details (marked with `@cpython_only`).

* Add lots of assertions

Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
2024-06-21 17:19:31 +02:00
..
check_extension_modules.py gh-104773: Remove the msilib package (GH-104911) 2023-05-24 20:06:00 -05:00
deepfreeze.py GH-111485: Delete the old generator code. (GH-113321) 2023-12-21 12:46:28 +00:00
freeze_modules.py gh-120417: Remove unused imports in Tools (#120623) 2024-06-17 18:09:26 +02:00
generate_global_objects.py gh-113993: Allow interned strings to be mortal, and fix related issues (GH-120520) 2024-06-21 17:19:31 +02:00
generate_levenshtein_examples.py gh-99016: Make build scripts compatible with Python 3.8 (GH-99017) 2022-11-02 20:30:09 +02:00
generate_re_casefix.py gh-97669: Create Tools/build/ directory (#97963) 2022-10-17 12:01:00 +02:00
generate_sbom.py gh-112844: Update CPE references for external dependencies (#118521) 2024-05-20 13:27:09 -04:00
generate_sre_constants.py gh-97669: Create Tools/build/ directory (#97963) 2022-10-17 12:01:00 +02:00
generate_stdlib_module_names.py gh-76785: Rename _xxsubinterpreters to _interpreters (gh-117791) 2024-04-24 16:18:24 +00:00
generate_token.py gh-107211: No longer export internal functions (7) (#108425) 2023-08-24 17:40:56 +02:00
mypy.ini gh-112302: Add Software Bill-of-Materials (SBOM) tracking for dependencies (#112303) 2023-12-07 18:01:58 +02:00
parse_html5_entities.py gh-97669: Create Tools/build/ directory (#97963) 2022-10-17 12:01:00 +02:00
regen-configure.sh gh-112088: Run autoreconf in GHA check_generated_files (#112090) 2023-11-15 21:47:14 +01:00
smelly.py gh-90815: Add mimalloc memory allocator (#109914) 2023-10-30 15:43:11 +00:00
stable_abi.py gh-116303: Skip test module dependent tests if test modules are unavailable (#117341) 2024-04-03 15:11:36 +02:00
umarshal.py Fix a typo in umarshal.py (#108803) 2023-09-06 09:06:41 +00:00
update_file.py gh-97669: Create Tools/build/ directory (#97963) 2022-10-17 12:01:00 +02:00
verify_ensurepip_wheels.py GH-80789: Get rid of the `ensurepip` infra for many wheels (#109245) 2024-01-30 01:25:31 +00:00