Commit Graph

3348 Commits

Author SHA1 Message Date
Christian Heimes
ecfff63e06
bpo-40280: Disable AF_UNIX, AF_PACKET, SO_REUSE* on Emscripten (#31829)
Emscripten's socket emulation is limited. AF_UNIX, AF_PACKET, setsockopt(), and most SO_* constants are not supported.
2022-03-11 23:25:14 +01:00
Christian Heimes
de554d6e02
bpo-40280: Skip more tests/features that don't apply to Emscripten (GH-31791)
- fd inheritance can't be modified because Emscripten doesn't support subprocesses anyway.
- setpriority always fails
- geteuid no longer causes problems with latest emsdk
- umask is a stub
- geteuid / getuid always return 0, but process cannot chown to random uid.
2022-03-10 13:43:40 +01:00
Kumar Aditya
8714b6fa27
bpo-46881: Statically allocate and initialize the latin1 characters. (GH-31616) 2022-03-09 15:02:00 -08:00
Christian Heimes
5081e78efd
bpo-40280: Block more non-working syscalls in Emscripten (GH-31757)
- getgroups always fails.
- geteuid and getegid always return 0 (root), which confuse tarfile and
  tests.
- hardlinks (link, linkat) always fails.
- non-encodable file names are not supported by NODERAWFS layer.
- mark more tests with dependency on subprocess and multiprocessing.
  Mocking does not work if the module fails to import.
2022-03-08 12:17:30 +01:00
Steve Dower
77446d2aa5
bpo-46948: Fix CVE-2022-26488 by ensuring the Windows Installer correctly uses the install path during repair (GH-31726) 2022-03-07 17:23:11 +00:00
Luca Chiodini
7f07b5ee9c
Update grammar_grapher with the new forced (&&) directive (#31704) 2022-03-06 15:26:28 +01:00
Serhiy Storchaka
6927632492
Remove trailing spaces (GH-31695) 2022-03-05 17:47:00 +02:00
Steve Dower
8f31bf4698
bpo-46744: Move Windows ARM64 installation directory to correct ProgramFiles (GH-31677) 2022-03-04 00:50:42 +00:00
Inada Naoki
9833bb91e4
bpo-46845: Reduce dict size when all keys are Unicode (GH-31564) 2022-03-02 08:09:28 +09:00
Eric Snow
21099fc064
bpo-46712: Let generate_global_objects.py Run on Earlier Python Versions (gh-31637)
https://bugs.python.org/issue46712
2022-03-01 14:29:54 -07:00
Eric Snow
08deed1af5
bpo-46753: Add the empty tuple to the _PyRuntimeState.global_objects. (gh-31345)
https://bugs.python.org/issue46753
2022-02-28 15:15:48 -07:00
Mark Shannon
4558af5a8f
bpo-46841: Move the cache for LOAD_GLOBAL inline. (GH-31575) 2022-02-28 12:56:29 +00:00
Steve Dower
da7d99a4de
bpo-46567: Add Tcl/Tk build for Windows ARM64 (GH-31574) 2022-02-28 12:06:43 +00:00
Kumar Aditya
0d9b565e62
Propagate errors (however unlikely) from _Py_Deepfreeze_Init() (GH-31596) 2022-02-26 08:35:03 -08:00
Kumar Aditya
eb002dbe0d
bpo-46712: Share global string identifiers in deepfreeze (GH-31261)
Where appropriate, deepfreeze.c now uses `&_Py_ID(blah)` references instead of locally defining constants. This saves some space.
2022-02-25 10:05:24 -08:00
Brandt Bucher
0f41aac109
bpo-46841: Use *inline* caching for BINARY_OP (GH-31543) 2022-02-25 12:11:34 +00:00
Petr Viktorin
2c228a7b8f
bpo-46748: Don't import <stdbool.h> in public headers (GH-31553)
<stdbool.h> is the standard/modern way to define embedd/extends Python free to define bool, true and false, but there are existing applications that use slightly different redefinitions, which fail if the header is included.

It's OK to use stdbool outside the public headers, though.

https://bugs.python.org/issue46748
2022-02-25 09:25:54 +01:00
Kumar Aditya
4dc746310b
bpo-46430: Fix memory leak in interned strings of deep-frozen modules (GH-31549) 2022-02-24 17:54:06 +01:00
Victor Stinner
042f31da55
bpo-45459: C API uses type names rather than structure names (GH-31528)
Thanks to the new pytypedefs.h, it becomes to use type names like
PyObject rather like structure names like "struct _object".
2022-02-24 17:51:59 +01:00
Eric Snow
1f455361ec
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366)
https://bugs.python.org/issue46765
2022-02-22 17:23:51 -07:00
Mark Shannon
09487c11a5
Fix reporting of specialization stats. (GH-31503) 2022-02-22 17:18:10 +00:00
Mark Shannon
1320d56ff7
Remove pair-counts from specialization stats. (GH-31500) 2022-02-22 15:44:31 +00:00
Eric Snow
4d8a515d19
bpo-46541: Scan Fewer Files in generate_global_objects.py (gh-31364)
https://bugs.python.org/issue46541
2022-02-15 20:07:11 -07:00
Eric Snow
6c89589486
bpo-46541: Drop the check for orphaned global strings. (gh-31363)
https://bugs.python.org/issue46541
2022-02-15 20:06:38 -07:00
Eric Snow
12360aa159
bpo-46541: Discover the global strings. (gh-31346)
Instead of manually enumerating the global strings in generate_global_objects.py, we extrapolate the list from usage of _Py_ID() and _Py_STR() in the source files.

This is partly inspired by gh-31261.

https://bugs.python.org/issue46541
2022-02-14 17:36:51 -07:00
Eric Snow
1d9036425e
Regen the global objects using PYTHON_FOR_REGEN. (gh-31344)
https://bugs.python.org/issue46541
2022-02-14 16:07:09 -07:00
Mark Shannon
0ade875ebe
Add pair counts to stats output and summary. (GH-31324) 2022-02-14 15:53:38 +00:00
Eric Snow
80e4f262aa
bpo-36876: Make sure the c-analyzer is checking all the source files.' (gh-31264)
https://bugs.python.org/issue36876
2022-02-10 16:14:19 -07:00
Pablo Galindo Salgado
390459de6d
Allow the parser to avoid nested processing of invalid rules (GH-31252) 2022-02-10 13:12:14 +00:00
Mark Shannon
b0662ae5c8
Add stats for PRECALL_FUNCTION. (GH-31250) 2022-02-10 11:47:52 +00:00
Mark Shannon
cfc1cecd7b
Right justify numeric columns in stats summary script. (GH-31234) 2022-02-10 11:01:18 +00:00
Eric Snow
cb68788dca
bpo-36876: Minor cleanup to c-analyzer "ignored" data.' (gh-31239)
https://bugs.python.org/issue36876
2022-02-09 17:10:53 -07:00
Kumar Aditya
c0a5ebeb12
bpo-46430: Intern strings in deep-frozen modules (GH-30683) 2022-02-09 08:52:42 -08:00
Mark Shannon
f71a69aa92
bpo-46072: Output stats as markdown with collapsible sections. (GH-31228) 2022-02-09 12:30:26 +00:00
Eric Snow
77bab59c8a
bpo-36876: Update the c-analyzer whitelist. (gh-31225)
This change adds variables that had been added since the last time the whitelist was updated. It also cleans up the list a little.

https://bugs.python.org/issue36876
2022-02-08 18:37:57 -07:00
Eric Snow
81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Eric Snow
c018d3037b
bpo-45952: Get the C analyzer tool working again. (gh-31220)
https://bugs.python.org/issue45952
2022-02-08 12:37:53 -07:00
Eric Snow
1e6214dbd6
bpo-45952: Get the C analyzer tool working again. (gh-31219)
https://bugs.python.org/issue45952
2022-02-08 11:42:17 -07:00
Mark Shannon
328fe3fd20
Print summary stats for overall success of specialization. (GH-31211) 2022-02-08 11:50:02 +00:00
Christian Heimes
96b344c2f1
bpo-40280: Address more test failures on Emscripten (GH-31050)
Co-authored-by: Brett Cannon <brett@python.org>
2022-02-05 20:52:01 +01:00
Kumar Aditya
9d4161a60c
bpo-46608: Fix argument parsing in freeze_modules.py (GH-31131) 2022-02-05 07:59:48 -08:00
Kumar Aditya
bf95ff91f2
bpo-46608: exclude marshalled-frozen data if deep-freezing to save 300 KB space (GH-31074)
This reduces the size of the data segment by **300 KB** of the executable because if the modules are deep-frozen then the marshalled frozen data just wastes space. This was inspired by comment by @gvanrossum in https://github.com/python/cpython/pull/29118#issuecomment-958521863. Note: There is a new option `--deepfreeze-only` in `freeze_modules.py` to change this behavior, it is on be default to save disk space.
```console 
# du -s ./python before
27892   ./python
# du -s ./python after
27524   ./python
```

Automerge-Triggered-By: GH:ericsnowcurrently
2022-02-04 09:57:03 -08:00
Mark Shannon
832876b992
Add miss stats for specialized instructions. (GH-31108) 2022-02-04 09:56:46 +00:00
Manish Kumar ⛄
ba650af7d6
Optimize images by IMGbot (GH-21348)
Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com>
2022-02-04 15:49:43 +09:00
Gregory P. Smith
164a017e13
bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (#31015)
Disable compiler optimization within test_peg_generator.

This speed up test_peg_generator by always disabling compiler
optimizations by using -O0 or equivalent when the test is building its
own C extensions.

A build not using --with-pydebug in order to speed up test execution
winds up with this test taking a very long time as it would do
repeated compilation of parser C code using the same optimization
flags as CPython was built with.

This speeds the test up 6-8x on gps-raspbian.

Also incorporate's #31017's win32 conditional and flags.

Co-authored-by: Kumar Aditya kumaraditya303
2022-02-02 12:15:16 -08:00
Mark Shannon
0d05da1fbf
Add specialization stats for FOR_ITER. (GH-31079) 2022-02-02 15:56:47 +00:00
Mark Shannon
187930f74c
bpo-46072: Add some frame stats. (GH-31060) 2022-02-02 11:01:33 +00:00
Mark Shannon
48be46ec1f
bpo-46072: Add some object layout and allocation stats (GH-31051) 2022-02-01 15:05:18 +00:00
Steve Dower
45faf151c6
bpo-33125: Enables building traditional installer for Windows ARM64 (GH-30885)
Also makes a few general improvements to the build process and removes some dead code.
2022-01-28 16:48:06 +00:00
Mark Shannon
90ab138bbd
bpo-46072: Add simple stats for Python calls. (GH-30989) 2022-01-28 15:20:33 +00:00