Commit Graph

3134 Commits

Author SHA1 Message Date
Victor Stinner
a9c03d7fb7
bpo-43456: Remove _xxsubinterpreters from sys.stdlib_module_names (GH-24814) 2021-03-10 12:10:01 +01:00
Victor Stinner
307745aa42
bpo-43445: Add frozen modules to sys.stdlib_module_names (GH-24798)
Add frozen modules to sys.stdlib_module_names. For example, add
"_frozen_importlib" and "_frozen_importlib_external" names.

Add "list_frozen" command to Programs/_testembed.
2021-03-10 11:14:07 +01:00
Jozef Grajciar
c994ffe695
bpo-11717: fix ssize_t redefinition error when targeting 32bit Windows app (GH-24479) 2021-03-01 11:18:33 +00:00
Nicholas Sim
4a6bf276ed
bpo-35134: Move non-limited C API files to Include/cpython/ (GH-24561)
Include/{odictobject.h,parser_interface.h,picklebufobject.h,pydebug.h,pyfpe.h}
into Include/cpython/.

Parser: peg_api: include Python.h instead of parser_interface.h.
2021-02-19 15:55:46 +01:00
Neil Schemenauer
ffa55d21b4
bpo-39448: Add regen-frozen makefile target. (GH-18174)
Add the "regen-frozen" makefile target that regenerates the code for the
frozen __hello__ module.
2021-02-18 16:49:12 -08:00
Nicholas Sim
366dc3a135
bpo-35134: Move Include/{pyarena.h,pyctype.h} to Include/cpython/ (GH-24550)
Move non-limited C API headers pyarena.h and pyctype.h
into Include/cpython/ directory.
2021-02-17 19:30:31 +01:00
Victor Stinner
801bb0b503
bpo-43103: Add configure --without-static-libpython (GH-24418)
Add a new configure --without-static-libpython option to not build
the libpythonMAJOR.MINOR.a static library and not install the
python.o object file.

Fix smelly.py and stable_abi.py tools when libpython3.10.a is
missing.
2021-02-17 11:14:42 +01:00
Pablo Galindo
58fb156edd
bpo-42997: Improve error message for missing : before suites (GH-24292)
* Add to the peg generator a new directive ('&&') that allows to expect
  a token and hard fail the parsing if the token is not found. This
  allows to quickly emmit syntax errors for missing tokens.

* Use the new grammar element to hard-fail if the ':' is missing before
  suites.
2021-02-02 19:54:22 +00:00
Victor Stinner
64fc105b2d
bpo-42955: Remove sub-packages from sys.stdlib_module_names (GH-24353) 2021-01-28 00:03:23 +01:00
Victor Stinner
9852cb3811
bpo-42955: Rename module_names to sys.stdlib_module_names (GH-24332)
* Rename _Py_module_names to _Py_stdlib_module_names.
* Rename Python/module_names.h to Python/stdlib_module_names.h.
2021-01-25 23:12:50 +01:00
Victor Stinner
db584bdad3
bpo-42955: Add sys.modules_names (GH-24238)
Add sys.module_names, containing the list of the standard library
module names.
2021-01-25 13:24:42 +01:00
Victor Stinner
cad8020cb8
bpo-42955: Add Python/module_names.h (GH-24258)
Add a private list of all stdlib modules: _Py_module_names.

* Add Tools/scripts/generate_module_names.py script.
* Makefile: Add "make regen-module-names" command.
* setup.py: Add --list-module-names option.
* GitHub Action and Travis CI also runs "make regen-module-names",
  not ony "make regen-all", to ensure that the module names remains
  up to date.
2021-01-19 23:04:49 +01:00
Victor Stinner
0e2a0f72cc
bpo-42802: Remove distutils bdist_wininst command (GH-24043)
The distutils bdist_wininst command deprecated in Python 3.8 has been
removed. The distutils bidst_wheel command is now recommended to
distribute binary packages on Windows.

* Remove Lib/distutils/command/bdist_wininst.py
* Remove PC/bdist_wininst/ project
* Remove Lib/distutils/command/wininst-*.exe programs
* Remove all references to bdist_wininst
2021-01-09 00:35:01 +01:00
Steve Dower
af4cd16479
Do not remove x bit from published directories (GH-24101) 2021-01-04 20:56:32 +00:00
Pablo Galindo
3bcc4ead3f
Add small validator utility for PEG grammars (GH-23519) 2020-12-26 19:11:29 +00:00
Eric Snow
5ae9be68d9
bpo-36876: [c-analyzer tool] Additional CLI updates for "capi" command. (gh-23929)
https://bugs.python.org/issue36876
2020-12-25 15:57:30 -07:00
Eric Snow
7ec59d8861
bpo-36876: [c-analyzer tool] Add a "capi" subcommand to the c-analyzer tool. (gh-23918)
This will help identify which C-API items will need to be updated for subinterpreter support.

https://bugs.python.org/issue36876
2020-12-24 11:04:19 -07:00
Augusto Hack
b57ada98da
closes bpo-42726: gdb libpython: InstanceProxy support for py3 (GH-23912)
On Fedora 31 gdb is using python 3.7.9, calling `proxyval` on an instance with a dictionary fails because of the `dict.iteritems` usage. This PR changes the code to be compatible with py2 and py3.

This changed seemed small enough to not need an issue and news blurb, if one is required please let me know.

Automerge-Triggered-By: GH:benjaminp
2020-12-24 09:16:04 -08:00
Serhiy Storchaka
3d569fd6dc
bpo-42630: Improve error reporting in Tkinter for absent default root (GH-23781)
* Tkinter functions and constructors which need a default root window
  raise now RuntimeError with descriptive message instead of obscure
  AttributeError or NameError if it is not created yet or cannot
  be created automatically.

* Add tests for all functions which use default root window.

* Fix import in the pynche script.
2020-12-19 12:17:08 +02:00
Victor Stinner
1c653f17cb
bpo-42613: Fix freeze.py config directory (GH-23792)
Fix freeze.py tool to use the prope config and library directories.
2020-12-16 12:12:36 +01:00
Pablo Galindo
0911411e0c
Update stable ABI script for MacOS and update list of exported symbols (GH-23783) 2020-12-15 18:16:13 +00:00
Pablo Galindo
79c1849b9e
bpo-42545: Improve the error message in the stable API script (GH-23648) 2020-12-04 23:19:21 +00:00
Pablo Galindo
85f1dedb8d
bpo-42545: Check that all symbols in the limited ABI are exported (GH-23616) 2020-12-04 22:05:58 +00:00
Victor Stinner
ac7d0169d2
bpo-42212: smelly.py also checks the dynamic library (GH-23423)
The smelly.py script now also checks the Python dynamic library and
extension modules, not only the Python static library. Make also the
script more verbose: explain what it does.

The GitHub Action job now builds Python with the libpython dynamic
library.
2020-11-24 13:38:08 +01:00
Eric Snow
9f02b479e6
bpo-36876: [c-analyzer tool] Tighten up the results and output. (GH-23431)
We also update the "ignored" file with a temporary list of all known globals.
2020-11-20 15:39:28 -07:00
Victor Stinner
8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
Steve Dower
9b6934230c
bpo-40754: Adds _testinternalcapi to Windows installer for test suite (GH-23271) 2020-11-13 22:24:36 +00:00
Mark Shannon
877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
jack1142
bfc6b63102
bpo-36310: Allow pygettext.py to detect calls to gettext in f-strings. (GH-19875)
Adds support to Tools/i18n/pygettext.py for gettext calls in f-strings. This process is done by parsing the f-strings, processing each value, and flagging the ones which contain a gettext call.

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-11-10 01:50:45 +03:00
Yash Shete
1f73c320e2
bpo-41712: Avoid runaway regex match in upload scripts (GH-23166) 2020-11-09 17:38:09 +00:00
Steve Dower
db6434c474
Enable signing of nuget.org packages and update to supported timestamp server (GH-23132) 2020-11-03 22:31:49 +00:00
Lysandros Nikolaou
02cdfc93f8
bpo-42218: Correctly handle errors in left-recursive rules (GH-23065)
Left-recursive rules need to check for errors explicitly, since
even if the rule returns NULL, the parsing might continue and lead
to long-distance failures.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-10-31 20:31:41 +02:00
Eric Snow
4fe72090de
bpo-36876: Small adjustments to the C-analyzer tool. (GH-23045)
This is a little bit of clean-up, small fixes, and additional helpers prior to building an updated & accurate list of globals to eliminate.
2020-10-30 15:46:52 -06:00
Victor Stinner
b9ee4af4c6
bpo-42208: Fix test_gdb for gc_collect_main() name (GH-23041)
The gcmodule.c collect() function was renamed to gc_collect_main():
update gdb/libpython.py (python-gdb.py).
2020-10-30 21:09:48 +01:00
Victor Stinner
c310185c08
bpo-42161: Remove private _PyLong_Zero and _PyLong_One (GH-23003)
Use PyLong_FromLong(0) and PyLong_FromLong(1) of the public C API
instead. For Python internals, _PyLong_GetZero() and _PyLong_GetOne()
of pycore_long.h can be used.
2020-10-27 21:34:33 +01:00
Lysandros Nikolaou
bca7014032
bpo-42123: Run the parser two times and only enable invalid rules on the second run (GH-22111)
* Implement running the parser a second time for the errors messages

The first parser run is only responsible for detecting whether
there is a `SyntaxError` or not. If there isn't the AST gets returned.
Otherwise, the parser is run a second time with all the `invalid_*`
rules enabled so that all the customized error messages get produced.
2020-10-27 00:42:04 +02:00
Eric Snow
345cd37abe
bpo-36876: Fix the C analyzer tool. (GH-22841)
The original tool wasn't working right and it was simpler to create a new one, partially re-using some of the old code. At this point the tool runs properly on the master. (Try: ./python Tools/c-analyzer/c-analyzer.py analyze.)  It take ~40 seconds on my machine to analyze the full CPython code base.

Note that we'll need to iron out some OS-specific stuff (e.g. preprocessor). We're okay though since this tool isn't used yet in our workflow. We will also need to verify the analysis results in detail before activating the check in CI, though I'm pretty sure it's close.

https://bugs.python.org/issue36876
2020-10-22 18:42:51 -06:00
Pablo Galindo
47ecfd8030
bpo-41971: Fix test failure in test.test_tools.test_c_analyzer when mutating global state (GH-22652) 2020-10-11 21:34:51 +01:00
Serhiy Storchaka
9975cc5008
bpo-41985: Add _PyLong_FileDescriptor_Converter and AC converter for "fildes". (GH-22620) 2020-10-09 23:00:45 +03:00
Andre Delfino
e8a2076e14
Revert "Fix all Python Cookbook links (#22205)" (GH-22424)
This commit reverts commit ac0333e1e1 as the original links are working again and they provide extended features such as comments and alternative versions.
2020-09-27 01:47:25 +01:00
Victor Stinner
19c3ac92bf
bpo-41834: Remove _Py_CheckRecursionLimit variable (GH-22359)
Remove the global _Py_CheckRecursionLimit variable: it has been
replaced by ceval.recursion_limit of the PyInterpreterState
structure.

There is no need to keep the variable for the stable ABI, since
Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() were not usable
in Python 3.8 and older: these macros accessed PyThreadState members,
whereas the PyThreadState structure is opaque in the limited C API.
2020-09-23 14:04:57 +02:00
Serhiy Storchaka
dcfaa520c4
bpo-41715: Fix potential catastrofic backtracking in c_analyzer. (GH-22091) 2020-09-17 10:34:20 +03:00
Pablo Galindo
a5634c4067
bpo-41746: Add type information to asdl_seq objects (GH-22223)
* Add new capability to the PEG parser to type variable assignments. For instance:
```
       | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a }
```

* Add new sequence types from the asdl definition (automatically generated)
* Make `asdl_seq` type a generic aliasing pointer type.
* Create a new `asdl_generic_seq` for the generic case using `void*`.
* The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed.
* New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences.
* Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 19:42:00 +01:00
Andre Delfino
ac0333e1e1
Fix all Python Cookbook links (#22205) 2020-09-15 21:13:26 +01:00
Václav Slavík
7c11a9acca
bpo-41744: Package python.props with correct name in NuGet package (GH-22154)
NuGet automatically includes .props file from the build directory in the
target using the package, but only if the .props file has the correct
name: it must be $(id).props

Rename python.props correspondingly in all the nuspec variants. Also
keep python.props as it were for backward compatibility.
2020-09-14 20:30:15 +01:00
Pablo Galindo
e55a0e971b
Fix 'gather' rules in the python parser generator (GH-22021)
Currently, empty sequences in gather rules make the conditional for
gather rules fail as empty sequences evaluate as "False". We need to
explicitly check for "None" (the failure condition) to avoid false
negatives.
2020-09-03 15:29:55 +01:00
Steve Dower
6444ca9469
bpo-41526: Fixed layout of final page of the installer (GH-21871) 2020-08-14 00:35:52 +01:00
Victor Stinner
fabd7bb8e0
bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822)
Automerge-Triggered-By: @tiran
2020-08-11 06:26:59 -07:00
Mark Shannon
582aaf19e8
bpo-41463: Generate information about jumps from 'opcode.py' rather than duplicating it in 'compile.c' (GH-21714)
Generate information about jumps from 'opcode.py' rather than duplicate it in 'compile.c'
2020-08-04 17:30:11 +01:00
Steve Dower
37a06cbe5c
bpo-41412 and bpo-40948: Windows installer updates (GH-21656)
Prevent installation on Windows 8 and earlier.
Download UCRT on demand when required (non-updated Windows 8.1 only)
Add reference to py launcher to post-install message
2020-07-28 16:35:46 +01:00