Commit Graph

6993 Commits

Author SHA1 Message Date
junyixie
75048c8a38
bpo-43441: Fix _PyType_ClearCache() for subinterpreters (GH-24822)
_PyType_ClearCache() now only resets next_version_tag in the main interpreter.

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-03-13 14:38:36 +01:00
Pablo Galindo
0632b1012d
bpo-42128: Add __match_args__ to structseq-based classes (GH-24732) 2021-03-04 00:03:44 +00:00
Jozef Grajciar
c994ffe695
bpo-11717: fix ssize_t redefinition error when targeting 32bit Windows app (GH-24479) 2021-03-01 11:18:33 +00:00
Dennis Sweeney
73a85c4e1d
bpo-41972: Use the two-way algorithm for string searching (GH-22904)
Implement an enhanced variant of Crochemore and Perrin's Two-Way string searching algorithm, which reduces worst-case time from quadratic (the product of the string and pattern lengths) to linear. This applies to forward searches (like``find``, ``index``, ``replace``); the algorithm for reverse searches (like ``rfind``) is not changed.

Co-authored-by: Tim Peters <tim.peters@gmail.com>
2021-02-28 12:20:50 -06:00
Brandt Bucher
145bf269df
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Talin <viridia@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 14:51:55 -08:00
Alex
cc02b4f2e8
bpo-38302: __pow__/__rpow__ now called when __ipow__ returns NotImplemented (#16459) 2021-02-26 11:58:39 -08:00
Dennis Sweeney
b19855bb6f
bpo-42808: Add PyType_Type.tp_vectorcall for type(obj) performance (GH-24058) 2021-02-22 11:59:16 +09:00
Victor Stinner
d5fc998737
bpo-42093: Cleanup _PyDict_GetItemHint() (GH-24582)
* No longer save/restore the current exception. It is no longer used
  with an exception raised.
* No longer clear the current exception on error: it's now up to the
  caller.
2021-02-21 12:02:04 +01:00
Pablo Galindo
d439fb304c
bpo-43277: Add PySet_CheckExact to the C-API (GH-24598)
For some mysterious reason we have PySet_Check, PyFrozenSet_Check, PyAnySet_Check, PyAnySet_CheckExact and PyFrozenSet_CheckExact but no PySet_CheckExact.
2021-02-20 18:03:08 +00:00
Victor Stinner
46496f9d12
bpo-42990: Functions inherit current builtins (GH-24564)
The types.FunctionType constructor now inherits the current builtins
if the globals dictionary has no "__builtins__" key, rather than
using {"None": None} as builtins: same behavior as eval() and exec()
functions.

Defining a function with "def function(...): ..." in Python is not
affected, globals cannot be overriden with this syntax: it also
inherits the current builtins.

PyFrame_New(), PyEval_EvalCode(), PyEval_EvalCodeEx(),
PyFunction_New() and PyFunction_NewWithQualName() now inherits the
current builtins namespace if the globals dictionary has no
"__builtins__" key.

* Add _PyEval_GetBuiltins() function.
* _PyEval_BuiltinsFromGlobals() now uses _PyEval_GetBuiltins() if
  builtins cannot be found in globals.
* Add tstate parameter to _PyEval_BuiltinsFromGlobals().
2021-02-20 15:17:18 +01:00
Victor Stinner
bcb094b41f
bpo-43268: Pass interp rather than tstate to internal functions (GH-24580)
Pass the current interpreter (interp) rather than the current Python
thread state (tstate) to internal functions which only use the
interpreter.

Modified functions:

* _PyXXX_Fini() and _PyXXX_ClearFreeList() functions
* _PyEval_SignalAsyncExc(), make_pending_calls()
* _PySys_GetObject(), sys_set_object(), sys_set_object_id(), sys_set_object_str()
* should_audit(), set_flags_from_config(), make_flags()
* _PyAtExit_Call()
* init_stdio_encoding()
* etc.
2021-02-19 15:10:45 +01:00
Victor Stinner
101bf69ff1
bpo-43268: _Py_IsMainInterpreter() now expects interp (GH-24577)
The _Py_IsMainInterpreter() function now expects interp rather than
tstate.
2021-02-19 13:33:31 +01:00
Victor Stinner
44085a3fc9
bpo-42990: Refactor _PyFrame_New_NoTrack() (GH-24566)
* Refactor _PyFrame_New_NoTrack() and PyFunction_NewWithQualName()
  code.
* PyFrame_New() checks for _PyEval_BuiltinsFromGlobals() failure.
* Fix a ref leak in _PyEval_BuiltinsFromGlobals() error path.
* Complete PyFunction_GetModule() documentation: it returns a
  borrowed reference and it can return NULL.
* Move _PyEval_BuiltinsFromGlobals() definition to the internal C
  API.
* PyFunction_NewWithQualName() uses _Py_IDENTIFIER() API for the
  "__name__" string to make it compatible with subinterpreters.
2021-02-18 19:20:16 +01:00
Victor Stinner
a3c3ffa68e
bpo-42990: Add __builtins__ attribute to functions (GH-24559)
Expose the new PyFunctionObject.func_builtins member in Python as a
new __builtins__ attribute on functions.

Document also the behavior change in What's New in Python 3.10.
2021-02-18 12:35:37 +01:00
Erlend Egeberg Aasland
cd80f430da
bpo-40170: Always define PyExceptionClass_Name() as a function (GH-24553)
Remove macro variant of PyExceptionClass_Name().
2021-02-17 11:51:08 +01:00
Erlend Egeberg Aasland
cc54001c2e
bpo-40170: Always define PyIter_Check() as a function (GH-24548) 2021-02-16 16:05:58 +01:00
Erlend Egeberg Aasland
871eb4237b
bpo-40170: Convert PyDescr_IsData() to static inline function (GH-24535) 2021-02-16 08:50:00 +01:00
Mark Shannon
0332e569c1
bpo-42990: Further refactoring of PyEval_ functions. (GH-24368)
* Further refactoring of PyEval_EvalCode and friends. Break into make-frame, and eval-frame parts.

* Simplify function vector call using new _PyEval_Vector.

* Remove unused internal functions: _PyEval_EvalCodeWithName and _PyEval_EvalCode.

* Don't use legacy function PyEval_EvalCodeEx.
2021-02-01 10:42:03 +00:00
Pablo Galindo
86e322f141
bpo-40455: Fix gcc10+ warning about writing into a section of offset 0 (GH-24384) 2021-01-30 22:54:22 +09:00
Victor Stinner
a6192635f1
bpo-42979: Use _Py_CheckSlotResult() to check slots result (GH-24356)
When Python is built in debug mode (with C assertions), calling a
type slot like sq_length (__len__() in Python) now fails with a fatal
error if the slot succeeded with an exception set, or failed with no
exception set. The error message contains the slot, the type name,
and the current exception (if an exception is set).

* Check the result of all slots using _Py_CheckSlotResult().
* No longer pass op_name to ternary_op() in release mode.
* Replace operator with dunder Python method name in error messages.
  For example, replace "*" with "__mul__".
* Fix compiler_exit_scope() when an exception is set.
* Fix bytearray.extend() when an exception is set: don't call
  bytearray_setslice() with an exception set.
2021-01-29 16:53:03 +01:00
Mark Shannon
d6c33fbd34
bpo-42990: Introduce 'frame constructor' struct to simplify API for PyEval_CodeEval and friends (GH-24298)
* Introduce 'frame constructor' to simplify API for frame creation

* Embed struct using a macro to conform to PEP 7
2021-01-29 13:24:55 +00:00
Victor Stinner
c9b8e9c421
bpo-42979: Enhance abstract.c assertions checking slot result (GH-24352)
* bpo-42979: Enhance abstract.c assertions checking slot result

Add _Py_CheckSlotResult() function which fails with a fatal error if
a slot function succeeded with an exception set or failed with no
exception set: write the slot name, the type name and the current
exception (if an exception is set).
2021-01-27 16:39:16 +00:00
Victor Stinner
250035d134
bpo-42923: Dump extension modules on fatal error (GH-24207)
The Py_FatalError() function and the faulthandler module now dump the
list of extension modules on a fatal error.

Add _Py_DumpExtensionModules() and _PyModule_IsExtension() internal
functions.
2021-01-18 20:47:13 +01:00
Tobias Holl
61d8c54f43
bpo-42924: Fix incorrect copy in bytearray_repeat (GH-24208)
Before, using the * operator to repeat a bytearray would copy data from the start of
the internal buffer (ob_bytes) and not from the start of the actual data (ob_start).
2021-01-13 18:16:40 +02:00
Mark Shannon
ee9f98d9f4
bpo-42823: Fix frame lineno when frame.f_trace is set (GH-24099)
* Add test for frame.f_lineno with/without tracing.

* Make sure that frame.f_lineno is correct regardless of whether frame.f_trace is set.

* Update importlib

* Add NEWS
2021-01-05 12:04:10 +00:00
Zackery Spytz
5d3553b0a8
bpo-42814: Fix undefined behavior in Objects/genericaliasobject.c (GH-24073)
In is_typing_name(), va_end() is not always called before the
function returns.  It is undefined behavior to call va_start()
without also calling va_end().
2021-01-03 13:18:25 +01:00
Ken Jin
49cd68fb1e
bpo-42195: Disallow isinstance/issubclass for subclasses of genericaliases in Union (GH-24059)
Previously this didn't raise an error. Now it will:
```python
from collections.abc import Callable
isinstance(int, list | Callable[..., str])
```
Also added tests in Union since there were previously none for stuff like ``isinstance(list, list | list[int])`` either.

Backport to 3.9 not required.

Automerge-Triggered-By: GH:gvanrossum
2021-01-02 08:19:15 -08:00
Yurii Karabas
c56387f80c
bpo-27794: Add name attribute to property class (GH-23967) 2020-12-30 01:51:24 -08:00
Petr Viktorin
056c08211b
bpo-40052: Fix alignment issue in PyVectorcall_Function() (GH-23999)
```
In file included from /usr/include/python3.8/Python.h:147:
In file included from /usr/include/python3.8/abstract.h:837:
/usr/include/python3.8/cpython/abstract.h:91:11: error: cast from 'char *' to 'vectorcallfunc *'
(aka 'struct _object *(**)(struct _object *, struct _object *const *, unsigned long, struct _object *)')
increases required alignment from 1 to 8 [-Werror,-Wcast-align]

    ptr = (vectorcallfunc*)(((char *)callable) + offset);
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
```
Co-Authored-By: Andreas Schneider <asn@cryptomilk.org>
Co-Authored-By: Antoine Pitrou <antoine@python.org>
2020-12-29 15:32:07 -08:00
Pablo Galindo
a6d63a20df
Fix compiler warnings regarding loss of data (GH-23983) 2020-12-29 00:28:09 +00:00
Victor Stinner
f4507231e3
bpo-42745: finalize_interp_types() calls _PyType_Fini() (GH-23953)
Call _PyType_Fini() in subinterpreters.

Fix reference leaks in subinterpreters.
2020-12-26 20:26:08 +01:00
Victor Stinner
ea251806b8
bpo-40521: Per-interpreter interned strings (GH-20085)
Make the Unicode dictionary of interned strings compatible with
subinterpreters.

Remove the INTERN_NAME_STRINGS macro in typeobject.c: names are
always now interned (even if EXPERIMENTAL_ISOLATED_SUBINTERPRETERS
macro is defined).

_PyUnicode_ClearInterned() now uses PyDict_Next() to no longer
allocate memory, to ensure that the interned dictionary is cleared.
2020-12-26 02:58:33 +01:00
Victor Stinner
4101018488
bpo-42745: Make the type cache per-interpreter (GH-23947)
Make the type attribute lookup cache per-interpreter.

Add private _PyType_InitCache() function, called by PyInterpreterState_New().

Continue to share next_version_tag between interpreters, since static
types are still shared by interpreters.

Remove MCACHE macro: the cache is no longer disabled if the
EXPERIMENTAL_ISOLATED_SUBINTERPRETERS macro is defined.
2020-12-26 01:45:43 +01:00
Victor Stinner
ba3d67c2fb
bpo-39465: Fix _PyUnicode_FromId() for subinterpreters (GH-20058)
Make _PyUnicode_FromId() function compatible with subinterpreters.
Each interpreter now has an array of identifier objects (interned
strings decoded from UTF-8).

* Add PyInterpreterState.unicode.identifiers: array of identifiers
  objects.
* Add _PyRuntimeState.unicode_ids used to allocate unique indexes
  to _Py_Identifier.
* Rewrite the _Py_Identifier structure.

Microbenchmark on _PyUnicode_FromId(&PyId_a) with _Py_IDENTIFIER(a):

[ref] 2.42 ns +- 0.00 ns -> [atomic] 3.39 ns +- 0.00 ns: 1.40x slower

This change adds 1 ns per _PyUnicode_FromId() call in average.
2020-12-26 00:41:46 +01:00
kj
73607be686
bpo-41559: Implement PEP 612 - Add ParamSpec and Concatenate to typing (#23702) 2020-12-23 20:33:48 -08:00
kj
463c7d3d14
bpo-42195: Ensure consistency of Callable's __args__ in collections.abc and typing (GH-23060) 2020-12-13 10:38:24 -08:00
kj
804d6893b8
bpo-42576: Raise TypeError when passing in keyword arguments to GenericAlias (GH-23656)
Use `_PyArg_NoKeywords` instead of `_PyArg_NoKwnames` when checking the `kwds` tuple when creating `GenericAlias`. This fixes an interpreter crash when passing in keyword arguments to `GenericAlias`'s constructor.

Needs backport to 3.9.

Automerge-Triggered-By: GH:gvanrossum
2020-12-05 08:02:14 -08:00
Brandt Bucher
226a012d1c
bpo-42536: GC track recycled tuples (GH-23623)
Several built-in and standard library types now ensure that their internal result tuples are always tracked by the garbage collector:

- collections.OrderedDict.items
- dict.items
- enumerate
- functools.reduce
- itertools.combinations
- itertools.combinations_with_replacement
- itertools.permutations
- itertools.product
- itertools.zip_longest
- zip

Previously, they could have become untracked by a prior garbage collection.
2020-12-04 19:45:57 -08:00
Serhiy Storchaka
2ad93821a6
bpo-42431: Fix outdated bytes comments (GH-23458)
Also move definitions of internal macros F_LJUST etc to private header.
2020-12-03 12:46:16 +02:00
Victor Stinner
32bd68c839
bpo-42519: Replace PyObject_MALLOC() with PyObject_Malloc() (GH-23587)
No longer use deprecated aliases to functions:

* Replace PyObject_MALLOC() with PyObject_Malloc()
* Replace PyObject_REALLOC() with PyObject_Realloc()
* Replace PyObject_FREE() with PyObject_Free()
* Replace PyObject_Del() with PyObject_Free()
* Replace PyObject_DEL() with PyObject_Free()
2020-12-01 10:37:39 +01:00
Victor Stinner
00d7abd7ef
bpo-42519: Replace PyMem_MALLOC() with PyMem_Malloc() (GH-23586)
No longer use deprecated aliases to functions:

* Replace PyMem_MALLOC() with PyMem_Malloc()
* Replace PyMem_REALLOC() with PyMem_Realloc()
* Replace PyMem_FREE() with PyMem_Free()
* Replace PyMem_Del() with PyMem_Free()
* Replace PyMem_DEL() with PyMem_Free()

Modify also the PyMem_DEL() macro to use directly PyMem_Free().
2020-12-01 09:56:42 +01:00
Yurii Karabas
7301979b23
bpo-42202: Store func annotations as a tuple (GH-23316)
Reduce memory footprint and improve performance of loading modules having many func annotations.

  >>> sys.getsizeof({"a":"int","b":"int","return":"int"})
  232
  >>> sys.getsizeof(("a","int","b","int","return","int"))
  88

The tuple is converted into dict on the fly when `func.__annotations__` is accessed first.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2020-11-25 19:43:18 +09:00
Hai Shi
ed1a5a5bac
bpo-40170: Hide impl detail of Py_TRASHCAN_BEGIN macro (GH-23235)
The Py_TRASHCAN_BEGIN macro no longer accesses PyTypeObject attributes,
but now can get the condition by calling the new private
_PyTrash_cond() function which hides implementation details.
2020-11-24 23:03:31 +01:00
Serhiy Storchaka
313467efdc
bpo-42435: Speed up comparison of bytes and bytearray object (GH--23461)
* Speed up comparison of bytes objects with non-bytes objects when
  option -b is specified.
* Speed up comparison of bytarray objects with non-buffer object.
2020-11-22 22:00:53 +02:00
Serhiy Storchaka
686c203cd4
bpo-42423: Accept single base class in PyType_FromModuleAndSpec() (GH-23441) 2020-11-22 13:25:02 +02:00
Serhiy Storchaka
ed1007c0d7
bpo-42425: Fix possible leak in initialization of errmap for OSError (GH-23446) 2020-11-21 19:17:46 +02:00
Serhiy Storchaka
1db76394ea
bpo-42412: Fix possible leaks and check arguments in PyType_FromModuleAndSpec() (GH-23410)
* There were leaks if Py_tp_bases is used more than once or if some call is
  failed before setting tp_bases.
* There was a crash if the bases argument or the Py_tp_bases slot is not a tuple.
* The documentation was not accurate.
2020-11-21 12:02:53 +02:00
Christian Heimes
07f2adedf0
bpo-40998: Address compiler warnings found by ubsan (GH-20929)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: GH:tiran
2020-11-18 07:38:53 -08:00
kj
384b7a4bd9
bpo-42332: Add weakref slot to types.GenericAlias (GH-23250)
Automerge-Triggered-By: GH:gvanrossum
2020-11-15 19:27:23 -08:00
Mark Shannon
877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00