Commit Graph

28034 Commits

Author SHA1 Message Date
Victor Stinner
1f7e42131d
gh-109054: configure checks if libatomic is needed (#109101)
Fix building the _testcapi extension on Linux AArch64 which requires
linking to libatomic when <cpython/pyatomic.h> is used: the
_Py_atomic_or_uint64() function requires libatomic
__atomic_fetch_or_8() on this platform.

The configure script now checks if linking to libatomic is needed and
generates a new LIBATOMIC variable used to build the _testcapi
extension.

Building the _testcapi extension now uses the LIBATOMIC variable in
its LDFLAGS, since Modules/_testcapi/pyatomic.c uses
<cpython/pyatomic.h>.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-09-08 23:14:33 +02:00
Pablo Galindo Salgado
5bda2f637e
gh-109114: Relax the check for invalid lambdas inside f-strings to avoid false positives (#109121) 2023-09-08 17:00:23 +00:00
Pablo Galindo Salgado
6275c67ea6
gh-106922: Fix error location for constructs with spaces and parentheses (#108959) 2023-09-08 17:18:35 +01:00
Mark Shannon
15d4c9fabc
GH-108716: Turn off deep-freezing of code objects. (GH-108722) 2023-09-08 10:34:40 +01:00
Ethan Furman
c74e440168
gh-109022: [Enum] require names=() to create empty enum type (GH-109048)
add guard so that ``Enum('bar')`` raises a TypeError instead of
creating a new enum class called `bar`.  To create the new but
empty class, use:

    huh = Enum('bar', names=())
2023-09-07 18:19:03 -07:00
Carl Meyer
f2584eade3
gh-108732: include comprehension locals in frame.f_locals (#109026)
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-07 08:56:43 -06:00
Mark Dickinson
b72251de93
gh-102823: Document return type of floor division on floats (#102824)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-09-07 14:20:33 +00:00
Serhiy Storchaka
b2729e93e9
gh-88943: Improve syntax error for non-ASCII character that follows a numerical literal (GH-109081)
It now points on the invalid non-ASCII character, not on the valid numerical literal.
2023-09-07 17:00:13 +03:00
Mark Shannon
0858328ca2
GH-108614: Add RESUME_CHECK instruction (GH-108630) 2023-09-07 14:39:03 +01:00
Ijtaba Hussain
3e53ac9903
gh-103186: Suppress and assert expected RuntimeWarnings in test_sys_settrace (GH-103244)
Caused as a result of frame manipulation where locals are never assigned / initialised.
2023-09-07 10:41:38 +03:00
Victor Stinner
a52a350977
gh-109015: Add test.support.socket_helper.tcp_blackhole() (#109016)
Skip test_asyncio, test_imaplib and test_socket tests if FreeBSD TCP
blackhole is enabled (net.inet.tcp.blackhole=2).
2023-09-07 01:58:03 +02:00
Brandt Bucher
6971e40c2e
GH-104584: Restore frame->stacktop on optimizer error (GH-108953) 2023-09-06 13:59:50 -07:00
Serhiy Storchaka
6f3c138dfa
gh-108751: Add copy.replace() function (GH-108752)
It creates a modified copy of an object by calling the object's
__replace__() method.

It is a generalization of dataclasses.replace(), named tuple's _replace()
method and replace() methods in various classes, and supports all these
stdlib classes.
2023-09-06 23:55:42 +03:00
Victor Stinner
db1ee6a19a
gh-108740: Fix "make regen-all" race condition (#108741)
Fix a race condition in "make regen-all". The deepfreeze.c source and
files generated by Argument Clinic are now generated or updated
before generating "global objects". Previously, some identifiers may
miss depending on the order in which these files were generated.

* "make regen-global-objects": Make sure that deepfreeze.c is
  generated and up to date, and always run "make clinic".
* "make clinic" no longer runs generate_global_objects.py script.
* "make regen-deepfreeze" now only updates deepfreeze.c (C file).
  It doesn't build deepfreeze.o (object) anymore.
* Remove misleading messages in "make regen-global-objects" and
  "make clinic". They are now outdated, these commands are now
  safe to use.
* Document generates files in Doc/using/configure.rst.

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-09-06 20:09:21 +02:00
Victor Stinner
8ff1142578
gh-108851: Fix tomllib recursion tests (#108853)
* Add get_recursion_available() and get_recursion_depth() functions
  to the test.support module.
* Change infinite_recursion() default max_depth from 75 to 100.
* Fix test_tomllib recursion tests for WASI buildbots: reduce the
  recursion limit and compute the maximum nested array/dict depending
  on the current available recursion limit.
* test.pythoninfo logs sys.getrecursionlimit().
* Enhance test_sys tests on sys.getrecursionlimit()
  and sys.setrecursionlimit().
2023-09-06 17:34:31 +02:00
Ed Maste
2cd170db40
gh-91960: Add FreeBSD build and test using Cirrus-CI (#91961)
Cirrus-CI is a hosted CI service that supports FreeBSD, Linux, macOS,
and Windows.  Add a .cirrus.yml configuration file to provide CI coverage
on pull requests for FreeBSD 13.2.

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-06 16:57:40 +02:00
Victor Stinner
fbce43a251
gh-91960: Skip test_gdb if gdb cannot retrive Python frames (#108999)
Skip test_gdb if gdb is unable to retrieve Python frame objects: if a
frame is "<optimized out>". When Python is built with "clang -Og",
gdb can fail to retrive the 'frame' parameter of
_PyEval_EvalFrameDefault(). In this case, tests like py_bt() are
likely to fail. Without getting access to Python frames,
python-gdb.py is mostly clueless on retrieving the Python traceback.
Moreover, test_gdb is no longer skipped on macOS if Python is built
with Clang.
2023-09-06 14:34:35 +00:00
Victor Stinner
cd2ef21b07
gh-108962: Skip test_tempfile.test_flags() if not supported (#108964)
Skip test_tempfile.test_flags() if chflags() fails with "OSError:
[Errno 45] Operation not supported" (ex: on FreeBSD 13).
2023-09-05 21:59:40 +00:00
Anthony Shaw
2c4c26c4ce
gh-108469: Update ast.unparse for unescaped quote support from PEP701 [3.12] (#108553)
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-09-05 21:01:23 +01:00
Zachary Ware
deea7c8268
gh-107565: Update Windows build to use OpenSSL 3.0.10 (GH-108928)
Also clean up some intermediate NEWS entries about previous versions.
2023-09-05 16:03:06 +00:00
Mark Shannon
5a2a046151
GH-108390: Prevent non-local events being set with sys.monitoring.set_local_events() (GH-108420) 2023-09-05 08:03:53 +01:00
Serhiy Storchaka
04a0830b00
gh-89392: Remove support of test_main() in libregrtest (GH-108876) 2023-09-05 08:36:43 +03:00
Tian Gao
6304d983a0
gh-108463: Make expressions/statements work as expected in pdb (#108464) 2023-09-04 21:44:40 +00:00
Victor Stinner
5a79d2ae57
Revert "gh-46376: Return existing pointer when possible in ctypes (#1… (#108688)
This reverts commit 08447b5deb.

Revert also _ctypes.c changes of the PyDict_ContainsString() change,
commit 6726626646.
2023-09-04 11:21:47 +02:00
Victor Stinner
31c2945f14
gh-108834: regrtest reruns failed tests in subprocesses (#108839)
When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.

Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.

Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.

Changes:

* Fix and enhance statistics in regrtest summary. Add "(filtered)"
  when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
  using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
  filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
  the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.
2023-09-03 21:37:15 +00:00
Victor Stinner
03c4080c71
gh-108765: Python.h no longer includes <ctype.h> (#108831)
Remove <ctype.h> in C files which don't use it; only sre.c and
_decimal.c still use it.

Remove _PY_PORT_CTYPE_UTF8_ISSUE code from pyport.h:

* Code added by commit b5047fd019
  in 2004 for MacOSX and FreeBSD.
* Test removed by commit 52ddaefb6b
  in 2007, since Python str type now uses locale independent
  functions like Py_ISALPHA() and Py_TOLOWER() and the Unicode
  database.

Modules/_sre/sre.c replaces _PY_PORT_CTYPE_UTF8_ISSUE with new
functions: sre_isalnum(), sre_tolower(), sre_toupper().

Remove unused includes:

* _localemodule.c: remove <stdio.h>.
* getargs.c: remove <float.h>.
* dynload_win.c: remove <direct.h>, it no longer calls _getcwd()
  since commit fb1f68ed7c (in 2001).
2023-09-03 18:54:27 +02:00
Виталий Дмитриев
55846099b1
Fix duplicated words 'Be be' (GH-108815) 2023-09-03 16:35:13 +03:00
Victor Stinner
0e6d582b3b
gh-63760: Don't declare gethostname() on Solaris (#108817)
Since 2005, Solaris defines gethostname(). socketmodule.c no longer
has to define gethostname() for Solaris.

Oracle Solaris and OpenSolaris have patches to remove the
gethostname() definition in Python:

* https://github.com/oracle/solaris-userland/blob/master/components/python/python37/patches/15-gethostname.patch
* https://github.com/OpenIndiana/oi-userland/blob/oi/hipster/components/python/python37/patches/15-gethostname.patch
* https://github.com/omniosorg/omnios-build/blob/master/build/python27/patches/24-gethostname.patch

Co-authored-by: Jakub Kulík <Kulikjak@gmail.com>
2023-09-02 17:54:59 +00:00
Victor Stinner
f5ddbeeab7
gh-108822: Add Changelog entry for regrtest statistics (#108821) 2023-09-02 19:26:20 +02:00
Victor Stinner
e7de0c5901
gh-108765: Python.h no longer includes <sys/time.h> (#108775)
Python.h no longer includes <time.h>, <sys/select.h> and <sys/time.h>
standard header files.

* Add <time.h> include to xxsubtype.c.
* Add <sys/time.h> include to posixmodule.c and semaphore.c.
* readline.c includes <sys/select.h> instead of <sys/time.h>.
* resource.c no longer includes <time.h> and <sys/time.h>.
2023-09-02 17:51:19 +02:00
Barney Gale
bdc3c884cd
GH-78722: Raise exceptions from pathlib.Path.iterdir() without delay. (#107320)
`pathlib.Path.iterdir()` now immediately raises any `OSError`
exception from `os.listdir()`, rather than waiting until its
result is iterated over.
2023-09-02 16:08:03 +01:00
Victor Stinner
594b00057e
gh-108765: Python.h no longer includes <unistd.h> (#108783) 2023-09-02 16:50:18 +02:00
Victor Stinner
4f9b706c6f
gh-108794: doctest counts skipped tests (#108795)
* Add 'skipped' attribute to TestResults.
* Add 'skips' attribute to DocTestRunner.
* Rename private DocTestRunner._name2ft attribute
  to DocTestRunner._stats.
* Use f-string for string formatting.
* Add some tests.
* Document DocTestRunner attributes and its API for statistics.
* Document TestResults class.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-09-02 16:42:07 +02:00
Victor Stinner
4ba18099b7
gh-108765: Python.h no longer includes <ieeefp.h> (#108781)
Remove also the HAVE_IEEEFP_H macro: remove ieeefp.h from the
AC_CHECK_HEADERS() check of configure.ac.
2023-09-02 15:48:32 +02:00
Alex Waygood
a1cbace91b
gh-105509: Simplify implementation of typing.Annotated (#105510) 2023-09-01 13:57:25 -07:00
Irit Katriel
844f4c2e12
gh-108727: Fix segfault due to missing tp_dealloc definition for CounterOptimizer_Type (GH-108734) 2023-09-01 10:16:09 +01:00
Nikita Sobolev
044b8b3b6a
gh-107805: Fix signatures of module-level generated functions in turtle (#107807)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-09-01 01:18:15 -07:00
Victor Stinner
2bd960b579
gh-108337: Add pyatomic.h header (#108701)
This adds a new header that provides atomic operations on common data
types. The intention is that this will be exposed through Python.h,
although that is not the case yet. The only immediate use is in
the test file.

Co-authored-by: Sam Gross <colesbury@gmail.com>
2023-08-31 21:41:18 +00:00
Ethan Furman
d48760b2f1
gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ (GH-108704)
When overriding the `__new__` method of an enum, the underlying data type should be created directly; i.e. .

    member = object.__new__(cls)
    member = int.__new__(cls, value)
    member = str.__new__(cls, value)

Calling `super().__new__()` finds the lookup version of `Enum.__new__`, and will now raise an exception when detected.
2023-08-31 12:45:12 -07:00
Victor Stinner
13a00078b8
gh-108634: Py_TRACE_REFS uses a hash table (#108663)
Python built with "configure --with-trace-refs" (tracing references)
is now ABI compatible with Python release build and debug build.
Moreover, it now also supports the Limited API.

Change Py_TRACE_REFS build:

* Remove _PyObject_EXTRA_INIT macro.
* The PyObject structure no longer has two extra members (_ob_prev
  and _ob_next).
* Use a hash table (_Py_hashtable_t) to trace references (all
  objects): PyInterpreterState.object_state.refchain.
* Py_TRACE_REFS build is now ABI compatible with release build and
  debug build.
* Limited C API extensions can now be built with Py_TRACE_REFS:
  xxlimited, xxlimited_35, _testclinic_limited.
* No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2()
  functions to PyModule_Create2TraceRefs() and
  PyModule_FromDefAndSpec2TraceRefs().
* _Py_PrintReferenceAddresses() is now called before
  finalize_interp_delete() which deletes the refchain hash table.
* test_tracemalloc find_trace() now also filters by size to ignore
  the memory allocated by _PyRefchain_Trace().

Test changes for Py_TRACE_REFS:

* Add test.support.Py_TRACE_REFS constant.
* Add test_sys.test_getobjects() to test sys.getobjects() function.
* test_exceptions skips test_recursion_normalizing_with_no_memory()
  and test_memory_error_in_PyErr_PrintEx() if Python is built with
  Py_TRACE_REFS.
* test_repl skips test_no_memory().
* test_capi skisp test_set_nomemory().
2023-08-31 18:33:34 +02:00
Carl Meyer
d52c4482a8
gh-108654: restore comprehension locals before handling exception (#108659)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2023-08-30 17:50:50 -06:00
albanD
add8d45cbe
gh-108520: Fix bad fork detection in nested multiprocessing use case (#108568)
gh-107275 introduced a regression where a SemLock would fail being passed along nested child processes, as the `is_fork_ctx` attribute would be left missing after the first deserialization.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-08-30 17:07:41 +00:00
Erlend E. Aasland
2a3926fa51
gh-108590: Revert gh-108657 (commit 400a1cebc) (#108686)
Reverted per Serhiy's request.
2023-08-30 14:53:10 +00:00
Victor Stinner
2928e5dc65
gh-108494: Argument Clinic: Document how to generate code that uses the limited C API (#108584)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-08-30 16:02:48 +02:00
Corvin
400a1cebc7
gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (#108657)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-30 09:06:21 +00:00
Erlend E. Aasland
0b0c1d046c
gh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (#108632)
Deprecate passing the callback callable by keyword for the following
sqlite3.Connection APIs:

- set_authorizer(authorizer_callback)
- set_progress_handler(progress_handler, ...)
- set_trace_callback(trace_callback)

The affected parameters will become positional-only in Python 3.15.
2023-08-29 22:02:12 +02:00
Guido van Rossum
59e46932c8
gh-108488: Initialize JUMP_BACKWARD cache to 0, not 17 (#108591)
This mis-initialization caused the executor optimization to kick in sooner than intended. It also set the lower 4 bits of the counter to `1` -- those bits are supposed to be reserved (the actual counter is in the upper 12 bits).
2023-08-29 18:14:56 +00:00
Brett Cannon
5f85b443f7
GH-106176, GH-104702: Fix reference leak when importing across multiple threads (#108497) 2023-08-29 09:17:25 +02:00
Matthias Bussonnier
f75cefd402
gh-106670: Allow Pdb to move between chained exceptions (#106676) 2023-08-28 18:31:03 +00:00
Guido van Rossum
47d7eba889
gh-108487: Move assert(self != NULL) down beyond DEOPT_IF() (#108510) 2023-08-28 10:17:00 -07:00