Commit Graph

13871 Commits

Author SHA1 Message Date
Zackery Spytz
73c9326563
gh-80109: Fix io.TextIOWrapper dropping the internal buffer during write() (GH-22535)
io.TextIOWrapper was dropping the internal decoding buffer
during read() and write() calls.
2024-01-08 12:33:34 +02:00
Rami
84d1f76092
gh-89532: Remove LibreSSL workarounds (#28728)
Remove LibreSSL specific workaround ifdefs from `_ssl.c` and delete the non-version-specific `_ssl_data.h` file (relevant for OpenSSL < 1.1.1, which we no longer support per PEP 644).

Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2024-01-06 23:25:58 +00:00
Sam Gross
d0f0308a37
gh-113750: Fix object resurrection in free-threaded builds (gh-113751)
gh-113750: Fix object resurrection on free-threaded builds

This avoids the undesired re-initializing of fields like `ob_gc_bits`,
`ob_mutex`, and `ob_tid` when an object is resurrected due to its
finalizer being called.

This change has no effect on the default (with GIL) build.
2024-01-06 12:12:26 +09:00
Sam Gross
99854ce170
gh-113688: Split up gcmodule.c (gh-113715)
This splits part of Modules/gcmodule.c of into Python/gc.c, which
now contains the core garbage collection implementation. The Python
module remain in the Modules/gcmodule.c file.
2024-01-05 12:17:16 -08:00
Christopher Chavez
f637b44dd2
gh-111178: Avoid calling functions from incompatible pointer types in _tkinter.c (GH-112893)
Fix undefined behavior warnings (UBSan  -fsanitize=function).
2024-01-02 15:51:32 +01:00
Neil Schemenauer
b2566d89ce
GH-113633: Use module state structure for _testcapi. (GH-113634)
Use module state structure for _testcapi.
2024-01-01 23:04:09 +00:00
Ronald Oussoren
d0b0e3d2ef
gh-113536: Expose os.waitid on macOS (#113542)
* gh-113536: Expose `os.waitid` on macOS

This API has been available on macOS for a long time, but was
explicitly excluded due to unspecified problems with the API
in ancient versions of macOS.

* Document that the API is available on macOS starting in Python 3.13
2024-01-01 19:38:29 +01:00
Jeffrey Kintscher
5f3cc90a12
gh-62260: Fix ctypes.Structure subclassing with multiple layers (GH-13374)
The length field of StgDictObject for Structure class contains now
the total number of items in ffi_type_pointer.elements (excluding
the trailing null).

The old behavior of using the number of elements in the parent class can
cause the array to be truncated when it is copied, especially when there
are multiple layers of subclassing.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-01-01 18:24:24 +02:00
Kirill Podoprigora
cf34b7704b
gh-103092: Make `pyexpat` module importable in sub-interpreters (#113555) 2023-12-29 18:43:46 +05:30
Zackery Spytz
f108468970
bpo-11102: Make configure enable major(), makedev(), and minor() on HP-UX (GH-19856)
Always include <sys/types.h> before <sys/sysmacros.h>.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-12-28 12:47:44 +02:00
Kirill Podoprigora
f1676867b5
gh-103092: Make _elementtree module importable in sub-interpreters (#113434)
Enable imports of _elementtree module in sub-interpreters
2023-12-28 14:42:21 +05:30
David Benjamin
af2b8f6845
gh-113332: Simplify calls to SSL_(CTX_)set_verify in _ssl.c (#113333)
_ssl.c currently tries to preserve the verification callback, but at no
point does it ever set one. Just pass in NULL.
2023-12-26 16:35:41 -05:00
Serhiy Storchaka
1f06baeabd
gh-113191: Add support of os.fchmod() on Windows (GH-113192)
Also support a file descriptor in os.chmod().
2023-12-24 10:57:11 +00:00
Kirill Podoprigora
894f0e573d
gh-111784: Fix two segfaults in the elementtree module (GH-113405)
First fix resolve situation when pyexpat module (which contains expat_CAPI
capsule) deallocates before _elementtree, so we need to hold a strong
reference to pyexpat module to.

Second fix resolve situation when module state is deallocated before
deallocation of XMLParser instances, which uses module state to clear
some stuff.
2023-12-24 10:57:41 +02:00
Donghee Na
57b7e52790
gh-112205: Support docstring for @getter (#113160)
---------

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-12-20 21:52:12 +09:00
Gregory P. Smith
c895403de0
gh-113119: Fix the macOS framework installer build (#113268)
`--enable-framework` builds were failing.  we apparently do not have good CI & buildbot coverage here.
2023-12-18 21:18:30 -08:00
Itamar Oren
2feec0fc7f
gh-113039: Avoid using leading dots in the include path for frozen getpath.py (GH-113022) 2023-12-18 17:04:40 +00:00
Jakub Kulík
2b93f52242
gh-113117: Support posix_spawn in subprocess.Popen with close_fds=True (#113118)
Add support for `os.POSIX_SPAWN_CLOSEFROM` and
`posix_spawn_file_actions_addclosefrom_np` and have the `subprocess` module use
them when available.  This means `posix_spawn` can now be used in the default
`close_fds=True` situation on many platforms.

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2023-12-17 21:34:57 +00:00
Carson Radtke
cfa25fe3e3
gh-113149: Improve error message when JSON has trailing comma (GH-113227) 2023-12-17 20:52:26 +02:00
Jakub Kulík
48c907a15c
gh-113119 fix environment handling in subprocess.Popen when posix_spawn is used (#113120)
* Allow posix_spawn to inherit environment form parent environ variable.

With this change, posix_spawn call can behave similarly to execv with regards to environments when used in subprocess functions.
2023-12-17 05:19:05 +00:00
Raymond Hettinger
1583c40be9
gh-113202: Add a strict option to itertools.batched() (gh-113203) 2023-12-16 09:13:50 -06:00
Sam Gross
5ae75e1be2
gh-111964: Add _PyRWMutex a "readers-writer" lock (gh-112859)
This adds `_PyRWMutex`, a "readers-writer" lock, which wil be used to
serialize global stop-the-world pauses with per-interpreter pauses.
2023-12-15 18:56:55 -07:00
Serhiy Storchaka
e365c943f2
gh-113172: Fix compiler warnings in Modules/_xxinterpqueuesmodule.c (GH-113173)
Fix compiler waarnings in Modules/_xxinterpqueuesmodule.c
2023-12-15 17:36:25 +02:00
Zackery Spytz
a723a13bf1
bpo-36796: Clean the error handling in _testcapimodule.c (GH-13085) 2023-12-14 19:06:53 +00:00
Steve Dower
fd81afc624
gh-86179: Avoid making case-only changes when calculating realpath() during initialization (GH-113077) 2023-12-14 15:16:39 +00:00
Serhiy Storchaka
29f7eb4859
gh-59616: Support os.chmod(follow_symlinks=True) and os.lchmod() on Windows (GH-113049) 2023-12-14 13:28:37 +02:00
Donghee Na
23a5711100
gh-112205: Update textio module to use @getter as possible. (gh-113095) 2023-12-14 10:26:46 +00:00
Serhiy Storchaka
bb36f72efc
gh-111049: Fix crash during garbage collection of the BytesIO buffer object (GH-111221) 2023-12-14 10:04:23 +00:00
Steve Dower
fddc829236
gh-86179: Implement realpath() on Windows for getpath.py calculations (GH-113033) 2023-12-13 23:41:43 +00:00
Diego Russo
6644ca45cd
gh-110190: Fix ctypes structs with array on PPCLE64 (GH-112959)
Fix the same issue of PR #112604 on PPC64LE platform
Refactor tests to make easier to add more platfroms if needed.
2023-12-13 17:08:15 +01:00
Steve Dower
79dad03747
gh-111650: Ensure pyconfig.h includes Py_GIL_DISABLED on Windows (GH-112778) 2023-12-13 15:38:45 +00:00
Donghee Na
498a096a51
gh-112205: Support @setter annotation from AC (gh-112922)
---------

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-12-13 14:00:34 +00:00
Sam Gross
a3c031884d
gh-112723: Call PyThreadState_Clear() from the correct interpreter (#112776)
The `PyThreadState_Clear()` function must only be called with the GIL
held and must be called from the same interpreter as the passed in
thread state. Otherwise, any Python objects on the thread state may be
destroyed using the wrong interpreter, leading to memory corruption.

This is also important for `Py_GIL_DISABLED` builds because free lists
will be associated with PyThreadStates and cleared in
`PyThreadState_Clear()`.

This fixes two places that called `PyThreadState_Clear()` from the wrong
interpreter and adds an assertion to `PyThreadState_Clear()`.
2023-12-12 17:20:21 -07:00
Eric Snow
9898e61041
gh-76785: Add Interpreter.prepare_main() (gh-113021)
This is one of the last pieces to get test.support.interpreters in sync with PEP 734.
2023-12-12 18:06:06 +00:00
Eric Snow
a49b427b02
gh-76785: More Fixes for test.support.interpreters (gh-113012)
This brings the module (along with the associated extension modules) mostly in sync with PEP 734.  There are only a few small things to wrap up.
2023-12-12 17:43:30 +00:00
Eric Snow
86a77f4e1a
gh-76785: Fixes for test.support.interpreters (gh-112982)
This involves a number of changes for PEP 734.
2023-12-12 08:24:31 -07:00
Ronald Oussoren
f26bfe4b25
gh-51944: fix type and missing addition in gh-112823 (#112996)
Fix the defition of VDSUSP and add CCTS_OFLOW (which
was mentioned in NEWS, but not actually addded)
2023-12-12 13:57:45 +01:00
Ronald Oussoren
0d2fe6bab0
gh-87286: Add a number of LOG_* constants to syslog (#24432)
* bpo-43120: Add a number of LOG_* constants to syslog

This adds a number of syslog facilities to the syslogmodule.c.
These values are available on macOS.

* Switch contant documentation to the data directive

This fixes a CI warning and matches the pattern
used in the documentation for ``os``.

* Update Doc/library/syslog.rst


Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-12-12 10:25:51 +01:00
Sam Gross
d70e27f258
gh-112529: Use atomic operations for gcstate->collecting (#112533)
* gh-112529: Use atomic operations for `gcstate->collecting`

The `collecting` field in `GCState` is used to prevent overlapping garbage
collections within the same interpreter. This is updated to use atomic
operations in order to be thread-safe in `--disable-gil` builds.

The GC code is refactored a bit to support this. More of the logic is pushed
down to `gc_collect_main()` so that we can safely order the logic setting
`collecting`, the selection of the generation, and the invocation of callbacks
with respect to the atomic operations and the (future) stop-the-world pauses.

The change uses atomic operations for both `--disable-gil` and the default
build (with the GIL) to avoid extra `#ifdef` guards and ease the maintenance
burden.
2023-12-11 18:33:21 +00:00
Brad Larsen
eb27c9a99e
Add a fuzzer for Py_CompileStringExFlags (#111721) 2023-12-10 12:16:15 -05:00
Ronald Oussoren
4b125dd31a
gh-51944: Add missing macOS constants to termios (#112823)
* gh-51944: Add some macOS constants to termios

This changeset adds all public constants in <termio.h>
and <sys/termios.h> on macOS that weren't present
already.

Based on the macOS 14.2 SDK

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-12-07 11:27:30 +01:00
Pablo Galindo Salgado
16448cab44
gh-112730: Use color to highlight error locations (gh-112732)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-12-06 23:29:54 +01:00
Christopher Chavez
b920d6ceaa
gh-111178: Define visitproc callback functions properly and remove unnecessary casts in gcmodule.c (#112687) 2023-12-06 15:30:37 +00:00
Victor Stinner
828451dfde
gh-111545: Add Py_HashPointer() function (#112096)
* Implement _Py_HashPointerRaw() as a static inline function.
* Add Py_HashPointer() tests to test_capi.test_hash.
* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
2023-12-06 15:09:22 +01:00
Diego Russo
bc68f4a4ab
gh-110190: Fix ctypes structs with array on Arm (#112604)
Set MAX_STRUCT_SIZE to 32 in stgdict.c when on Arm platforms.
This because on Arm platforms structs with at most 4 elements of any
floating point type values can be passed through registers. If the type
is double the maximum size of the struct is 32 bytes.
On x86-64 Linux, it's maximum 16 bytes hence we need to differentiate.
2023-12-05 16:07:50 +01:00
Tian Gao
e7e1116a78
gh-105323: Remove WITH_APPLE_EDITLINE to use the same declaration for all editline (gh-112513) 2023-12-05 23:52:28 +09:00
Gregory P. Smith
9fe7655c6c
gh-112334: Restore subprocess's use of vfork() & fix extra_groups=[] behavior (#112617)
Restore `subprocess`'s intended use of `vfork()` by default for performance on Linux;
also fixes the behavior of `extra_groups=[]` which was unintentionally broken in 3.12.0:

Fixed a performance regression in 3.12's :mod:`subprocess` on Linux where it
would no longer use the fast-path ``vfork()`` system call when it could have
due to a logic bug, instead falling back to the safe but slower ``fork()``.

Also fixed a security bug introduced in 3.12.0.  If a value of ``extra_groups=[]``
was passed to :mod:`subprocess.Popen` or related APIs, the underlying
``setgroups(0, NULL)`` system call to clear the groups list would not be made
in the child process prior to ``exec()``.

The security issue was identified via code inspection in the process of
fixing the first bug.  Thanks to @vain for the detailed report and
analysis in the initial bug on Github.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-12-04 15:08:19 -08:00
Victor Stinner
c5fa8a54db
gh-112535: Add test on _Py_ThreadId() (#112709)
Add also test.support.Py_GIL_DISABLED constant.
2023-12-04 22:40:06 +00:00
Gregory P. Smith
4eddb4c9d9
gh-105967: Work around a macOS bug, limit zlib C library crc32 API calls to 1gig (#112615)
Work around a macOS bug, limit zlib crc32 calls to 1GiB.

Without this, `zlib.crc32` and `binascii.crc32` could produce incorrect
results on multi-gigabyte inputs depending on the macOS version's Apple
supplied zlib implementation.
2023-12-04 12:04:05 -08:00
Serhiy Storchaka
6ca9d3e017
gh-109786: Fix leaks and crash when re-enter itertools.pairwise.__next__() (GH-109788) 2023-12-04 11:47:55 +00:00