Commit Graph

2800 Commits

Author SHA1 Message Date
xdegaye
559bad1a70 bpo-38841: Skip asyncio test_create_datagram_endpoint_existing_sock_unix (GH-17294)
on platforms lacking a functional bind() for named unix domain sockets



https://bugs.python.org/issue38841



Automerge-Triggered-By: @asvetlov
2019-11-20 12:02:07 -08:00
Steve Dower
de148f263f
bpo-33125: Add support for building and releasing Windows ARM64 packages (GH-16828)
Note that the support is not actually enabled yet, and so we won't be publishing these packages. However, for those who want to build it themselves (even by reusing the Azure Pipelines definition), it's now relatively easy to enable.
2019-11-20 09:30:47 -08:00
Federico Bond
be5c79e033 bpo-38821: Fix crash in argparse when using gettext (GH-17192) 2019-11-20 15:29:29 +02:00
Victor Stinner
4dedd0f0dd
bpo-37340: Remove PyMethod_ClearFreeList() and PyCFunction_ClearFreeList() (GH-17284)
Remove PyMethod_ClearFreeList() and PyCFunction_ClearFreeList()
functions: the free lists of bound method objects have been removed.

Remove also _PyMethod_Fini() and _PyCFunction_Fini() functions.
2019-11-20 12:59:12 +01:00
Victor Stinner
7247407c35
bpo-36854: Move _PyRuntimeState.gc to PyInterpreterState (GH-17287)
* Rename _PyGC_InitializeRuntime() to _PyGC_InitState()
* finalize_interp_clear() now also calls _PyGC_Fini() in
  subinterpreters (clear the GC state).
2019-11-20 12:25:50 +01:00
Victor Stinner
488d02a241
bpo-38835: Exclude PyFPE macros from the stable API (GH-17228)
Exclude PyFPE_START_PROTECT() and PyFPE_END_PROTECT() macros of
pyfpe.h from Py_LIMITED_API (stable API).
2019-11-20 12:17:09 +01:00
Terry Jan Reedy
b8462477bf
bpo-38636: Fix IDLE tab toggle and file indent width (GH-17008)
These Format menu functions (default shortcuts Alt-T and Alt-U)
were mistakenly disabled in 3.7.5 and 3.8.0.
2019-11-20 01:18:39 -05:00
Benjamin Peterson
7483451577
closes bpo-38712: Add signal.pidfd_send_signal. (GH-17070)
This exposes a Linux-specific syscall for sending a signal to a process
identified by a file descriptor rather than a pid.

For simplicity, we don't support the siginfo_t parameter to the syscall. This
parameter allows implementing a pidfd version of rt_sigqueueinfo(2), which
Python also doesn't support.
2019-11-19 20:39:14 -08:00
Victor Stinner
be143ec996
bpo-38835: Don't use PyFPE_START_PROTECT and PyFPE_END_PROTECT (GH-17231)
The PyFPE_START_PROTECT() and PyFPE_END_PROTECT() macros are empty:
they have been doing nothing for the last year  (since commit
735ae8d139), so stop using them.
2019-11-20 02:51:30 +01:00
Łukasz Langa
1c5a71a7dd
Merge tag 'v3.9.0a1' 2019-11-20 02:05:28 +01:00
Pablo Galindo
e0cd8aa70a
bpo-37957: Allow regrtest to receive a file with test (and subtests) to ignore (GH-16989)
When building Python in some uncommon platforms there are some known tests that will fail. Right now, the test suite has the ability to ignore entire tests using the -x option and to receive a filter file using the --matchfile filter. The problem with the --matchfile option is that it receives a file with patterns to accept and when you want to ignore a couple of tests and subtests, is too cumbersome to lists ALL tests that are not the ones that you want to accept and he problem with -x is that is not easy to ignore just a subtests that fail and the whole test needs to be ignored.

For these reasons, add a new option to allow to ignore a list of test and subtests for these situations.
2019-11-19 23:46:49 +00:00
Jake Tesler
c6b20be85c bpo-38707: Fix for multiprocessing.Process MainThread.native_id (GH-17088)
This PR implements a fix for `multiprocessing.Process` objects; the error occurs when Processes are created using either `fork` or `forkserver` as the `start_method`.

In these instances, the `MainThread` of the newly created `Process` object retains all attributes from its parent's `MainThread` object, including the `native_id` attribute. The resulting behavior is such that the new process' `MainThread` captures an incorrect/outdated `native_id` (the parent's instead of its own). 

This change forces the Process object to update its `native_id` attribute during the bootstrap process.

cc @vstinner





https://bugs.python.org/issue38707



Automerge-Triggered-By: @pitrou
2019-11-19 11:50:12 -08:00
Adam Johnson
892221bfa0 bpo-38839: Fix some unused functions in tests (GH-17189) 2019-11-19 11:45:20 -08:00
Vincent Michel
8e0de2a480 bpo-35409: Ignore GeneratorExit in async_gen_athrow_throw (GH-14755)
Ignore `GeneratorExit` exceptions when throwing an exception into the `aclose` coroutine of an asynchronous generator.





https://bugs.python.org/issue35409
2019-11-19 05:53:52 -08:00
Łukasz Langa
fd757083df
Python 3.9.0a1 2019-11-19 12:17:21 +01:00
Batuhan Taşkaya
24555ce2f9 bpo-21767: explicitly mention abc support in functools.singledispatch docs (#17171) 2019-11-19 09:16:46 +01:00
Tomás Farías
fe75b62575 bpo-38807: Add os.PathLike to exception message raised by _check_arg_types (#17160) 2019-11-18 21:54:00 -08:00
Steve Dower
00923c6399
bpo-38622: Add missing audit events for ctypes module (GH-17158) 2019-11-18 11:32:46 -08:00
jsnklln
e243bae999 bpo-38722: Runpy use io.open_code() (GH-17234)
https://bugs.python.org/issue38722



Automerge-Triggered-By: @taleinat
2019-11-18 11:11:13 -08:00
Tal Einat
ee703cbb41 bpo-38809: Windows build scripts use python.exe from virtual envs (GH-17164)
https://bugs.python.org/issue38809
2019-11-18 09:32:25 -08:00
Victor Stinner
04394df74b
bpo-38631: Avoid Py_FatalError() in float.__getformat__() (GH-17232)
Replace Py_FatalError() with a regular RuntimeError exception in
float.__getformat__().
2019-11-18 17:39:48 +01:00
Victor Stinner
59c80889ff
Revert "bpo-38811: Check for presence of os.link method in pathlib. (GH-17170)" (#17219)
This reverts commit 111772fc27.
2019-11-18 12:26:37 +01:00
Zackery Spytz
2bc343417a bpo-36589: Fix the error handling in curses.update_lines_cols(). (GH-12766)
Return None instead of 1.
2019-11-17 19:10:13 +02:00
Toke Høiland-Jørgensen
111772fc27 bpo-38811: Check for presence of os.link method in pathlib. (GH-17170)
Fix also the Path.symplink() method implementation for the case when
symlinks are not supported.
2019-11-17 19:06:38 +02:00
Andrey Doroschenko
645005e947 bpo-38724: Implement subprocess.Popen.__repr__ (GH-17151) 2019-11-17 16:08:31 +02:00
Serhiy Storchaka
a0652328a2
bpo-28286: Deprecate opening GzipFile for writing implicitly. (GH-16417)
Always specify the mode argument for writing.
2019-11-16 18:56:57 +02:00
Serhiy Storchaka
bd44a7ead9
bpo-38650: Constify PyStructSequence_UnnamedField. (GH-17005)
Make it a constant and referring to a constant string.
2019-11-16 18:55:29 +02:00
Serhiy Storchaka
5fd5cb8d85
bpo-38639: Optimize floor(), ceil() and trunc() for floats. (GH-16991) 2019-11-16 18:00:57 +02:00
Eric Snow
73cdb0c6b2 bpo-38816: Add notes in the C-API docs about fork in subinterpreters. (GH-17176)
The C-API docs are a bit sparse on the interplay between C `fork()` and the CPython runtime.  This change adds some more information on the subject.


https://bugs.python.org/issue38816
2019-11-15 13:28:54 -08:00
Steve Dower
abde52cd8e
bpo-38453: Ensure ntpath.realpath correctly resolves relative paths (GH-16967)
Ensure isabs() is always True for \\?\ prefixed paths
Avoid unnecessary usage of readlink() to avoid resolving broken links incorrectly
Ensure shutil tests run in test directory
2019-11-15 09:49:21 -08:00
Phil Connell
b22030073b bpo-38778: Document that os.fork is not allowed in subinterpreters (GH-17123)
Small docs update for [bpo-34651](https://bugs.python.org/issue34651).

Other references to fork (e.g. the PyOS.*Fork functions or discussions of fork() when embedding Python) point back to os.fork, so I don't think any other updates are needed.





https://bugs.python.org/issue38778



Automerge-Triggered-By: @ericsnowcurrently
2019-11-15 08:56:03 -08:00
Andrey Doroschenko
e8acc865a3 bpo-38351: Modernize email examples from %-formatting to f-strings (GH-17162) 2019-11-15 11:03:46 +02:00
Benjamin Peterson
3ccdd9b180
closes bpo-38692: Add a pidfd child process watcher to asyncio. (GH-17069) 2019-11-13 19:08:50 -08:00
Andrew Svetlov
dad6be5ffe bpo-38785: Prevent asyncio from crashing (GH-17144)
if parent `__init__` is not called from a constructor of object derived from `asyncio.Future`



https://bugs.python.org/issue38785
2019-11-13 13:36:46 -08:00
Kirill
61289d4366 bpo-38786: Add parsing of https links to pydoc (GH-17143) 2019-11-13 18:13:52 +02:00
Zackery Spytz
9c2844927d bpo-4630: Add cursor no-blink option for IDLE (GH-16960)
This immediately toggles shell, editor, and output windows, but does not affect other input widgets.
2019-11-13 02:13:33 -05:00
Benjamin Peterson
74fa9f723f
closes bpo-27805: Ignore ESPIPE in initializing seek of append-mode files. (GH-17112)
This change, which follows the behavior of C stdio's fdopen and Python 2's file object, allows pipes to be opened in append mode.
2019-11-12 14:51:34 -08:00
jsnklln
d593881505 bpo-38723: Pdb._runscript should use io.open_code() instead of open() (GH-17127)
Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
2019-11-12 14:42:47 -08:00
Zackery Spytz
c8b53dc3d8 bpo-26353: IDLE adds an unneeded newline when saving a shell window (GH-17103) 2019-11-12 05:54:10 -05:00
Manjusaka
051ff526b5 bpo-38565: add new cache_parameters method for lru_cache (GH-16916) 2019-11-11 23:30:18 -08:00
Brandt Bucher
a0ed99bca8 bpo-38438: Simplify argparse "star nargs" usage. (GH-17106) 2019-11-11 12:47:48 -08:00
Raymond Hettinger
84ac437658
bpo-38761: Register WeakSet as a MutableSet (GH-17104) 2019-11-10 20:12:04 -08:00
Inada Naoki
6cbc84fb99
bpo-38613: Optimize set operations of dict keys. (GH-16961) 2019-11-08 00:59:04 +09:00
l0rb
519cb8772a bpo-38716: stop rotating handlers from setting inherited namer and rotator to None (GH-17072) 2019-11-06 21:21:40 +00:00
Benjamin Peterson
5c0c325453 closes bpo-38713: Expose P_PIDFD in os if it's defined. (GH-17071)
https://bugs.python.org/issue38713
2019-11-05 21:58:31 -08:00
Benjamin Peterson
6c4c45efae
bpo-38692: Add os.pidfd_open. (GH-17063) 2019-11-05 19:21:29 -08:00
Jeroen Demeyer
bf17d41826 bpo-37645: add new function _PyObject_FunctionStr() (GH-14890)
Additional note: the `method_check_args` function in `Objects/descrobject.c` is written in such a way that it applies to all kinds of descriptors. In particular, a future re-implementation of `wrapper_descriptor` could use that code.

CC @vstinner @encukou 


https://bugs.python.org/issue37645



Automerge-Triggered-By: @encukou
2019-11-05 07:48:04 -08:00
Eddie Elizondo
b3966639d2 bpo-35381 Remove all static state from posixmodule (GH-15892)
After #9665, this moves the remaining types in posixmodule to be heap-allocated to make it compatible with PEP384 as well as modifying all the type accessors to fully make the type opaque.

The original PR that got messed up a rebase: https://github.com/python/cpython/pull/10854. All the issues in that commit have now been addressed since https://github.com/python/cpython/pull/11661 got committed.

This change also removes any state from the data segment and onto the module state itself.


https://bugs.python.org/issue35381



Automerge-Triggered-By: @encukou
2019-11-05 07:16:14 -08:00
Benjamin Peterson
62161ce989
closes bpo-37633: Reëxport some function compatibility wrappers for macros in `pythonrun.h`. (GH-17056) 2019-11-04 21:34:14 -08:00
Victor Stinner
f4b1e3d7c6
bpo-38644: Add Py_EnterRecursiveCall() to the limited API (GH-17046)
Provide Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() as
regular functions for the limited API. Previously, there were defined
as macros, but these macros didn't work with the limited API which
cannot access PyThreadState.recursion_depth field.

Remove _Py_CheckRecursionLimit from the stable ABI.

Add Include/cpython/ceval.h header file.
2019-11-04 19:48:34 +01:00
Alexandru Ardelean
6552563b3d bpo-38684: haslib: fix build when Blake2 not enabled in OpenSSL (#17043) 2019-11-04 15:55:56 +01:00
Ram Rachum
8d4fef4ee2 bpo-38422: Clarify docstrings of pathlib suffix(es) (GH-16679)
Whenever I use `path.suffix` I have to check again whether it includes the dot or not. I decided to add it to the docstring so I won't have to keep checking. 


https://bugs.python.org/issue38422



Automerge-Triggered-By: @pitrou
2019-11-02 09:46:24 -07:00
Anthony Sottile
b32cb97bce bpo-38312: Add curses.{get,set}_escdelay and curses.{get,set}_tabsize. (GH-16938) 2019-10-31 11:13:48 +02:00
Lucas Cimon
b15100fe7d bpo-38586: setting logging.Handler .name property in fileConfig (GH-16918) 2019-10-31 08:06:25 +00:00
Victor Stinner
a4ed6ed9f3
bpo-38614: Increase asyncio test_communicate() timeout (GH-16995)
Fix test_communicate() of test_asyncio.test_subprocess: use
support.LONG_TIMEOUT (5 minutes), instead of 1 minute.
2019-10-30 16:00:44 +01:00
Pablo Galindo
6c3e66a34b
bpo-38640: Allow break and continue in always false while loops (GH-16992) 2019-10-30 11:53:26 +00:00
Victor Stinner
24c6258269
bpo-38614: Add timeout constants to test.support (GH-16964)
Add timeout constants to test.support:

* LOOPBACK_TIMEOUT
* INTERNET_TIMEOUT
* SHORT_TIMEOUT
* LONG_TIMEOUT
2019-10-30 12:41:43 +01:00
Serhiy Storchaka
865c3b257f
bpo-28029: Make "".replace("", s, n) returning s for any n != 0. (GH-16981) 2019-10-30 12:03:53 +02:00
Daniel Hillier
da6ce58dd5 bpo-36993: Improve error reporting for zipfiles with bad zip64 extra data. (GH-14656) 2019-10-29 09:24:18 +02:00
Steve Dower
0ac6137dd3
bpo-38589: Fixes HTML Help shortcut when Windows is not installed to C drive (GH-16968) 2019-10-28 14:34:25 -07:00
Zackery Spytz
edb172a872 bpo-38519: Internal include files missing on Windows (GH-16921) 2019-10-28 10:03:27 -07:00
Victor Stinner
e471e72977
bpo-37330: open() no longer accept 'U' in file mode (GH-16959)
open(), io.open(), codecs.open() and fileinput.FileInput no longer
accept "U" ("universal newline") in the file mode. This flag was
deprecated since Python 3.3.
2019-10-28 15:40:08 +01:00
Dong-hee Na
3bfc8e0fcc bpo-38602: Add fcntl.F_OFD_XXXX for fcntlmodule (GH-16956) 2019-10-28 09:31:15 +02:00
Marco Rougeth
85c6f8c65c bpo-38592 Add pt-br switcher to Python Docs website (GH-16924)
* 📜🤖 Added by blurb_it.
2019-10-27 09:30:12 +01:00
Serhiy Storchaka
5c32af7522
bpo-38334: Fix seeking backward on an encrypted zipfile.ZipExtFile. (GH-16937)
Test by Daniel Hillier.
2019-10-27 10:22:14 +02:00
Terry Jan Reedy
e3f90b217a
bpo-38598: Do not try to compile IDLE shell or output windows (GH-16939) 2019-10-26 21:15:10 -04:00
Serhiy Storchaka
26ae9f6d3d
bpo-38535: Fix positions for AST nodes for calls without arguments in decorators. (GH-16861) 2019-10-26 16:46:05 +03:00
Zsolt Dollenstein
96b06aefe2 bpo-33348: parse expressions after * and ** in lib2to3 (GH-6586)
These are valid even in python 2.7


https://bugs.python.org/issue33348



Automerge-Triggered-By: @gpshead
2019-10-23 23:19:07 -07:00
Girts
a01ba333af bpo-30618: add readlink to pathlib.Path (GH-8285)
This adds a "readlink" method to pathlib.Path objects that calls through
to os.readlink.


https://bugs.python.org/issue30618



Automerge-Triggered-By: @gpshead
2019-10-23 14:18:40 -07:00
Victor Stinner
1b53a24fb4
bpo-34679: ProactorEventLoop only uses set_wakeup_fd() in main thread (GH-16901)
bpo-34679, bpo-38563: asyncio.ProactorEventLoop.close() now only calls
signal.set_wakeup_fd() in the main thread.
2019-10-23 17:25:29 +02:00
Victor Stinner
028f7349a0
bpo-37415: Fix stdatomic.h header check for ICC compiler (GH-16717)
Fix stdatomic.h header check for ICC compiler: the ICC implementation
lacks atomic_uintptr_t type which is needed by Python.

Test:

* atomic_int and atomic_uintptr_t types
* atomic_load_explicit() and atomic_store_explicit()
* memory_order_relaxed and memory_order_seq_cst constants

But don't test ATOMIC_VAR_INIT(): it's not used in Python.
2019-10-22 21:53:50 +02:00
Tyler Kieft
f548a3e4a2 bpo-38303: Make audioop extension module PEP-384 compatible (GH-16497)
https://bugs.python.org/issue38303



Automerge-Triggered-By: @encukou
2019-10-22 06:07:03 -07:00
Serhiy Storchaka
10ecbadb79
bpo-31202: Preserve case of literal parts in Path.glob() on Windows. (GH-16860) 2019-10-21 20:37:15 +03:00
Serhiy Storchaka
5bc6a7c06e
bpo-38540: Fix possible leak in PyArg_Parse for "es#" and "et#". (GH-16869) 2019-10-21 11:35:07 +03:00
Dong-hee Na
2eba6ad7bf bpo-38493: Add os.CLD_KILLED and os.CLD_STOPPED. (GH-16821) 2019-10-21 10:01:05 +03:00
Hai Shi
06cd5b6acd bpo-38465: Convert the type of exports counters to Py_ssize_t. (GH-16746) 2019-10-21 09:31:46 +03:00
Raymond Hettinger
58ccd201fa
bpo-36321: Fix misspelled attribute name in namedtuple() (GH-16858) 2019-10-20 10:19:47 -07:00
Dong-hee Na
24dc2f8c56 bpo-38525: Fix a segmentation fault when using reverse iterators of empty dict (GH-16846)
The reverse iterator for empty dictionaries was not handling correctly shared-key dictionaries.
2019-10-19 21:01:08 +01:00
Raymond Hettinger
5eabec022b
bpo-38521: Fix error in NormalDist.__eq__() (GH-16840) 2019-10-18 14:20:35 -07:00
Victor Stinner
ecb035cd14
bpo-38502: regrtest uses process groups if available (GH-16829)
test.regrtest now uses process groups in the multiprocessing mode
(-jN command line option) if process groups are available: if
os.setsid() and os.killpg() functions are available.
2019-10-18 15:49:08 +02:00
Tim Graham
5a88d50ff0 bpo-27657: Fix urlparse() with numeric paths (#661)
* bpo-27657: Fix urlparse() with numeric paths

Revert parsing decision from bpo-754016 in favor of the documented
consensus in bpo-16932 of how to treat strings without a // to
designate the netloc.

* bpo-22891: Remove urlsplit() optimization for 'http' prefixed inputs.
2019-10-18 06:07:20 -07:00
Steve Dower
fbe3c76c7c bpo-38418: Fixes audit event for os.system to be named 'os.system' (GH-16670)
https://bugs.python.org/issue38418



Automerge-Triggered-By: @zooba
2019-10-18 00:52:15 -07:00
Gregory P. Smith
f33c57d5c7
bpo-33604: Raise TypeError on missing hmac arg. (GH-16805)
Also updates the documentation to clarify the situation surrounding
the digestmod parameter that is required despite its position in the
argument list as of 3.8.0 as well as removing old python2 era
references to "binary strings".

We indavertently had this raise ValueError in 3.8.0 for the missing
arg.  This is not considered an API change as no reasonable code would
be catching this missing argument error in order to handle it.
2019-10-17 20:30:42 -07:00
Inada Naoki
9c11029bb4
bpo-37587: json: Use _PyUnicodeWriter when scanning string. (GH-15591) 2019-10-17 16:12:41 +09:00
Steve Dower
7aebbd1182
bpo-38492: Remove pythonw.exe dependency on the Microsoft C++ runtime (GH-16824) 2019-10-16 10:27:17 -07:00
Victor Stinner
fab4ef2df0
bpo-35998: Fix test_asyncio.test_start_tls_server_1() (GH-16815)
main() is now responsible to send the ANSWER, rather than
ServerProto. main() now waits until it got the HELLO before sending
the ANSWER over the new transport.

Previously, there was a race condition between main() replacing the
protocol and the protocol sending the ANSWER once it gets the HELLO.

TLSv1.3 was disabled for the test: reenable it.
2019-10-16 02:36:42 +02:00
Joannah Nanjekye
3cd21aa6a1
bpo-38468 : Refactor python-config (#16749)
* refactor python-config

* 📜🤖 Added by blurb_it.

* Update Misc/NEWS.d/next/Build/2019-10-13-16-18-24.bpo-38468.buCO84.rst

Co-Authored-By: Pablo Galindo <Pablogsal@gmail.com>

* Use getvar for pyver

* Update news entry with affected file.

* Update Misc/NEWS.d/next/Build/2019-10-13-16-18-24.bpo-38468.buCO84.rst

Co-Authored-By: Victor Stinner <vstinner@python.org>

* Update 2019-10-13-16-18-24.bpo-38468.buCO84.rst
2019-10-15 11:18:47 -03:00
Julien Danjou
8d59eb1b66 bpo-37961, tracemalloc: add Traceback.total_nframe (GH-15545)
Add a total_nframe field to the traces collected by the tracemalloc module.
This field indicates the original number of frames before it was truncated.
2019-10-15 14:00:16 +02:00
Pablo Galindo
f3ef06a7cb
bpo-38478: Correctly handle keyword argument with same name as positional-only parameter (GH-16800) 2019-10-15 12:40:02 +01:00
Victor Stinner
eb1dda2b56 bpo-38470: Fix test_compileall.test_compile_dir_maxlevels() (GH-16789)
Fix test_compile_dir_maxlevels() on Windows without long path
support: only create 3 subdirectories instead of between 20 and 100
subdirectories.

Fix also compile_dir() to use the current sys.getrecursionlimit()
value as the default maxlevels value, rather than using
sys.getrecursionlimit() value read at startup.
2019-10-15 11:26:13 +02:00
Vinay Sajip
0b60f64e43
bpo-11410: Standardize and use symbol visibility attributes across POSIX and Windows. (GH-16347) 2019-10-15 08:26:12 +01:00
Stein Karlsen
aad2ee0156 bpo-32498: urllib.parse.unquote also accepts bytes (GH-7768) 2019-10-14 13:36:29 +03:00
Pablo Galindo
fd5c414880
bpo-38469: Handle named expression scope with global/nonlocal keywords (GH-16755)
The symbol table handing of PEP572's assignment expressions is not resolving correctly the scope of some variables in presence of global/nonlocal keywords in conjunction with comprehensions.
2019-10-14 05:18:05 +01:00
Pablo Galindo
466326dcdf
bpo-38379: Don't block collection of unreachable objects when some objects resurrect (GH-16687)
Currently if any finalizer invoked during garbage collection resurrects any object, the gc gives up and aborts the collection. Although finalizers are assured to only run once per object, this behaviour of the gc can lead to an ever-increasing memory situation if new resurrecting objects are allocated in every new gc collection.

To avoid this, recompute what objects among the unreachable set need to be resurrected and what objects can be safely collected. In this way, resurrecting objects will not block the collection of other objects in the unreachable set.
2019-10-13 16:48:59 +01:00
Samuel Colvin
793cb85437 bpo-38431: Fix __repr__ method of InitVar to work with typing objects. (GH-16702) 2019-10-13 14:45:36 +03:00
Serhiy Storchaka
140a7d1f35
bpo-38378: Rename parameters "out" and "in" of os.sendfile(). (GH-16742)
They conflicted with keyword "in".

Also rename positional-only parameters of private os._fcopyfile()
for consistency.
2019-10-13 11:59:31 +03:00
Gregory P. Smith
f3751efb5c
bpo-38417: Add umask support to subprocess (GH-16726)
On POSIX systems, allow the umask to be set in the child process before we exec.
2019-10-12 13:24:56 -07:00
Samuel Colvin
822922af90 bpo-35800: Deprecate smtpd.MailmanProxy (GH-11675)
Since `smtpd.MailmanProxy` is already broken, it is not formally deprecated in 3.9. It will be removed in 3.10.


https://bugs.python.org/issue35800
2019-10-12 10:24:26 -07:00
Abhilash Raj
19a3d87300 bpo-38449: Revert "bpo-22347: Update mimetypes.guess_type to allow oper parsing of URLs (GH-15522)" (GH-16724)
This reverts commit 87bd2071c7.



https://bugs.python.org/issue38449
2019-10-11 22:41:35 -07:00
Ruediger Pluem
2b7dc40b2a bpo-38347: find pathfix for Python scripts whose name contain a '-' (GH-16536)
pathfix.py: Assume all files that end on '.py' are Python scripts when working recursively.
2019-10-11 15:36:50 +02:00
Pablo Galindo
320dd504dd
bpo-38437: Activate GC_DEBUG when PY_DEBUG is set (GH-16707) 2019-10-10 22:45:17 +01:00
Ronan Lamy
7bb14316b8 bpo-38109: Add missing constants to Lib/stat.py (GH-16665)
Add missing stat.S_IFDOOR, stat.S_IFPORT, stat.S_IFWHT,
stat.S_ISDOOR, stat.S_ISPORT, and stat.S_ISWHT values to
the Python implementation of the stat module.
2019-10-10 09:34:46 +02:00
Tim Peters
ecbf35f933
bpo-38379: don't claim objects are collected when they aren't (#16658)
* bpo-38379:  when a finalizer resurrects an object,
nothing is actually collected in this run of gc.
Change the stats to relect that truth.
2019-10-09 12:37:30 -05:00
Antonio Gutierrez
594e2edfb5 closes bpo-36161: Use thread-safe ttyname_r instead of ttyname. (GH-14868)
Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
2019-10-08 19:19:48 -07:00
Victor Stinner
0ec618af98
bpo-37531: regrtest ignores output on timeout (GH-16659)
bpo-37531, bpo-38207: On timeout, regrtest no longer attempts to call
`popen.communicate() again: it can hang until all child processes
using stdout and stderr pipes completes. Kill the worker process and
ignores its output.

Reenable test_regrtest.test_multiprocessing_timeout().

bpo-37531: Change also the faulthandler timeout of the main process
from 1 minute to 5 minutes, for Python slowest buildbots.
2019-10-08 18:45:43 +02:00
Pablo Galindo
10cd00a9e3
bpo-38395: Fix ownership in weakref.proxy methods (GH-16632)
The implementation of weakref.proxy's methods call back into the Python
API using a borrowed references of the weakly referenced object
(acquired via PyWeakref_GET_OBJECT). This API call may delete the last
reference to the object (either directly or via GC), leaving a dangling
pointer, which can be subsequently dereferenced.

To fix this, claim a temporary ownership of the referenced object when
calling the appropriate method. Some functions because at the moment they
do not need to access the borrowed referent, but to protect against
future changes to these functions, ownership need to be fixed in
all potentially affected methods.
2019-10-08 16:30:50 +01:00
Victor Stinner
03ab6b4fc6
bpo-38118: Ignore Valgrind false alarm in PyUnicode_Decode() (GH-16651)
Valgrind emits "Conditional jump or move depends on uninitialised
value(s)" false alarms on GCC builtin strcmp() function. The GCC code
is correct.

Valgrind bug: https://bugs.kde.org/show_bug.cgi?id=264936
2019-10-08 15:42:17 +02:00
Serhiy Storchaka
13abda4100
bpo-38405: Make nested subclasses of typing.NamedTuple pickleable. (GH-16641) 2019-10-08 16:29:52 +03:00
Serhiy Storchaka
b690a2759e
bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)
It now escapes them with a backslash, as the regular Python interpreter.
Added the "errors" field to the standard streams.
2019-10-08 14:32:25 +03:00
Serhiy Storchaka
d05b000c6b
bpo-38371: Tkinter: deprecate the split() method. (GH-16584) 2019-10-08 14:31:35 +03:00
Antonio Gutierrez
0d3fe8ae49 closes bpo-38402: Check error of primitive crypt/crypt_r. (GH-16599)
Checks also for encryption algorithms methods not supported in different
OSs.

Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
2019-10-07 21:22:17 -07:00
Victor Stinner
1b18455695
bpo-38392: PyObject_GC_Track() validates object in debug mode (GH-16615)
In debug mode, PyObject_GC_Track() now calls tp_traverse() of the
object type to ensure that the object is valid: test that objects
visited by tp_traverse() are valid.

Fix pyexpat.c: only track the parser in the GC once the parser is
fully initialized.
2019-10-08 00:09:31 +02:00
James Abel
e310af9e29 bpo-38344: Fix syntax in activate.bat (GH-16533) 2019-10-07 14:07:19 -07:00
Ricardo Bánffy
15ae75d660 bpo-38294: Add list of no-longer-escaped chars to re.escape documentation. (GH-16442)
Prior to 3.7, re.escape escaped many characters that don't have
special meaning in Python, but that use to require escaping in other
tools and languages. This commit aims to make it clear which characters
were, but are no longer escaped.
2019-10-07 23:54:35 +03:00
Victor Stinner
6876257eaa
bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.

Modify the following functions to also work in release mode:

* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

Other changes:

* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
  (equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
  with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
  attempt to render repr() at the end.
2019-10-07 18:42:01 +02:00
Serhiy Storchaka
ef092fe990
bpo-25988: Do not expose abstract collection classes in the collections module. (GH-10596) 2019-10-07 12:10:15 +03:00
Dong-hee Na
c38e725d17 bpo-38210: Fix intersection operation with dict view and iterator. (GH-16602) 2019-10-06 14:28:33 +03:00
Andrei Troie
65dcc8a8dc bpo-38332: Catch KeyError from unknown cte in encoded-word. (GH-16503)
KeyError should cause a failure in parsing the encoded word and should be caught and raised as a _InvalidEWError instead.
2019-10-05 09:19:15 -07:00
nde
3faf826e58 bpo-38341: Add SMTPNotSupportedError in the exports of smtplib (#16525)
Add SMTPNotSupportedError in the exports of smtplib

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
2019-10-04 17:30:58 -07:00
Joannah Nanjekye
8855e47d09 bpo-38266: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal (GH-16558)
Revert the removal of PyThreadState_DeleteCurrent() with documentation.
2019-10-04 13:35:42 +02:00
Serhiy Storchaka
06cb94bc84
bpo-13153: Use OS native encoding for converting between Python and Tcl. (GH-16545)
On Windows use UTF-16 (or UTF-32 for 32-bit Tcl_UniChar) with the
"surrogatepass" error handler for converting to/from Tcl Unicode objects.

On Linux use UTF-8 with the "surrogateescape" error handler for converting
to/from Tcl String objects.

Converting strings from Tcl to Python and back now never fails
(except MemoryError).
2019-10-04 13:09:52 +03:00
Steve Dower
353fb1ecbf
bpo-38359: Ensures pyw.exe launcher reads correct registry key (GH-16561) 2019-10-03 08:31:21 -07:00
Steve Dower
a0e3d27e4e
bpo-38355: Fix ntpath.realpath failing on sys.executable (GH-16551) 2019-10-03 08:31:03 -07:00
Victor Stinner
5e0ea7540f
bpo-37474: Don't call fedisableexcept() on FreeBSD (GH-16515)
On FreeBSD, Python no longer calls fedisableexcept() at startup to
control the floating point control mode. The call became useless
since FreeBSD 6: it became the default mode.
2019-10-01 13:12:29 +02:00
Victor Stinner
3c30a76f3d
bpo-38304: Remove PyConfig.struct_size (GH-16500) (GH-16508)
For now, we'll rely on the fact that the config structures aren't covered by the stable ABI.

We may revisit this in the future if we further explore the idea of offering a stable embedding API.

(cherry picked from commit bdace21b76)
2019-10-01 10:56:37 +02:00
Giampaolo Rodola
94e165096f
bpo-38319: Fix shutil._fastcopy_sendfile(): set sendfile() max block size (GH-16491) 2019-10-01 11:40:54 +08:00
Maxwell A McKinnon
cf57cabef8 bpo-32689: Updates shutil.move to allow for Path objects to be used as source arg (GH-15326)
Important work originally done by @emilyemorehouse two years ago and nearly ready to go in.

This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwise wonderful pathlib and PEP519. https://stackoverflow.com/questions/33625931/copy-file-with-pathlib-in-python.

This adds the outstanding test request from that PR @vstinner (https://github.com/python/cpython/pull/5393).

Test fails without the change, passes with it, along with every other test in test_shutil.

Some variants were experimented with to make the one line change and the most performant one was picked.


# Added Test for PathLike directory destination, the current fail case

```
Lib/test/test_shutil.py::TestMove::test_move_file_pathlike FAILED                                                               [100%]

============================================================== FAILURES ===============================================================
__________________________________________________ TestMove.test_move_file_pathlike ___________________________________________________

self = <test.test_shutil.TestMove testMethod=test_move_file_pathlike>

    def test_move_file_pathlike(self):
        # Move a file to another location on the same filesystem.
        src = pathlib.Path(self.src_file)
>       self._check_move_file(src, self.dst_dir, self.dst_file)

Lib/test/test_shutil.py:1563:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Lib/test/test_shutil.py:1545: in _check_move_file
    shutil.move(src, dst)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:562: in move
    real_dst = os.path.join(dst, _basename(src))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = PosixPath('/var/folders/r2/psq74t5x3nbfzlph8bh2pvdw0000gn/T/tmp9ie0wh9_/foo')

    def _basename(path):
        # A basename() variant which first strips the trailing slash, if present.
        # Thus we always get the last component of the path, even for directories.
        sep = os.path.sep + (os.path.altsep or '')
>       return os.path.basename(path.rstrip(sep))
E       AttributeError: 'PosixPath' object has no attribute 'rstrip'

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:526: AttributeError
============================================== 1 failed, 102 deselected in 0.30 seconds ===============================================
```

After change:

```
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/maxwellmckinnon/dev/cpython
plugins: cov-2.7.1, mock-1.10.4
collected 103 items / 102 deselected / 1 selected

Lib/test/test_shutil.py::TestMove::test_move_file_pathlike PASSED                                                               [100%]

============================================== 1 passed, 102 deselected in 0.06 seconds ===============================================
```

Running all the tests in test_shutil.py
```
╰─ pytest Lib/test/test_shutil.py -v
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/maxwellmckinnon/dev/cpython
plugins: cov-2.7.1, mock-1.10.4
collected 103 items

Lib/test/test_shutil.py::TestShutil::test_chown PASSED                                                                          [  0%]
Lib/test/test_shutil.py::TestShutil::test_copy PASSED                                                                           [  1%]
...
Lib/test/test_shutil.py::TermsizeTests::test_stty_match SKIPPED                                                                 [ 99%]
Lib/test/test_shutil.py::PublicAPITests::test_module_all_attribute PASSED                                                       [100%]

================================================ 96 passed, 7 skipped in 1.25 seconds =================================================
```

# Performance Considerations
Is it considered poor form to get rid of _basename altogether and make use of pathlib in the move function? I'm not sure if the idea is for all these modules to strictly avoid circular dependencies. They are already using os.path which is just as much a citizen in 3.8 as pathlib right?

e.g.

`real_dst = os.path.join(dst, _basename(src))`
becomes
`real_dst = Path(dst) / Path(src).name`

I've looked around and familiarized myself, and I now think importing pathlib here is fine. My only remaining concern is that of performance.

Here's the performance difference for this step. 

```
In [46]: %timeit real_dst = os.path.join("a/b/c", _basename('b/'))
2.71 µs ± 62.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [47]: %timeit real_dst = Path("a/b/c") / Path('b/').name
12.4 µs ± 65.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
```

Is 10us significant or insignificant compared to the least expensive operation this function will do? I don't know. Let's find out.

```
In [55]: %timeit os.rename('/tmp/a/a.txt', '/tmp/a/b.txt'); os.rename('/tmp/a/b.txt', '/tmp/a/a.txt')
124 µs ± 2.18 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```
62us to rename. 10us seems significant enough that we wouldn't want to favor the Path sugar suggestion. 16% speed decrease from adding the 10us.

What do people think? I was hoping to get to use pathlib.Path here, but I suspect for this low level move, it should be as fast as possible, and 16% is not worth one line of sugary code to me.



https://bugs.python.org/issue32689



Automerge-Triggered-By: @gvanrossum
2019-09-30 19:41:16 -07:00
Neil Schemenauer
bcda460baf Clear weakrefs in garbage found by the GC (#16495)
Fix a bug due to the interaction of weakrefs and the cyclic garbage
collector. We must clear any weakrefs in garbage in order to prevent
their callbacks from executing and causing a crash.
2019-09-30 19:06:45 +02:00
Yury Selivanov
fc4a044a3c
bpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel (#7468) 2019-09-29 22:59:11 -07:00
Yury Selivanov
6758e6e12a
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
See https://bugs.python.org/issue38242 for more details
2019-09-29 21:59:55 -07:00
Lisa Roach
3667e1ee6c
bpo-38163: Child mocks detect their type as sync or async (GH-16471) 2019-09-29 21:56:47 -07:00
Lisa Roach
25e115ec00
bpo-38161: Removes _AwaitEvent from AsyncMock. (GH-16443) 2019-09-29 21:01:28 -07:00
Victor Stinner
fb4ae152a9
bpo-38317: Fix PyConfig.warnoptions priority (GH-16478)
Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.

* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
  with "_Py".
* test_embed: add test_init_warnoptions().
2019-09-30 01:40:17 +02:00
Andrew Svetlov
58498bc717
bpo-38019: correctly handle pause/resume reading of closed asyncio unix pipe (GH-16472) 2019-09-29 15:00:35 +03:00
Lisa Roach
9a7d951950
bpo-38108: Makes mock objects inherit from Base (GH-16060) 2019-09-28 18:42:44 -07:00
Brandt Bucher
f185a73249 bpo-38310: Predict BUILD_MAP_UNPACK_WITH_CALL -> CALL_FUNCTION_EX. (GH-16467) 2019-09-28 17:12:49 -07:00
T. Wouters
c8165036f3 bpo-38115: Deal with invalid bytecode offsets in lnotab (GH-16079)
Document that lnotab can contain invalid bytecode offsets (because of
terrible reasons that are difficult to fix). Make dis.findlinestarts()
ignore invalid offsets in lnotab. All other uses of lnotab in CPython
(various reimplementations of addr2line or line2addr in Python, C and gdb)
already ignore this, because they take an address to look for, instead.

Add tests for the result of dis.findlinestarts() on wacky constructs in
test_peepholer.py, because it's the easiest place to add them.
2019-09-28 07:49:15 -07:00
Jason R. Coombs
7774d7831e
bpo-38216, bpo-36274: Allow subclasses to separately override validation and encoding behavior (GH-16448)
* bpo-38216: Allow bypassing input validation

* bpo-36274: Also allow the URL encoding to be overridden.

* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.

* Call with skip_host to avoid tripping on the host checking in the URL.

* Remove obsolete comment.

* Make _prepare_path_encoding its own attr.

This makes overriding just that simpler.

Also, don't use the := operator to make backporting easier.

* Add a news entry.

* _prepare_path_encoding -> _encode_prepared_path()

* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
2019-09-28 08:32:01 -04:00
Victor Stinner
441b10cf28
bpo-38304: Add PyConfig.struct_size (GH-16451)
Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.

* Replace private _config_version field with public struct_size field
  in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
  PyPreConfig_InitPythonConfig()
  return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
  _PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
  _PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
  instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.
2019-09-28 04:28:35 +02:00
Jesús Cea
52d1b86bde
bpo-38301: In Solaris family, we must be sure to use '-D_REENTRANT' (#16446) 2019-09-28 03:44:32 +02:00
Dong-hee Na
e8650a4f8c bpo-38243, xmlrpc.server: Escape the server_title (GH-16373)
Escape the server title of xmlrpc.server.DocXMLRPCServer
when rendering the document page as HTML.
2019-09-27 21:59:37 +02:00
Michael Felt
0bcbfa43d5 bpo-28009: Fix uuid.uuid1() and uuid.get_node() on AIX (GH-8672) 2019-09-26 22:43:15 +03:00
Christian Heimes
df6ac7e2b8 bpo-38275: Skip ssl tests for disabled versions (GH-16386)
test_ssl now handles disabled TLS/SSL versions better. OpenSSL's crypto
policy and run-time settings are recognized and tests for disabled versions
are skipped.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38275
2019-09-26 08:02:59 -07:00
Victor Stinner
64b4a3a2de
bpo-38239: Fix test_gdb for Link Time Optimization (LTO) (GH-16422) 2019-09-26 16:54:13 +02:00
Lumír 'Frenzy' Balhar
8e7bb991de bpo-38112: Compileall improvements (GH-16012)
* Raise the limit of maximum path depth to actual  recursion limit

* Add posibilities to adjust a path compiled in .pyc  file.

Now, you can:
- Strip a part of path from a beggining of path into compiled file
   example "-s /test /test/build/real/test.py" → "build/real/test.py"
- Append some new path to a beggining of path into compiled file
   example "-p /boo real/test.py" → "/boo/real/test.py"

You can also use both options in the same time. In that case,
striping is done before appending.

* Add a possibility to specify multiple optimization levels

Each optimization level then leads to separated compiled file.
Use `action='append'` instead of `nargs='+'` for the -o option.
Instead of `-o 0 1 2`, specify `-o 0 -o 1 -o 2`. It's more to type,
but much more explicit.

* Add a symlinks limitation feature

This feature allows us to limit byte-compilation of symbolic
links if they are pointing outside specified dir (build root
for example).
2019-09-26 08:28:26 +02:00
Benjamin Peterson
52b9408038
closes bpo-38174: Update vendored expat library to 2.2.8. (GH-16346)
Fixes CVE-2019-15903. See full changelog at https://github.com/libexpat/libexpat/blob/R_2_2_8/expat/Changes.
2019-09-25 21:33:58 -07:00
Christian Heimes
df69e75edc
bpo-38142: Updated _hashopenssl.c to be PEP 384 compliant (#16071)
* Updated _hashopenssl.c to be PEP 384 compliant
* Remove refleak test from test_hashlib. The updated type no longer accepts random arguments to __init__.
2019-09-25 23:03:30 +02:00
Christian Heimes
bfd0c963d8 bpo-38271: encrypt private key test files with AES256 (GH-16385)
The private keys for test_ssl were encrypted with 3DES in traditional
PKCS#5 format. 3DES and the digest algorithm of PKCS#5 are blocked by
some strict crypto policies. Use PKCS#8 format with AES256 encryption
instead.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38271



Automerge-Triggered-By: @tiran
2019-09-25 08:55:02 -07:00
Christian Heimes
c64a1a61e6 bpo-38270: Check for hash digest algorithms and avoid MD5 (GH-16382)
Make it easier to run and test Python on systems with restrict crypto policies:

* add requires_hashdigest to test.support to check if a hash digest algorithm is available and working
* avoid MD5 in test_hmac
* replace MD5 with SHA256 in test_tarfile
* mark network tests that require MD5 for MD5-based digest auth or CRAM-MD5


https://bugs.python.org/issue38270
2019-09-25 07:30:20 -07:00
PatrikKopkan
1dc1acbd73 bpo-37064: Add option -a to pathfix.py tool (GH-15717)
Add option -a to Tools/Scripts/pathfix.py script: add flags.
2019-09-25 14:26:28 +02:00
Yury Selivanov
edad4d89e3 bpo-38248: Fix inconsistent immediate asyncio.Task cancellation (GH-16330) 2019-09-25 03:32:08 -07:00
Serhiy Storchaka
f163aeaa8c
bpo-38219: Optimize dict creating and updating by a dict. (GH-16268) 2019-09-25 09:47:00 +03:00
Dong-hee Na
ad7736faf5 bpo-38265: Update os.pread to accept the length type as Py_ssize_t. (GH-16359) 2019-09-25 08:47:04 +03:00
Samuel Freilich
b5a7a4f0c2 bpo-36871: Handle spec errors in assert_has_calls (GH-16005)
The fix in PR 13261 handled the underlying issue about the spec for specific methods not being applied correctly, but it didn't fix the issue that was causing the misleading error message.

The code currently grabs a list of responses from _call_matcher (which may include exceptions). But it doesn't reach inside the list when checking if the result is an exception. This results in a misleading error message when one of the provided calls does not match the spec.


https://bugs.python.org/issue36871



Automerge-Triggered-By: @gpshead
2019-09-24 12:08:31 -07:00
Victor Stinner
1ce152a42e
bpo-38234: Py_SetPath() uses the program full path (GH-16357)
Py_SetPath() now sets sys.executable to the program full path
(Py_GetProgramFullPath()), rather than to the program name
(Py_GetProgramName()).

Fix also memory leaks in pathconfig_set_from_config().
2019-09-24 17:44:15 +02:00
Victor Stinner
b0e1ae5f54
bpo-37123: multiprocessing test_mymanager() accepts SIGTERM (GH-16349)
Multiprocessing test test_mymanager() now also expects -SIGTERM, not
only exitcode 0.

bpo-30356: BaseManager._finalize_manager() sends SIGTERM to the
manager process if it takes longer than 1 second to stop, which
happens on slow buildbots.
2019-09-24 14:19:48 +02:00
Victor Stinner
99799c7220
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
Multiprocessing tests: increase test_queue_feeder_donot_stop_onexc()
timeout from 1 to 60 seconds.
2019-09-24 12:47:49 +02:00
Lisa Roach
ef04851775
bpo-38136: Updates await_count and call_count to be different things (GH-16192) 2019-09-23 20:49:40 -07:00
Victor Stinner
9c42f8cda5
bpo-38234: Fix _PyConfig_InitPathConfig() (GH-16335)
* _PyConfig_InitPathConfig() now starts by copying the global path
  configuration, and then override values set in PyConfig.
* _PyPathConfig_Calculate() implementations no longer override
  _PyPathConfig fields which are already computed. For example,
  if _PyPathConfig.prefix is not NULL, leave it unchanged.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() doesn't
  call _PyPathConfig_Calculate() anymore.
* _PyPathConfig_Calculate() no longer uses PyConfig,
  except to initialize PyCalculatePath structure.
* pathconfig_calculate(): remove useless temporary
  "_PyPathConfig new_config" variable.
* calculate_module_search_path(): remove hack to workaround memory
  allocation failure, call Py_FatalError() instead.
* Fix get_program_full_path(): handle memory allocation failure.
2019-09-23 18:47:29 +02:00
Victor Stinner
fcdb027234
bpo-38236: Dump path config at first import error (GH-16300)
Python now dumps path configuration if it fails to import the Python
codecs of the filesystem and stdio encodings.
2019-09-23 14:45:47 +02:00
Ammar Askar
87d6cd3604 bpo-38237: Make pow's arguments have more descriptive names and be keyword passable (GH-16302)
Edit: `math.pow` changes removed on Mark's request.


https://bugs.python.org/issue38237



Automerge-Triggered-By: @rhettinger
2019-09-20 21:28:49 -07:00
Victor Stinner
e267793aa4
bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298)
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() now
  uses its value.
* Py_Initialize() now longer copies path configuration from PyConfig
  to the global path configuration (_Py_path_config).
2019-09-21 01:50:16 +02:00
Victor Stinner
b1542583be
bpo-38205: Py_UNREACHABLE() calls Py_FatalError() (GH-16290) 2019-09-20 23:10:16 +02:00
Ram Rachum
9c2682efc6 bpo-37937: Mention frame.f_trace in sys.settrace docs (GH-15439)
Mention frame.f_trace in sys.settrace docs, as well as the fact you still
need to call `sys.settrace` to enable the tracing machinery before setting
`frame.f_trace` will have any effect.
2019-09-21 01:19:02 +10:00
Lisa Roach
8b03f943c3
bpo-38093: Correctly returns AsyncMock for async subclasses. (GH-15947) 2019-09-19 21:04:18 -07:00
Michael Selik
2702638eab bpo-34002: Minor efficiency and clarity improvements in email package. (GH-7999)
* Check intersection of two sets explicitly

Comparing ``len(a) > ``len(a - b)`` is essentially looking for an
intersection between the two sets. If set ``b`` does not intersect ``a``
then ``len(a - b)`` will be equal to ``len(a)``. This logic is more
clearly expressed as ``a & b``.

* Change while/pop to a for-loop

Copying the list, then repeatedly popping the first element was
unnecessarily slow. I also cleaned up a couple other inefficiencies.
There's no need to unpack a tuple, then re-pack and append it. The list
can be created with the first element instead of empty. Secondly, the
``endswith`` method returns a bool, so there's no need for an if-
statement to set ``encoding`` to True or False.

* Use set.intersection to check for intersections

``a.intersection(b)`` method is more clear of purpose than ``not
a.isdisjoint(b)`` and avoids an unnecessary set construction that ``a &
set(b)`` performs.

* Use not isdisjoint instead of intersection

While it reads slightly worse, the isdisjoint method will stop when it
finds a counterexample and returns a bool, rather than looping over the
entire iterable and constructing a new set.
2019-09-19 20:25:55 -07:00
Eddie Elizondo
3368f3c6ae bpo-38140: Make dict and weakref offsets opaque for C heap types (#16076)
* Make dict and weakref offsets opaque for C heap types

* Add news
2019-09-19 17:29:05 +01:00
t k
96b1c59c71 bpo-38155: Add __all__ to datetime module (GH-16203)
https://bugs.python.org/issue38155
2019-09-19 14:34:41 +01:00
Kyle Stanley
9fdc64cf12 bpo-34037: Fix test_asyncio failure and add loop.shutdown_default_executor() (GH-15735) 2019-09-19 15:47:22 +03:00
Serhiy Storchaka
8fc5839a9d
bpo-38191: Turn warnings into errors in NamedTuple() and TypedDict(). (GH-16238) 2019-09-17 22:41:55 +03:00
Serhiy Storchaka
2bf31ccab3
bpo-38191: Accept arbitrary keyword names in NamedTuple() and TypedDict(). (GH-16222)
This includes such names as "cls", "self", "typename", "_typename",
"fields" and "_fields".
Passing positional arguments by keyword is deprecated.
2019-09-17 21:22:00 +03:00
Andrew Svetlov
c275312a62 bpo-38013: make async_generator_athrow object tolerant to throwing exceptions (GH-16070)
Even when the helper is not started yet.

This behavior follows conventional generator one.
There is no reason for `async_generator_athrow` to handle `gen.throw()` differently.



https://bugs.python.org/issue38013
2019-09-17 05:59:49 -07:00
Abraham Toriz Cruz
5f5f11faf9 bpo-37828: Fix default mock_name in unittest.mock.assert_called error (GH-16166)
In the format string for assert_called the evaluation order is incorrect and hence for mock's without name, 'None' is printed whereas it should be 'mock' like for other messages. The error message is ("Expected '%s' to have been called." % self._mock_name or 'mock').
2019-09-17 12:16:08 +01:00
Serhiy Storchaka
f669581a95
bpo-38185: Fixed case-insensitive string comparison in sqlite3.Row indexing. (GH-16190) 2019-09-17 09:20:56 +03:00
Terry Jan Reedy
dfd34a9cd5
bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212)
One happens when EditorWindow.close is called twice.
Printing a traceback, when IDLE is run from a terminal,
is useless and annoying.
2019-09-17 02:05:04 -04:00
Terry Jan Reedy
0048afc16a
bpo-38183: Test_idle ignores user config directory GH-16198)
It no longer tries to create or access .idlerc or any files within.
Users must run IDLE to discover problems with saving settings.
2019-09-16 19:04:21 -04:00
Christian Heimes
724f1a5723 bpo-33936: Don't call obsolete init methods with OpenSSL 1.1.0+ (GH-16140)
``OPENSSL_VERSION_1_1`` was never defined in ``_hashopenssl.c``. 


https://bugs.python.org/issue33936
2019-09-16 12:10:05 -07:00
Serhiy Storchaka
8debfa5040
bpo-38175: Fix a memory leak in comparison of sqlite3.Row objects. (GH-16155) 2019-09-16 20:15:18 +03:00
Steve Dower
3ab73f6bbf
bpo-38117: Update bundled Windows OpenSSL to 1.1.1d (GH-16184) 2019-09-16 15:21:16 +01:00
Ned Deily
24d1597e43
bpo-38117: Updated OpenSSL to 1.1.1d in macOS installer. (GH-16170) 2019-09-16 03:38:27 +01:00
Dino Viehland
f919054e53
bpo-38116: Convert select module to PEP-384 (#15971)
* Convert select module to PEP-384

Summary: Do the necessary versions to be Pyro-compatible, including migrating `PyType_Ready` to `PyType_FromSpec` and moving static data into a new `_selectstate` struct.

* 📜🤖 Added by blurb_it.

* Fixup Mac OS/X build
2019-09-14 15:20:27 +01:00
Serhiy Storchaka
279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Serhiy Storchaka
bf169915ec
bpo-38005: Fixed comparing and creating of InterpreterID and ChannelID. (GH-15652)
* Fix a crash in comparing with float (and maybe other crashes).
* They are now never equal to strings and non-integer numbers.
* Comparison with a large number no longer raises OverflowError.
* Arbitrary exceptions no longer silenced in constructors and comparisons.
* TypeError raised in the constructor contains now the name of the type.
* Accept only ChannelID and int-like objects in channel functions.
* Accept only InterpreterId, int-like objects and str in the InterpreterId constructor.
* Accept int-like objects, not just int in interpreter related functions.
2019-09-13 22:50:27 +03:00
plokmijnuhby
e082e7cbe4 bpo-37953: Fix ForwardRef hash and equality checks (GH-15400)
Ideally if we stick a ForwardRef in a dictionary we would like to reliably be able to get it out again.

https://bugs.python.org/issue37953
2019-09-13 20:40:54 +01:00
Steve Dower
f2b7556ef8 bpo-38092: Reduce overhead when using multiprocessing in a Windows virtual environment (GH-16098)
https://bugs.python.org/issue38092
2019-09-13 09:40:19 -07:00
Gregory P. Smith
5b9ff7a0dc
bpo-34706: Preserve subclassing in inspect.Signature.from_callable (GH-16108)
https://bugs.python.org/issue34706

Specifically in the case of a class that does not override its
constructor signature inherited from object.

These are Buck Evan @bukzor's changes cherrypicked from GH-9344.
2019-09-13 17:13:51 +01:00
Joannah Nanjekye
2ccb50cf77 bpo-37449: Move ensurepip off of pkgutil and to importlib.resources (GH-15109)
Move ensurepip off of pkgutil and to importlib.resources.


https://bugs.python.org/issue37449
2019-09-13 09:01:20 -07:00
Elizabeth Uselton
d6a9d17d8b bpo-37555: Update _CallList.__contains__ to respect ANY (#14700)
* Flip equality to use mock calls' __eq__

* bpo-37555: Regression test demonstrating assert_has_calls not working with ANY and spec_set

Co-authored-by: Neal Finne <neal@nealfinne.com>

* Revert "Flip equality to use mock calls' __eq__"

This reverts commit 94ddf54c5a.

* bpo-37555: Add regression tests for mock ANY ordering issues

Add regression tests for whether __eq__ is order agnostic on _Call and _CallList, which is useful for comparisons involving ANY, especially if the ANY comparison is to a class not defaulting __eq__ to NotImplemented.

Co-authored-by: Neal Finne <neal@nealfinne.com>

* bpo-37555: Fix _CallList and _Call order sensitivity

_Call and _CallList depend on ordering to correctly process that an object being compared to ANY with __eq__ should return True. This fix updates the comparison to check both a == b and b == a and return True if either condition is met, fixing situations from the tests in the previous two commits where assertEqual would not be commutative if checking _Call or _CallList objects. This seems like a reasonable fix considering that the Python data model specifies that if an object doesn't know how to compare itself to another object it should return NotImplemented, and that on getting NotImplemented from a == b, it should try b == a, implying that good behavior for __eq__ is commutative. This also flips the order of comparison in _CallList's __contains__ method, guaranteeing ANY will be on the left and have it's __eq__ called for equality checking, fixing the interaction between assert_has_calls and ANY.

Co-author: Neal Finne <neal@neal.finne.com>

* bpo-37555: Ensure _call_matcher returns _Call object

* Adding ACK and news entry

* bpo-37555: Replacing __eq__ with == to sidestep NotImplemented

bool(NotImplemented) returns True, so it's necessary to use ==
instead of __eq__ in this comparison.

* bpo-37555: cleaning up changes unnecessary to the final product

* bpo-37555: Fixed call on bound arguments to respect args and kwargs

* Revert "bpo-37555: Add regression tests for mock ANY ordering issues"

This reverts commit 49c5310ad4.

* Revert "bpo-37555: cleaning up changes unnecessary to the final product"

This reverts commit 18e964ba01.

* Revert "bpo-37555: Replacing __eq__ with == to sidestep NotImplemented"

This reverts commit f295eaca5b.

* Revert "bpo-37555: Fix _CallList and _Call order sensitivity"

This reverts commit 874fb697b8.

* Updated NEWS.d

* bpo-37555: Add tests checking every function using _call_matcher both with and without spec

* bpo-37555: Ensure all assert methods using _call_matcher are actually passing calls

* Remove AnyCompare and use call objects everywhere.

* Revert "Remove AnyCompare and use call objects everywhere."

This reverts commit 24973c0b32.

* Check for exception in assert_any_await
2019-09-13 16:54:32 +01:00
Zackery Spytz
b761e3aed1 bpo-25068: urllib.request.ProxyHandler now lowercases the dict keys (GH-13489) 2019-09-13 15:07:07 +01:00
Doyle Rowland
0519d497b0 closes bpo-36002: Use AC_PATH_TOOL to find llvm-profdata and llvm-ar. (GH-14998) 2019-09-13 14:38:07 +01:00
Christian Heimes
995b5d38e7 bpo-38153: Normalize hashlib algorithm names (GH-16083)
Signed-off-by: Christian Heimes <christian@python.org>
2019-09-13 14:31:19 +01:00
Andrew Svetlov
9eb35ab0d7
bpo-38148: Add slots to asyncio transports (GH-16077)
* bpo-38148: Add slots to asyncio transports

* Update Misc/NEWS.d/next/Library/2019-09-13-08-55-43.bpo-38148.Lnww6D.rst

Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
2019-09-13 15:18:46 +03:00
Ashwin Ramaswami
ff2e182865 bpo-12707: deprecate info(), geturl(), getcode() methods in favor of headers, url, and status properties for HTTPResponse and addinfourl (GH-11447)
Co-Authored-By: epicfaace <aramaswamis@gmail.com>
2019-09-13 12:40:07 +01:00
Xtreak
bb41147eab bpo-12144: Handle cookies with expires attribute in CookieJar.make_cookies (GH-13921)
Handle time comparison for cookies with `expires` attribute when `CookieJar.make_cookies` is called.

Co-authored-by: Demian Brecht <demianbrecht@gmail.com>


https://bugs.python.org/issue12144



Automerge-Triggered-By: @asvetlov
2019-09-13 04:29:00 -07:00
Rémi Lapeyre
6a517c6749 bpo-8538: Add support for boolean actions to argparse (GH-11478)
Co-Authored-By: remilapeyre <remi.lapeyre@henki.fr>
2019-09-13 11:17:43 +01:00
Dino Viehland
04f0bbfbed bpo-38075: Port _randommodule.c to PEP-384 (GH-15798)
- Migrate `Random_Type` to `PyType_FromSpec`
- To simulate an old use of `PyLong_Type.tp_as_number->nb_absolute`, I added
  code to the module init function to stash `int.__abs__` for later
  use. Ideally we'd use `PyType_GetSlot()` instead, but it doesn't currently
  work for static types in CPython, and implementing it just for this case
  doesn't seem worth it.
- Do exact check for long and dispatch to PyNumber_Absolute, use vector call when not exact.
2019-09-13 03:12:27 -07:00
Jakub Kulík
42671aea2d bpo-37785: Fix xgettext warning in argparse (GH-15161) 2019-09-13 11:25:32 +02:00
Christian Heimes
7cad53e6b0 bpo-9216: Add usedforsecurity to hashlib constructors (GH-16044)
The usedforsecurity keyword only argument added to the hash constructors is useful for FIPS builds and similar restrictive environment with non-technical requirements that legacy algorithms be forbidden by their implementations without being explicitly annotated as not being used for any security related purposes.  Linux distros with FIPS support benefit from this being standard rather than making up their own way(s) to do it.

Contributed and Signed-off-by: Christian Heimes christian@python.org
2019-09-12 19:30:00 -05:00
Steve Dower
ed93a8852d
bpo-38133: Allow py.exe launcher to locate installations from the Microsoft Store (GH-16025) 2019-09-12 18:16:50 +01:00
Patrick McLean
2b2ead7438 bpo-36046: Add user and group parameters to subprocess (GH-11950)
* subprocess: Add user, group and extra_groups paremeters to subprocess.Popen

This adds a `user` parameter to the Popen constructor that will call
setreuid() in the child before calling exec(). This allows processes
running as root to safely drop privileges before running the subprocess
without having to use a preexec_fn.

This also adds a `group` parameter that will call setregid() in
the child process before calling exec().

Finally an `extra_groups` parameter was added that will call
setgroups() to set the supplimental groups.
2019-09-12 18:15:44 +01:00