Commit Graph

348 Commits

Author SHA1 Message Date
Greg Price
3a4f66707e Cut disused recode_encoding logic in _PyBytes_DecodeEscape. (GH-16013)
All call sites pass NULL for `recode_encoding`, so this path is
completely untested.  That's been true since before Python 3.0.
It adds significant complexity to this logic, so it's best to
take it out.

All call sites now have a literal NULL, and that's been true since
commit 768921cf3 eliminated a conditional (`foo ? bar : NULL`) at
the call site in Python/ast.c where we're parsing a bytes literal.
But even before then, that condition `foo` had been a constant
since unadorned string literals started meaning Unicode, in commit
572dbf8f1 aka v3.0a1~1035 .

The `unicode` parameter is already unused, so mark it as unused too.
The code that acted on it was also taken out before Python 3.0, in
commit 8d30cc014 aka v3.0a1~1031 .

The function (PyBytes_DecodeEscape) is exposed in the API, but it's
never been documented.
2019-09-12 19:12:22 +01:00
Sergey Fedoseev
afdeb189e9 Remove unneeded assignment in PyBytes_Concat() (GH-15274)
The `wb.len = -1` assignment is unneeded since its introduction in 161d695fb0 as `PyObject_GetBuffer` always fills it in.
2019-09-10 17:11:10 +01:00
Greg Price
0711642eec Cut tricky goto that isn't needed, in _PyBytes_DecodeEscape. (GH-15825)
This is the sort of `goto` that requires the reader to stare hard at
the code to unpick what it's doing.

On doing so, the answer is... not very much!

* It jumps from the bottom of the loop to almost the top; the effect
  is to bypass the loop condition `s < end` and also the
  `if`-condition `*s != '\\'`, acting as if both are true.

* We've just decremented `s`, after incrementing it in the `switch`
  condition.  So it has the same value as when `s == end` failed.
  Before that was another increment... and before that we had
  `s < end`.  So `s < end` true, then increment, then `s == end`
  false... that means `s < end` is still true.

* Also this means `s` points to the same character as it did for the
  `switch` condition.  And there was a `case '\\'`, which we didn't
  hit -- so `*s != '\\'` is also true.

* That means this has no effect on the behavior!  The most it might do
  is an optimization -- we get to skip those two checks, because (as
  just proven above) we know they're true.

* But gosh, this is the *invalid escape sequence* path.  This does not
  seem like the kind of code path that calls for extreme optimization
  tricks.

So, take the `goto` and the label out.

Perhaps the compiler will notice the exact same facts we showed above,
and generate identical code.  Or perhaps it won't!  That'll be OK.

But then, crucially, if some future edit to this loop causes the
reasoning above to *stop* holding true... the compiler will adjust
this jump accordingly.  One of us fallible humans might not.
2019-09-10 09:51:04 +01:00
Victor Stinner
bed4817d52
Make PyXXX_Fini() functions private (GH-15531)
For example, rename PyTuple_Fini() to _PyTuple_Fini().

These functions are only declared in the internal C API.
2019-08-27 00:12:32 +02:00
Jeroen Demeyer
196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Jeroen Demeyer
530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Gregory P. Smith
0c2f930564
bpo-22385: Support output separators in hex methods. (#13578)
* bpo-22385: Support output separators in hex methods.

Also in binascii.hexlify aka b2a_hex.

The underlying implementation behind all hex generation in CPython uses the
same pystrhex.c implementation.  This adds support to bytes, bytearray,
and memoryview objects.

The binascii module functions exist rather than being slated for deprecation
because they return bytes rather than requiring an intermediate step through a
str object.

This change was inspired by MicroPython which supports sep in its binascii
implementation (and does not yet support the .hex methods).

https://bugs.python.org/issue22385
2019-05-29 11:46:58 -07:00
Inada Naoki
7d408697a9
remove unnecessary tp_dealloc (GH-13647) 2019-05-29 17:23:27 +09:00
Victor Stinner
331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
David Carlier
27ee0f8551 Fix couple of dead code paths (GH-7418) 2019-05-17 19:46:22 -04:00
Zackery Spytz
14514d9084 bpo-36946: Fix possible signed integer overflow when handling slices. (GH-13375)
The final addition (cur += step) may overflow, so use size_t for "cur".
"cur" is always positive (even for negative steps), so it is safe to use
size_t here.

Co-Authored-By: Martin Panter <vadmium+py@gmail.com>
2019-05-17 10:13:03 +03:00
Victor Stinner
c96be811fa
bpo-36900: Replace global conf vars with config (GH-13299)
Replace global configuration variables with core_config read from the
current interpreter.

Cleanup dynload_hpux.c.
2019-05-14 17:34:56 +02:00
Serhiy Storchaka
d53fe5f407
bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264) 2019-03-13 22:59:55 +02:00
Serhiy Storchaka
d586ccb04f
bpo-35552: Fix reading past the end in PyUnicode_FromFormat() and PyBytes_FromFormat(). (GH-11276)
Format characters "%s" and "%V" in PyUnicode_FromFormat() and "%s" in PyBytes_FromFormat()
no longer read memory past the limit if precision is specified.
2019-01-12 10:30:35 +02:00
Serhiy Storchaka
44cc4822bb
bpo-33817: Fix _PyBytes_Resize() for empty bytes object. (GH-11516)
Add also tests for PyUnicode_FromFormat() and PyBytes_FromFormat()
with empty result.
2019-01-12 09:22:29 +02:00
Serhiy Storchaka
bb86bf4c4e
bpo-35444: Unify and optimize the helper for getting a builtin object. (GH-11047)
This speeds up pickling of some iterators.

This fixes also error handling in pickling methods when fail to
look up builtin "getattr".
2018-12-11 08:28:18 +02:00
Victor Stinner
b509d52083
bpo-35059: PyObject_INIT() casts to PyObject* (GH-10674)
PyObject_INIT() and PyObject_INIT_VAR() now cast their first argument
to PyObject*, as done in Python 3.7.

Revert partially commit b4435e20a9.
2018-11-23 14:27:38 +01:00
Victor Stinner
bcda8f1d42
bpo-35081: Add Include/internal/pycore_object.h (GH-10640)
Move _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() from
Include/objimpl.h to Include/internal/pycore_object.h.
2018-11-21 22:27:47 +01:00
Victor Stinner
621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner
27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Pablo Galindo
49c75a8086
bpo-35064 prefix smelly symbols that appear with COUNT_ALLOCS with _Py_ (GH-10152)
Configuring python with ./configure --with-pydebug CFLAGS="-D COUNT_ALLOCS -O0"
makes "make smelly" fail as some symbols were being exported without the "Py_" or
"_Py" prefixes.
2018-10-28 15:02:17 +00:00
Victor Stinner
b4435e20a9
bpo-35059: Convert PyObject_INIT() to function (GH-10077)
* Convert PyObject_INIT() and PyObject_INIT_VAR() macros to static
  inline functions.
* Fix usage of these functions: cast to PyObject* or PyVarObject*.
2018-10-26 14:35:00 +02:00
Serhiy Storchaka
2c2044e789
bpo-34984: Improve error messages for bytes and bytearray constructors. (GH-9874) 2018-10-21 15:29:12 +03:00
Serhiy Storchaka
914f9a078f
bpo-34973: Fix crash in bytes constructor. (GH-9841)
Constructing bytes from mutating list could cause a crash.
2018-10-21 15:25:53 +03:00
Serhiy Storchaka
e890421e33
bpo-34974: Do not replace unexpected errors in bytes() and bytearray(). (GH-9852)
bytes and bytearray constructors converted unexpected exceptions
(e.g. MemoryError and KeyboardInterrupt) to TypeError.
2018-10-15 00:02:57 +03:00
Zackery Spytz
96c5932794 bpo-34879: Fix a possible null pointer dereference in bytesobject.c (GH-9683)
formatfloat() was not checking if PyBytes_FromStringAndSize()
failed, which could lead to a null pointer dereference in
_PyBytes_FormatEx().
2018-10-03 09:01:30 +03:00
Alexey Izbyshev
ccd9975267 bpo-34436: Fix check that disables overallocation for the last fmt specifier (GH-8826)
Reported by Svace static analyzer.
2018-08-23 09:50:52 +02:00
Victor Stinner
53b7d4e402
bpo-34170: Add _PyCoreConfig.bytes_warning (GH-8447)
Add more fields to _PyCoreConfig:

* _check_hash_pycs_mode
* bytes_warning
* debug
* inspect
* interactive
* legacy_windows_fs_encoding
* legacy_windows_stdio
* optimization_level
* quiet
* unbuffered_stdio
* user_site_directory
* verbose
* write_bytecode

Changes:

* Remove pymain_get_global_config() and pymain_set_global_config()
  which became useless. These functions have been replaced by
  _PyCoreConfig_GetGlobalConfig() and
  _PyCoreConfig_SetGlobalConfig().
* sys.flags.dont_write_bytecode value is now restricted to 1 even if
  -B option is specified multiple times on the command line.
* PyThreadState_Clear() now uses the config from the current
  interpreter rather than using global Py_VerboseFlag
2018-07-25 01:37:05 +02:00
Tal Einat
c929df3b96
bpo-20180: complete AC conversion of Objects/stringlib/transmogrify.h (GH-8039)
* converted bytes methods: expandtabs, ljust, rjust, center, zfill
* updated char_convertor to properly set the C default value
2018-07-06 13:17:38 +03:00
Siddhesh Poyarekar
55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
INADA Naoki
a49ac99029
bpo-32677: Add .isascii() to str, bytes and bytearray (GH-5342) 2018-01-27 14:06:21 +09:00
stratakis
e8b1965639 bpo-23699: Use a macro to reduce boilerplate code in rich comparison functions (GH-793) 2017-11-02 20:32:54 +10:00
Serhiy Storchaka
a2314283ff
bpo-20047: Make bytearray methods partition() and rpartition() rejecting (#4158)
separators that are not bytes-like objects.
2017-10-29 02:11:54 +03:00
Serhiy Storchaka
56cb465cc9 bpo-31825: Fixed OverflowError in the 'unicode-escape' codec (#4058)
and in codecs.escape_decode() when decode an escaped non-ascii byte.
2017-10-20 17:08:15 +03:00
Barry Warsaw
b2e5794870 bpo-31338 (#3374)
* Add Py_UNREACHABLE() as an alias to abort().
* Use Py_UNREACHABLE() instead of assert(0)
* Convert more unreachable code to use Py_UNREACHABLE()
* Document Py_UNREACHABLE() and a few other macros.
2017-09-14 18:13:16 -07:00
Eric Snow
2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Serhiy Storchaka
b879fe82e7 Expand the PySlice_GetIndicesEx macro. (#1023) 2017-04-08 09:53:51 +03:00
Serhiy Storchaka
6b5a9ec478 bpo-29116: Fix error messages for concatenating bytes and bytearray with unsupported type. (#709) 2017-03-19 19:47:02 +02:00
Xiang Zhang
7e2a54cdd9 bpo-28856: Let %b format for bytes support objects that follow the buffer protocol (GH-546) 2017-03-14 15:07:15 +08:00
Serhiy Storchaka
202fda55c2 bpo-24037: Add Argument Clinic converter bool(accept={int}). (#485) 2017-03-12 10:10:47 +02:00
Serhiy Storchaka
9f8ad3f39e bpo-29568: Disable any characters between two percents for escaped percent "%%" in the format string for classic string formatting. (GH-513) 2017-03-08 11:51:19 +08:00
Xiang Zhang
b76ad5121e bpo-29714: Fix a regression that bytes format may fail when containing zero bytes inside. (GH-499) 2017-03-06 17:17:05 +08:00
INADA Naoki
7ed28a8914 Merge 3.6 2017-01-06 17:44:43 +09:00
INADA Naoki
a634e23209 Issue #29159: Fix regression in bytes(x) when x.__index__() raises Exception. 2017-01-06 17:32:01 +09:00
Serhiy Storchaka
dd1da7f74a Issue #28927: bytes.fromhex() and bytearray.fromhex() now ignore all ASCII
whitespace, not only spaces.  Patch by Robert Xiao.
2016-12-19 18:51:37 +02:00
Serhiy Storchaka
9d16b616d5 Issue #29000: Fixed bytes formatting of octals with zero padding in alternate
form.
2016-12-17 22:15:10 +02:00
Serhiy Storchaka
025f8953f1 Issue #29000: Fixed bytes formatting of octals with zero padding in alternate
form.
2016-12-17 22:13:05 +02:00
Serhiy Storchaka
b1a1619bf0 Issue #29000: Fixed bytes formatting of octals with zero padding in alternate
form.
2016-12-17 21:48:03 +02:00
Victor Stinner
f17c3de263 Use _PyObject_CallNoArg()
Replace:
    PyObject_CallFunctionObjArgs(callable, NULL)
with:
    _PyObject_CallNoArg(callable)
2016-12-06 18:46:19 +01:00
Victor Stinner
de4ae3d486 Backed out changeset b9c9691c72c5
Issue #28858: The change b9c9691c72c5 introduced a regression. It seems like
_PyObject_CallArg1() uses more stack memory than
PyObject_CallFunctionObjArgs().
2016-12-04 22:59:09 +01:00