Commit Graph

123 Commits

Author SHA1 Message Date
Inada Naoki
6773203487
bpo-47000: Add locale.getencoding() (GH-32068) 2022-04-09 09:54:54 +09:00
Victor Stinner
00d7abd7ef
bpo-42519: Replace PyMem_MALLOC() with PyMem_Malloc() (GH-23586)
No longer use deprecated aliases to functions:

* Replace PyMem_MALLOC() with PyMem_Malloc()
* Replace PyMem_REALLOC() with PyMem_Realloc()
* Replace PyMem_FREE() with PyMem_Free()
* Replace PyMem_Del() with PyMem_Free()
* Replace PyMem_DEL() with PyMem_Free()

Modify also the PyMem_DEL() macro to use directly PyMem_Free().
2020-12-01 09:56:42 +01:00
Kevin Adler
c79667ff79
bpo-42087: Remove support for AIX 5.3 and below (GH-22830)
As AIX 5.3 and below do not support thread_cputime, it was decided in
https://bugs.python.org/issue40680 to require AIX 6.1 and above. This
commit removes workarounds for — and references to — older, unsupported
AIX versions.
2020-11-16 16:16:10 +01:00
Victor Stinner
82458b6cdb
bpo-42236: Enhance _locale._get_locale_encoding() (GH-23083)
* Rename _Py_GetLocaleEncoding() to _Py_GetLocaleEncodingObject()
* Add _Py_GetLocaleEncoding() which returns a wchar_t* string to
  share code between _Py_GetLocaleEncodingObject()
  and config_get_locale_encoding().
* _Py_GetLocaleEncodingObject() now decodes nl_langinfo(CODESET)
  from the current locale encoding with surrogateescape,
  rather than using UTF-8.
2020-11-01 20:59:35 +01:00
Victor Stinner
b62bdf71ea
bpo-42208: Add _locale._get_locale_encoding() (GH-23052)
* Add a new _locale._get_locale_encoding() function to get the
  current locale encoding.
* Modify locale.getpreferredencoding() to use it.
* Remove the _bootlocale module.
2020-10-31 01:32:11 +01:00
TIGirardi
f2312037e3
bpo-38324: Fix test__locale.py Windows failures (GH-20529)
Use wide-char _W_* fields of lconv structure on Windows
Remove "ps_AF" from test__locale.known_numerics on Windows
2020-10-20 12:39:52 +01:00
Zackery Spytz
bbceef6851
bpo-20183: Convert _locale to the Argument Clinic (GH-14201) 2020-07-15 12:07:34 +03:00
Serhiy Storchaka
8f87eefe7f
bpo-39943: Add the const qualifier to pointers on non-mutable PyBytes data. (GH-19472) 2020-04-12 14:58:27 +03:00
Hai Shi
7a6f3bcc43
bpo-1635741: Fix refleak in _locale init error handling (GH-19307) 2020-04-02 20:00:47 +02:00
Hai Shi
13397ee47d
bpo-39824: Convert PyModule_GetState() to get_module_state() (GH-19076)
Automerge-Triggered-By: @vstinner
2020-03-19 10:11:33 -07:00
Victor Stinner
5b1ef200d3
bpo-39824: module_traverse() don't call m_traverse if md_state=NULL (GH-18738)
Extension modules: m_traverse, m_clear and m_free functions of
PyModuleDef are no longer called if the module state was requested
but is not allocated yet. This is the case immediately after the
module is created and before the module is executed (Py_mod_exec
function). More precisely, these functions are not called if m_size is
greater than 0 and the module state (as returned by
PyModule_GetState()) is NULL.

Extension modules without module state (m_size <= 0) are not affected.

Co-Authored-By: Petr Viktorin <encukou@gmail.com>
2020-03-17 18:09:46 +01:00
Hai Shi
a158168a78
bpo-1635741: Port _locale extension module to multiphase initialization (PEP 489) (GH-18358)
Co-authored-by: Petr Viktorin <pviktori@redhat.com>
2020-03-11 17:46:06 +01:00
Serhiy Storchaka
d53fe5f407
bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264) 2019-03-13 22:59:55 +02:00
Zackery Spytz
99d56b5356 bpo-35441: Remove dead and buggy code related to PyList_SetItem(). (GH-11033)
In _localemodule.c and selectmodule.c, remove dead code that would
cause double decrefs if run.

In addition, replace PyList_SetItem() with PyList_SET_ITEM() in cases
where a new list is populated and there is no possibility of an error.

In addition, check if the list changed size in the loop in array_array_fromlist().
2018-12-08 16:16:55 +02:00
Victor Stinner
02e6bf7f20
bpo-28604: Fix localeconv() for different LC_MONETARY (GH-10606)
locale.localeconv() now sets temporarily the LC_CTYPE locale to the
LC_MONETARY locale if the two locales are different and monetary
strings are non-ASCII. This temporary change affects other threads.

Changes:

* locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode
  monetary fields.
* Add LocaleInfo.grouping_buffer: copy localeconv() grouping string
  since it can be replaced anytime if a different thread calls
  localeconv().
* _Py_GetLocaleconvNumeric() now requires a "struct lconv *"
  structure, so locale.localeconv() now longer calls localeconv()
  twice. Moreover, the function now requires all arguments to be
  non-NULL.
* Rename STATIC_LOCALE_INFO_INIT to LocaleInfo_STATIC_INIT.
* Move _Py_GetLocaleconvNumeric() definition from fileutils.h
  to pycore_fileutils.h. pycore_fileutils.h now includes locale.h.
* The _locale module is now built with Py_BUILD_CORE defined.
2018-11-20 16:20:16 +01:00
Victor Stinner
9e4994d410
bpo-34485: Enhance init_sys_streams() (GH-8978)
Python now gets the locale encoding with C code to initialize the encoding
of standard streams like sys.stdout. Moreover, the encoding is now
initialized to the Python codec name to get a normalized encoding name and
to ensure that the codec is loaded. The change avoids importing
_bootlocale and _locale modules at startup by default.

When the PYTHONIOENCODING environment variable only contains an encoding,
the error handler is now is now set explicitly to "strict".

Rename also get_default_standard_stream_error_handler() to
get_stdio_errors().

Reduce the buffer to format the "cpXXX" string (Windows locale encoding).
2018-08-28 23:26:33 +02:00
Siddhesh Poyarekar
55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Victor Stinner
cb064fc232
bpo-31900: Fix localeconv() encoding for LC_NUMERIC (#4174)
* Add _Py_GetLocaleconvNumeric() function: decode decimal_point and
  thousands_sep fields of localeconv() from the LC_NUMERIC encoding,
  rather than decoding from the LC_CTYPE encoding.
* Modify locale.localeconv() and "n" formatter of str.format() (for
  int, float and complex to use _Py_GetLocaleconvNumeric()
  internally.
2018-01-15 15:58:02 +01:00
Victor Stinner
7ed7aead95
bpo-29240: Fix locale encodings in UTF-8 Mode (#5170)
Modify locale.localeconv(), time.tzname, os.strerror() and other
functions to ignore the UTF-8 Mode: always use the current locale
encoding.

Changes:

* Add _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx(). On decoding or
  encoding error, they return the position of the error and an error
  message which are used to raise Unicode errors in
  PyUnicode_DecodeLocale() and PyUnicode_EncodeLocale().
* Replace _Py_DecodeCurrentLocale() with _Py_DecodeLocaleEx().
* PyUnicode_DecodeLocale() now uses _Py_DecodeLocaleEx() for all
  cases, especially for the strict error handler.
* Add _Py_DecodeUTF8Ex(): return more information on decoding error
  and supports the strict error handler.
* Rename _Py_EncodeUTF8_surrogateescape() to _Py_EncodeUTF8Ex().
* Replace _Py_EncodeCurrentLocale() with _Py_EncodeLocaleEx().
* Ignore the UTF-8 mode to encode/decode localeconv(), strerror()
  and time zone name.
* Remove PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize()
  and PyUnicode_EncodeLocale() now ignore the UTF-8 mode: always use
  the "current" locale.
* Remove _PyUnicode_DecodeCurrentLocale(),
  _PyUnicode_DecodeCurrentLocaleAndSize() and
  _PyUnicode_EncodeCurrentLocale().
2018-01-15 10:45:49 +01:00
Serhiy Storchaka
f7eae0adfc [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302)
Based on patch by Victor Stinner.

Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters.
2017-06-28 08:30:06 +03:00
Benjamin Peterson
ad4a0cc519 allow the first call to wcsxfrm to return ERANGE (#536)
If the output buffer provided to wcsxfrm is too small, errno is set to ERANGE. We should not error out in that case.
2017-03-07 22:24:44 -08:00
Serhiy Storchaka
be487a65f1 bpo-15954: Check return code of wcsxfrm(). (#508) 2017-03-06 21:21:41 +02:00
Martin Panter
d508d00919 Issue #28139: Merge indentation fixes from 3.5 into 3.6 2016-09-17 07:59:14 +00:00
Martin Panter
6d57fe1c23 Issue #28139: Fix messed up indentation
Also update the classmethod and staticmethod doc strings and comments to
match the RST documentation.
2016-09-17 03:26:16 +00:00
Christian Heimes
ff4fddde57 Add NULL checks to the initializer of the locale module
The _locale module was using old-style APIs to set numeric module
constants from macros. The new way requires less code and properly
checks for NULL.

CID 1295027
2016-09-09 00:24:12 +02:00
Serhiy Storchaka
ef1585eb9a Issue #25923: Added more const qualifiers to signatures of static and private functions. 2015-12-25 20:01:53 +02:00
Serhiy Storchaka
1a1ff29659 Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer
overflows.  Added few missed PyErr_NoMemory().
2015-02-16 13:28:22 +02:00
Victor Stinner
f38a5c28e0 Cleanup locale.localeconv(): move Py_DECREF() closer to the error 2013-10-29 19:28:20 +01:00
Victor Stinner
d594f24fbc Issue #18408: Fix locale.localeconv(), handle PyDict_SetItemString() failure 2013-07-17 00:55:57 +02:00
Kristjan Valur Jonsson
85634d7a2e Issue #14909: A number of places were using PyMem_Realloc() apis and
PyObject_GC_Resize() with incorrect error handling.  In case of errors,
the original object would be leaked.  This checkin fixes those cases.
2012-05-31 09:37:31 +00:00
Victor Stinner
a9c895d497 PyUnicode_DecodeLocale() second argument is now a char*, no more an int 2012-02-14 02:33:38 +01:00
Victor Stinner
af02e1c85a Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
* PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale() decode a string
   from the current locale encoding
 * _Py_char2wchar() writes an "error code" in the size argument to indicate
   if the function failed because of memory allocation failure or because of a
   decoding error. The function doesn't write the error message directly to
   stderr.
 * Fix time.strftime() (if wcsftime() is missing): decode strftime() result
   from the current locale encoding, not from the filesystem encoding.
2011-12-16 23:56:01 +01:00
Victor Stinner
ce0e7daf41 Remove temporary hacks for the issue #13441 2011-11-21 16:01:27 +01:00
Victor Stinner
a697b37300 Another temporary hack to debug the issue #13441
Dump the wchar_t that we are going to decode and dump the locale
2011-11-21 15:41:17 +01:00
Victor Stinner
da29cc36aa Issue #13441: _PyUnicode_CheckConsistency() dumps the string if the maximum
character is bigger than U+10FFFF and locale.localeconv() dumps the string
before decoding it.

Temporary hack to debug the issue #13441.
2011-11-21 14:31:41 +01:00
Victor Stinner
2c5d3cbfb8 Fix a compiler warning in _locale 2011-10-11 22:35:52 +02:00
Victor Stinner
6364927cec PyLocale_strxfrm() uses the new Unicode API 2011-09-29 23:32:06 +02:00
Victor Stinner
beb4135b8c PyUnicode_AsWideCharString() takes a PyObject*, not a PyUnicodeObject*
All unicode functions uses PyObject* except PyUnicode_AsWideChar(). Fix the
prototype for the new function PyUnicode_AsWideCharString().
2010-10-07 01:02:42 +00:00
Victor Stinner
449057f2fa Issue #9979: Use PyUnicode_AsWideCharString() for _locale.strcoll()
It simplifies the code and prepare the surrogates support.
2010-09-29 10:30:43 +00:00
Daniel Stutzbach
8515eaefda Issue 8781: On systems a signed 4-byte wchar_t and a 4-byte Py_UNICODE, use memcpy to convert between the two (as already done when wchar_t is unsigned) 2010-08-24 21:57:33 +00:00
Victor Stinner
9e19ca42d3 locale.bindtextdomain(): use PyUnicode_FSConverter() to parse the filename 2010-06-11 22:09:51 +00:00
Antoine Pitrou
f95a1b3c53 Recorded merge of revisions 81029 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

  Untabify C files. Will watch buildbots.
........
2010-05-09 15:52:27 +00:00
Amaury Forgeot d'Arc
64f3ca4206 Merged revisions 76625 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r76625 | amaury.forgeotdarc | 2009-12-01 22:51:04 +0100 (mar., 01 déc. 2009) | 3 lines

  #7419: Fix a crash on Windows in locale.setlocale() when the category
  is outside the allowed range.
........
2009-12-01 21:59:18 +00:00
Georg Brandl
9137391d59 Only declare variable when it's used. 2009-10-27 21:37:48 +00:00
Antoine Pitrou
6a448d4c2e Issue #7080: locale.strxfrm() raises a MemoryError on 64-bit non-Windows
platforms, and assorted locale fixes by Derk Drukker.
2009-10-19 19:43:09 +00:00
Mark Dickinson
3360401980 Merged revisions 74843 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r74843 | mark.dickinson | 2009-09-16 21:26:31 +0100 (Wed, 16 Sep 2009) | 4 lines

  Remove outdated include; this include was breaking OS X builds using
  non-Apple gcc4.3 and gcc4.4 (because CoreFoundation/CoreFoundation.h
  won't compile under non-Apple gcc).
........
2009-09-16 20:32:35 +00:00
Ronald Oussoren
fe8a3d6eeb Fix for issue 6202 2009-06-07 15:29:46 +00:00
Martin v. Löwis
db1c399fe0 Issue #6093: Fix off-by-one error in locale.strxfrm. 2009-05-23 10:38:26 +00:00
Mark Dickinson
211c625829 Issue #1717, stage 2: remove uses of tp_compare in Modules and most
Objects.
2009-02-01 10:28:51 +00:00
Antoine Pitrou
fff953048f Issue #3696: Error parsing arguments on OpenBSD <= 4.4 and Cygwin.
Patch by Amaury Forgeot d'Arc, reviewed by me.
2008-09-03 18:58:51 +00:00