mirrors/cpython
0
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2024-11-24 02:15:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

bpo-33710: Deprecate l*gettext() and related functions in the gettext module. (GH-10139)

Browse Source 
They return encoded bytes and are Python 2 artifacts.
This commit is contained in:
Serhiy Storchaka 2018-10-27 08:00:41 +03:00 committed by GitHub
parent d9bff4e81b
commit fec35c99aa
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 233 additions and 67 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
Doc/library/gettext.rst
View File

@ -53,6 +53,8 @@ class-based API instead.
and :func:`ldngettext` functions.
If *codeset* is omitted, then the current binding is returned.
.. deprecated-removed:: 3.8 3.10
.. function:: textdomain(domain=None)
@ -112,9 +114,9 @@ class-based API instead.
Unicode strings instead, since most Python applications will want to
manipulate human readable text as strings instead of bytes. Further,
it's possible that you may get unexpected Unicode-related exceptions
if there are encoding problems with the translated strings. It is
possible that the ``l*()`` functions will be deprecated in future Python
versions due to their inherent problems and limitations.
if there are encoding problems with the translated strings.
.. deprecated-removed:: 3.8 3.10
Note that GNU :program:`gettext` also defines a :func:`dcgettext` method, but
@ -192,6 +194,9 @@ class can also install themselves in the built-in namespace as the function
.. versionchanged:: 3.3
:exc:`IOError` used to be raised instead of :exc:`OSError`.
.. deprecated-removed:: 3.8 3.10
The *codeset* parameter.
.. function:: install(domain, localedir=None, codeset=None, names=None)
@ -212,6 +217,9 @@ class can also install themselves in the built-in namespace as the function
builtins namespace, so it is easily accessible in all modules of your
application.
.. deprecated-removed:: 3.8 3.10
The *codeset* parameter.
The :class:`NullTranslations` class
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -272,6 +280,8 @@ are the methods of :class:`!NullTranslations`:
These methods should be avoided in Python 3. See the warning for the
:func:`lgettext` function.
.. deprecated-removed:: 3.8 3.10
.. method:: info()
@ -288,11 +298,15 @@ are the methods of :class:`!NullTranslations`:
Return the encoding used to return translated messages in :meth:`.lgettext`
and :meth:`.lngettext`.
.. deprecated-removed:: 3.8 3.10
.. method:: set_output_charset(charset)
Change the encoding used to return translated messages.
.. deprecated-removed:: 3.8 3.10
.. method:: install(names=None)
@ -393,6 +407,8 @@ unexpected, or if other problems occur while reading the file, instantiating a
These methods should be avoided in Python 3. See the warning for the
:func:`lgettext` function.
.. deprecated-removed:: 3.8 3.10
Solaris message catalog support
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

17
Doc/whatsnew/3.8.rst
View File

@ -295,6 +295,23 @@ Deprecated
versions. :class:`~ast.Constant` should be used instead.
(Contributed by Serhiy Storchaka in :issue:`32892`.)
* The following functions and methods are deprecated in the :mod:`gettext`
module: :func:`~gettext.lgettext`, :func:`~gettext.ldgettext`,
:func:`~gettext.lngettext` and :func:`~gettext.ldngettext`.
They return encoded bytes, and it's possible that you will get unexpected
Unicode-related exceptions if there are encoding problems with the
translated strings. It's much better to use alternatives which return
Unicode strings in Python 3. These functions have been broken for a long time.
Function :func:`~gettext.bind_textdomain_codeset`, methods
:meth:`~gettext.NullTranslations.output_charset` and
:meth:`~gettext.NullTranslations.set_output_charset`, and the *codeset*
parameter of functions :func:`~gettext.translation` and
:func:`~gettext.install` are also deprecated, since they are only used for
for the ``l*gettext()`` functions.
(Contributed by Serhiy Storchaka in :issue:`33710`.)
Removed
=======

95
Lib/gettext.py
View File

@ -274,8 +274,14 @@ class NullTranslations:
return message
def lgettext(self, message):
import warnings
warnings.warn('lgettext() is deprecated, use gettext() instead',
DeprecationWarning, 2)
if self._fallback:
return self._fallback.lgettext(message)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\blgettext\b.*',
DeprecationWarning)
return self._fallback.lgettext(message)
if self._output_charset:
return message.encode(self._output_charset)
return message.encode(locale.getpreferredencoding())
@ -289,8 +295,14 @@ class NullTranslations:
return msgid2
def lngettext(self, msgid1, msgid2, n):
import warnings
warnings.warn('lngettext() is deprecated, use ngettext() instead',
DeprecationWarning, 2)
if self._fallback:
return self._fallback.lngettext(msgid1, msgid2, n)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\blngettext\b.*',
DeprecationWarning)
return self._fallback.lngettext(msgid1, msgid2, n)
if n == 1:
tmsg = msgid1
else:
@ -306,9 +318,15 @@ class NullTranslations:
return self._charset
def output_charset(self):
import warnings
warnings.warn('output_charset() is deprecated',
DeprecationWarning, 2)
return self._output_charset
def set_output_charset(self, charset):
import warnings
warnings.warn('set_output_charset() is deprecated',
DeprecationWarning, 2)
self._output_charset = charset
def install(self, names=None):
@ -424,6 +442,9 @@ class GNUTranslations(NullTranslations):
transidx += 8
def lgettext(self, message):
import warnings
warnings.warn('lgettext() is deprecated, use gettext() instead',
DeprecationWarning, 2)
missing = object()
tmsg = self._catalog.get(message, missing)
if tmsg is missing:
@ -435,6 +456,9 @@ class GNUTranslations(NullTranslations):
return tmsg.encode(locale.getpreferredencoding())
def lngettext(self, msgid1, msgid2, n):
import warnings
warnings.warn('lngettext() is deprecated, use ngettext() instead',
DeprecationWarning, 2)
try:
tmsg = self._catalog[(msgid1, self.plural(n))]
except KeyError:
@ -510,9 +534,10 @@ def find(domain, localedir=None, languages=None, all=False):
# a mapping between absolute .mo file path and Translation object
_translations = {}
_unspecified = ['unspecified']
def translation(domain, localedir=None, languages=None,
class_=None, fallback=False, codeset=None):
class_=None, fallback=False, codeset=_unspecified):
if class_ is None:
class_ = GNUTranslations
mofiles = find(domain, localedir, languages, all=True)
@ -538,8 +563,15 @@ def translation(domain, localedir=None, languages=None,
# are not used.
import copy
t = copy.copy(t)
if codeset:
t.set_output_charset(codeset)
if codeset is not _unspecified:
import warnings
warnings.warn('parameter codeset is deprecated',
DeprecationWarning, 2)
if codeset:
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\bset_output_charset\b.*',
DeprecationWarning)
t.set_output_charset(codeset)
if result is None:
result = t
else:
@ -547,7 +579,7 @@ def translation(domain, localedir=None, languages=None,
return result
def install(domain, localedir=None, codeset=None, names=None):
def install(domain, localedir=None, codeset=_unspecified, names=None):
t = translation(domain, localedir, fallback=True, codeset=codeset)
t.install(names)
@ -576,6 +608,9 @@ def bindtextdomain(domain, localedir=None):
def bind_textdomain_codeset(domain, codeset=None):
import warnings
warnings.warn('bind_textdomain_codeset() is deprecated',
DeprecationWarning, 2)
global _localecodesets
if codeset is not None:
_localecodesets[domain] = codeset
@ -584,24 +619,31 @@ def bind_textdomain_codeset(domain, codeset=None):
def dgettext(domain, message):
try:
t = translation(domain, _localedirs.get(domain, None),
codeset=_localecodesets.get(domain))
t = translation(domain, _localedirs.get(domain, None))
except OSError:
return message
return t.gettext(message)
def ldgettext(domain, message):
import warnings
warnings.warn('ldgettext() is deprecated, use dgettext() instead',
DeprecationWarning, 2)
codeset = _localecodesets.get(domain)
try:
t = translation(domain, _localedirs.get(domain, None), codeset=codeset)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\bparameter codeset\b.*',
DeprecationWarning)
t = translation(domain, _localedirs.get(domain, None), codeset=codeset)
except OSError:
return message.encode(codeset or locale.getpreferredencoding())
return t.lgettext(message)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\blgettext\b.*',
DeprecationWarning)
return t.lgettext(message)
def dngettext(domain, msgid1, msgid2, n):
try:
t = translation(domain, _localedirs.get(domain, None),
codeset=_localecodesets.get(domain))
t = translation(domain, _localedirs.get(domain, None))
except OSError:
if n == 1:
return msgid1
@ -610,28 +652,49 @@ def dngettext(domain, msgid1, msgid2, n):
return t.ngettext(msgid1, msgid2, n)
def ldngettext(domain, msgid1, msgid2, n):
import warnings
warnings.warn('ldngettext() is deprecated, use dngettext() instead',
DeprecationWarning, 2)
codeset = _localecodesets.get(domain)
try:
t = translation(domain, _localedirs.get(domain, None), codeset=codeset)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\bparameter codeset\b.*',
DeprecationWarning)
t = translation(domain, _localedirs.get(domain, None), codeset=codeset)
except OSError:
if n == 1:
tmsg = msgid1
else:
tmsg = msgid2
return tmsg.encode(codeset or locale.getpreferredencoding())
return t.lngettext(msgid1, msgid2, n)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\blngettext\b.*',
DeprecationWarning)
return t.lngettext(msgid1, msgid2, n)
def gettext(message):
return dgettext(_current_domain, message)
def lgettext(message):
return ldgettext(_current_domain, message)
import warnings
warnings.warn('lgettext() is deprecated, use gettext() instead',
DeprecationWarning, 2)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\bldgettext\b.*',
DeprecationWarning)
return ldgettext(_current_domain, message)
def ngettext(msgid1, msgid2, n):
return dngettext(_current_domain, msgid1, msgid2, n)
def lngettext(msgid1, msgid2, n):
return ldngettext(_current_domain, msgid1, msgid2, n)
import warnings
warnings.warn('lngettext() is deprecated, use ngettext() instead',
DeprecationWarning, 2)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', r'.*\bldngettext\b.*',
DeprecationWarning)
return ldngettext(_current_domain, msgid1, msgid2, n)
# dcgettext() has been deemed unnecessary and is not implemented.

162
Lib/test/test_gettext.py
View File

@ -1,5 +1,6 @@
import os
import base64
import contextlib
import gettext
import locale
import unittest
@ -461,116 +462,174 @@ class LGettextTestCase(GettextBaseTest):
GettextBaseTest.setUp(self)
self.mofile = MOFILE
@contextlib.contextmanager
def assertDeprecated(self, name):
with self.assertWarnsRegex(DeprecationWarning,
fr'^{name}\(\) is deprecated'):
yield
def test_lgettext(self):
lgettext = gettext.lgettext
ldgettext = gettext.ldgettext
self.assertEqual(lgettext('mullusk'), b'bacon')
self.assertEqual(lgettext('spam'), b'spam')
self.assertEqual(ldgettext('gettext', 'mullusk'), b'bacon')
self.assertEqual(ldgettext('gettext', 'spam'), b'spam')
with self.assertDeprecated('lgettext'):
self.assertEqual(lgettext('mullusk'), b'bacon')
with self.assertDeprecated('lgettext'):
self.assertEqual(lgettext('spam'), b'spam')
with self.assertDeprecated('ldgettext'):
self.assertEqual(ldgettext('gettext', 'mullusk'), b'bacon')
with self.assertDeprecated('ldgettext'):
self.assertEqual(ldgettext('gettext', 'spam'), b'spam')
def test_lgettext_2(self):
with open(self.mofile, 'rb') as fp:
t = gettext.GNUTranslations(fp)
lgettext = t.lgettext
self.assertEqual(lgettext('mullusk'), b'bacon')
self.assertEqual(lgettext('spam'), b'spam')
with self.assertDeprecated('lgettext'):
self.assertEqual(lgettext('mullusk'), b'bacon')
with self.assertDeprecated('lgettext'):
self.assertEqual(lgettext('spam'), b'spam')
def test_lgettext_bind_textdomain_codeset(self):
lgettext = gettext.lgettext
ldgettext = gettext.ldgettext
saved_codeset = gettext.bind_textdomain_codeset('gettext')
with self.assertDeprecated('bind_textdomain_codeset'):
saved_codeset = gettext.bind_textdomain_codeset('gettext')
try:
gettext.bind_textdomain_codeset('gettext', 'utf-16')
self.assertEqual(lgettext('mullusk'), 'bacon'.encode('utf-16'))
self.assertEqual(lgettext('spam'), 'spam'.encode('utf-16'))
self.assertEqual(ldgettext('gettext', 'mullusk'), 'bacon'.encode('utf-16'))
self.assertEqual(ldgettext('gettext', 'spam'), 'spam'.encode('utf-16'))
with self.assertDeprecated('bind_textdomain_codeset'):
gettext.bind_textdomain_codeset('gettext', 'utf-16')
with self.assertDeprecated('lgettext'):
self.assertEqual(lgettext('mullusk'), 'bacon'.encode('utf-16'))
with self.assertDeprecated('lgettext'):
self.assertEqual(lgettext('spam'), 'spam'.encode('utf-16'))
with self.assertDeprecated('ldgettext'):
self.assertEqual(ldgettext('gettext', 'mullusk'), 'bacon'.encode('utf-16'))
with self.assertDeprecated('ldgettext'):
self.assertEqual(ldgettext('gettext', 'spam'), 'spam'.encode('utf-16'))
finally:
del gettext._localecodesets['gettext']
gettext.bind_textdomain_codeset('gettext', saved_codeset)
with self.assertDeprecated('bind_textdomain_codeset'):
gettext.bind_textdomain_codeset('gettext', saved_codeset)
def test_lgettext_output_encoding(self):
with open(self.mofile, 'rb') as fp:
t = gettext.GNUTranslations(fp)
lgettext = t.lgettext
t.set_output_charset('utf-16')
self.assertEqual(lgettext('mullusk'), 'bacon'.encode('utf-16'))
self.assertEqual(lgettext('spam'), 'spam'.encode('utf-16'))
with self.assertDeprecated('set_output_charset'):
t.set_output_charset('utf-16')
with self.assertDeprecated('lgettext'):
self.assertEqual(lgettext('mullusk'), 'bacon'.encode('utf-16'))
with self.assertDeprecated('lgettext'):
self.assertEqual(lgettext('spam'), 'spam'.encode('utf-16'))
def test_lngettext(self):
lngettext = gettext.lngettext
ldngettext = gettext.ldngettext
x = lngettext('There is %s file', 'There are %s files', 1)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s file', 'There are %s files', 1)
self.assertEqual(x, b'Hay %s fichero')
x = lngettext('There is %s file', 'There are %s files', 2)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s file', 'There are %s files', 2)
self.assertEqual(x, b'Hay %s ficheros')
x = lngettext('There is %s directory', 'There are %s directories', 1)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s directory', 'There are %s directories', 1)
self.assertEqual(x, b'There is %s directory')
x = lngettext('There is %s directory', 'There are %s directories', 2)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s directory', 'There are %s directories', 2)
self.assertEqual(x, b'There are %s directories')
x = ldngettext('gettext', 'There is %s file', 'There are %s files', 1)
with self.assertDeprecated('ldngettext'):
x = ldngettext('gettext', 'There is %s file', 'There are %s files', 1)
self.assertEqual(x, b'Hay %s fichero')
x = ldngettext('gettext', 'There is %s file', 'There are %s files', 2)
with self.assertDeprecated('ldngettext'):
x = ldngettext('gettext', 'There is %s file', 'There are %s files', 2)
self.assertEqual(x, b'Hay %s ficheros')
x = ldngettext('gettext', 'There is %s directory', 'There are %s directories', 1)
with self.assertDeprecated('ldngettext'):
x = ldngettext('gettext', 'There is %s directory', 'There are %s directories', 1)
self.assertEqual(x, b'There is %s directory')
x = ldngettext('gettext', 'There is %s directory', 'There are %s directories', 2)
with self.assertDeprecated('ldngettext'):
x = ldngettext('gettext', 'There is %s directory', 'There are %s directories', 2)
self.assertEqual(x, b'There are %s directories')
def test_lngettext_2(self):
with open(self.mofile, 'rb') as fp:
t = gettext.GNUTranslations(fp)
lngettext = t.lngettext
x = lngettext('There is %s file', 'There are %s files', 1)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s file', 'There are %s files', 1)
self.assertEqual(x, b'Hay %s fichero')
x = lngettext('There is %s file', 'There are %s files', 2)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s file', 'There are %s files', 2)
self.assertEqual(x, b'Hay %s ficheros')
x = lngettext('There is %s directory', 'There are %s directories', 1)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s directory', 'There are %s directories', 1)
self.assertEqual(x, b'There is %s directory')
x = lngettext('There is %s directory', 'There are %s directories', 2)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s directory', 'There are %s directories', 2)
self.assertEqual(x, b'There are %s directories')
def test_lngettext_bind_textdomain_codeset(self):
lngettext = gettext.lngettext
ldngettext = gettext.ldngettext
saved_codeset = gettext.bind_textdomain_codeset('gettext')
with self.assertDeprecated('bind_textdomain_codeset'):
saved_codeset = gettext.bind_textdomain_codeset('gettext')
try:
gettext.bind_textdomain_codeset('gettext', 'utf-16')
x = lngettext('There is %s file', 'There are %s files', 1)
with self.assertDeprecated('bind_textdomain_codeset'):
gettext.bind_textdomain_codeset('gettext', 'utf-16')
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s file', 'There are %s files', 1)
self.assertEqual(x, 'Hay %s fichero'.encode('utf-16'))
x = lngettext('There is %s file', 'There are %s files', 2)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s file', 'There are %s files', 2)
self.assertEqual(x, 'Hay %s ficheros'.encode('utf-16'))
x = lngettext('There is %s directory', 'There are %s directories', 1)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s directory', 'There are %s directories', 1)
self.assertEqual(x, 'There is %s directory'.encode('utf-16'))
x = lngettext('There is %s directory', 'There are %s directories', 2)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s directory', 'There are %s directories', 2)
self.assertEqual(x, 'There are %s directories'.encode('utf-16'))
x = ldngettext('gettext', 'There is %s file', 'There are %s files', 1)
with self.assertDeprecated('ldngettext'):
x = ldngettext('gettext', 'There is %s file', 'There are %s files', 1)
self.assertEqual(x, 'Hay %s fichero'.encode('utf-16'))
x = ldngettext('gettext', 'There is %s file', 'There are %s files', 2)
with self.assertDeprecated('ldngettext'):
x = ldngettext('gettext', 'There is %s file', 'There are %s files', 2)
self.assertEqual(x, 'Hay %s ficheros'.encode('utf-16'))
x = ldngettext('gettext', 'There is %s directory', 'There are %s directories', 1)
with self.assertDeprecated('ldngettext'):
x = ldngettext('gettext', 'There is %s directory', 'There are %s directories', 1)
self.assertEqual(x, 'There is %s directory'.encode('utf-16'))
x = ldngettext('gettext', 'There is %s directory', 'There are %s directories', 2)
with self.assertDeprecated('ldngettext'):
x = ldngettext('gettext', 'There is %s directory', 'There are %s directories', 2)
self.assertEqual(x, 'There are %s directories'.encode('utf-16'))
finally:
del gettext._localecodesets['gettext']
gettext.bind_textdomain_codeset('gettext', saved_codeset)
with self.assertDeprecated('bind_textdomain_codeset'):
gettext.bind_textdomain_codeset('gettext', saved_codeset)
def test_lngettext_output_encoding(self):
with open(self.mofile, 'rb') as fp:
t = gettext.GNUTranslations(fp)
lngettext = t.lngettext
t.set_output_charset('utf-16')
x = lngettext('There is %s file', 'There are %s files', 1)
with self.assertDeprecated('set_output_charset'):
t.set_output_charset('utf-16')
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s file', 'There are %s files', 1)
self.assertEqual(x, 'Hay %s fichero'.encode('utf-16'))
x = lngettext('There is %s file', 'There are %s files', 2)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s file', 'There are %s files', 2)
self.assertEqual(x, 'Hay %s ficheros'.encode('utf-16'))
x = lngettext('There is %s directory', 'There are %s directories', 1)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s directory', 'There are %s directories', 1)
self.assertEqual(x, 'There is %s directory'.encode('utf-16'))
x = lngettext('There is %s directory', 'There are %s directories', 2)
with self.assertDeprecated('lngettext'):
x = lngettext('There is %s directory', 'There are %s directories', 2)
self.assertEqual(x, 'There are %s directories'.encode('utf-16'))
def test_output_encoding(self):
with open(self.mofile, 'rb') as fp:
t = gettext.GNUTranslations(fp)
with self.assertDeprecated('set_output_charset'):
t.set_output_charset('utf-16')
with self.assertDeprecated('output_charset'):
self.assertEqual(t.output_charset(), 'utf-16')
class GNUTranslationParsingTest(GettextBaseTest):
def test_plural_form_error_issue17898(self):
@ -642,6 +701,16 @@ class GettextCacheTestCase(GettextBaseTest):
self.assertEqual(len(gettext._translations), 2)
self.assertEqual(t.__class__, DummyGNUTranslations)
# Test deprecated parameter codeset
with self.assertWarnsRegex(DeprecationWarning, 'parameter codeset'):
t = gettext.translation('gettext', self.localedir,
class_=DummyGNUTranslations,
codeset='utf-16')
self.assertEqual(len(gettext._translations), 2)
self.assertEqual(t.__class__, DummyGNUTranslations)
with self.assertWarns(DeprecationWarning):
self.assertEqual(t.output_charset(), 'utf-16')
class MiscTestCase(unittest.TestCase):
def test__all__(self):
@ -649,11 +718,8 @@ class MiscTestCase(unittest.TestCase):
support.check__all__(self, gettext, blacklist=blacklist)
def test_main():
support.run_unittest(__name__)
if __name__ == '__main__':
test_main()
unittest.main()
# For reference, here's the .po file used to created the GNU_MO_DATA above.

4
Misc/NEWS.d/next/Library/2018-10-26-21-12-55.bpo-33710.Q5oXc6.rst Normal file
View File

@ -0,0 +1,4 @@
Deprecated ``l*gettext()`` functions and methods in the :mod:`gettext`
module. They return encoded bytes instead of Unicode strings and are
artifacts from Python 2 times. Also deprecated functions and methods related
to setting the charset for ``l*gettext()`` functions and methods.