2008-03-19 13:04:44 +08:00
|
|
|
# Copyright 2006 Google, Inc. All Rights Reserved.
|
|
|
|
# Licensed to PSF under a Contributor Agreement.
|
|
|
|
|
|
|
|
"""Export the Python grammar and symbols."""
|
|
|
|
|
|
|
|
# Python imports
|
|
|
|
import os
|
|
|
|
|
|
|
|
# Local imports
|
|
|
|
from .pgen2 import token
|
|
|
|
from .pgen2 import driver
|
|
|
|
from . import pytree
|
|
|
|
|
|
|
|
# The grammar file
|
|
|
|
_GRAMMAR_FILE = os.path.join(os.path.dirname(__file__), "Grammar.txt")
|
Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r83852 | benjamin.peterson | 2010-08-08 15:45:44 -0500 (Sun, 08 Aug 2010) | 1 line
wrap with parens
........
r83853 | benjamin.peterson | 2010-08-08 15:46:31 -0500 (Sun, 08 Aug 2010) | 1 line
use parens
........
r83857 | benjamin.peterson | 2010-08-08 15:59:49 -0500 (Sun, 08 Aug 2010) | 1 line
things which use touch_import should be pre order
........
r84042 | george.boutsioukis | 2010-08-14 16:10:19 -0500 (Sat, 14 Aug 2010) | 2 lines
This revision incorporates into the 2to3 tool the new, faster, tree matching algorithm developed during a GSOC project. The algorithm resides in the two added modules, btm_matcher and btm_utils. New code has been added to drive the new matching process in refactor.py and a few minor changes were made in other modules. A BM_compatible flag(False by default) has been added in fixer_base and it is set to True in most of the current fixers.
........
r84216 | benjamin.peterson | 2010-08-19 16:44:05 -0500 (Thu, 19 Aug 2010) | 1 line
allow star_expr in testlist_gexp
........
r84274 | benjamin.peterson | 2010-08-22 18:40:46 -0500 (Sun, 22 Aug 2010) | 1 line
wrap long line
........
r84275 | benjamin.peterson | 2010-08-22 18:42:22 -0500 (Sun, 22 Aug 2010) | 1 line
cleanup
........
r84276 | benjamin.peterson | 2010-08-22 18:51:01 -0500 (Sun, 22 Aug 2010) | 1 line
when there's a None value and a traceback, don't call type with it #9661
........
r84375 | george.boutsioukis | 2010-08-31 08:38:53 -0500 (Tue, 31 Aug 2010) | 3 lines
Idiomatic code changes & stylistic issues fixed in the BottomMatcher module. Thanks to Benjamin Peterson for taking the time to review the code.
........
r85388 | benjamin.peterson | 2010-10-12 17:27:44 -0500 (Tue, 12 Oct 2010) | 1 line
fix urllib fixer with multiple as imports on a line #10069
........
r85478 | benjamin.peterson | 2010-10-14 08:09:56 -0500 (Thu, 14 Oct 2010) | 1 line
stop abusing docstrings
........
r85506 | benjamin.peterson | 2010-10-14 17:45:19 -0500 (Thu, 14 Oct 2010) | 1 line
kill sibling import
........
r85507 | benjamin.peterson | 2010-10-14 17:54:15 -0500 (Thu, 14 Oct 2010) | 1 line
remove trailing whitespace
........
r85508 | benjamin.peterson | 2010-10-14 17:55:28 -0500 (Thu, 14 Oct 2010) | 1 line
typo
........
2010-10-15 07:00:04 +08:00
|
|
|
_PATTERN_GRAMMAR_FILE = os.path.join(os.path.dirname(__file__),
|
|
|
|
"PatternGrammar.txt")
|
2008-03-19 13:04:44 +08:00
|
|
|
|
|
|
|
|
|
|
|
class Symbols(object):
|
|
|
|
|
|
|
|
def __init__(self, grammar):
|
|
|
|
"""Initializer.
|
|
|
|
|
|
|
|
Creates an attribute for each grammar symbol (nonterminal),
|
|
|
|
whose value is the symbol's type (an int >= 256).
|
|
|
|
"""
|
2008-03-19 13:33:36 +08:00
|
|
|
for name, symbol in grammar.symbol2number.items():
|
2008-03-19 13:04:44 +08:00
|
|
|
setattr(self, name, symbol)
|
|
|
|
|
|
|
|
|
|
|
|
python_grammar = driver.load_grammar(_GRAMMAR_FILE)
|
Merged revisions 74114 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
................
r74114 | benjamin.peterson | 2009-07-20 10:33:09 -0500 (Mon, 20 Jul 2009) | 110 lines
Merged revisions 73771,73811,73840,73842,73848-73849,73861,73957-73960,73964-73969,73972-73974,73977,73981,73984,74065,74113 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r73771 | benjamin.peterson | 2009-07-02 10:56:55 -0500 (Thu, 02 Jul 2009) | 1 line
force the imports fixer to be run after the import one #6400
........
r73811 | benjamin.peterson | 2009-07-03 09:03:14 -0500 (Fri, 03 Jul 2009) | 1 line
check for sep, not pathsep when looking for a subpackage #6408
........
r73840 | benjamin.peterson | 2009-07-04 09:52:28 -0500 (Sat, 04 Jul 2009) | 1 line
don't print diffs by default; it's annoying
........
r73842 | benjamin.peterson | 2009-07-04 09:58:46 -0500 (Sat, 04 Jul 2009) | 1 line
complain when not showing diffs or writing
........
r73848 | alexandre.vassalotti | 2009-07-04 23:38:19 -0500 (Sat, 04 Jul 2009) | 2 lines
Fix test_refactor_stdin to handle print_output() method with 4 arguments.
........
r73849 | alexandre.vassalotti | 2009-07-04 23:43:18 -0500 (Sat, 04 Jul 2009) | 5 lines
Issue 2370: Add fixer for the removal of operator.isCallable() and
operator.sequenceIncludes().
Patch contributed by Jeff Balogh (and updated by me).
........
r73861 | benjamin.peterson | 2009-07-05 09:15:53 -0500 (Sun, 05 Jul 2009) | 1 line
cleanup and use unicode where appropiate
........
r73957 | benjamin.peterson | 2009-07-11 15:49:56 -0500 (Sat, 11 Jul 2009) | 1 line
fix calls to str() with unicode()
........
r73958 | benjamin.peterson | 2009-07-11 15:51:51 -0500 (Sat, 11 Jul 2009) | 1 line
more str() -> unicode()
........
r73959 | benjamin.peterson | 2009-07-11 16:40:08 -0500 (Sat, 11 Jul 2009) | 1 line
add tests for refactor_dir()
........
r73960 | benjamin.peterson | 2009-07-11 16:44:32 -0500 (Sat, 11 Jul 2009) | 1 line
don't parse files just because they end with 'py' (no dot)
........
r73964 | benjamin.peterson | 2009-07-11 17:30:15 -0500 (Sat, 11 Jul 2009) | 1 line
simplify
........
r73965 | benjamin.peterson | 2009-07-11 17:31:30 -0500 (Sat, 11 Jul 2009) | 1 line
remove usage of get_prefix()
........
r73966 | benjamin.peterson | 2009-07-11 17:33:35 -0500 (Sat, 11 Jul 2009) | 1 line
revert unintended change in 73965
........
r73967 | benjamin.peterson | 2009-07-11 17:34:44 -0500 (Sat, 11 Jul 2009) | 1 line
avoid expensive checks and assume the node did change
........
r73968 | benjamin.peterson | 2009-07-11 20:46:46 -0500 (Sat, 11 Jul 2009) | 1 line
use a regular dict for the heads to avoid adding lists in the loop
........
r73969 | benjamin.peterson | 2009-07-11 20:50:43 -0500 (Sat, 11 Jul 2009) | 1 line
prefix headnode functions with '_'
........
r73972 | benjamin.peterson | 2009-07-11 21:25:45 -0500 (Sat, 11 Jul 2009) | 1 line
try to make the head node dict as sparse as possible
........
r73973 | benjamin.peterson | 2009-07-11 21:59:49 -0500 (Sat, 11 Jul 2009) | 1 line
a better idea; add an option to *not* print diffs
........
r73974 | benjamin.peterson | 2009-07-11 22:00:29 -0500 (Sat, 11 Jul 2009) | 1 line
add space
........
r73977 | benjamin.peterson | 2009-07-12 10:16:07 -0500 (Sun, 12 Jul 2009) | 1 line
update get_headnode_dict tests for recent changes
........
r73981 | benjamin.peterson | 2009-07-12 12:06:39 -0500 (Sun, 12 Jul 2009) | 4 lines
detect when "from __future__ import print_function" is given
Deprecate the 'print_function' option and the -p flag
........
r73984 | benjamin.peterson | 2009-07-12 16:16:37 -0500 (Sun, 12 Jul 2009) | 1 line
add tests for Call; thanks Joe Amenta
........
r74065 | benjamin.peterson | 2009-07-17 12:52:49 -0500 (Fri, 17 Jul 2009) | 1 line
pathname2url and url2pathname are in urllib.request not urllib.parse #6496
........
r74113 | benjamin.peterson | 2009-07-20 08:56:57 -0500 (Mon, 20 Jul 2009) | 1 line
fix deprecation warnings in tests
........
................
2009-07-21 00:42:03 +08:00
|
|
|
|
2008-03-19 13:04:44 +08:00
|
|
|
python_symbols = Symbols(python_grammar)
|
Merged revisions 74114 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
................
r74114 | benjamin.peterson | 2009-07-20 10:33:09 -0500 (Mon, 20 Jul 2009) | 110 lines
Merged revisions 73771,73811,73840,73842,73848-73849,73861,73957-73960,73964-73969,73972-73974,73977,73981,73984,74065,74113 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r73771 | benjamin.peterson | 2009-07-02 10:56:55 -0500 (Thu, 02 Jul 2009) | 1 line
force the imports fixer to be run after the import one #6400
........
r73811 | benjamin.peterson | 2009-07-03 09:03:14 -0500 (Fri, 03 Jul 2009) | 1 line
check for sep, not pathsep when looking for a subpackage #6408
........
r73840 | benjamin.peterson | 2009-07-04 09:52:28 -0500 (Sat, 04 Jul 2009) | 1 line
don't print diffs by default; it's annoying
........
r73842 | benjamin.peterson | 2009-07-04 09:58:46 -0500 (Sat, 04 Jul 2009) | 1 line
complain when not showing diffs or writing
........
r73848 | alexandre.vassalotti | 2009-07-04 23:38:19 -0500 (Sat, 04 Jul 2009) | 2 lines
Fix test_refactor_stdin to handle print_output() method with 4 arguments.
........
r73849 | alexandre.vassalotti | 2009-07-04 23:43:18 -0500 (Sat, 04 Jul 2009) | 5 lines
Issue 2370: Add fixer for the removal of operator.isCallable() and
operator.sequenceIncludes().
Patch contributed by Jeff Balogh (and updated by me).
........
r73861 | benjamin.peterson | 2009-07-05 09:15:53 -0500 (Sun, 05 Jul 2009) | 1 line
cleanup and use unicode where appropiate
........
r73957 | benjamin.peterson | 2009-07-11 15:49:56 -0500 (Sat, 11 Jul 2009) | 1 line
fix calls to str() with unicode()
........
r73958 | benjamin.peterson | 2009-07-11 15:51:51 -0500 (Sat, 11 Jul 2009) | 1 line
more str() -> unicode()
........
r73959 | benjamin.peterson | 2009-07-11 16:40:08 -0500 (Sat, 11 Jul 2009) | 1 line
add tests for refactor_dir()
........
r73960 | benjamin.peterson | 2009-07-11 16:44:32 -0500 (Sat, 11 Jul 2009) | 1 line
don't parse files just because they end with 'py' (no dot)
........
r73964 | benjamin.peterson | 2009-07-11 17:30:15 -0500 (Sat, 11 Jul 2009) | 1 line
simplify
........
r73965 | benjamin.peterson | 2009-07-11 17:31:30 -0500 (Sat, 11 Jul 2009) | 1 line
remove usage of get_prefix()
........
r73966 | benjamin.peterson | 2009-07-11 17:33:35 -0500 (Sat, 11 Jul 2009) | 1 line
revert unintended change in 73965
........
r73967 | benjamin.peterson | 2009-07-11 17:34:44 -0500 (Sat, 11 Jul 2009) | 1 line
avoid expensive checks and assume the node did change
........
r73968 | benjamin.peterson | 2009-07-11 20:46:46 -0500 (Sat, 11 Jul 2009) | 1 line
use a regular dict for the heads to avoid adding lists in the loop
........
r73969 | benjamin.peterson | 2009-07-11 20:50:43 -0500 (Sat, 11 Jul 2009) | 1 line
prefix headnode functions with '_'
........
r73972 | benjamin.peterson | 2009-07-11 21:25:45 -0500 (Sat, 11 Jul 2009) | 1 line
try to make the head node dict as sparse as possible
........
r73973 | benjamin.peterson | 2009-07-11 21:59:49 -0500 (Sat, 11 Jul 2009) | 1 line
a better idea; add an option to *not* print diffs
........
r73974 | benjamin.peterson | 2009-07-11 22:00:29 -0500 (Sat, 11 Jul 2009) | 1 line
add space
........
r73977 | benjamin.peterson | 2009-07-12 10:16:07 -0500 (Sun, 12 Jul 2009) | 1 line
update get_headnode_dict tests for recent changes
........
r73981 | benjamin.peterson | 2009-07-12 12:06:39 -0500 (Sun, 12 Jul 2009) | 4 lines
detect when "from __future__ import print_function" is given
Deprecate the 'print_function' option and the -p flag
........
r73984 | benjamin.peterson | 2009-07-12 16:16:37 -0500 (Sun, 12 Jul 2009) | 1 line
add tests for Call; thanks Joe Amenta
........
r74065 | benjamin.peterson | 2009-07-17 12:52:49 -0500 (Fri, 17 Jul 2009) | 1 line
pathname2url and url2pathname are in urllib.request not urllib.parse #6496
........
r74113 | benjamin.peterson | 2009-07-20 08:56:57 -0500 (Mon, 20 Jul 2009) | 1 line
fix deprecation warnings in tests
........
................
2009-07-21 00:42:03 +08:00
|
|
|
|
|
|
|
python_grammar_no_print_statement = python_grammar.copy()
|
|
|
|
del python_grammar_no_print_statement.keywords["print"]
|
Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r83852 | benjamin.peterson | 2010-08-08 15:45:44 -0500 (Sun, 08 Aug 2010) | 1 line
wrap with parens
........
r83853 | benjamin.peterson | 2010-08-08 15:46:31 -0500 (Sun, 08 Aug 2010) | 1 line
use parens
........
r83857 | benjamin.peterson | 2010-08-08 15:59:49 -0500 (Sun, 08 Aug 2010) | 1 line
things which use touch_import should be pre order
........
r84042 | george.boutsioukis | 2010-08-14 16:10:19 -0500 (Sat, 14 Aug 2010) | 2 lines
This revision incorporates into the 2to3 tool the new, faster, tree matching algorithm developed during a GSOC project. The algorithm resides in the two added modules, btm_matcher and btm_utils. New code has been added to drive the new matching process in refactor.py and a few minor changes were made in other modules. A BM_compatible flag(False by default) has been added in fixer_base and it is set to True in most of the current fixers.
........
r84216 | benjamin.peterson | 2010-08-19 16:44:05 -0500 (Thu, 19 Aug 2010) | 1 line
allow star_expr in testlist_gexp
........
r84274 | benjamin.peterson | 2010-08-22 18:40:46 -0500 (Sun, 22 Aug 2010) | 1 line
wrap long line
........
r84275 | benjamin.peterson | 2010-08-22 18:42:22 -0500 (Sun, 22 Aug 2010) | 1 line
cleanup
........
r84276 | benjamin.peterson | 2010-08-22 18:51:01 -0500 (Sun, 22 Aug 2010) | 1 line
when there's a None value and a traceback, don't call type with it #9661
........
r84375 | george.boutsioukis | 2010-08-31 08:38:53 -0500 (Tue, 31 Aug 2010) | 3 lines
Idiomatic code changes & stylistic issues fixed in the BottomMatcher module. Thanks to Benjamin Peterson for taking the time to review the code.
........
r85388 | benjamin.peterson | 2010-10-12 17:27:44 -0500 (Tue, 12 Oct 2010) | 1 line
fix urllib fixer with multiple as imports on a line #10069
........
r85478 | benjamin.peterson | 2010-10-14 08:09:56 -0500 (Thu, 14 Oct 2010) | 1 line
stop abusing docstrings
........
r85506 | benjamin.peterson | 2010-10-14 17:45:19 -0500 (Thu, 14 Oct 2010) | 1 line
kill sibling import
........
r85507 | benjamin.peterson | 2010-10-14 17:54:15 -0500 (Thu, 14 Oct 2010) | 1 line
remove trailing whitespace
........
r85508 | benjamin.peterson | 2010-10-14 17:55:28 -0500 (Thu, 14 Oct 2010) | 1 line
typo
........
2010-10-15 07:00:04 +08:00
|
|
|
|
|
|
|
pattern_grammar = driver.load_grammar(_PATTERN_GRAMMAR_FILE)
|
|
|
|
pattern_symbols = Symbols(pattern_grammar)
|