mirror of
https://github.com/python/cpython.git
synced 2024-11-24 18:34:43 +08:00
495da29225
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.) https://bugs.python.org/issue35975
68 lines
1.5 KiB
Plaintext
68 lines
1.5 KiB
Plaintext
ENDMARKER
|
|
NAME
|
|
NUMBER
|
|
STRING
|
|
NEWLINE
|
|
INDENT
|
|
DEDENT
|
|
|
|
LPAR '('
|
|
RPAR ')'
|
|
LSQB '['
|
|
RSQB ']'
|
|
COLON ':'
|
|
COMMA ','
|
|
SEMI ';'
|
|
PLUS '+'
|
|
MINUS '-'
|
|
STAR '*'
|
|
SLASH '/'
|
|
VBAR '|'
|
|
AMPER '&'
|
|
LESS '<'
|
|
GREATER '>'
|
|
EQUAL '='
|
|
DOT '.'
|
|
PERCENT '%'
|
|
LBRACE '{'
|
|
RBRACE '}'
|
|
EQEQUAL '=='
|
|
NOTEQUAL '!='
|
|
LESSEQUAL '<='
|
|
GREATEREQUAL '>='
|
|
TILDE '~'
|
|
CIRCUMFLEX '^'
|
|
LEFTSHIFT '<<'
|
|
RIGHTSHIFT '>>'
|
|
DOUBLESTAR '**'
|
|
PLUSEQUAL '+='
|
|
MINEQUAL '-='
|
|
STAREQUAL '*='
|
|
SLASHEQUAL '/='
|
|
PERCENTEQUAL '%='
|
|
AMPEREQUAL '&='
|
|
VBAREQUAL '|='
|
|
CIRCUMFLEXEQUAL '^='
|
|
LEFTSHIFTEQUAL '<<='
|
|
RIGHTSHIFTEQUAL '>>='
|
|
DOUBLESTAREQUAL '**='
|
|
DOUBLESLASH '//'
|
|
DOUBLESLASHEQUAL '//='
|
|
AT '@'
|
|
ATEQUAL '@='
|
|
RARROW '->'
|
|
ELLIPSIS '...'
|
|
COLONEQUAL ':='
|
|
|
|
OP
|
|
AWAIT
|
|
ASYNC
|
|
TYPE_IGNORE
|
|
TYPE_COMMENT
|
|
ERRORTOKEN
|
|
|
|
# These aren't used by the C tokenizer but are needed for tokenize.py
|
|
COMMENT
|
|
NL
|
|
ENCODING
|