This is the first instalment of a set which goal is to track locations
of tokens across macro expansions. Tom Tromey did the original work
and attached the patch to PR preprocessor/7263. This opus is a
derivative of that original work.
This patch modifies the linemap module of libcpp to add virtual
locations support.
A virtual location is a mapped location that can resolve to several
different physical locations. It can always resolve to the spelling
location of a token. For tokens resulting from macro expansion it can
resolve to:
- either the location of the expansion point of the macro.
- or the location of the token in the definition of the
macro
- or, if the token is an argument of a function-like macro,
the location of the use of the matching macro parameter in
the definition of the macro
The patch creates a new type of line map called a macro map. For every
single macro expansion, there is a macro map that generates a virtual
location for every single resulting token of the expansion.
The good old type of line map we all know is now called an ordinary
map. That one still encodes spelling locations as it has always had.
As a result linemap_lookup as been extended to return a macro map when
given a virtual location resulting from a macro expansion. The layout
of structs line_map has changed to support this new type of map. So
did the layout of struct line_maps. Accessor macros have been
introduced to avoid messing with the implementation details of these
datastructures directly. This helped already as we have been testing
different ways of arranging these datastructure. Having to constantly
adjust client code that is too tied with the internals of line_map and
line_maps would have been even more painful.
Of course, many new public functions have been added to the linemap
module to handle the resolution of virtual locations.
This patch introduces the infrastructure but no part of the compiler
uses virtual locations yet.
However the client code of the linemap data structures has been
adjusted as per the changes. E.g, it's not anymore reliable for a
client code to manipulate struct line_map directly if it just wants to
deal with spelling locations, because struct line_map can now
represent a macro map as well. In that case, it's better to use the
convenient API to resolve the initial (possibly virtual) location to a
spelling location (or to an ordinary map) and use that.
This is the reason why the patch adjusts the Java, Ada and Fortran
front ends.
Also, note that virtual locations are not supposed to be ordered for
relations '<' and '>' anymore. To test if a virtual location appears
"before" another one, one has to use a new operator exposed by the
line map interface. The patch updates the only spot (in the
diagnostics module) I have found that was making the assumption that
locations were ordered for these relations. This is the only change
that introduces a use of the new line map API in this patch, so I am
adding a regression test for it only.
From-SVN: r180081
Change "if (E) free (E);" to "free (E);" everywhere except in the
libgo/, intl/, zlib/ and classpath/ directories.
Also transform equivalent variants like
"if (E != NULL) free (E);" and allow an extra cast on the
argument to free. Otherwise, the tested and freed "E"
expressions must be identical, modulo white space.
From-SVN: r172785
gcc:
* c-lex.c (c_lex_with_flags): Expect cpp_hashnode in
tok->val.node.node.
libcpp:
* include/cpplib.h (enum cpp_token_fld_kind): Add
CPP_TOKEN_FLD_TOKEN_NO.
(struct cpp_macro_arg, struct cpp_identifier): Define.
(union cpp_token_u): Use struct cpp_identifier for identifiers.
Use struct cpp_macro_arg for macro arguments. Add token_no for
CPP_PASTE token numbers.
* directives.c (_cpp_handle_directive, lex_macro_node, do_pragma,
do_pragma_poison, parse_assertion): Use val.node.node in place of
val.node.
* expr.c (parse_defined, eval_token): Use val.node.node in place
of val.node.
* lex.c (cpp_ideq, _cpp_lex_direct, cpp_token_len,
cpp_spell_token, cpp_output_token, _cpp_equiv_tokens,
cpp_token_val_index): Use val.macro_arg.arg_no or val.token_no in
place of val.arg_no. Use val.node.node in place of val.node.
* macro.c (replace_args, cpp_get_token, parse_params,
lex_expansion_token, create_iso_definition, cpp_macro_definition):
Use val.macro_arg.arg_no or val.token_no in place of val.arg_no.
Use val.node.node in place of val.node.
From-SVN: r147341
libcpp:
PR preprocessor/20078
* include/cpp-id-data.h (struct cpp_macro): Add extra_tokens
field.
* include/cpplib.h (SP_DIGRAPH, SP_PREV_WHITE): Define.
(struct cpp_token): Change flags to unsigned short.
* lex.c (_cpp_lex_direct): Initialize arg_no for CPP_PASTE tokens.
(_cpp_equiv_tokens): Check arg_no for CPP_PASTE tokens.
(cpp_token_val_index): Return CPP_TOKEN_FLD_ARG_NO for CPP_PASTE
tokens.
* macro.c (macro_real_token_count): New.
(enter_macro_context, replace_args): Use macro_real_token_count.
(create_iso_definition): Record whitespace surrounding and digraph
spelling of # and ## tokens using SP_PREV_WHITE and SP_DIGRAPH.
Set extra_tokens and save CPP_PASTE tokens with arg_no set for
multiple consecutive ## tokens.
(_cpp_create_definition): Initialize extra_tokens.
(cpp_macro_definition): Use macro_real_token_count.
gcc/testsuite:
* gcc.dg/cpp/paste16.c, gcc.dg/cpp/redef4.c: New tests.
From-SVN: r146352
libcpp:
PR preprocessor/31869
* macro.c (stringify_arg): Handle NULL source token in padding
token where previous padding token did not have source token with
preceding whitespace.
gcc/testsuite:
* gcc.dg/cpp/strify5.c: New test.
From-SVN: r145989
PR target/39558
* macro.c (cpp_get_token): If macro_to_expand returns NULL
and used some tokens, add CPP_PADDING before next token.
* gcc.target/powerpc/altivec-29.c: New test.
From-SVN: r145297
2008-07-21 Manuel Lopez-Ibanez <manu@gcc.gnu.org>
* include/line-map.h (linenum_type): New typedef.
(struct line_map): Use it.
(SOURCE_LINE): Second arguments is a LOCATION not a LINE.
(SOURCE_COLUMN): Likewise.
* macro.c (_cpp_builtin_macro_text): Use linenum_type. Don't store
source_location values in a variable of type linenum_type.
* directives.c (struct if_stack): Use linenum_type.
(strtoul_for_line): Rename as strtolinenum.
(do_line): Use linenum_type.
(do_linemarker): Use linenum_type and strtolinenum.
(_cpp_do_file_change): Use linenum_t.
* line-map.c (linemap_add): Likewise.
(linemap_line_start): Likewise.
* traditional.c (struct fun_macro): 'line' is a source_location.
* errors.c (print_location): Use linenum_type.
* directives-only.c (_cpp_preprocess_dir_only): Likewise.
* internal.h (CPP_INCREMENT_LINE): Likewise.
* lex.c (_cpp_skip_block_comment): Use source_location.
From-SVN: r138026
2008-06-13 Andrew Haley <aph@redhat.com>
PR preprocessor/33305
* macro.c (replace_args): Print a warning for empty macro
arguments in C89 and C++.
From-SVN: r137414
libcpp/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
* include/cpp-id-data.h (UC): Was U, conflicts with U... literal.
* include/cpplib.h (CHAR16, CHAR32, STRING16, STRING32): New tokens.
(struct cpp_options): Added uliterals.
(cpp_interpret_string): Update prototype.
(cpp_interpret_string_notranslate): Idem.
* charset.c (init_iconv_desc): New width member in cset_converter.
(cpp_init_iconv): Add support for char{16,32}_cset_desc.
(convert_ucn): Idem.
(emit_numeric_escape): Idem.
(convert_hex): Idem.
(convert_oct): Idem.
(convert_escape): Idem.
(converter_for_type): New function.
(cpp_interpret_string): Use converter_for_type, support u and U prefix.
(cpp_interpret_string_notranslate): Match changed prototype.
(wide_str_to_charconst): Use converter_for_type.
(cpp_interpret_charconst): Add support for CPP_CHAR{16,32}.
* directives.c (linemarker_dir): Macro U changed to UC.
(parse_include): Idem.
(register_pragma_1): Idem.
(restore_registered_pragmas): Idem.
(get__Pragma_string): Support CPP_STRING{16,32}.
* expr.c (eval_token): Support CPP_CHAR{16,32}.
* init.c (struct lang_flags): Added uliterals.
(lang_defaults): Idem.
* internal.h (struct cset_converter) <width>: New field.
(struct cpp_reader) <char16_cset_desc>: Idem.
(struct cpp_reader) <char32_cset_desc>: Idem.
* lex.c (digraph_spellings): Macro U changed to UC.
(OP, TK): Idem.
(lex_string): Add support for u'...', U'...', u... and U....
(_cpp_lex_direct): Idem.
* macro.c (_cpp_builtin_macro_text): Macro U changed to UC.
(stringify_arg): Support CPP_CHAR{16,32} and CPP_STRING{16,32}.
gcc/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
* c-common.c (CHAR16_TYPE, CHAR32_TYPE): New macros.
(fname_as_string): Match updated cpp_interpret_string prototype.
(fix_string_type): Support char16_t* and char32_t*.
(c_common_nodes_and_builtins): Add char16_t and char32_t (and
derivative) nodes. Register as builtin if C++0x.
(c_parse_error): Support CPP_CHAR{16,32}.
* c-common.h (RID_CHAR16, RID_CHAR32): New elements.
(enum c_tree_index) <CTI_CHAR16_TYPE, CTI_SIGNED_CHAR16_TYPE,
CTI_UNSIGNED_CHAR16_TYPE, CTI_CHAR32_TYPE, CTI_SIGNED_CHAR32_TYPE,
CTI_UNSIGNED_CHAR32_TYPE, CTI_CHAR16_ARRAY_TYPE,
CTI_CHAR32_ARRAY_TYPE>: New elements.
(char16_type_node, signed_char16_type_node, unsigned_char16_type_node,
char32_type_node, signed_char32_type_node, char16_array_type_node,
char32_array_type_node): New defines.
* c-lex.c (cb_ident): Match updated cpp_interpret_string prototype.
(c_lex_with_flags): Support CPP_CHAR{16,32} and CPP_STRING{16,32}.
(lex_string): Support CPP_STRING{16,32}, match updated
cpp_interpret_string and cpp_interpret_string_notranslate prototypes.
(lex_charconst): Support CPP_CHAR{16,32}.
* c-parser.c (c_parser_postfix_expression): Support CPP_CHAR{16,32}
and CPP_STRING{16,32}.
gcc/cp/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
* cvt.c (type_promotes_to): Support char16_t and char32_t.
* decl.c (grokdeclarator): Disallow signed/unsigned/short/long on
char16_t and char32_t.
* lex.c (reswords): Add char16_t and char32_t (for c++0x).
* mangle.c (write_builtin_type): Mangle char16_t/char32_t as vendor
extended builtin type u8char32_t.
* parser.c (cp_lexer_next_token_is_decl_specifier_keyword): Support
RID_CHAR{16,32}.
(cp_lexer_print_token): Support CPP_STRING{16,32}.
(cp_parser_is_string_literal): Idem.
(cp_parser_string_literal): Idem.
(cp_parser_primary_expression): Support CPP_CHAR{16,32} and
CPP_STRING{16,32}.
(cp_parser_simple_type_specifier): Support RID_CHAR{16,32}.
* tree.c (char_type_p): Support char16_t and char32_t as char types.
* typeck.c (string_conv_p): Support char16_t and char32_t.
gcc/testsuite/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
Tests for char16_t and char32_t support.
* g++.dg/ext/utf-cvt.C: New
* g++.dg/ext/utf-cxx0x.C: New
* g++.dg/ext/utf-cxx98.C: New
* g++.dg/ext/utf-dflt.C: New
* g++.dg/ext/utf-gnuxx0x.C: New
* g++.dg/ext/utf-gnuxx98.C: New
* g++.dg/ext/utf-mangle.C: New
* g++.dg/ext/utf-typedef-cxx0x.C: New
* g++.dg/ext/utf-typedef-
* g++.dg/ext/utf-typespec.C: New
* g++.dg/ext/utf16-1.C: New
* g++.dg/ext/utf16-2.C: New
* g++.dg/ext/utf16-3.C: New
* g++.dg/ext/utf16-4.C: New
* g++.dg/ext/utf32-1.C: New
* g++.dg/ext/utf32-2.C: New
* g++.dg/ext/utf32-3.C: New
* g++.dg/ext/utf32-4.C: New
* gcc.dg/utf-cvt.c: New
* gcc.dg/utf-dflt.c: New
* gcc.dg/utf16-1.c: New
* gcc.dg/utf16-2.c: New
* gcc.dg/utf16-3.c: New
* gcc.dg/utf16-4.c: New
* gcc.dg/utf32-1.c: New
* gcc.dg/utf32-2.c: New
* gcc.dg/utf32-3.c: New
* gcc.dg/utf32-4.c: New
libiberty/ChangeLog:
2008-04-14 Kris Van Hees <kris.van.hees@oracle.com>
* testsuite/demangle-expected: Added tests for char16_t and char32_t.
From-SVN: r134438
PR preprocessor/34692
* macro.c (collect_args): Add pragma_buff argument. Push
CPP_PRAGMA ... CPP_PRAGMA_EOL tokens to *pragma_buff, rather
than into arguments. Reset prevent_expansion and parsing_args
state at CPP_PRAGMA_EOL/CPP_EOF.
(funlike_invocation_p): Add pragma_buff argument, pass it through
to collect_args.
(enter_macro_context): Add result argument. Adjust
funlike_invocation_p caller. Emit all deferred pragma tokens
gathered during collect_args before the expansion, add a padding
token. Return 2 instead of 1 if any pragma tokens were prepended.
(cpp_get_token): If enter_macro_context returns 2, don't return
a padding token, instead cycle to grab CPP_PRAGMA token.
* directives.c (_cpp_handle_directive): If was_parsing_args
in deferred pragma, leave parsing_args and prevent_expansion as is.
* gcc.dg/cpp/pr34692.c: New test.
* gcc.dg/gomp/pr34692.c: New test.
From-SVN: r131819
2007-05-23 Simon Martin <simartin@users.sourceforge.net>
PR preprocessor/20077
* macro.c (create_iso_definition): Fixed the method to determine whether
the token-pasting operator appears at the beginning or the end of a macro.
From-SVN: r125000
PR preprocessor/28709
* macro.c (paste_tokens): Do error reporting here, use BUF with the
spelled LHS token as opposed to spelling it again.
(paste_all_tokens): Don't report errors here, just break on failure.
* gcc.dg/cpp/paste14.c: New test.
From-SVN: r117664
PR preprocessor/27746
* directives.c (do_pragma): Handle pragma with valid namespace
and invalid name coming from macro expansion.
* directives.c (destringize_and_run): Initialize next field in
context.
PR c/27747
PR c++/27748
* directives.c (destringize_and_run): Set NO_EXPAND on the
tokens.
* macro.c (_cpp_backup_tokens): Fix comment typo.
testsuite/
PR c/27747
* gcc.dg/cpp/_Pragma6.c: New test.
PR c++/27748
* g++.dg/cpp/_Pragma1.C: New test.
PR preprocessor/27746
* gcc.dg/gomp/macro-3.c: New test.
* gcc.dg/gomp/macro-4.c: New test.
* g++.dg/gomp/macro-3.C: New test.
* g++.dg/gomp/macro-4.C: New test.
From-SVN: r114519
PR preprocessor/25717
* init.c (cpp_init_builtins): If __STDC__ will not change value
between system headers and other sources, define it as a normal
macro rather than a builtin.
* macro.c (_cpp_builtin_macro_text) <case BT_STDC>: Only check
cpp_in_system_header condition.
From-SVN: r110144
* configure.ac: Check declarations for asprintf and vasprintf.
* config.in: Regenerate.
* configure: Likewise.
* charset.c (conversion_loop): Use XRESIZEVEC.
(convert_no_conversion): Likewise.
(convert_using_iconv): Likewise.
(init_iconv_desc): Cast return value of alloca.
(cpp_host_to_exec_charset): Use XNEWVEC.
(emit_numeric_escape): Use XRESIZEVEC.
(cpp_interpret_string): Use XNEWVEC.
(cpp_interpret_string): Use XRESIZEVEC.
(_cpp_interpret_identifier): Cast return value of alloca.
(_cpp_convert_input): Use XNEWVEC and XRESIZEVEC.
* directives.c (glue_header_name): Use XNEWVEC and XRESIZEVEC.
(parse_include): Use XNEWVEC.
(insert_pragma_entry): Rename local variable "new" to
"new_entry".
(save_registered_pragmas): Cast return value of xmemdup.
(destringize_and_run): Same for alloca.
(parse_assertion): Likewise.
(do_assert): Cast allocated storage to proper type.
(cpp_define): Likewise.
(_cpp_define_builtin): Likewise.
(cpp_undef): Likewise.
(handle_assertion): Likewise.
(cpp_push_buffer): Rename local variable "new" to "new_buffer".
* expr.c (CPP_UPLUS): Cast value to type cpp_ttype.
(CPP_UMINUS): Likewise.
(struct cpp_operator): Rename from struct operator.
(_cpp_expand_op_stack): Use XRESIZEVEC.
* files.c (pch_open_file): Use XNEWVEC.
(pch_open_file): Use XRESIZEVEC.
(read_file_guts): Use XNEWVEC and XRESIZEVEC.
(dir_name_of_file): Use XNEWVEC.
(make_cpp_file): Use XCNEW.
(make_cpp_dir): Likewise.
(allocate_file_hash_entries): USE XNEWVEC.
(cpp_included): Cast return value of htab_find_with_hash.
(append_file_to_dir): Use XNEWVEC.
(read_filename_string): Likewise. Use XRESIZEVEC too.
(read_name_map): Cast return value of alloca. Use XRESIZEVEC.
(remap_filename): Use XNEWVEC.
(struct pchf_entry): Move definition out of struct pchf_data.
(_cpp_save_file_entries): Use XCNEWVAR.
(_cpp_read_file_entries): Use XNEWVAR.
* identifiers.c (alloc_node): Use XOBNEW.
* init.c (cpp_create_reader): Use XCNEW.
(cpp_init_builtins): Cast of b->value to enum builtin_type.
(read_original_directory): Cast return value of alloca.
* lex.c (add_line_note): Use XRESIZEVEC.
(warn_about_normalization): Use XNEWVEC.
(_cpp_lex_direct): Cast node->directive_index to (enum cpp_ttype).
(new_buff): Use XNEWVEC.
* line-map.c (linemap_add): Use XRESIZEVEC.
* macro.c (builtin_macro): Cast return value of alloca.
(paste_tokens): Likewise.
(expand_arg): Use XNEWVEC and XRESIZEVEC.
(_cpp_save_parameter): Use XRESIZEVEC.
(create_iso_definition): Cast allocated storage to proper type.
(_cpp_create_definition): Likewise.
(cpp_macro_definition): Use XRESIZEVEC.
* makedepend.c (add_clm): Use XNEW.
(add_dir): Likewise.
* mkdeps.c (munge): Use XNEWVEC.
(deps_init): Use XCNEW.
(deps_add_target): Use XRESIZEVEC.
(deps_add_default_target): Cast return value of alloca.
(deps_add_dep): Use XRESIZEVEC.
(deps_add_vpath): Likewise. Use XNEWVEC too.
(deps_restore): Likewise.
* pch.c (save_idents): Use XNEW and XNEWVEC.
(cpp_save_state): Use XNEW.
(count_defs): Cast return value of htab_find.
(write_defs): Likewise.
(cpp_write_pch_deps): Use XNEWVEC.
(collect_ht_nodes): Use XRESIZEVEC.
(cpp_valid_state): Use XNEWVEC.
(save_macros): Use XRESIZEVEC. Cast return value of xmemdup.
* symtab.c (ht_create): Use XCNEW.
(ht_lookup_with_hash): Cast return value of obstack_copy0.
(ht_expand): Use XCNEWVEC.
* system.h (HAVE_DESIGNATED_INITIALIZERS): False if __cplusplus.
(bool): Do not define if __cplusplus.
From-SVN: r100295
PR preprocessor/19475
* macro.c (create_iso_definition): For < ISO C99, don't
pedwarn if there is no whitespace between macro name and its
replacement, but the replacement starts with a basic character
set character.
* gcc.dg/cpp/macspace1.c: New test.
* gcc.dg/cpp/macspace2.c: New test.
From-SVN: r97652
PR bootstrap/20282
PR bootstrap/20305
* macro.c (replace_args, cpp_get_token): Copy whole
cpp_token_u instead of just cpp_string field from it.
From-SVN: r95887
2005-02-15 Eric Christopher <echristo@redhat.com>
PR preprocessor/19077
* macro.c (cpp_macro_definition): Move handling of whitespace
to PREV_WHITE conditional. Remove overloading of len
variable.
2005-02-15 Eric Christopher <echristo@redhat.com>
* gcc.dg/cpp/20050215-1.c: New file.
From-SVN: r95080
2004-09-09 Matt Austern <austern@apple.com>
Zack Weinberg <zack@codesourcery.com>
* include/cpplib.h (TTYPE_TABLE): Remove CPP_ and SPELL_
prefixes throughout. Add entry for PRAGMA. Remove
unnecessary "= 0" from EQ.
(enum cpp_ttype): Adjust OP and TK definitions to restore
prefixes, via token-paste.
(CPP_LAST_EQ, CPP_FIRST_DIGRAPH, CPP_LAST_PUNCTUATOR, CPP_LAST_CPP_OP):
Change from #defines to additional cpp_ttype enumerators.
(struct cpp_options): Add defer_pragmas.
(cpp_handle_deferred_pragma): Prototype new interface.
* internal.h (struct cpp_reader): Add directive_result.
* directives.c (struct pragma_entry): Add is_internal field;
give boolean fields type bool.
(start_directive): Initialize pfile->directive_result.type.
(_cpp_do__Pragma): Likewise.
(run_directive): Do not crash if pfile->buffer->prev is NULL.
(insert_pragma_entry): Add 'internal' argument; set new->is_internal
from it.
(register_pragma): New static function, bulk of former
cpp_register_pragma here; add 'internal' argument, pass along
to insert_pragma_entry.
(cpp_register_pragma): Now a wrapper around register_pragma which
always passes false for 'internal' argument.
(_cpp_init_internal_pragmas): Call register_pragma directly, passing
true for 'internal'.
(do_pragma): If CPP_OPTION (pfile, defer_pragmas) and this isn't
an internal pragma, save text till the end of the line as a CPP_PRAGMA
token instead of executing the pragma.
(cpp_handle_deferred_pragma): New interface.
* lex.c (token_spellings): Adjust OP and TK definitions to
match changes to cpplib.h.
(_cpp_lex_token): Check for a directive-result token and
return it if present.
(cpp_token_val_index): Handle CPP_PRAGMA.
* macro.c (cpp_builtin_macro_text): Correct comment.
(builtin_macro): Handle directive-result tokens from _cpp_do__Pragma.
From-SVN: r87247
libcpp:
* Makefile.am: Add makedepend.
* Makefile.in, aclocal.m4: Regenerate.
* charset.c: Insert a space to avoid a warning.
* directives.c: Include mkdeps.h.
(_cpp_handle_directive): Reenable macro expander if appropriate.
(undefine_macros): Inline body of _cpp_free_definition for speed.
Do not call undef callback or _cpp_warn_if_unused_macro.
(cpp_get_deps): New interface.
* files.c (search_cache): Add pfile argument. Check for file
that would be found by "" or <> search here...
(_cpp_find_file): ...not here. Correct recorded start_dir of
files found by directory-of-current-file search that would be
found by "" or <> search.
* init.c (cpp_add_dependency_target): Delete.
* internal.h (struct lexer_state): Add discarding_output flag.
* lex.c (lex_identifier): Compute hash function while scanning.
* macro.c (cpp_scan_nooutput): Disable macro expansion outside
directives.
* makedepend.c: New file.
* mkdeps.c (struct deps): Add vpath vector.
(apply_vpath, deps_add_vpath): New function.
(deps_free): Free vpath vector.
(deps_add_dep, deps_add_target): Use apply_vpath.
* symtab.c (calc_hash): Use HT_HASHSTEP and HT_FINISH.
(ht_lookup_with_hash): New function.
* cpplib.h, mkdeps.h: Update prototypes.
* symtab.h: Update prototypes.
(HT_HASHSTEP, HT_FINISH): New macros.
gcc:
* Makefile.in (MKDEPS_H): New shorthand.
(c-opts.o): Update dependencies.
* c-opts.c: Include mkdeps.h.
(handle_deferred_opts): Use cpp_get_deps and deps_add_target,
not cpp_add_dependency_target.
From-SVN: r82654
