mirror of
https://gcc.gnu.org/git/gcc.git
synced 2024-11-27 05:44:15 +08:00
diagnostics: escape non-ASCII source bytes for certain diagnostics
This patch adds support to GCC's diagnostic subsystem for escaping certain
bytes and Unicode characters when quoting source code.
Specifically, this patch adds a new flag rich_location::m_escape_on_output
which is a hint from a diagnostic that non-ASCII bytes in the pertinent
lines of the user's source code should be escaped when printed.
The patch sets this for the following diagnostics:
- when complaining about stray bytes in the program (when these
are non-printable)
- when complaining about "null character(s) ignored");
- for -Wnormalized= (and generate source ranges for such warnings)
The escaping is controlled by a new option:
-fdiagnostics-escape-format=[unicode|bytes]
For example, consider a diagnostic involing a source line containing the
string "before" followed by the Unicode character U+03C0 ("GREEK SMALL
LETTER PI", with UTF-8 encoding 0xCF 0x80) followed by the byte 0xBF
(a stray UTF-8 trailing byte), followed by the string "after", where the
diagnostic highlights the U+03C0 character.
By default, this line will be printed verbatim to the user when
reporting a diagnostic at it, as:
beforeπXafter
^
(using X for the stray byte to avoid putting invalid UTF-8 in this
commit message)
If the diagnostic sets the "escape" flag, it will be printed as:
before<U+03C0><BF>after
^~~~~~~~
with -fdiagnostics-escape-format=unicode (the default), or as:
before<CF><80><BF>after
^~~~~~~~
if the user supplies -fdiagnostics-escape-format=bytes.
This only affects how the source is printed; it does not affect
how column numbers that are printed (as per -fdiagnostics-column-unit=
and -fdiagnostics-column-origin=).
gcc/c-family/ChangeLog:
* c-lex.c (c_lex_with_flags): When complaining about non-printable
CPP_OTHER tokens, set the "escape on output" flag.
gcc/ChangeLog:
* common.opt (fdiagnostics-escape-format=): New.
(diagnostics_escape_format): New enum.
(DIAGNOSTICS_ESCAPE_FORMAT_UNICODE): New enum value.
(DIAGNOSTICS_ESCAPE_FORMAT_BYTES): Likewise.
* diagnostic-format-json.cc (json_end_diagnostic): Add
"escape-source" attribute.
* diagnostic-show-locus.c
(exploc_with_display_col::exploc_with_display_col): Replace
"tabstop" param with a cpp_char_column_policy and add an "aspect"
param. Use these to compute m_display_col accordingly.
(struct char_display_policy): New struct.
(layout::m_policy): New field.
(layout::m_escape_on_output): New field.
(def_policy): New function.
(make_range): Update for changes to exploc_with_display_col ctor.
(default_print_decoded_ch): New.
(width_per_escaped_byte): New.
(escape_as_bytes_width): New.
(escape_as_bytes_print): New.
(escape_as_unicode_width): New.
(escape_as_unicode_print): New.
(make_policy): New.
(layout::layout): Initialize new fields. Update m_exploc ctor
call for above change to ctor.
(layout::maybe_add_location_range): Update for changes to
exploc_with_display_col ctor.
(layout::calculate_x_offset_display): Update for change to
cpp_display_width.
(layout::print_source_line): Pass policy
to cpp_display_width_computation. Capture cpp_decoded_char when
calling process_next_codepoint. Move printing of source code to
m_policy.m_print_cb.
(line_label::line_label): Pass in policy rather than context.
(layout::print_any_labels): Update for change to line_label ctor.
(get_affected_range): Pass in policy rather than context, updating
calls to location_compute_display_column accordingly.
(get_printed_columns): Likewise, also for cpp_display_width.
(correction::correction): Pass in policy rather than tabstop.
(correction::compute_display_cols): Pass m_policy rather than
m_tabstop to cpp_display_width.
(correction::m_tabstop): Replace with...
(correction::m_policy): ...this.
(line_corrections::line_corrections): Pass in policy rather than
context.
(line_corrections::m_context): Replace with...
(line_corrections::m_policy): ...this.
(line_corrections::add_hint): Update to use m_policy rather than
m_context.
(line_corrections::add_hint): Likewise.
(layout::print_trailing_fixits): Likewise.
(selftest::test_display_widths): New.
(selftest::test_layout_x_offset_display_utf8): Update to use
policy rather than tabstop.
(selftest::test_one_liner_labels_utf8): Add test of escaping
source lines.
(selftest::test_diagnostic_show_locus_one_liner_utf8): Update to
use policy rather than tabstop.
(selftest::test_overlapped_fixit_printing): Likewise.
(selftest::test_overlapped_fixit_printing_utf8): Likewise.
(selftest::test_overlapped_fixit_printing_2): Likewise.
(selftest::test_tab_expansion): Likewise.
(selftest::test_escaping_bytes_1): New.
(selftest::test_escaping_bytes_2): New.
(selftest::diagnostic_show_locus_c_tests): Call the new tests.
* diagnostic.c (diagnostic_initialize): Initialize
context->escape_format.
(convert_column_unit): Update to use default character width policy.
(selftest::test_diagnostic_get_location_text): Likewise.
* diagnostic.h (enum diagnostics_escape_format): New enum.
(diagnostic_context::escape_format): New field.
* doc/invoke.texi (-fdiagnostics-escape-format=): New option.
(-fdiagnostics-format=): Add "escape-source" attribute to examples
of JSON output, and document it.
* input.c (location_compute_display_column): Pass in "policy"
rather than "tabstop", passing to
cpp_byte_column_to_display_column.
(selftest::test_cpp_utf8): Update to use cpp_char_column_policy.
* input.h (class cpp_char_column_policy): New forward decl.
(location_compute_display_column): Pass in "policy" rather than
"tabstop".
* opts.c (common_handle_option): Handle
OPT_fdiagnostics_escape_format_.
* selftest.c (temp_source_file::temp_source_file): New ctor
overload taking a size_t.
* selftest.h (temp_source_file::temp_source_file): Likewise.
gcc/testsuite/ChangeLog:
* c-c++-common/diagnostic-format-json-1.c: Add regexp to consume
"escape-source" attribute.
* c-c++-common/diagnostic-format-json-2.c: Likewise.
* c-c++-common/diagnostic-format-json-3.c: Likewise.
* c-c++-common/diagnostic-format-json-4.c: Likewise, twice.
* c-c++-common/diagnostic-format-json-5.c: Likewise.
* gcc.dg/cpp/warn-normalized-4-bytes.c: New test.
* gcc.dg/cpp/warn-normalized-4-unicode.c: New test.
* gcc.dg/encoding-issues-bytes.c: New test.
* gcc.dg/encoding-issues-unicode.c: New test.
* gfortran.dg/diagnostic-format-json-1.F90: Add regexp to consume
"escape-source" attribute.
* gfortran.dg/diagnostic-format-json-2.F90: Likewise.
* gfortran.dg/diagnostic-format-json-3.F90: Likewise.
libcpp/ChangeLog:
* charset.c (convert_escape): Use encoding_rich_location when
complaining about nonprintable unknown escape sequences.
(cpp_display_width_computation::::cpp_display_width_computation):
Pass in policy rather than tabstop.
(cpp_display_width_computation::process_next_codepoint): Add "out"
param and populate *out if non-NULL.
(cpp_display_width_computation::advance_display_cols): Pass NULL
to process_next_codepoint.
(cpp_byte_column_to_display_column): Pass in policy rather than
tabstop. Pass NULL to process_next_codepoint.
(cpp_display_column_to_byte_column): Pass in policy rather than
tabstop.
* errors.c (cpp_diagnostic_get_current_location): New function,
splitting out the logic from...
(cpp_diagnostic): ...here.
(cpp_warning_at): New function.
(cpp_pedwarning_at): New function.
* include/cpplib.h (cpp_warning_at): New decl for rich_location.
(cpp_pedwarning_at): Likewise.
(struct cpp_decoded_char): New.
(struct cpp_char_column_policy): New.
(cpp_display_width_computation::cpp_display_width_computation):
Replace "tabstop" param with "policy".
(cpp_display_width_computation::process_next_codepoint): Add "out"
param.
(cpp_display_width_computation::m_tabstop): Replace with...
(cpp_display_width_computation::m_policy): ...this.
(cpp_byte_column_to_display_column): Replace "tabstop" param with
"policy".
(cpp_display_width): Likewise.
(cpp_display_column_to_byte_column): Likewise.
* include/line-map.h (rich_location::escape_on_output_p): New.
(rich_location::set_escape_on_output): New.
(rich_location::m_escape_on_output): New.
* internal.h (cpp_diagnostic_get_current_location): New decl.
(class encoding_rich_location): New.
* lex.c (skip_whitespace): Use encoding_rich_location when
complaining about null characters.
(warn_about_normalization): Generate a source range when
complaining about improperly normalized tokens, rather than just a
point, and use encoding_rich_location so that the source code
is escaped on printing.
* line-map.c (rich_location::rich_location): Initialize
m_escape_on_output.
Signed-off-by: David Malcolm <dmalcolm@redhat.com>
This commit is contained in:
David Malcolm
parent
91bac9fed5
commit
bd5e882cf6
@ -616,7 +616,11 @@ c_lex_with_flags (tree *value, location_t *loc, unsigned char *cpp_flags,
|
||||
else if (ISGRAPH (c))
|
||||
error_at (*loc, "stray %qc in program", (int) c);
|
||||
else
|
||||
error_at (*loc, "stray %<\\%o%> in program", (int) c);
|
||||
{
|
||||
rich_location rich_loc (line_table, *loc);
|
||||
rich_loc.set_escape_on_output (true);
|
||||
error_at (&rich_loc, "stray %<\\%o%> in program", (int) c);
|
||||
}
|
||||
}
|
||||
goto retry;
|
||||
|
||||
|
||||
@ -1365,6 +1365,10 @@ fdiagnostics-format=
|
||||
Common Joined RejectNegative Enum(diagnostics_output_format)
|
||||
-fdiagnostics-format=[text|json] Select output format.
|
||||
|
||||
fdiagnostics-escape-format=
|
||||
Common Joined RejectNegative Enum(diagnostics_escape_format)
|
||||
-fdiagnostics-escape-format=[unicode|bytes] Select how to escape non-printable-ASCII bytes in the source for diagnostics that suggest it.
|
||||
|
||||
; Required for these enum values.
|
||||
SourceInclude
|
||||
diagnostic.h
|
||||
@ -1378,6 +1382,15 @@ Enum(diagnostics_column_unit) String(display) Value(DIAGNOSTICS_COLUMN_UNIT_DISP
|
||||
EnumValue
|
||||
Enum(diagnostics_column_unit) String(byte) Value(DIAGNOSTICS_COLUMN_UNIT_BYTE)
|
||||
|
||||
Enum
|
||||
Name(diagnostics_escape_format) Type(int)
|
||||
|
||||
EnumValue
|
||||
Enum(diagnostics_escape_format) String(unicode) Value(DIAGNOSTICS_ESCAPE_FORMAT_UNICODE)
|
||||
|
||||
EnumValue
|
||||
Enum(diagnostics_escape_format) String(bytes) Value(DIAGNOSTICS_ESCAPE_FORMAT_BYTES)
|
||||
|
||||
Enum
|
||||
Name(diagnostics_output_format) Type(int)
|
||||
|
||||
|
||||
@ -264,6 +264,9 @@ json_end_diagnostic (diagnostic_context *context, diagnostic_info *diagnostic,
|
||||
json::value *path_value = context->make_json_for_path (context, path);
|
||||
diag_obj->set ("path", path_value);
|
||||
}
|
||||
|
||||
diag_obj->set ("escape-source",
|
||||
new json::literal (richloc->escape_on_output_p ()));
|
||||
}
|
||||
|
||||
/* No-op implementation of "begin_group_cb" for JSON output. */
|
||||
|
||||
@ -175,10 +175,26 @@ enum column_unit {
|
||||
class exploc_with_display_col : public expanded_location
|
||||
{
|
||||
public:
|
||||
exploc_with_display_col (const expanded_location &exploc, int tabstop)
|
||||
: expanded_location (exploc),
|
||||
m_display_col (location_compute_display_column (exploc, tabstop))
|
||||
{}
|
||||
exploc_with_display_col (const expanded_location &exploc,
|
||||
const cpp_char_column_policy &policy,
|
||||
enum location_aspect aspect)
|
||||
: expanded_location (exploc),
|
||||
m_display_col (location_compute_display_column (exploc, policy))
|
||||
{
|
||||
if (exploc.column > 0)
|
||||
{
|
||||
/* m_display_col is now the final column of the byte.
|
||||
If escaping has happened, we may want the first column instead. */
|
||||
if (aspect != LOCATION_ASPECT_FINISH)
|
||||
{
|
||||
expanded_location prev_exploc (exploc);
|
||||
prev_exploc.column--;
|
||||
int prev_display_col
|
||||
= (location_compute_display_column (prev_exploc, policy));
|
||||
m_display_col = prev_display_col + 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
int m_display_col;
|
||||
};
|
||||
@ -313,6 +329,31 @@ test_line_span ()
|
||||
|
||||
#endif /* #if CHECKING_P */
|
||||
|
||||
/* A bundle of information containing how to print unicode
|
||||
characters and bytes when quoting source code.
|
||||
|
||||
Provides a unified place to support escaping some subset
|
||||
of characters to some format.
|
||||
|
||||
Extends char_column_policy; printing is split out to avoid
|
||||
libcpp having to know about pretty_printer. */
|
||||
|
||||
struct char_display_policy : public cpp_char_column_policy
|
||||
{
|
||||
public:
|
||||
char_display_policy (int tabstop,
|
||||
int (*width_cb) (cppchar_t c),
|
||||
void (*print_cb) (pretty_printer *pp,
|
||||
const cpp_decoded_char &cp))
|
||||
: cpp_char_column_policy (tabstop, width_cb),
|
||||
m_print_cb (print_cb)
|
||||
{
|
||||
}
|
||||
|
||||
void (*m_print_cb) (pretty_printer *pp,
|
||||
const cpp_decoded_char &cp);
|
||||
};
|
||||
|
||||
/* A class to control the overall layout when printing a diagnostic.
|
||||
|
||||
The layout is determined within the constructor.
|
||||
@ -345,6 +386,8 @@ class layout
|
||||
|
||||
void print_line (linenum_type row);
|
||||
|
||||
void on_bad_codepoint (const char *ptr, cppchar_t ch, size_t ch_sz);
|
||||
|
||||
private:
|
||||
bool will_show_line_p (linenum_type row) const;
|
||||
void print_leading_fixits (linenum_type row);
|
||||
@ -386,6 +429,7 @@ class layout
|
||||
private:
|
||||
diagnostic_context *m_context;
|
||||
pretty_printer *m_pp;
|
||||
char_display_policy m_policy;
|
||||
location_t m_primary_loc;
|
||||
exploc_with_display_col m_exploc;
|
||||
colorizer m_colorizer;
|
||||
@ -398,6 +442,7 @@ class layout
|
||||
auto_vec <line_span> m_line_spans;
|
||||
int m_linenum_width;
|
||||
int m_x_offset_display;
|
||||
bool m_escape_on_output;
|
||||
};
|
||||
|
||||
/* Implementation of "class colorizer". */
|
||||
@ -646,6 +691,11 @@ layout_range::intersects_line_p (linenum_type row) const
|
||||
/* Default for when we don't care what the tab expansion is set to. */
|
||||
static const int def_tabstop = 8;
|
||||
|
||||
static cpp_char_column_policy def_policy ()
|
||||
{
|
||||
return cpp_char_column_policy (8, cpp_wcwidth);
|
||||
}
|
||||
|
||||
/* Create some expanded locations for testing layout_range. The filename
|
||||
member of the explocs is set to the empty string. This member will only be
|
||||
inspected by the calls to location_compute_display_column() made from the
|
||||
@ -662,10 +712,13 @@ make_range (int start_line, int start_col, int end_line, int end_col)
|
||||
= {"", start_line, start_col, NULL, false};
|
||||
const expanded_location finish_exploc
|
||||
= {"", end_line, end_col, NULL, false};
|
||||
return layout_range (exploc_with_display_col (start_exploc, def_tabstop),
|
||||
exploc_with_display_col (finish_exploc, def_tabstop),
|
||||
return layout_range (exploc_with_display_col (start_exploc, def_policy (),
|
||||
LOCATION_ASPECT_START),
|
||||
exploc_with_display_col (finish_exploc, def_policy (),
|
||||
LOCATION_ASPECT_FINISH),
|
||||
SHOW_RANGE_WITHOUT_CARET,
|
||||
exploc_with_display_col (start_exploc, def_tabstop),
|
||||
exploc_with_display_col (start_exploc, def_policy (),
|
||||
LOCATION_ASPECT_CARET),
|
||||
0, NULL);
|
||||
}
|
||||
|
||||
@ -959,6 +1012,164 @@ fixit_cmp (const void *p_a, const void *p_b)
|
||||
return hint_a->get_start_loc () - hint_b->get_start_loc ();
|
||||
}
|
||||
|
||||
/* Callbacks for use when not escaping the source. */
|
||||
|
||||
/* The default callback for char_column_policy::m_width_cb is cpp_wcwidth. */
|
||||
|
||||
/* Callback for char_display_policy::m_print_cb for printing source chars
|
||||
when not escaping the source. */
|
||||
|
||||
static void
|
||||
default_print_decoded_ch (pretty_printer *pp,
|
||||
const cpp_decoded_char &decoded_ch)
|
||||
{
|
||||
for (const char *ptr = decoded_ch.m_start_byte;
|
||||
ptr != decoded_ch.m_next_byte; ptr++)
|
||||
{
|
||||
if (*ptr == '\0' || *ptr == '\r')
|
||||
{
|
||||
pp_space (pp);
|
||||
continue;
|
||||
}
|
||||
|
||||
pp_character (pp, *ptr);
|
||||
}
|
||||
}
|
||||
|
||||
/* Callbacks for use with DIAGNOSTICS_ESCAPE_FORMAT_BYTES. */
|
||||
|
||||
static const int width_per_escaped_byte = 4;
|
||||
|
||||
/* Callback for char_column_policy::m_width_cb for determining the
|
||||
display width when escaping with DIAGNOSTICS_ESCAPE_FORMAT_BYTES. */
|
||||
|
||||
static int
|
||||
escape_as_bytes_width (cppchar_t ch)
|
||||
{
|
||||
if (ch < 0x80 && ISPRINT (ch))
|
||||
return cpp_wcwidth (ch);
|
||||
else
|
||||
{
|
||||
if (ch <= 0x7F) return 1 * width_per_escaped_byte;
|
||||
if (ch <= 0x7FF) return 2 * width_per_escaped_byte;
|
||||
if (ch <= 0xFFFF) return 3 * width_per_escaped_byte;
|
||||
return 4 * width_per_escaped_byte;
|
||||
}
|
||||
}
|
||||
|
||||
/* Callback for char_display_policy::m_print_cb for printing source chars
|
||||
when escaping with DIAGNOSTICS_ESCAPE_FORMAT_BYTES. */
|
||||
|
||||
static void
|
||||
escape_as_bytes_print (pretty_printer *pp,
|
||||
const cpp_decoded_char &decoded_ch)
|
||||
{
|
||||
if (!decoded_ch.m_valid_ch)
|
||||
{
|
||||
for (const char *iter = decoded_ch.m_start_byte;
|
||||
iter != decoded_ch.m_next_byte; ++iter)
|
||||
{
|
||||
char buf[16];
|
||||
sprintf (buf, "<%02x>", (unsigned char)*iter);
|
||||
pp_string (pp, buf);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
cppchar_t ch = decoded_ch.m_ch;
|
||||
if (ch < 0x80 && ISPRINT (ch))
|
||||
pp_character (pp, ch);
|
||||
else
|
||||
{
|
||||
for (const char *iter = decoded_ch.m_start_byte;
|
||||
iter < decoded_ch.m_next_byte; ++iter)
|
||||
{
|
||||
char buf[16];
|
||||
sprintf (buf, "<%02x>", (unsigned char)*iter);
|
||||
pp_string (pp, buf);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Callbacks for use with DIAGNOSTICS_ESCAPE_FORMAT_UNICODE. */
|
||||
|
||||
/* Callback for char_column_policy::m_width_cb for determining the
|
||||
display width when escaping with DIAGNOSTICS_ESCAPE_FORMAT_UNICODE. */
|
||||
|
||||
static int
|
||||
escape_as_unicode_width (cppchar_t ch)
|
||||
{
|
||||
if (ch < 0x80 && ISPRINT (ch))
|
||||
return cpp_wcwidth (ch);
|
||||
else
|
||||
{
|
||||
// Width of "<U+%04x>"
|
||||
if (ch > 0xfffff)
|
||||
return 10;
|
||||
else if (ch > 0xffff)
|
||||
return 9;
|
||||
else
|
||||
return 8;
|
||||
}
|
||||
}
|
||||
|
||||
/* Callback for char_display_policy::m_print_cb for printing source chars
|
||||
when escaping with DIAGNOSTICS_ESCAPE_FORMAT_UNICODE. */
|
||||
|
||||
static void
|
||||
escape_as_unicode_print (pretty_printer *pp,
|
||||
const cpp_decoded_char &decoded_ch)
|
||||
{
|
||||
if (!decoded_ch.m_valid_ch)
|
||||
{
|
||||
escape_as_bytes_print (pp, decoded_ch);
|
||||
return;
|
||||
}
|
||||
|
||||
cppchar_t ch = decoded_ch.m_ch;
|
||||
if (ch < 0x80 && ISPRINT (ch))
|
||||
pp_character (pp, ch);
|
||||
else
|
||||
{
|
||||
char buf[16];
|
||||
sprintf (buf, "<U+%04X>", ch);
|
||||
pp_string (pp, buf);
|
||||
}
|
||||
}
|
||||
|
||||
/* Populate a char_display_policy based on DC and RICHLOC. */
|
||||
|
||||
static char_display_policy
|
||||
make_policy (const diagnostic_context &dc,
|
||||
const rich_location &richloc)
|
||||
{
|
||||
/* The default is to not escape non-ASCII bytes. */
|
||||
char_display_policy result
|
||||
(dc.tabstop, cpp_wcwidth, default_print_decoded_ch);
|
||||
|
||||
/* If the diagnostic suggests escaping non-ASCII bytes, then
|
||||
use policy from user-supplied options. */
|
||||
if (richloc.escape_on_output_p ())
|
||||
{
|
||||
result.m_undecoded_byte_width = width_per_escaped_byte;
|
||||
switch (dc.escape_format)
|
||||
{
|
||||
default:
|
||||
gcc_unreachable ();
|
||||
case DIAGNOSTICS_ESCAPE_FORMAT_UNICODE:
|
||||
result.m_width_cb = escape_as_unicode_width;
|
||||
result.m_print_cb = escape_as_unicode_print;
|
||||
break;
|
||||
case DIAGNOSTICS_ESCAPE_FORMAT_BYTES:
|
||||
result.m_width_cb = escape_as_bytes_width;
|
||||
result.m_print_cb = escape_as_bytes_print;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/* Implementation of class layout. */
|
||||
|
||||
/* Constructor for class layout.
|
||||
@ -975,8 +1186,10 @@ layout::layout (diagnostic_context * context,
|
||||
diagnostic_t diagnostic_kind)
|
||||
: m_context (context),
|
||||
m_pp (context->printer),
|
||||
m_policy (make_policy (*context, *richloc)),
|
||||
m_primary_loc (richloc->get_range (0)->m_loc),
|
||||
m_exploc (richloc->get_expanded_location (0), context->tabstop),
|
||||
m_exploc (richloc->get_expanded_location (0), m_policy,
|
||||
LOCATION_ASPECT_CARET),
|
||||
m_colorizer (context, diagnostic_kind),
|
||||
m_colorize_source_p (context->colorize_source_p),
|
||||
m_show_labels_p (context->show_labels_p),
|
||||
@ -986,7 +1199,8 @@ layout::layout (diagnostic_context * context,
|
||||
m_fixit_hints (richloc->get_num_fixit_hints ()),
|
||||
m_line_spans (1 + richloc->get_num_locations ()),
|
||||
m_linenum_width (0),
|
||||
m_x_offset_display (0)
|
||||
m_x_offset_display (0),
|
||||
m_escape_on_output (richloc->escape_on_output_p ())
|
||||
{
|
||||
for (unsigned int idx = 0; idx < richloc->get_num_locations (); idx++)
|
||||
{
|
||||
@ -1072,10 +1286,13 @@ layout::maybe_add_location_range (const location_range *loc_range,
|
||||
|
||||
/* Everything is now known to be in the correct source file,
|
||||
but it may require further sanitization. */
|
||||
layout_range ri (exploc_with_display_col (start, m_context->tabstop),
|
||||
exploc_with_display_col (finish, m_context->tabstop),
|
||||
layout_range ri (exploc_with_display_col (start, m_policy,
|
||||
LOCATION_ASPECT_START),
|
||||
exploc_with_display_col (finish, m_policy,
|
||||
LOCATION_ASPECT_FINISH),
|
||||
loc_range->m_range_display_kind,
|
||||
exploc_with_display_col (caret, m_context->tabstop),
|
||||
exploc_with_display_col (caret, m_policy,
|
||||
LOCATION_ASPECT_CARET),
|
||||
original_idx, loc_range->m_label);
|
||||
|
||||
/* If we have a range that finishes before it starts (perhaps
|
||||
@ -1409,7 +1626,7 @@ layout::calculate_x_offset_display ()
|
||||
= get_line_bytes_without_trailing_whitespace (line.get_buffer (),
|
||||
line.length ());
|
||||
int eol_display_column
|
||||
= cpp_display_width (line.get_buffer (), line_bytes, m_context->tabstop);
|
||||
= cpp_display_width (line.get_buffer (), line_bytes, m_policy);
|
||||
if (caret_display_column > eol_display_column
|
||||
|| !caret_display_column)
|
||||
{
|
||||
@ -1488,7 +1705,7 @@ layout::print_source_line (linenum_type row, const char *line, int line_bytes)
|
||||
/* This object helps to keep track of which display column we are at, which is
|
||||
necessary for computing the line bounds in display units, for doing
|
||||
tab expansion, and for implementing m_x_offset_display. */
|
||||
cpp_display_width_computation dw (line, line_bytes, m_context->tabstop);
|
||||
cpp_display_width_computation dw (line, line_bytes, m_policy);
|
||||
|
||||
/* Skip the first m_x_offset_display display columns. In case the leading
|
||||
portion that will be skipped ends with a character with wcwidth > 1, then
|
||||
@ -1536,7 +1753,8 @@ layout::print_source_line (linenum_type row, const char *line, int line_bytes)
|
||||
tabs and replacing some control bytes with spaces as necessary. */
|
||||
const char *c = dw.next_byte ();
|
||||
const int start_disp_col = dw.display_cols_processed () + 1;
|
||||
const int this_display_width = dw.process_next_codepoint ();
|
||||
cpp_decoded_char cp;
|
||||
const int this_display_width = dw.process_next_codepoint (&cp);
|
||||
if (*c == '\t')
|
||||
{
|
||||
/* The returned display width is the number of spaces into which the
|
||||
@ -1545,15 +1763,6 @@ layout::print_source_line (linenum_type row, const char *line, int line_bytes)
|
||||
pp_space (m_pp);
|
||||
continue;
|
||||
}
|
||||
if (*c == '\0' || *c == '\r')
|
||||
{
|
||||
/* cpp_wcwidth() promises to return 1 for all control bytes, and we
|
||||
want to output these as a single space too, so this case is
|
||||
actually the same as the '\t' case. */
|
||||
gcc_assert (this_display_width == 1);
|
||||
pp_space (m_pp);
|
||||
continue;
|
||||
}
|
||||
|
||||
/* We have a (possibly multibyte) character to output; update the line
|
||||
bounds if it is not whitespace. */
|
||||
@ -1565,7 +1774,8 @@ layout::print_source_line (linenum_type row, const char *line, int line_bytes)
|
||||
}
|
||||
|
||||
/* Output the character. */
|
||||
while (c != dw.next_byte ()) pp_character (m_pp, *c++);
|
||||
m_policy.m_print_cb (m_pp, cp);
|
||||
c = dw.next_byte ();
|
||||
}
|
||||
print_newline ();
|
||||
return lbounds;
|
||||
@ -1664,14 +1874,14 @@ layout::print_annotation_line (linenum_type row, const line_bounds lbounds)
|
||||
class line_label
|
||||
{
|
||||
public:
|
||||
line_label (diagnostic_context *context, int state_idx, int column,
|
||||
line_label (const cpp_char_column_policy &policy,
|
||||
int state_idx, int column,
|
||||
label_text text)
|
||||
: m_state_idx (state_idx), m_column (column),
|
||||
m_text (text), m_label_line (0), m_has_vbar (true)
|
||||
{
|
||||
const int bytes = strlen (text.m_buffer);
|
||||
m_display_width
|
||||
= cpp_display_width (text.m_buffer, bytes, context->tabstop);
|
||||
m_display_width = cpp_display_width (text.m_buffer, bytes, policy);
|
||||
}
|
||||
|
||||
/* Sorting is primarily by column, then by state index. */
|
||||
@ -1731,7 +1941,7 @@ layout::print_any_labels (linenum_type row)
|
||||
if (text.m_buffer == NULL)
|
||||
continue;
|
||||
|
||||
labels.safe_push (line_label (m_context, i, disp_col, text));
|
||||
labels.safe_push (line_label (m_policy, i, disp_col, text));
|
||||
}
|
||||
}
|
||||
|
||||
@ -2011,7 +2221,7 @@ public:
|
||||
|
||||
/* Get the range of bytes or display columns that HINT would affect. */
|
||||
static column_range
|
||||
get_affected_range (diagnostic_context *context,
|
||||
get_affected_range (const cpp_char_column_policy &policy,
|
||||
const fixit_hint *hint, enum column_unit col_unit)
|
||||
{
|
||||
expanded_location exploc_start = expand_location (hint->get_start_loc ());
|
||||
@ -2022,13 +2232,11 @@ get_affected_range (diagnostic_context *context,
|
||||
int finish_column;
|
||||
if (col_unit == CU_DISPLAY_COLS)
|
||||
{
|
||||
start_column
|
||||
= location_compute_display_column (exploc_start, context->tabstop);
|
||||
start_column = location_compute_display_column (exploc_start, policy);
|
||||
if (hint->insertion_p ())
|
||||
finish_column = start_column - 1;
|
||||
else
|
||||
finish_column
|
||||
= location_compute_display_column (exploc_finish, context->tabstop);
|
||||
finish_column = location_compute_display_column (exploc_finish, policy);
|
||||
}
|
||||
else
|
||||
{
|
||||
@ -2041,12 +2249,13 @@ get_affected_range (diagnostic_context *context,
|
||||
/* Get the range of display columns that would be printed for HINT. */
|
||||
|
||||
static column_range
|
||||
get_printed_columns (diagnostic_context *context, const fixit_hint *hint)
|
||||
get_printed_columns (const cpp_char_column_policy &policy,
|
||||
const fixit_hint *hint)
|
||||
{
|
||||
expanded_location exploc = expand_location (hint->get_start_loc ());
|
||||
int start_column = location_compute_display_column (exploc, context->tabstop);
|
||||
int start_column = location_compute_display_column (exploc, policy);
|
||||
int hint_width = cpp_display_width (hint->get_string (), hint->get_length (),
|
||||
context->tabstop);
|
||||
policy);
|
||||
int final_hint_column = start_column + hint_width - 1;
|
||||
if (hint->insertion_p ())
|
||||
{
|
||||
@ -2056,8 +2265,7 @@ get_printed_columns (diagnostic_context *context, const fixit_hint *hint)
|
||||
{
|
||||
exploc = expand_location (hint->get_next_loc ());
|
||||
--exploc.column;
|
||||
int finish_column
|
||||
= location_compute_display_column (exploc, context->tabstop);
|
||||
int finish_column = location_compute_display_column (exploc, policy);
|
||||
return column_range (start_column,
|
||||
MAX (finish_column, final_hint_column));
|
||||
}
|
||||
@ -2075,13 +2283,13 @@ public:
|
||||
column_range affected_columns,
|
||||
column_range printed_columns,
|
||||
const char *new_text, size_t new_text_len,
|
||||
int tabstop)
|
||||
const cpp_char_column_policy &policy)
|
||||
: m_affected_bytes (affected_bytes),
|
||||
m_affected_columns (affected_columns),
|
||||
m_printed_columns (printed_columns),
|
||||
m_text (xstrdup (new_text)),
|
||||
m_byte_length (new_text_len),
|
||||
m_tabstop (tabstop),
|
||||
m_policy (policy),
|
||||
m_alloc_sz (new_text_len + 1)
|
||||
{
|
||||
compute_display_cols ();
|
||||
@ -2099,7 +2307,7 @@ public:
|
||||
|
||||
void compute_display_cols ()
|
||||
{
|
||||
m_display_cols = cpp_display_width (m_text, m_byte_length, m_tabstop);
|
||||
m_display_cols = cpp_display_width (m_text, m_byte_length, m_policy);
|
||||
}
|
||||
|
||||
void overwrite (int dst_offset, const char_span &src_span)
|
||||
@ -2127,7 +2335,7 @@ public:
|
||||
char *m_text;
|
||||
size_t m_byte_length; /* Not including null-terminator. */
|
||||
int m_display_cols;
|
||||
int m_tabstop;
|
||||
const cpp_char_column_policy &m_policy;
|
||||
size_t m_alloc_sz;
|
||||
};
|
||||
|
||||
@ -2163,15 +2371,16 @@ correction::ensure_terminated ()
|
||||
class line_corrections
|
||||
{
|
||||
public:
|
||||
line_corrections (diagnostic_context *context, const char *filename,
|
||||
line_corrections (const char_display_policy &policy,
|
||||
const char *filename,
|
||||
linenum_type row)
|
||||
: m_context (context), m_filename (filename), m_row (row)
|
||||
: m_policy (policy), m_filename (filename), m_row (row)
|
||||
{}
|
||||
~line_corrections ();
|
||||
|
||||
void add_hint (const fixit_hint *hint);
|
||||
|
||||
diagnostic_context *m_context;
|
||||
const char_display_policy &m_policy;
|
||||
const char *m_filename;
|
||||
linenum_type m_row;
|
||||
auto_vec <correction *> m_corrections;
|
||||
@ -2217,10 +2426,10 @@ source_line::source_line (const char *filename, int line)
|
||||
void
|
||||
line_corrections::add_hint (const fixit_hint *hint)
|
||||
{
|
||||
column_range affected_bytes = get_affected_range (m_context, hint, CU_BYTES);
|
||||
column_range affected_columns = get_affected_range (m_context, hint,
|
||||
column_range affected_bytes = get_affected_range (m_policy, hint, CU_BYTES);
|
||||
column_range affected_columns = get_affected_range (m_policy, hint,
|
||||
CU_DISPLAY_COLS);
|
||||
column_range printed_columns = get_printed_columns (m_context, hint);
|
||||
column_range printed_columns = get_printed_columns (m_policy, hint);
|
||||
|
||||
/* Potentially consolidate. */
|
||||
if (!m_corrections.is_empty ())
|
||||
@ -2289,7 +2498,7 @@ line_corrections::add_hint (const fixit_hint *hint)
|
||||
printed_columns,
|
||||
hint->get_string (),
|
||||
hint->get_length (),
|
||||
m_context->tabstop));
|
||||
m_policy));
|
||||
}
|
||||
|
||||
/* If there are any fixit hints on source line ROW, print them.
|
||||
@ -2303,7 +2512,7 @@ layout::print_trailing_fixits (linenum_type row)
|
||||
{
|
||||
/* Build a list of correction instances for the line,
|
||||
potentially consolidating hints (for the sake of readability). */
|
||||
line_corrections corrections (m_context, m_exploc.file, row);
|
||||
line_corrections corrections (m_policy, m_exploc.file, row);
|
||||
for (unsigned int i = 0; i < m_fixit_hints.length (); i++)
|
||||
{
|
||||
const fixit_hint *hint = m_fixit_hints[i];
|
||||
@ -2646,6 +2855,59 @@ namespace selftest {
|
||||
|
||||
/* Selftests for diagnostic_show_locus. */
|
||||
|
||||
/* Verify that cpp_display_width correctly handles escaping. */
|
||||
|
||||
static void
|
||||
test_display_widths ()
|
||||
{
|
||||
gcc_rich_location richloc (UNKNOWN_LOCATION);
|
||||
|
||||
/* U+03C0 "GREEK SMALL LETTER PI". */
|
||||
const char *pi = "\xCF\x80";
|
||||
/* U+1F642 "SLIGHTLY SMILING FACE". */
|
||||
const char *emoji = "\xF0\x9F\x99\x82";
|
||||
/* Stray trailing byte of a UTF-8 character. */
|
||||
const char *stray = "\xBF";
|
||||
/* U+10FFFF. */
|
||||
const char *max_codepoint = "\xF4\x8F\xBF\xBF";
|
||||
|
||||
/* No escaping. */
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
char_display_policy policy (make_policy (dc, richloc));
|
||||
ASSERT_EQ (cpp_display_width (pi, strlen (pi), policy), 1);
|
||||
ASSERT_EQ (cpp_display_width (emoji, strlen (emoji), policy), 2);
|
||||
ASSERT_EQ (cpp_display_width (stray, strlen (stray), policy), 1);
|
||||
/* Don't check width of U+10FFFF; it's in a private use plane. */
|
||||
}
|
||||
|
||||
richloc.set_escape_on_output (true);
|
||||
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
dc.escape_format = DIAGNOSTICS_ESCAPE_FORMAT_UNICODE;
|
||||
char_display_policy policy (make_policy (dc, richloc));
|
||||
ASSERT_EQ (cpp_display_width (pi, strlen (pi), policy), 8);
|
||||
ASSERT_EQ (cpp_display_width (emoji, strlen (emoji), policy), 9);
|
||||
ASSERT_EQ (cpp_display_width (stray, strlen (stray), policy), 4);
|
||||
ASSERT_EQ (cpp_display_width (max_codepoint, strlen (max_codepoint),
|
||||
policy),
|
||||
strlen ("<U+10FFFF>"));
|
||||
}
|
||||
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
dc.escape_format = DIAGNOSTICS_ESCAPE_FORMAT_BYTES;
|
||||
char_display_policy policy (make_policy (dc, richloc));
|
||||
ASSERT_EQ (cpp_display_width (pi, strlen (pi), policy), 8);
|
||||
ASSERT_EQ (cpp_display_width (emoji, strlen (emoji), policy), 16);
|
||||
ASSERT_EQ (cpp_display_width (stray, strlen (stray), policy), 4);
|
||||
ASSERT_EQ (cpp_display_width (max_codepoint, strlen (max_codepoint),
|
||||
policy),
|
||||
16);
|
||||
}
|
||||
}
|
||||
|
||||
/* For precise tests of the layout, make clear where the source line will
|
||||
start. test_left_margin sets the total byte count from the left side of the
|
||||
screen to the start of source lines, after the line number and the separator,
|
||||
@ -2715,10 +2977,10 @@ test_layout_x_offset_display_utf8 (const line_table_case &case_)
|
||||
char_span lspan = location_get_source_line (tmp.get_filename (), 1);
|
||||
ASSERT_EQ (line_display_cols,
|
||||
cpp_display_width (lspan.get_buffer (), lspan.length (),
|
||||
def_tabstop));
|
||||
def_policy ()));
|
||||
ASSERT_EQ (line_display_cols,
|
||||
location_compute_display_column (expand_location (line_end),
|
||||
def_tabstop));
|
||||
def_policy ()));
|
||||
ASSERT_EQ (0, memcmp (lspan.get_buffer () + (emoji_col - 1),
|
||||
"\xf0\x9f\x98\x82\xf0\x9f\x98\x82", 8));
|
||||
|
||||
@ -2866,12 +3128,13 @@ test_layout_x_offset_display_tab (const line_table_case &case_)
|
||||
ASSERT_EQ ('\t', *(lspan.get_buffer () + (tab_col - 1)));
|
||||
for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
|
||||
{
|
||||
cpp_char_column_policy policy (tabstop, cpp_wcwidth);
|
||||
ASSERT_EQ (line_bytes + extra_width[tabstop],
|
||||
cpp_display_width (lspan.get_buffer (), lspan.length (),
|
||||
tabstop));
|
||||
policy));
|
||||
ASSERT_EQ (line_bytes + extra_width[tabstop],
|
||||
location_compute_display_column (expand_location (line_end),
|
||||
tabstop));
|
||||
policy));
|
||||
}
|
||||
|
||||
/* Check that the tab is expanded to the expected number of spaces. */
|
||||
@ -4003,6 +4266,43 @@ test_one_liner_labels_utf8 ()
|
||||
" bb\xf0\x9f\x98\x82\xf0\x9f\x98\x82\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
|
||||
/* Example of escaping the source lines. */
|
||||
{
|
||||
text_range_label label0 ("label 0\xf0\x9f\x98\x82");
|
||||
text_range_label label1 ("label 1\xcf\x80");
|
||||
text_range_label label2 ("label 2\xcf\x80");
|
||||
gcc_rich_location richloc (foo, &label0);
|
||||
richloc.add_range (bar, SHOW_RANGE_WITHOUT_CARET, &label1);
|
||||
richloc.add_range (field, SHOW_RANGE_WITHOUT_CARET, &label2);
|
||||
richloc.set_escape_on_output (true);
|
||||
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
dc.escape_format = DIAGNOSTICS_ESCAPE_FORMAT_UNICODE;
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
ASSERT_STREQ (" <U+1F602>_foo = <U+03C0>_bar.<U+1F602>_field<U+03C0>;\n"
|
||||
" ^~~~~~~~~~~~~ ~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~\n"
|
||||
" | | |\n"
|
||||
" | | label 2\xcf\x80\n"
|
||||
" | label 1\xcf\x80\n"
|
||||
" label 0\xf0\x9f\x98\x82\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
dc.escape_format = DIAGNOSTICS_ESCAPE_FORMAT_BYTES;
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
ASSERT_STREQ
|
||||
(" <f0><9f><98><82>_foo = <cf><80>_bar.<f0><9f><98><82>_field<cf><80>;\n"
|
||||
" ^~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n"
|
||||
" | | |\n"
|
||||
" | | label 2\xcf\x80\n"
|
||||
" | label 1\xcf\x80\n"
|
||||
" label 0\xf0\x9f\x98\x82\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Make sure that colorization codes don't interrupt a multibyte
|
||||
@ -4057,9 +4357,9 @@ test_diagnostic_show_locus_one_liner_utf8 (const line_table_case &case_)
|
||||
|
||||
char_span lspan = location_get_source_line (tmp.get_filename (), 1);
|
||||
ASSERT_EQ (25, cpp_display_width (lspan.get_buffer (), lspan.length (),
|
||||
def_tabstop));
|
||||
def_policy ()));
|
||||
ASSERT_EQ (25, location_compute_display_column (expand_location (line_end),
|
||||
def_tabstop));
|
||||
def_policy ()));
|
||||
|
||||
test_one_liner_simple_caret_utf8 ();
|
||||
test_one_liner_caret_and_range_utf8 ();
|
||||
@ -4445,30 +4745,31 @@ test_overlapped_fixit_printing (const line_table_case &case_)
|
||||
pp_formatted_text (dc.printer));
|
||||
|
||||
/* Unit-test the line_corrections machinery. */
|
||||
char_display_policy policy (make_policy (dc, richloc));
|
||||
ASSERT_EQ (3, richloc.get_num_fixit_hints ());
|
||||
const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
|
||||
ASSERT_EQ (column_range (12, 12),
|
||||
get_affected_range (&dc, hint_0, CU_BYTES));
|
||||
get_affected_range (policy, hint_0, CU_BYTES));
|
||||
ASSERT_EQ (column_range (12, 12),
|
||||
get_affected_range (&dc, hint_0, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (12, 22), get_printed_columns (&dc, hint_0));
|
||||
get_affected_range (policy, hint_0, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (12, 22), get_printed_columns (policy, hint_0));
|
||||
const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
|
||||
ASSERT_EQ (column_range (18, 18),
|
||||
get_affected_range (&dc, hint_1, CU_BYTES));
|
||||
get_affected_range (policy, hint_1, CU_BYTES));
|
||||
ASSERT_EQ (column_range (18, 18),
|
||||
get_affected_range (&dc, hint_1, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (18, 20), get_printed_columns (&dc, hint_1));
|
||||
get_affected_range (policy, hint_1, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (18, 20), get_printed_columns (policy, hint_1));
|
||||
const fixit_hint *hint_2 = richloc.get_fixit_hint (2);
|
||||
ASSERT_EQ (column_range (29, 28),
|
||||
get_affected_range (&dc, hint_2, CU_BYTES));
|
||||
get_affected_range (policy, hint_2, CU_BYTES));
|
||||
ASSERT_EQ (column_range (29, 28),
|
||||
get_affected_range (&dc, hint_2, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (29, 29), get_printed_columns (&dc, hint_2));
|
||||
get_affected_range (policy, hint_2, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (29, 29), get_printed_columns (policy, hint_2));
|
||||
|
||||
/* Add each hint in turn to a line_corrections instance,
|
||||
and verify that they are consolidated into one correction instance
|
||||
as expected. */
|
||||
line_corrections lc (&dc, tmp.get_filename (), 1);
|
||||
line_corrections lc (policy, tmp.get_filename (), 1);
|
||||
|
||||
/* The first replace hint by itself. */
|
||||
lc.add_hint (hint_0);
|
||||
@ -4660,30 +4961,31 @@ test_overlapped_fixit_printing_utf8 (const line_table_case &case_)
|
||||
pp_formatted_text (dc.printer));
|
||||
|
||||
/* Unit-test the line_corrections machinery. */
|
||||
char_display_policy policy (make_policy (dc, richloc));
|
||||
ASSERT_EQ (3, richloc.get_num_fixit_hints ());
|
||||
const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
|
||||
ASSERT_EQ (column_range (14, 14),
|
||||
get_affected_range (&dc, hint_0, CU_BYTES));
|
||||
get_affected_range (policy, hint_0, CU_BYTES));
|
||||
ASSERT_EQ (column_range (12, 12),
|
||||
get_affected_range (&dc, hint_0, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (12, 22), get_printed_columns (&dc, hint_0));
|
||||
get_affected_range (policy, hint_0, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (12, 22), get_printed_columns (policy, hint_0));
|
||||
const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
|
||||
ASSERT_EQ (column_range (22, 22),
|
||||
get_affected_range (&dc, hint_1, CU_BYTES));
|
||||
get_affected_range (policy, hint_1, CU_BYTES));
|
||||
ASSERT_EQ (column_range (18, 18),
|
||||
get_affected_range (&dc, hint_1, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (18, 20), get_printed_columns (&dc, hint_1));
|
||||
get_affected_range (policy, hint_1, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (18, 20), get_printed_columns (policy, hint_1));
|
||||
const fixit_hint *hint_2 = richloc.get_fixit_hint (2);
|
||||
ASSERT_EQ (column_range (35, 34),
|
||||
get_affected_range (&dc, hint_2, CU_BYTES));
|
||||
get_affected_range (policy, hint_2, CU_BYTES));
|
||||
ASSERT_EQ (column_range (30, 29),
|
||||
get_affected_range (&dc, hint_2, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (30, 30), get_printed_columns (&dc, hint_2));
|
||||
get_affected_range (policy, hint_2, CU_DISPLAY_COLS));
|
||||
ASSERT_EQ (column_range (30, 30), get_printed_columns (policy, hint_2));
|
||||
|
||||
/* Add each hint in turn to a line_corrections instance,
|
||||
and verify that they are consolidated into one correction instance
|
||||
as expected. */
|
||||
line_corrections lc (&dc, tmp.get_filename (), 1);
|
||||
line_corrections lc (policy, tmp.get_filename (), 1);
|
||||
|
||||
/* The first replace hint by itself. */
|
||||
lc.add_hint (hint_0);
|
||||
@ -4877,15 +5179,16 @@ test_overlapped_fixit_printing_2 (const line_table_case &case_)
|
||||
richloc.add_fixit_insert_before (col_21, "}");
|
||||
|
||||
/* These fixits should be accepted; they can't be consolidated. */
|
||||
char_display_policy policy (make_policy (dc, richloc));
|
||||
ASSERT_EQ (2, richloc.get_num_fixit_hints ());
|
||||
const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
|
||||
ASSERT_EQ (column_range (23, 22),
|
||||
get_affected_range (&dc, hint_0, CU_BYTES));
|
||||
ASSERT_EQ (column_range (23, 23), get_printed_columns (&dc, hint_0));
|
||||
get_affected_range (policy, hint_0, CU_BYTES));
|
||||
ASSERT_EQ (column_range (23, 23), get_printed_columns (policy, hint_0));
|
||||
const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
|
||||
ASSERT_EQ (column_range (21, 20),
|
||||
get_affected_range (&dc, hint_1, CU_BYTES));
|
||||
ASSERT_EQ (column_range (21, 21), get_printed_columns (&dc, hint_1));
|
||||
get_affected_range (policy, hint_1, CU_BYTES));
|
||||
ASSERT_EQ (column_range (21, 21), get_printed_columns (policy, hint_1));
|
||||
|
||||
/* Verify that they're printed correctly. */
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
@ -5152,10 +5455,11 @@ test_tab_expansion (const line_table_case &case_)
|
||||
....................123 45678901234 56789012345 columns */
|
||||
|
||||
const int tabstop = 8;
|
||||
cpp_char_column_policy policy (tabstop, cpp_wcwidth);
|
||||
const int first_non_ws_byte_col = 7;
|
||||
const int right_quote_byte_col = 15;
|
||||
const int last_byte_col = 25;
|
||||
ASSERT_EQ (35, cpp_display_width (content, last_byte_col, tabstop));
|
||||
ASSERT_EQ (35, cpp_display_width (content, last_byte_col, policy));
|
||||
|
||||
temp_source_file tmp (SELFTEST_LOCATION, ".c", content);
|
||||
line_table_test ltt (case_);
|
||||
@ -5198,6 +5502,114 @@ test_tab_expansion (const line_table_case &case_)
|
||||
}
|
||||
}
|
||||
|
||||
/* Verify that the escaping machinery can cope with a variety of different
|
||||
invalid bytes. */
|
||||
|
||||
static void
|
||||
test_escaping_bytes_1 (const line_table_case &case_)
|
||||
{
|
||||
const char content[] = "before\0\1\2\3\r\x80\xff""after\n";
|
||||
const size_t sz = sizeof (content);
|
||||
temp_source_file tmp (SELFTEST_LOCATION, ".c", content, sz);
|
||||
line_table_test ltt (case_);
|
||||
const line_map_ordinary *ord_map = linemap_check_ordinary
|
||||
(linemap_add (line_table, LC_ENTER, false, tmp.get_filename (), 0));
|
||||
linemap_line_start (line_table, 1, 100);
|
||||
|
||||
location_t finish
|
||||
= linemap_position_for_line_and_column (line_table, ord_map, 1,
|
||||
strlen (content));
|
||||
|
||||
if (finish > LINE_MAP_MAX_LOCATION_WITH_COLS)
|
||||
return;
|
||||
|
||||
/* Locations of the NUL and \r bytes. */
|
||||
location_t nul_loc
|
||||
= linemap_position_for_line_and_column (line_table, ord_map, 1, 7);
|
||||
location_t r_loc
|
||||
= linemap_position_for_line_and_column (line_table, ord_map, 1, 11);
|
||||
gcc_rich_location richloc (nul_loc);
|
||||
richloc.add_range (r_loc);
|
||||
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
ASSERT_STREQ (" before \1\2\3 \x80\xff""after\n"
|
||||
" ^ ~\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
richloc.set_escape_on_output (true);
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
dc.escape_format = DIAGNOSTICS_ESCAPE_FORMAT_UNICODE;
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
ASSERT_STREQ
|
||||
(" before<U+0000><U+0001><U+0002><U+0003><U+000D><80><ff>after\n"
|
||||
" ^~~~~~~~ ~~~~~~~~\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
dc.escape_format = DIAGNOSTICS_ESCAPE_FORMAT_BYTES;
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
ASSERT_STREQ (" before<00><01><02><03><0d><80><ff>after\n"
|
||||
" ^~~~ ~~~~\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
}
|
||||
|
||||
/* As above, but verify that we handle the initial byte of a line
|
||||
correctly. */
|
||||
|
||||
static void
|
||||
test_escaping_bytes_2 (const line_table_case &case_)
|
||||
{
|
||||
const char content[] = "\0after\n";
|
||||
const size_t sz = sizeof (content);
|
||||
temp_source_file tmp (SELFTEST_LOCATION, ".c", content, sz);
|
||||
line_table_test ltt (case_);
|
||||
const line_map_ordinary *ord_map = linemap_check_ordinary
|
||||
(linemap_add (line_table, LC_ENTER, false, tmp.get_filename (), 0));
|
||||
linemap_line_start (line_table, 1, 100);
|
||||
|
||||
location_t finish
|
||||
= linemap_position_for_line_and_column (line_table, ord_map, 1,
|
||||
strlen (content));
|
||||
|
||||
if (finish > LINE_MAP_MAX_LOCATION_WITH_COLS)
|
||||
return;
|
||||
|
||||
/* Location of the NUL byte. */
|
||||
location_t nul_loc
|
||||
= linemap_position_for_line_and_column (line_table, ord_map, 1, 1);
|
||||
gcc_rich_location richloc (nul_loc);
|
||||
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
ASSERT_STREQ (" after\n"
|
||||
" ^\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
richloc.set_escape_on_output (true);
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
dc.escape_format = DIAGNOSTICS_ESCAPE_FORMAT_UNICODE;
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
ASSERT_STREQ (" <U+0000>after\n"
|
||||
" ^~~~~~~~\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
{
|
||||
test_diagnostic_context dc;
|
||||
dc.escape_format = DIAGNOSTICS_ESCAPE_FORMAT_BYTES;
|
||||
diagnostic_show_locus (&dc, &richloc, DK_ERROR);
|
||||
ASSERT_STREQ (" <00>after\n"
|
||||
" ^~~~\n",
|
||||
pp_formatted_text (dc.printer));
|
||||
}
|
||||
}
|
||||
|
||||
/* Verify that line numbers are correctly printed for the case of
|
||||
a multiline range in which the width of the line numbers changes
|
||||
(e.g. from "9" to "10"). */
|
||||
@ -5254,6 +5666,8 @@ diagnostic_show_locus_c_tests ()
|
||||
test_layout_range_for_single_line ();
|
||||
test_layout_range_for_multiple_lines ();
|
||||
|
||||
test_display_widths ();
|
||||
|
||||
for_each_line_table_case (test_layout_x_offset_display_utf8);
|
||||
for_each_line_table_case (test_layout_x_offset_display_tab);
|
||||
|
||||
@ -5274,6 +5688,8 @@ diagnostic_show_locus_c_tests ()
|
||||
for_each_line_table_case (test_fixit_replace_containing_newline);
|
||||
for_each_line_table_case (test_fixit_deletion_affecting_newline);
|
||||
for_each_line_table_case (test_tab_expansion);
|
||||
for_each_line_table_case (test_escaping_bytes_1);
|
||||
for_each_line_table_case (test_escaping_bytes_2);
|
||||
|
||||
test_line_numbers_multiline_range ();
|
||||
}
|
||||
|
||||
@ -230,6 +230,7 @@ diagnostic_initialize (diagnostic_context *context, int n_opts)
|
||||
context->column_unit = DIAGNOSTICS_COLUMN_UNIT_DISPLAY;
|
||||
context->column_origin = 1;
|
||||
context->tabstop = 8;
|
||||
context->escape_format = DIAGNOSTICS_ESCAPE_FORMAT_UNICODE;
|
||||
context->edit_context_ptr = NULL;
|
||||
context->diagnostic_group_nesting_depth = 0;
|
||||
context->diagnostic_group_emission_count = 0;
|
||||
@ -393,7 +394,10 @@ convert_column_unit (enum diagnostics_column_unit column_unit,
|
||||
gcc_unreachable ();
|
||||
|
||||
case DIAGNOSTICS_COLUMN_UNIT_DISPLAY:
|
||||
return location_compute_display_column (s, tabstop);
|
||||
{
|
||||
cpp_char_column_policy policy (tabstop, cpp_wcwidth);
|
||||
return location_compute_display_column (s, policy);
|
||||
}
|
||||
|
||||
case DIAGNOSTICS_COLUMN_UNIT_BYTE:
|
||||
return s.column;
|
||||
@ -2373,8 +2377,8 @@ test_diagnostic_get_location_text ()
|
||||
const char *const content = "smile \xf0\x9f\x98\x82\n";
|
||||
const int line_bytes = strlen (content) - 1;
|
||||
const int def_tabstop = 8;
|
||||
const int display_width = cpp_display_width (content, line_bytes,
|
||||
def_tabstop);
|
||||
const cpp_char_column_policy policy (def_tabstop, cpp_wcwidth);
|
||||
const int display_width = cpp_display_width (content, line_bytes, policy);
|
||||
ASSERT_EQ (line_bytes - 2, display_width);
|
||||
temp_source_file tmp (SELFTEST_LOCATION, ".c", content);
|
||||
const char *const fname = tmp.get_filename ();
|
||||
|
||||
@ -38,6 +38,20 @@ enum diagnostics_column_unit
|
||||
DIAGNOSTICS_COLUMN_UNIT_BYTE
|
||||
};
|
||||
|
||||
/* An enum for controlling how to print non-ASCII characters/bytes when
|
||||
a diagnostic suggests escaping the source code on output. */
|
||||
|
||||
enum diagnostics_escape_format
|
||||
{
|
||||
/* Escape non-ASCII Unicode characters in the form <U+XXXX> and
|
||||
non-UTF-8 bytes in the form <XX>. */
|
||||
DIAGNOSTICS_ESCAPE_FORMAT_UNICODE,
|
||||
|
||||
/* Escape non-ASCII bytes in the form <XX> (thus showing the underlying
|
||||
encoding of non-ASCII Unicode characters). */
|
||||
DIAGNOSTICS_ESCAPE_FORMAT_BYTES
|
||||
};
|
||||
|
||||
/* Enum for overriding the standard output format. */
|
||||
|
||||
enum diagnostics_output_format
|
||||
@ -340,6 +354,10 @@ struct diagnostic_context
|
||||
/* The size of the tabstop for tab expansion. */
|
||||
int tabstop;
|
||||
|
||||
/* How should non-ASCII/non-printable bytes be escaped when
|
||||
a diagnostic suggests escaping the source code on output. */
|
||||
enum diagnostics_escape_format escape_format;
|
||||
|
||||
/* If non-NULL, an edit_context to which fix-it hints should be
|
||||
applied, for generating patches. */
|
||||
edit_context *edit_context_ptr;
|
||||
|
||||
@ -312,7 +312,8 @@ Objective-C and Objective-C++ Dialects}.
|
||||
-fdiagnostics-show-path-depths @gol
|
||||
-fno-show-column @gol
|
||||
-fdiagnostics-column-unit=@r{[}display@r{|}byte@r{]} @gol
|
||||
-fdiagnostics-column-origin=@var{origin}}
|
||||
-fdiagnostics-column-origin=@var{origin} @gol
|
||||
-fdiagnostics-escape-format=@r{[}unicode@r{|}bytes@r{]}}
|
||||
|
||||
@item Warning Options
|
||||
@xref{Warning Options,,Options to Request or Suppress Warnings}.
|
||||
@ -5126,6 +5127,38 @@ first column. The default value of 1 corresponds to traditional GCC
|
||||
behavior and to the GNU style guide. Some utilities may perform better with an
|
||||
origin of 0; any non-negative value may be specified.
|
||||
|
||||
@item -fdiagnostics-escape-format=@var{FORMAT}
|
||||
@opindex fdiagnostics-escape-format
|
||||
When GCC prints pertinent source lines for a diagnostic it normally attempts
|
||||
to print the source bytes directly. However, some diagnostics relate to encoding
|
||||
issues in the source file, such as malformed UTF-8, or issues with Unicode
|
||||
normalization. These diagnostics are flagged so that GCC will escape bytes
|
||||
that are not printable ASCII when printing their pertinent source lines.
|
||||
|
||||
This option controls how such bytes should be escaped.
|
||||
|
||||
The default @var{FORMAT}, @samp{unicode} displays Unicode characters that
|
||||
are not printable ASCII in the form @samp{<U+XXXX>}, and bytes that do not
|
||||
correspond to a Unicode character validly-encoded in UTF-8-encoded will be
|
||||
displayed as hexadecimal in the form @samp{<XX>}.
|
||||
|
||||
For example, a source line containing the string @samp{before} followed by the
|
||||
Unicode character U+03C0 (``GREEK SMALL LETTER PI'', with UTF-8 encoding
|
||||
0xCF 0x80) followed by the byte 0xBF (a stray UTF-8 trailing byte), followed by
|
||||
the string @samp{after} will be printed for such a diagnostic as:
|
||||
|
||||
@smallexample
|
||||
before<U+03C0><BF>after
|
||||
@end smallexample
|
||||
|
||||
Setting @var{FORMAT} to @samp{bytes} will display all non-printable-ASCII bytes
|
||||
in the form @samp{<XX>}, thus showing the underlying encoding of non-ASCII
|
||||
Unicode characters. For the example above, the following will be printed:
|
||||
|
||||
@smallexample
|
||||
before<CF><80><BF>after
|
||||
@end smallexample
|
||||
|
||||
@item -fdiagnostics-format=@var{FORMAT}
|
||||
@opindex fdiagnostics-format
|
||||
Select a different format for printing diagnostics.
|
||||
@ -5193,9 +5226,11 @@ might be printed in JSON form (after formatting) like this:
|
||||
@}
|
||||
@}
|
||||
],
|
||||
"escape-source": false,
|
||||
"message": "...this statement, but the latter is @dots{}"
|
||||
@}
|
||||
]
|
||||
"escape-source": false,
|
||||
"column-origin": 1,
|
||||
@}
|
||||
]
|
||||
@ -5281,6 +5316,7 @@ of the expression, which have labels. It might be printed in JSON form as:
|
||||
"label": "T @{aka struct t@}"
|
||||
@}
|
||||
],
|
||||
"escape-source": false,
|
||||
"message": "invalid operands to binary + @dots{}"
|
||||
@}
|
||||
@end smallexample
|
||||
@ -5334,6 +5370,7 @@ might be printed in JSON form as:
|
||||
@}
|
||||
@}
|
||||
],
|
||||
"escape-source": false,
|
||||
"message": "\u2018struct s\u2019 has no member named @dots{}"
|
||||
@}
|
||||
@end smallexample
|
||||
@ -5391,6 +5428,10 @@ For example, the intraprocedural example shown for
|
||||
]
|
||||
@end smallexample
|
||||
|
||||
Diagnostics have a boolean attribute @code{escape-source}, hinting whether
|
||||
non-ASCII bytes should be escaped when printing the pertinent lines of
|
||||
source code (@code{true} for diagnostics involving source encoding issues).
|
||||
|
||||
@end table
|
||||
|
||||
@node Warning Options
|
||||
|
||||
62
gcc/input.c
62
gcc/input.c
@ -1060,7 +1060,8 @@ make_location (location_t caret, source_range src_range)
|
||||
source line in order to calculate the display width. If that cannot be done
|
||||
for any reason, then returns the byte column as a fallback. */
|
||||
int
|
||||
location_compute_display_column (expanded_location exploc, int tabstop)
|
||||
location_compute_display_column (expanded_location exploc,
|
||||
const cpp_char_column_policy &policy)
|
||||
{
|
||||
if (!(exploc.file && *exploc.file && exploc.line && exploc.column))
|
||||
return exploc.column;
|
||||
@ -1068,7 +1069,7 @@ location_compute_display_column (expanded_location exploc, int tabstop)
|
||||
/* If line is NULL, this function returns exploc.column which is the
|
||||
desired fallback. */
|
||||
return cpp_byte_column_to_display_column (line.get_buffer (), line.length (),
|
||||
exploc.column, tabstop);
|
||||
exploc.column, policy);
|
||||
}
|
||||
|
||||
/* Dump statistics to stderr about the memory usage of the line_table
|
||||
@ -3767,43 +3768,50 @@ test_line_offset_overflow ()
|
||||
void test_cpp_utf8 ()
|
||||
{
|
||||
const int def_tabstop = 8;
|
||||
cpp_char_column_policy policy (def_tabstop, cpp_wcwidth);
|
||||
|
||||
/* Verify that wcwidth of invalid UTF-8 or control bytes is 1. */
|
||||
{
|
||||
int w_bad = cpp_display_width ("\xf0!\x9f!\x98!\x82!", 8, def_tabstop);
|
||||
int w_bad = cpp_display_width ("\xf0!\x9f!\x98!\x82!", 8, policy);
|
||||
ASSERT_EQ (8, w_bad);
|
||||
int w_ctrl = cpp_display_width ("\r\n\v\0\1", 5, def_tabstop);
|
||||
int w_ctrl = cpp_display_width ("\r\n\v\0\1", 5, policy);
|
||||
ASSERT_EQ (5, w_ctrl);
|
||||
}
|
||||
|
||||
/* Verify that wcwidth of valid UTF-8 is as expected. */
|
||||
{
|
||||
const int w_pi = cpp_display_width ("\xcf\x80", 2, def_tabstop);
|
||||
const int w_pi = cpp_display_width ("\xcf\x80", 2, policy);
|
||||
ASSERT_EQ (1, w_pi);
|
||||
const int w_emoji = cpp_display_width ("\xf0\x9f\x98\x82", 4, def_tabstop);
|
||||
const int w_emoji = cpp_display_width ("\xf0\x9f\x98\x82", 4, policy);
|
||||
ASSERT_EQ (2, w_emoji);
|
||||
const int w_umlaut_precomposed = cpp_display_width ("\xc3\xbf", 2,
|
||||
def_tabstop);
|
||||
policy);
|
||||
ASSERT_EQ (1, w_umlaut_precomposed);
|
||||
const int w_umlaut_combining = cpp_display_width ("y\xcc\x88", 3,
|
||||
def_tabstop);
|
||||
policy);
|
||||
ASSERT_EQ (1, w_umlaut_combining);
|
||||
const int w_han = cpp_display_width ("\xe4\xb8\xba", 3, def_tabstop);
|
||||
const int w_han = cpp_display_width ("\xe4\xb8\xba", 3, policy);
|
||||
ASSERT_EQ (2, w_han);
|
||||
const int w_ascii = cpp_display_width ("GCC", 3, def_tabstop);
|
||||
const int w_ascii = cpp_display_width ("GCC", 3, policy);
|
||||
ASSERT_EQ (3, w_ascii);
|
||||
const int w_mixed = cpp_display_width ("\xcf\x80 = 3.14 \xf0\x9f\x98\x82"
|
||||
"\x9f! \xe4\xb8\xba y\xcc\x88",
|
||||
24, def_tabstop);
|
||||
24, policy);
|
||||
ASSERT_EQ (18, w_mixed);
|
||||
}
|
||||
|
||||
/* Verify that display width properly expands tabs. */
|
||||
{
|
||||
const char *tstr = "\tabc\td";
|
||||
ASSERT_EQ (6, cpp_display_width (tstr, 6, 1));
|
||||
ASSERT_EQ (10, cpp_display_width (tstr, 6, 3));
|
||||
ASSERT_EQ (17, cpp_display_width (tstr, 6, 8));
|
||||
ASSERT_EQ (1, cpp_display_column_to_byte_column (tstr, 6, 7, 8));
|
||||
ASSERT_EQ (6, cpp_display_width (tstr, 6,
|
||||
cpp_char_column_policy (1, cpp_wcwidth)));
|
||||
ASSERT_EQ (10, cpp_display_width (tstr, 6,
|
||||
cpp_char_column_policy (3, cpp_wcwidth)));
|
||||
ASSERT_EQ (17, cpp_display_width (tstr, 6,
|
||||
cpp_char_column_policy (8, cpp_wcwidth)));
|
||||
ASSERT_EQ (1,
|
||||
cpp_display_column_to_byte_column
|
||||
(tstr, 6, 7, cpp_char_column_policy (8, cpp_wcwidth)));
|
||||
}
|
||||
|
||||
/* Verify that cpp_byte_column_to_display_column can go past the end,
|
||||
@ -3816,13 +3824,13 @@ void test_cpp_utf8 ()
|
||||
/* 111122223456
|
||||
Byte columns. */
|
||||
|
||||
ASSERT_EQ (5, cpp_display_width (str, 6, def_tabstop));
|
||||
ASSERT_EQ (5, cpp_display_width (str, 6, policy));
|
||||
ASSERT_EQ (105,
|
||||
cpp_byte_column_to_display_column (str, 6, 106, def_tabstop));
|
||||
cpp_byte_column_to_display_column (str, 6, 106, policy));
|
||||
ASSERT_EQ (10000,
|
||||
cpp_byte_column_to_display_column (NULL, 0, 10000, def_tabstop));
|
||||
cpp_byte_column_to_display_column (NULL, 0, 10000, policy));
|
||||
ASSERT_EQ (0,
|
||||
cpp_byte_column_to_display_column (NULL, 10000, 0, def_tabstop));
|
||||
cpp_byte_column_to_display_column (NULL, 10000, 0, policy));
|
||||
}
|
||||
|
||||
/* Verify that cpp_display_column_to_byte_column can go past the end,
|
||||
@ -3836,25 +3844,25 @@ void test_cpp_utf8 ()
|
||||
/* 000000000000000000000000000000000111111
|
||||
111122223333444456666777788889999012345
|
||||
Byte columns. */
|
||||
ASSERT_EQ (4, cpp_display_column_to_byte_column (str, 15, 2, def_tabstop));
|
||||
ASSERT_EQ (4, cpp_display_column_to_byte_column (str, 15, 2, policy));
|
||||
ASSERT_EQ (15,
|
||||
cpp_display_column_to_byte_column (str, 15, 11, def_tabstop));
|
||||
cpp_display_column_to_byte_column (str, 15, 11, policy));
|
||||
ASSERT_EQ (115,
|
||||
cpp_display_column_to_byte_column (str, 15, 111, def_tabstop));
|
||||
cpp_display_column_to_byte_column (str, 15, 111, policy));
|
||||
ASSERT_EQ (10000,
|
||||
cpp_display_column_to_byte_column (NULL, 0, 10000, def_tabstop));
|
||||
cpp_display_column_to_byte_column (NULL, 0, 10000, policy));
|
||||
ASSERT_EQ (0,
|
||||
cpp_display_column_to_byte_column (NULL, 10000, 0, def_tabstop));
|
||||
cpp_display_column_to_byte_column (NULL, 10000, 0, policy));
|
||||
|
||||
/* Verify that we do not interrupt a UTF-8 sequence. */
|
||||
ASSERT_EQ (4, cpp_display_column_to_byte_column (str, 15, 1, def_tabstop));
|
||||
ASSERT_EQ (4, cpp_display_column_to_byte_column (str, 15, 1, policy));
|
||||
|
||||
for (int byte_col = 1; byte_col <= 15; ++byte_col)
|
||||
{
|
||||
const int disp_col
|
||||
= cpp_byte_column_to_display_column (str, 15, byte_col, def_tabstop);
|
||||
= cpp_byte_column_to_display_column (str, 15, byte_col, policy);
|
||||
const int byte_col2
|
||||
= cpp_display_column_to_byte_column (str, 15, disp_col, def_tabstop);
|
||||
= cpp_display_column_to_byte_column (str, 15, disp_col, policy);
|
||||
|
||||
/* If we ask for the display column in the middle of a UTF-8
|
||||
sequence, it will return the length of the partial sequence,
|
||||
|
||||
@ -39,8 +39,11 @@ STATIC_ASSERT (BUILTINS_LOCATION < RESERVED_LOCATION_COUNT);
|
||||
extern bool is_location_from_builtin_token (location_t);
|
||||
extern expanded_location expand_location (location_t);
|
||||
|
||||
extern int location_compute_display_column (expanded_location exploc,
|
||||
int tabstop);
|
||||
class cpp_char_column_policy;
|
||||
|
||||
extern int
|
||||
location_compute_display_column (expanded_location exploc,
|
||||
const cpp_char_column_policy &policy);
|
||||
|
||||
/* A class capturing the bounds of a buffer, to allow for run-time
|
||||
bounds-checking in a checked build. */
|
||||
|
||||
@ -2734,6 +2734,10 @@ common_handle_option (struct gcc_options *opts,
|
||||
dc->column_origin = value;
|
||||
break;
|
||||
|
||||
case OPT_fdiagnostics_escape_format_:
|
||||
dc->escape_format = (enum diagnostics_escape_format)value;
|
||||
break;
|
||||
|
||||
case OPT_fdiagnostics_show_cwe:
|
||||
dc->show_cwe = value;
|
||||
break;
|
||||
|
||||
@ -192,6 +192,21 @@ temp_source_file::temp_source_file (const location &loc,
|
||||
fclose (out);
|
||||
}
|
||||
|
||||
/* As above, but with a size, to allow for NUL bytes in CONTENT. */
|
||||
|
||||
temp_source_file::temp_source_file (const location &loc,
|
||||
const char *suffix,
|
||||
const char *content,
|
||||
size_t sz)
|
||||
: named_temp_file (suffix)
|
||||
{
|
||||
FILE *out = fopen (get_filename (), "w");
|
||||
if (!out)
|
||||
fail_formatted (loc, "unable to open tempfile: %s", get_filename ());
|
||||
fwrite (content, sz, 1, out);
|
||||
fclose (out);
|
||||
}
|
||||
|
||||
/* Avoid introducing locale-specific differences in the results
|
||||
by hardcoding open_quote and close_quote. */
|
||||
|
||||
|
||||
@ -112,6 +112,8 @@ class temp_source_file : public named_temp_file
|
||||
public:
|
||||
temp_source_file (const location &loc, const char *suffix,
|
||||
const char *content);
|
||||
temp_source_file (const location &loc, const char *suffix,
|
||||
const char *content, size_t sz);
|
||||
};
|
||||
|
||||
/* RAII-style class for avoiding introducing locale-specific differences
|
||||
|
||||
@ -9,6 +9,7 @@
|
||||
|
||||
/* { dg-regexp "\"kind\": \"error\"" } */
|
||||
/* { dg-regexp "\"column-origin\": 1" } */
|
||||
/* { dg-regexp "\"escape-source\": false" } */
|
||||
/* { dg-regexp "\"message\": \"#error message\"" } */
|
||||
|
||||
/* { dg-regexp "\"caret\": \{" } */
|
||||
|
||||
@ -9,6 +9,7 @@
|
||||
|
||||
/* { dg-regexp "\"kind\": \"warning\"" } */
|
||||
/* { dg-regexp "\"column-origin\": 1" } */
|
||||
/* { dg-regexp "\"escape-source\": false" } */
|
||||
/* { dg-regexp "\"message\": \"#warning message\"" } */
|
||||
/* { dg-regexp "\"option\": \"-Wcpp\"" } */
|
||||
/* { dg-regexp "\"option_url\": \"https:\[^\n\r\"\]*#index-Wcpp\"" } */
|
||||
|
||||
@ -9,6 +9,7 @@
|
||||
|
||||
/* { dg-regexp "\"kind\": \"error\"" } */
|
||||
/* { dg-regexp "\"column-origin\": 1" } */
|
||||
/* { dg-regexp "\"escape-source\": false" } */
|
||||
/* { dg-regexp "\"message\": \"#warning message\"" } */
|
||||
/* { dg-regexp "\"option\": \"-Werror=cpp\"" } */
|
||||
/* { dg-regexp "\"option_url\": \"https:\[^\n\r\"\]*#index-Wcpp\"" } */
|
||||
|
||||
@ -19,6 +19,7 @@ int test (void)
|
||||
|
||||
/* { dg-regexp "\"kind\": \"note\"" } */
|
||||
/* { dg-regexp "\"message\": \"...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'\"" } */
|
||||
/* { dg-regexp "\"escape-source\": false" } */
|
||||
|
||||
/* { dg-regexp "\"caret\": \{" } */
|
||||
/* { dg-regexp "\"file\": \"\[^\n\r\"\]*diagnostic-format-json-4.c\"" } */
|
||||
@ -39,6 +40,7 @@ int test (void)
|
||||
/* { dg-regexp "\"kind\": \"warning\"" } */
|
||||
/* { dg-regexp "\"column-origin\": 1" } */
|
||||
/* { dg-regexp "\"message\": \"this 'if' clause does not guard...\"" } */
|
||||
/* { dg-regexp "\"escape-source\": false" } */
|
||||
/* { dg-regexp "\"option\": \"-Wmisleading-indentation\"" } */
|
||||
/* { dg-regexp "\"option_url\": \"https:\[^\n\r\"\]*#index-Wmisleading-indentation\"" } */
|
||||
|
||||
|
||||
@ -14,6 +14,7 @@ int test (struct s *ptr)
|
||||
|
||||
/* { dg-regexp "\"kind\": \"error\"" } */
|
||||
/* { dg-regexp "\"column-origin\": 1" } */
|
||||
/* { dg-regexp "\"escape-source\": false" } */
|
||||
/* { dg-regexp "\"message\": \".*\"" } */
|
||||
|
||||
/* Verify fix-it hints. */
|
||||
|
||||
21
gcc/testsuite/gcc.dg/cpp/warn-normalized-4-bytes.c
Normal file
21
gcc/testsuite/gcc.dg/cpp/warn-normalized-4-bytes.c
Normal file
@ -0,0 +1,21 @@
|
||||
// { dg-do preprocess }
|
||||
// { dg-options "-std=gnu99 -Werror=normalized=nfc -fdiagnostics-show-caret -fdiagnostics-escape-format=bytes" }
|
||||
/* { dg-message "some warnings being treated as errors" "" {target "*-*-*"} 0 } */
|
||||
|
||||
/* གྷ = U+0F43 TIBETAN LETTER GHA, which has decomposition "0F42 0FB7" i.e.
|
||||
U+0F42 TIBETAN LETTER GA: ག
|
||||
U+0FB7 TIBETAN SUBJOINED LETTER HA: ྷ
|
||||
|
||||
The UTF-8 encoding of U+0F43 TIBETAN LETTER GHA is: E0 BD 83. */
|
||||
|
||||
foo before_\u0F43_after bar // { dg-error "`before_.U00000f43_after' is not in NFC .-Werror=normalized=." }
|
||||
/* { dg-begin-multiline-output "" }
|
||||
foo before_\u0F43_after bar
|
||||
^~~~~~~~~~~~~~~~~~~
|
||||
{ dg-end-multiline-output "" } */
|
||||
|
||||
foo before_གྷ_after bar // { dg-error "`before_.U00000f43_after' is not in NFC .-Werror=normalized=." }
|
||||
/* { dg-begin-multiline-output "" }
|
||||
foo before_<e0><bd><83>_after bar
|
||||
^~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
{ dg-end-multiline-output "" } */
|
||||
19
gcc/testsuite/gcc.dg/cpp/warn-normalized-4-unicode.c
Normal file
19
gcc/testsuite/gcc.dg/cpp/warn-normalized-4-unicode.c
Normal file
@ -0,0 +1,19 @@
|
||||
// { dg-do preprocess }
|
||||
// { dg-options "-std=gnu99 -Werror=normalized=nfc -fdiagnostics-show-caret -fdiagnostics-escape-format=unicode" }
|
||||
/* { dg-message "some warnings being treated as errors" "" {target "*-*-*"} 0 } */
|
||||
|
||||
/* གྷ = U+0F43 TIBETAN LETTER GHA, which has decomposition "0F42 0FB7" i.e.
|
||||
U+0F42 TIBETAN LETTER GA: ག
|
||||
U+0FB7 TIBETAN SUBJOINED LETTER HA: ྷ */
|
||||
|
||||
foo before_\u0F43_after bar // { dg-error "`before_.U00000f43_after' is not in NFC .-Werror=normalized=." }
|
||||
/* { dg-begin-multiline-output "" }
|
||||
foo before_\u0F43_after bar
|
||||
^~~~~~~~~~~~~~~~~~~
|
||||
{ dg-end-multiline-output "" } */
|
||||
|
||||
foo before_གྷ_after bar // { dg-error "`before_.U00000f43_after' is not in NFC .-Werror=normalized=." }
|
||||
/* { dg-begin-multiline-output "" }
|
||||
foo before_<U+0F43>_after bar
|
||||
^~~~~~~~~~~~~~~~~~~~~
|
||||
{ dg-end-multiline-output "" } */
|
||||
BIN
gcc/testsuite/gcc.dg/encoding-issues-bytes.c
Normal file
BIN
gcc/testsuite/gcc.dg/encoding-issues-bytes.c
Normal file
Binary file not shown.
BIN
gcc/testsuite/gcc.dg/encoding-issues-unicode.c
Normal file
BIN
gcc/testsuite/gcc.dg/encoding-issues-unicode.c
Normal file
Binary file not shown.
@ -9,6 +9,7 @@
|
||||
|
||||
! { dg-regexp "\"kind\": \"error\"" }
|
||||
! { dg-regexp "\"column-origin\": 1" }
|
||||
! { dg-regexp "\"escape-source\": false" }
|
||||
! { dg-regexp "\"message\": \"#error message\"" }
|
||||
|
||||
! { dg-regexp "\"caret\": \{" }
|
||||
|
||||
@ -9,6 +9,7 @@
|
||||
|
||||
! { dg-regexp "\"kind\": \"warning\"" }
|
||||
! { dg-regexp "\"column-origin\": 1" }
|
||||
! { dg-regexp "\"escape-source\": false" }
|
||||
! { dg-regexp "\"message\": \"#warning message\"" }
|
||||
! { dg-regexp "\"option\": \"-Wcpp\"" }
|
||||
! { dg-regexp "\"option_url\": \"\[^\n\r\"\]*#index-Wcpp\"" }
|
||||
|
||||
@ -9,6 +9,7 @@
|
||||
|
||||
! { dg-regexp "\"kind\": \"error\"" }
|
||||
! { dg-regexp "\"column-origin\": 1" }
|
||||
! { dg-regexp "\"escape-source\": false" }
|
||||
! { dg-regexp "\"message\": \"#warning message\"" }
|
||||
! { dg-regexp "\"option\": \"-Werror=cpp\"" }
|
||||
! { dg-regexp "\"option_url\": \"\[^\n\r\"\]*#index-Wcpp\"" }
|
||||
|
||||
@ -1582,12 +1582,14 @@ convert_escape (cpp_reader *pfile, const uchar *from, const uchar *limit,
|
||||
"unknown escape sequence: '\\%c'", (int) c);
|
||||
else
|
||||
{
|
||||
encoding_rich_location rich_loc (pfile);
|
||||
|
||||
/* diagnostic.c does not support "%03o". When it does, this
|
||||
code can use %03o directly in the diagnostic again. */
|
||||
char buf[32];
|
||||
sprintf(buf, "%03o", (int) c);
|
||||
cpp_error (pfile, CPP_DL_PEDWARN,
|
||||
"unknown escape sequence: '\\%s'", buf);
|
||||
cpp_error_at (pfile, CPP_DL_PEDWARN, &rich_loc,
|
||||
"unknown escape sequence: '\\%s'", buf);
|
||||
}
|
||||
}
|
||||
|
||||
@ -2345,14 +2347,16 @@ cpp_string_location_reader::get_next ()
|
||||
}
|
||||
|
||||
cpp_display_width_computation::
|
||||
cpp_display_width_computation (const char *data, int data_length, int tabstop) :
|
||||
cpp_display_width_computation (const char *data, int data_length,
|
||||
const cpp_char_column_policy &policy) :
|
||||
m_begin (data),
|
||||
m_next (m_begin),
|
||||
m_bytes_left (data_length),
|
||||
m_tabstop (tabstop),
|
||||
m_policy (policy),
|
||||
m_display_cols (0)
|
||||
{
|
||||
gcc_assert (m_tabstop > 0);
|
||||
gcc_assert (policy.m_tabstop > 0);
|
||||
gcc_assert (policy.m_width_cb);
|
||||
}
|
||||
|
||||
|
||||
@ -2364,19 +2368,28 @@ cpp_display_width_computation (const char *data, int data_length, int tabstop) :
|
||||
point to a valid UTF-8-encoded sequence, then it will be treated as a single
|
||||
byte with display width 1. m_cur_display_col is the current display column,
|
||||
relative to which tab stops should be expanded. Returns the display width of
|
||||
the codepoint just processed. */
|
||||
the codepoint just processed.
|
||||
If OUT is non-NULL, it is populated. */
|
||||
|
||||
int
|
||||
cpp_display_width_computation::process_next_codepoint ()
|
||||
cpp_display_width_computation::process_next_codepoint (cpp_decoded_char *out)
|
||||
{
|
||||
cppchar_t c;
|
||||
int next_width;
|
||||
|
||||
if (out)
|
||||
out->m_start_byte = m_next;
|
||||
|
||||
if (*m_next == '\t')
|
||||
{
|
||||
++m_next;
|
||||
--m_bytes_left;
|
||||
next_width = m_tabstop - (m_display_cols % m_tabstop);
|
||||
next_width = m_policy.m_tabstop - (m_display_cols % m_policy.m_tabstop);
|
||||
if (out)
|
||||
{
|
||||
out->m_ch = '\t';
|
||||
out->m_valid_ch = true;
|
||||
}
|
||||
}
|
||||
else if (one_utf8_to_cppchar ((const uchar **) &m_next, &m_bytes_left, &c)
|
||||
!= 0)
|
||||
@ -2386,14 +2399,24 @@ cpp_display_width_computation::process_next_codepoint ()
|
||||
of one. */
|
||||
++m_next;
|
||||
--m_bytes_left;
|
||||
next_width = 1;
|
||||
next_width = m_policy.m_undecoded_byte_width;
|
||||
if (out)
|
||||
out->m_valid_ch = false;
|
||||
}
|
||||
else
|
||||
{
|
||||
/* one_utf8_to_cppchar() has updated m_next and m_bytes_left for us. */
|
||||
next_width = cpp_wcwidth (c);
|
||||
next_width = m_policy.m_width_cb (c);
|
||||
if (out)
|
||||
{
|
||||
out->m_ch = c;
|
||||
out->m_valid_ch = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (out)
|
||||
out->m_next_byte = m_next;
|
||||
|
||||
m_display_cols += next_width;
|
||||
return next_width;
|
||||
}
|
||||
@ -2409,7 +2432,7 @@ cpp_display_width_computation::advance_display_cols (int n)
|
||||
const int start = m_display_cols;
|
||||
const int target = start + n;
|
||||
while (m_display_cols < target && !done ())
|
||||
process_next_codepoint ();
|
||||
process_next_codepoint (NULL);
|
||||
return m_display_cols - start;
|
||||
}
|
||||
|
||||
@ -2417,29 +2440,33 @@ cpp_display_width_computation::advance_display_cols (int n)
|
||||
how many display columns are occupied by the first COLUMN bytes. COLUMN
|
||||
may exceed DATA_LENGTH, in which case the phantom bytes at the end are
|
||||
treated as if they have display width 1. Tabs are expanded to the next tab
|
||||
stop, relative to the start of DATA. */
|
||||
stop, relative to the start of DATA, and non-printable-ASCII characters
|
||||
will be escaped as per POLICY. */
|
||||
|
||||
int
|
||||
cpp_byte_column_to_display_column (const char *data, int data_length,
|
||||
int column, int tabstop)
|
||||
int column,
|
||||
const cpp_char_column_policy &policy)
|
||||
{
|
||||
const int offset = MAX (0, column - data_length);
|
||||
cpp_display_width_computation dw (data, column - offset, tabstop);
|
||||
cpp_display_width_computation dw (data, column - offset, policy);
|
||||
while (!dw.done ())
|
||||
dw.process_next_codepoint ();
|
||||
dw.process_next_codepoint (NULL);
|
||||
return dw.display_cols_processed () + offset;
|
||||
}
|
||||
|
||||
/* For the string of length DATA_LENGTH bytes that begins at DATA, compute
|
||||
the least number of bytes that will result in at least DISPLAY_COL display
|
||||
columns. The return value may exceed DATA_LENGTH if the entire string does
|
||||
not occupy enough display columns. */
|
||||
not occupy enough display columns. Non-printable-ASCII characters
|
||||
will be escaped as per POLICY. */
|
||||
|
||||
int
|
||||
cpp_display_column_to_byte_column (const char *data, int data_length,
|
||||
int display_col, int tabstop)
|
||||
int display_col,
|
||||
const cpp_char_column_policy &policy)
|
||||
{
|
||||
cpp_display_width_computation dw (data, data_length, tabstop);
|
||||
cpp_display_width_computation dw (data, data_length, policy);
|
||||
const int avail_display = dw.advance_display_cols (display_col);
|
||||
return dw.bytes_processed () + MAX (0, display_col - avail_display);
|
||||
}
|
||||
|
||||
@ -27,6 +27,31 @@ along with this program; see the file COPYING3. If not see
|
||||
#include "cpplib.h"
|
||||
#include "internal.h"
|
||||
|
||||
/* Get a location_t for the current location in PFILE,
|
||||
generally that of the previously lexed token. */
|
||||
|
||||
location_t
|
||||
cpp_diagnostic_get_current_location (cpp_reader *pfile)
|
||||
{
|
||||
if (CPP_OPTION (pfile, traditional))
|
||||
{
|
||||
if (pfile->state.in_directive)
|
||||
return pfile->directive_line;
|
||||
else
|
||||
return pfile->line_table->highest_line;
|
||||
}
|
||||
/* We don't want to refer to a token before the beginning of the
|
||||
current run -- that is invalid. */
|
||||
else if (pfile->cur_token == pfile->cur_run->base)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
else
|
||||
{
|
||||
return pfile->cur_token[-1].src_loc;
|
||||
}
|
||||
}
|
||||
|
||||
/* Print a diagnostic at the given location. */
|
||||
|
||||
ATTRIBUTE_FPTR_PRINTF(5,0)
|
||||
@ -52,25 +77,7 @@ cpp_diagnostic (cpp_reader * pfile, enum cpp_diagnostic_level level,
|
||||
enum cpp_warning_reason reason,
|
||||
const char *msgid, va_list *ap)
|
||||
{
|
||||
location_t src_loc;
|
||||
|
||||
if (CPP_OPTION (pfile, traditional))
|
||||
{
|
||||
if (pfile->state.in_directive)
|
||||
src_loc = pfile->directive_line;
|
||||
else
|
||||
src_loc = pfile->line_table->highest_line;
|
||||
}
|
||||
/* We don't want to refer to a token before the beginning of the
|
||||
current run -- that is invalid. */
|
||||
else if (pfile->cur_token == pfile->cur_run->base)
|
||||
{
|
||||
src_loc = 0;
|
||||
}
|
||||
else
|
||||
{
|
||||
src_loc = pfile->cur_token[-1].src_loc;
|
||||
}
|
||||
location_t src_loc = cpp_diagnostic_get_current_location (pfile);
|
||||
rich_location richloc (pfile->line_table, src_loc);
|
||||
return cpp_diagnostic_at (pfile, level, reason, &richloc, msgid, ap);
|
||||
}
|
||||
@ -144,6 +151,43 @@ cpp_warning_syshdr (cpp_reader * pfile, enum cpp_warning_reason reason,
|
||||
return ret;
|
||||
}
|
||||
|
||||
/* As cpp_warning above, but use RICHLOC as the location of the diagnostic. */
|
||||
|
||||
bool cpp_warning_at (cpp_reader *pfile, enum cpp_warning_reason reason,
|
||||
rich_location *richloc, const char *msgid, ...)
|
||||
{
|
||||
va_list ap;
|
||||
bool ret;
|
||||
|
||||
va_start (ap, msgid);
|
||||
|
||||
ret = cpp_diagnostic_at (pfile, CPP_DL_WARNING, reason, richloc,
|
||||
msgid, &ap);
|
||||
|
||||
va_end (ap);
|
||||
return ret;
|
||||
|
||||
}
|
||||
|
||||
/* As cpp_pedwarning above, but use RICHLOC as the location of the
|
||||
diagnostic. */
|
||||
|
||||
bool
|
||||
cpp_pedwarning_at (cpp_reader * pfile, enum cpp_warning_reason reason,
|
||||
rich_location *richloc, const char *msgid, ...)
|
||||
{
|
||||
va_list ap;
|
||||
bool ret;
|
||||
|
||||
va_start (ap, msgid);
|
||||
|
||||
ret = cpp_diagnostic_at (pfile, CPP_DL_PEDWARN, reason, richloc,
|
||||
msgid, &ap);
|
||||
|
||||
va_end (ap);
|
||||
return ret;
|
||||
}
|
||||
|
||||
/* Print a diagnostic at a specific location. */
|
||||
|
||||
ATTRIBUTE_FPTR_PRINTF(6,0)
|
||||
|
||||
@ -1268,6 +1268,14 @@ extern bool cpp_warning_syshdr (cpp_reader *, enum cpp_warning_reason reason,
|
||||
const char *msgid, ...)
|
||||
ATTRIBUTE_PRINTF_3;
|
||||
|
||||
/* As their counterparts above, but use RICHLOC. */
|
||||
extern bool cpp_warning_at (cpp_reader *, enum cpp_warning_reason,
|
||||
rich_location *richloc, const char *msgid, ...)
|
||||
ATTRIBUTE_PRINTF_4;
|
||||
extern bool cpp_pedwarning_at (cpp_reader *, enum cpp_warning_reason,
|
||||
rich_location *richloc, const char *msgid, ...)
|
||||
ATTRIBUTE_PRINTF_4;
|
||||
|
||||
/* Output a diagnostic with "MSGID: " preceding the
|
||||
error string of errno. No location is printed. */
|
||||
extern bool cpp_errno (cpp_reader *, enum cpp_diagnostic_level,
|
||||
@ -1442,43 +1450,95 @@ extern const char * cpp_get_userdef_suffix
|
||||
|
||||
/* In charset.c */
|
||||
|
||||
/* The result of attempting to decode a run of UTF-8 bytes. */
|
||||
|
||||
struct cpp_decoded_char
|
||||
{
|
||||
const char *m_start_byte;
|
||||
const char *m_next_byte;
|
||||
|
||||
bool m_valid_ch;
|
||||
cppchar_t m_ch;
|
||||
};
|
||||
|
||||
/* Information for mapping between code points and display columns.
|
||||
|
||||
This is a tabstop value, along with a callback for getting the
|
||||
widths of characters. Normally this callback is cpp_wcwidth, but we
|
||||
support other schemes for escaping non-ASCII unicode as a series of
|
||||
ASCII chars when printing the user's source code in diagnostic-show-locus.c
|
||||
|
||||
For example, consider:
|
||||
- the Unicode character U+03C0 "GREEK SMALL LETTER PI" (UTF-8: 0xCF 0x80)
|
||||
- the Unicode character U+1F642 "SLIGHTLY SMILING FACE"
|
||||
(UTF-8: 0xF0 0x9F 0x99 0x82)
|
||||
- the byte 0xBF (a stray trailing byte of a UTF-8 character)
|
||||
Normally U+03C0 would occupy one display column, U+1F642
|
||||
would occupy two display columns, and the stray byte would be
|
||||
printed verbatim as one display column.
|
||||
|
||||
However when escaping them as unicode code points as "<U+03C0>"
|
||||
and "<U+1F642>" they occupy 8 and 9 display columns respectively,
|
||||
and when escaping them as bytes as "<CF><80>" and "<F0><9F><99><82>"
|
||||
they occupy 8 and 16 display columns respectively. In both cases
|
||||
the stray byte is escaped to <BF> as 4 display columns. */
|
||||
|
||||
struct cpp_char_column_policy
|
||||
{
|
||||
cpp_char_column_policy (int tabstop,
|
||||
int (*width_cb) (cppchar_t c))
|
||||
: m_tabstop (tabstop),
|
||||
m_undecoded_byte_width (1),
|
||||
m_width_cb (width_cb)
|
||||
{}
|
||||
|
||||
int m_tabstop;
|
||||
/* Width in display columns of a stray byte that isn't decodable
|
||||
as UTF-8. */
|
||||
int m_undecoded_byte_width;
|
||||
int (*m_width_cb) (cppchar_t c);
|
||||
};
|
||||
|
||||
/* A class to manage the state while converting a UTF-8 sequence to cppchar_t
|
||||
and computing the display width one character at a time. */
|
||||
class cpp_display_width_computation {
|
||||
public:
|
||||
cpp_display_width_computation (const char *data, int data_length,
|
||||
int tabstop);
|
||||
const cpp_char_column_policy &policy);
|
||||
const char *next_byte () const { return m_next; }
|
||||
int bytes_processed () const { return m_next - m_begin; }
|
||||
int bytes_left () const { return m_bytes_left; }
|
||||
bool done () const { return !bytes_left (); }
|
||||
int display_cols_processed () const { return m_display_cols; }
|
||||
|
||||
int process_next_codepoint ();
|
||||
int process_next_codepoint (cpp_decoded_char *out);
|
||||
int advance_display_cols (int n);
|
||||
|
||||
private:
|
||||
const char *const m_begin;
|
||||
const char *m_next;
|
||||
size_t m_bytes_left;
|
||||
const int m_tabstop;
|
||||
const cpp_char_column_policy &m_policy;
|
||||
int m_display_cols;
|
||||
};
|
||||
|
||||
/* Convenience functions that are simple use cases for class
|
||||
cpp_display_width_computation. Tab characters will be expanded to spaces
|
||||
as determined by TABSTOP. */
|
||||
as determined by POLICY.m_tabstop, and non-printable-ASCII characters
|
||||
will be escaped as per POLICY. */
|
||||
|
||||
int cpp_byte_column_to_display_column (const char *data, int data_length,
|
||||
int column, int tabstop);
|
||||
int column,
|
||||
const cpp_char_column_policy &policy);
|
||||
inline int cpp_display_width (const char *data, int data_length,
|
||||
int tabstop)
|
||||
const cpp_char_column_policy &policy)
|
||||
{
|
||||
return cpp_byte_column_to_display_column (data, data_length, data_length,
|
||||
tabstop);
|
||||
policy);
|
||||
}
|
||||
int cpp_display_column_to_byte_column (const char *data, int data_length,
|
||||
int display_col, int tabstop);
|
||||
int display_col,
|
||||
const cpp_char_column_policy &policy);
|
||||
int cpp_wcwidth (cppchar_t c);
|
||||
|
||||
bool cpp_input_conversion_is_trivial (const char *input_charset);
|
||||
|
||||
@ -1787,6 +1787,18 @@ class rich_location
|
||||
const diagnostic_path *get_path () const { return m_path; }
|
||||
void set_path (const diagnostic_path *path) { m_path = path; }
|
||||
|
||||
/* A flag for hinting that the diagnostic involves character encoding
|
||||
issues, and thus that it will be helpful to the user if we show some
|
||||
representation of how the characters in the pertinent source lines
|
||||
are encoded.
|
||||
The default is false (i.e. do not escape).
|
||||
When set to true, non-ASCII bytes in the pertinent source lines will
|
||||
be escaped in a manner controlled by the user-supplied option
|
||||
-fdiagnostics-escape-format=, so that the user can better understand
|
||||
what's going on with the encoding in their source file. */
|
||||
bool escape_on_output_p () const { return m_escape_on_output; }
|
||||
void set_escape_on_output (bool flag) { m_escape_on_output = flag; }
|
||||
|
||||
private:
|
||||
bool reject_impossible_fixit (location_t where);
|
||||
void stop_supporting_fixits ();
|
||||
@ -1813,6 +1825,7 @@ protected:
|
||||
bool m_fixits_cannot_be_auto_applied;
|
||||
|
||||
const diagnostic_path *m_path;
|
||||
bool m_escape_on_output;
|
||||
};
|
||||
|
||||
/* A struct for the result of range_label::get_text: a NUL-terminated buffer
|
||||
|
||||
@ -769,6 +769,9 @@ extern void _cpp_do_file_change (cpp_reader *, enum lc_reason, const char *,
|
||||
extern void _cpp_pop_buffer (cpp_reader *);
|
||||
extern char *_cpp_bracket_include (cpp_reader *);
|
||||
|
||||
/* In errors.c */
|
||||
extern location_t cpp_diagnostic_get_current_location (cpp_reader *);
|
||||
|
||||
/* In traditional.c. */
|
||||
extern bool _cpp_scan_out_logical_line (cpp_reader *, cpp_macro *, bool);
|
||||
extern bool _cpp_read_logical_line_trad (cpp_reader *);
|
||||
@ -935,6 +938,26 @@ int linemap_get_expansion_line (class line_maps *,
|
||||
const char* linemap_get_expansion_filename (class line_maps *,
|
||||
location_t);
|
||||
|
||||
/* A subclass of rich_location for emitting a diagnostic
|
||||
at the current location of the reader, but flagging
|
||||
it with set_escape_on_output (true). */
|
||||
class encoding_rich_location : public rich_location
|
||||
{
|
||||
public:
|
||||
encoding_rich_location (cpp_reader *pfile)
|
||||
: rich_location (pfile->line_table,
|
||||
cpp_diagnostic_get_current_location (pfile))
|
||||
{
|
||||
set_escape_on_output (true);
|
||||
}
|
||||
|
||||
encoding_rich_location (cpp_reader *pfile, location_t loc)
|
||||
: rich_location (pfile->line_table, loc)
|
||||
{
|
||||
set_escape_on_output (true);
|
||||
}
|
||||
};
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
||||
38
libcpp/lex.c
38
libcpp/lex.c
@ -1268,7 +1268,11 @@ skip_whitespace (cpp_reader *pfile, cppchar_t c)
|
||||
while (is_nvspace (c));
|
||||
|
||||
if (saw_NUL)
|
||||
cpp_error (pfile, CPP_DL_WARNING, "null character(s) ignored");
|
||||
{
|
||||
encoding_rich_location rich_loc (pfile);
|
||||
cpp_error_at (pfile, CPP_DL_WARNING, &rich_loc,
|
||||
"null character(s) ignored");
|
||||
}
|
||||
|
||||
buffer->cur--;
|
||||
}
|
||||
@ -1297,6 +1301,28 @@ warn_about_normalization (cpp_reader *pfile,
|
||||
if (CPP_OPTION (pfile, warn_normalize) < NORMALIZE_STATE_RESULT (s)
|
||||
&& !pfile->state.skipping)
|
||||
{
|
||||
location_t loc = token->src_loc;
|
||||
|
||||
/* If possible, create a location range for the token. */
|
||||
if (loc >= RESERVED_LOCATION_COUNT
|
||||
&& token->type != CPP_EOF
|
||||
/* There must be no line notes to process. */
|
||||
&& (!(pfile->buffer->cur
|
||||
>= pfile->buffer->notes[pfile->buffer->cur_note].pos
|
||||
&& !pfile->overlaid_buffer)))
|
||||
{
|
||||
source_range tok_range;
|
||||
tok_range.m_start = loc;
|
||||
tok_range.m_finish
|
||||
= linemap_position_for_column (pfile->line_table,
|
||||
CPP_BUF_COLUMN (pfile->buffer,
|
||||
pfile->buffer->cur));
|
||||
loc = COMBINE_LOCATION_DATA (pfile->line_table,
|
||||
loc, tok_range, NULL);
|
||||
}
|
||||
|
||||
encoding_rich_location rich_loc (pfile, loc);
|
||||
|
||||
/* Make sure that the token is printed using UCNs, even
|
||||
if we'd otherwise happily print UTF-8. */
|
||||
unsigned char *buf = XNEWVEC (unsigned char, cpp_token_len (token));
|
||||
@ -1304,14 +1330,14 @@ warn_about_normalization (cpp_reader *pfile,
|
||||
|
||||
sz = cpp_spell_token (pfile, token, buf, false) - buf;
|
||||
if (NORMALIZE_STATE_RESULT (s) == normalized_C)
|
||||
cpp_warning_with_line (pfile, CPP_W_NORMALIZE, token->src_loc, 0,
|
||||
"`%.*s' is not in NFKC", (int) sz, buf);
|
||||
cpp_warning_at (pfile, CPP_W_NORMALIZE, &rich_loc,
|
||||
"`%.*s' is not in NFKC", (int) sz, buf);
|
||||
else if (CPP_OPTION (pfile, cxx23_identifiers))
|
||||
cpp_pedwarning_with_line (pfile, CPP_W_NORMALIZE, token->src_loc, 0,
|
||||
cpp_pedwarning_at (pfile, CPP_W_NORMALIZE, &rich_loc,
|
||||
"`%.*s' is not in NFC", (int) sz, buf);
|
||||
else
|
||||
cpp_warning_with_line (pfile, CPP_W_NORMALIZE, token->src_loc, 0,
|
||||
"`%.*s' is not in NFC", (int) sz, buf);
|
||||
cpp_warning_at (pfile, CPP_W_NORMALIZE, &rich_loc,
|
||||
"`%.*s' is not in NFC", (int) sz, buf);
|
||||
free (buf);
|
||||
}
|
||||
}
|
||||
|
||||
@ -2086,7 +2086,8 @@ rich_location::rich_location (line_maps *set, location_t loc,
|
||||
m_fixit_hints (),
|
||||
m_seen_impossible_fixit (false),
|
||||
m_fixits_cannot_be_auto_applied (false),
|
||||
m_path (NULL)
|
||||
m_path (NULL),
|
||||
m_escape_on_output (false)
|
||||
{
|
||||
add_range (loc, SHOW_RANGE_WITH_CARET, label);
|
||||
}
|
||||
|
||||
Loading…
Reference in New Issue
Block a user