A bug in ctf_dtd_delete led to refs in the string table to the
names of non-root-visible types not being removed when the DTD
was. This seems harmless, but actually it would lead to a write
down a pointer into freed memory if such a type was ctf_rollback()ed
over and then the dict was serialized (updating all the refs as the
strtab was serialized in turn).
Bug introduced in commit fe4c2d5563
("libctf: create: non-root-visible types should not appear in name tables")
which is included in binutils 2.35.
libctf/
* ctf-create.c (ctf_dtd_delete): Remove refs for all types
with names, not just root-visible ones.
The dict and archive opening code in libctf is somewhat unusual, because
unlike everything else, it cannot report errors by setting an error on the
dict, because in case of error there isn't one. They get passed an error
integer pointer that is set on error instead.
Inside ctf_bufopen this is implemented by calling ctf_set_open_errno and
passing it a positive error value. In turn this means that most things it
calls (including init_static_types) return zero on success and a *positive*
ECTF_* or errno value on error.
This trickles down to ctf_dynhash_insert_type, which is used by
init_static_types to add newly-detected types to the name tables. This was
returning the error value it received from a variety of functions without
alteration. ctf_dynhash_insert conformed to this contract by returning a
positive value on error (usually OOM), which is unfortunate for multiple
reasons:
- ctf_dynset_insert returns a *negative* value
- ctf_dynhash_insert and ctf_dynset_insert don't take an fp, so the value
they return is turned into the errno, so it had better be right, callers
don't just check for != 0 here
- more or less every single caller of ctf_dyn*_insert in libctf other than
ctf_dynhash_insert_type (and there are a *lot*, mostly in the
deduplicator) assumes that ctf_dynhash_insert returns a negative value
on error, even though it doesn't. In practice the only possible error is
OOM, but if OOM does happen we end up with a nonsense error value.
The simplest fix for this seems to be to make ctf_dynhash_insert and
ctf_dynset_insert conform to the usual interface contract: negative
values are errors. This in turn means that ctf_dynhash_insert_type
needs to change: let's make it consistent too, returning a negative
value on error, putting the error on the fp in non-negated form.
init_static_types_internal adapts to this by negating the error return from
ctf_dynhash_insert_type, so the value handed back to ctf_bufopen is still
positive: the new call site in ctf_track_enumerator does not need to change.
(The existing tests for this reliably detect when I get it wrong.
I know, because they did.)
libctf/
* ctf-hash.c (ctf_dynhash_insert): Negate return value.
(ctf_dynhash_insert_type): Set de-negated error on the dict:
return negated error.
* ctf-open.c (init_static_types_internal): Adapt to this change.
The recent change to detect duplicate enum values and return ECTF_DUPLICATE
when found turns out to perturb a great many callers. In particular, the
pahole-created kernel BTF has the same problem we historically did, and
gleefully emits duplicated enum constants in profusion. Handling the
resulting duplicate errors from BTF -> CTF converters reasonably is
unreasonably difficult (it amounts to forcing them to skip some types or
reimplement the deduplicator).
So let's step back a bit. What we care about mostly is that the
deduplicator treat enums with conflicting enumeration constants as
conflicting types: programs that want to look up enumeration constant ->
value mappings using the new APIs to do so might well want the same checks
to apply to any ctf_add_* operations they carry out (and since they're
*using* the new APIs, added at the same time as this restriction was
imposed, there is likely to be no negative consequence of this).
So we want some way to allow processes that know about duplicate detection
to opt into it, while allowing everyone else to stay clear of it: but we
want ctf_link to get this behaviour even if its caller has opted out.
So add a new concept to the API: dict-wide CTF flags, set via
ctf_dict_set_flag, obtained via ctf_dict_get_flag. They are not bitflags
but simple arbitrary integers and an on/off value, stored in an unspecified
manner (the one current flag, we translate into an LCTF_* flag value in the
internal ctf_dict ctf_flags word). If you pass in an invalid flag or value
you get a new ECTF_BADFLAG error, so the caller can easily tell whether
flags added in future are valid with a particular libctf or not.
We check this flag in ctf_add_enumerator, and set it around the link
(including on child per-CU dicts). The newish enumerator-iteration test is
souped up to check the semantics of the flag as well.
The fact that the flag can be set and unset at any time has curious
consequences. You can unset the flag, insert a pile of duplicates, then set
it and expect the new duplicates to be detected, not only by
ctf_add_enumerator but also by ctf_lookup_enumerator. This means we now
have to maintain the ctf_names and conflicting_enums enum-duplication
tracking as new enums are added, not purely as the dict is opened.
Move that code out of init_static_types_internal and into a new
ctf_track_enumerator function that addition can also call.
(None of this affects the file format or serialization machinery, which has
to be able to handle duplicate enumeration constants no matter what.)
include/
* ctf-api.h (CTF_ERRORS) [ECTF_BADFLAG]: New.
(ECTF_NERR): Update.
(CTF_STRICT_NO_DUP_ENUMERATORS): New flag.
(ctf_dict_set_flag): New function.
(ctf_dict_get_flag): Likewise.
libctf/
* ctf-impl.h (LCTF_STRICT_NO_DUP_ENUMERATORS): New flag.
(ctf_track_enumerator): Declare.
* ctf-dedup.c (ctf_dedup_emit_type): Set it.
* ctf-link.c (ctf_create_per_cu): Likewise.
(ctf_link_deduplicating_per_cu): Likewise.
(ctf_link): Likewise.
(ctf_link_write): Likewise.
* ctf-subr.c (ctf_dict_set_flag): New function.
(ctf_dict_get_flag): New function.
* ctf-open.c (init_static_types_internal): Move enum tracking to...
* ctf-create.c (ctf_track_enumerator): ... this new function.
(ctf_add_enumerator): Call it.
* libctf.ver: Add the new functions.
* testsuite/libctf-lookup/enumerator-iteration.c: Test them.
We set this flag at the top of ctf_link_write (to tell ctf_serialize, way
down under the archive file writing functions, to do the various link- time
serialization things like symbol filtering and the like), but we never
remember to clear it except on error. This is probably bad if you want to
serialize the dict yourself directly in the future after linking it (which
is... definitely a *possible* use of the API, if rather strange).
libctf/
* ctf-link.c (ctf_link_write): Clear LCTF_LINKING before exit.
libctf's dynsets are a straight wrapper around libiberty hashtab, storing
the key directly in the hashtab slot. However, we'd often like to be able
to store 0 and 1 (HTAB_EMPTY_ENTRY and HTAB_DELETED_ENTRY) in there, so we
move them out of the way and replace them with huge unlikely values
instead. Unfortunately we failed to do this replacement in one place, so
insertion of 0 or 1 ended up misinforming the hashtab machinery that an
entry was empty or deleted when it wasn't.
libctf/
* ctf-hash.c (ctf_dynset_insert): Call key_to_internal properly.
Drop an unnecessary variable, and fix a buggy comment.
No effect on generated code.
libctf/
* ctf-dedup.c (ctf_dedup_detect_name_ambiguity): Drop unnecessary
variable.
(ctf_dedup_rwalk_output_mapping): Fix comment.
This erorr doesn't just indicate that there is no parent dictionary
(that's routine, and true of all dicts that are parents themselves)
but that a parent is *needed* but wasn't found.
include/
* ctf-api.h (_CTF_ERRORS) [ECTF_NOPARENT]: Improve error message.
ld/
* testsuite/ld-ctf/diag-parname.d: Adjust.
Commit 483546ce4f ("libctf: make ctf_serialize() actually serialize")
accidentally broke dict compression. There were two bugs:
- ctf_arc_write_one_ctf was still making its own decision about
whether to compress the dict via direct ctf_size comparison, which is
unfortunate because now that it no longer calls ctf_serialize itself,
ctf_size is always zero when it does this: it should let the writing
functions decide on the threshold, which they contain code to do which is
simply not used for lack of one trivial wrapper to write to an fd and
also provide a compression threshold
- ctf_write_mem, the function underlying all writing as of the commit
above, was calling zlib's compressBound and avoiding compression if this
returned a value larger than the input. Unfortunately compressBound does
not do a trial compression and determine whether the result is
compressible: it just adds zlib header sizes to the value passed in, so
our test would *always* have concluded that the value was incompressible!
Avoid by simply always compressing if the raw size is larger than the
threshold: zlib is quite clever enough to avoid actually compressing
if the data is incompressible.
Add a testcase for this.
libctf/
* ctf-impl.h (ctf_write_thresholded): New...
* ctf-serialize.c (ctf_write_thresholded): ... defined here,
a wrapper around...
(ctf_write_mem): ... this. Don't check compressibility.
(ctf_compress_write): Reimplement as a ctf_write_thresholded
wrapper.
(ctf_write): Likewise.
* ctf-archive.c (arc_write_one_ctf): Just call
ctf_write_thresholded rather than trying to work out whether
to compress.
* testsuite/libctf-writable/ctf-compressed.*: New test.
If you deduplicate non-root-visible types, the resulting type should still
be non-root-visible! We were promoting all such types to root-visible, and
re-demoting them only if their names collided (which might happen on
cu-mapped links if multiple compilation units with conflicting types are
fused into one child dict).
This "worked" before now, in that linking at least didn't fail (if you don't
mind having your non-root flag value destroyed if you're adding
non-root-visible types), but now that conflicting enumerators cause their
containing enums to become conflicted (enums which might have *different
names*), this caused the linker to crash when it hit two enumerators with
conflicting values.
Not testable in ld because cu-mapped links are not exposed to ld, but can be
tested via direct creation of libraries and calls to ctf_link directly.
(This also tests the ctf_dump non-root type printout, which before now
was untested.)
libctf/
* ctf-dedup.c (ctf_dedup_emit_type): Non-root-visible input types
should be emitted as non-root-visible output types.
* testsuite/libctf-writable/ctf-nonroot-linking.c: New test.
* testsuite/libctf-writable/ctf-nonroot-linking.lk: New test.
The flag test when dumping non-root-visible tyeps was doubly wrong: the
flags word is a *bitfield* containing CTF_ADD_ROOT as one possible
value, so needs | and & testing, not just ==, and CTF_ADD_NONROOT is 0,
so cannot be tested for this way: one must check for the non-presence of
CTF_ADD_ROOT.
libctf/
* ctf-dump.c (ctf_dump_format_type): Fix non-root flag test.
In commit 149ce5c263 we introduced the concept of "movable" refs,
which are refs that can be moved in batches, to let us maintain valid ref
lists even when adding refs to blocks of memory that can be realloced (which
is any type containing a vlen which can expand, like names contained within
enum or struct members). Movable refs need a backpointer to the movable
refs dynhash for this dict; since non-movable refs are very common, we tried
to save memory by having a slightly bigger struct for moveable refs with a
backpointer in it, and casting appropriately, indicating which sort of ref
we were dealing with via a flag on the atom.
Unfortunately this doesn't work reliably, because you can perfectly well
have a string ("foo", say) which has both non-movable refs (say, an external
symbol and a variable name) and movable refs (say, a structure member name)
to the same atom. Indicate which struct we're dealing with with an atom
flag and suddenly you're casting a ctf_str_atom_ref to a
ctf_str_atom_ref_movable (which is bigger) and dereferencing random memory
off the end of it and interpreting it as a backpointer to the movable refs
dynhash. This is unlikely to work well.
So bite the bullet and split refs into two separate lists, one for movable
refs, one for immovable refs. It means some annoying code duplication, but
there's not very much of it, and it means we can keep the movable refs
hashtab (which in turn means we don't have to do linear searches to find all
relevant refs when moving refs, which in turn means that
structure/union/enum member additions remain amortized O(n) time, not
O(n^2).
Callers can now purge movable and non-movable refs independently of each
other. We don't use this yet, but a use is coming.
libctf/
* ctf-impl.h (CTF_STR_ATOM_MOVABLE): Delete.
(struct ctf_str_atom) [csa_movable_refs]: New.
(struct ctf_dict): Adjust comment.
(ctf_str_purge_refs): Add MOVABLE arg.
* ctf-string.c (ctf_str_purge_movable_atom_refs): Split out of...
(ctf_str_purge_atom_refs): ... this.
(ctf_str_free_atom): Call it.
(ctf_str_purge_one_atom_refs): Likewise.
(aref_create): Adjust accordingly.
(ctf_str_move_refs): Likewise.
(ctf_str_remove_ref): Remove movable refs too, including
deleting the ref from ctf_str_movable_refs.
(ctf_str_purge_refs): Add MOVABLE arg.
(ctf_str_update_refs): Update movable refs.
(ctf_str_write_strtab): Check, and purge, movable refs.
The PARENTS arg is carefully passed down through all the layers of hash
functions and then never used for anything. (In the distant past it was
used for cycle detection, but the algorithm eventually committed doesn't
need to do cycle detection...)
The PARENTS arg is still used by ctf_dedup_emit(), but even there we can
loosen the requirements and state that you can just leave entries
corresponding to dicts with no parents at zero (which will be useful
in an upcoming commit).
libctf/
* ctf-dedup.c (ctf_dedup_hash_type): Drop PARENTS arg.
(ctf_dedup_rhash_type): Likewise.
(ctf_dedup): Likewise.
(ctf_dedup_emit_struct_members): Mention what you can do to
PARENTS entries for parent dicts.
* ctf-impl.h (ctf_dedup): Adjust accordingly.
* ctf-link.c (ctf_link_deduplicating_per_cu): Likewise.
(ctf_link_deduplicating): Likewise.
The worry that caused this to not be supported was because we don't
bother endian-flipping version-related fields before checking them.
But they're all unsigned chars anyway, and don't need any flipping at
all.
This should be supported and should already work. Enable it.
libctf/
* ctf-open.c (ctf_bufopen): Don't prohibit foreign-endian
upgrades.
Add the MT ASE instruction operand types and encodings to the microMIPS
opcode table and enable the assembly of these instructions in GAS from
MIPSr2 onwards. Update the binutils and GAS testsuites accordingly.
References:
"MIPS Architecture for Programmers, Volume IV-f: The MIPS MT Module for
the microMIPS32 Architecture", MIPS Technologies, Inc., Document Number:
MD00768, Revision 1.12, July 16, 2013
Co-Authored-By: Maciej W. Rozycki <macro@redhat.com>
(cherry picked from commit 08e6af1bac)
Change TLS transition error messages from
a-argp-help.o(.text+0x12f): relocation R_X86_64_GOTTPOFF against `a' must be used in ADD or MOV onlyld: final link failed: bad value
to
a-argp-help.o(.text+0x12f): relocation R_X86_64_GOTTPOFF against `a' must be used in ADD or MOV only
ld: final link failed: bad value
PR ld/32017
* elfxx-x86.c (_bfd_x86_elf_link_report_tls_transition_error):
Add missing newlines.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
(cherry picked from commit f73f5173fa)
Change 2.42 to 2.43 for binutils 2.43 NEWS entries.
binutils/
* NEWS: Change 2.42 to 2.43 for 2.43 NEWS entries.
ld/
* NEWS: Change 2.42 to 2.43 for 2.43 NEWS entries.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
(cherry picked from commit e1b9d58e85)
The new test wasn't being run, and failed due to relocations against
.gnu.build.attributes being stripped by default strip behaviour.
We probably should be keeping these relocations, but I haven't made
that change here.
BTW, the new test fails on ia64-hpux but that's just a repeat of the
existing note-5 fail.
PR 31999
* testsuite/binutils-all/strip-16.d: strip with --strip-unneeded
and --merge-notes.
* testsuite/binutils-all/objcopy.exp: Run the new test. Sort
other strip tests.
When debugging gdb itself and trying to print a intrusive_list that has
more than one element, I get:
File "/home/simark/build/binutils-gdb-all-targets/gdb/gdb-gdb.py", line 365, in _children_generator
node_ptr = self._as_node_ptr(elem_ptr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simark/build/binutils-gdb-all-targets/gdb/gdb-gdb.py", line 345, in _as_node_ptr
assert elem_ptr.type.code == gdb.TYPE_CODE_PTR
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
This is because node_ptr is a typedef
(intrusive_list_base_iterator::pointer). Add a call to strip_typedefs
to get to the real type.
Enhance gdb.gdb/python-helper.exp with a test that would have caught
this bug.
Change-Id: I3eaca8de5ed06d05756ed979332b6a431e15b700
Approved-By: Andrew Burgess <aburgess@redhat.com>
A number of instructions in the regular MIPS opcode table are assembly
idioms for the MT thread context move MFTR and MTTR instructions, so
mark them as aliases accordingly. Add suitable test cases, which also
cover the PAUSE assembly idiom.
PAUSE is an assembly idiom for 'sll $0,$0,5', so mark it as an alias in
the regular MIPS opcode table, matching the microMIPS opcode table. A
test case will be supplied separately.
A number of coprocessor move encodings have been randomly sprinkled over
the regular MIPS and microMIPS opcode tables rather than where they'd be
expected following the alphabetic order. Fix the ordering, taking into
account precedence where it has to be observed for correct disassembly.
No functional change.
Make AL a shorthand for INSN2_ALIAS with the regular MIPS and microMIPS
opcode tables, just as with the MIPS16 opcode table, and use it
throughout. No functional change.
The semantics of the regular MIPS "+t" operand code is exactly the same
as that of the "E" operand code, so replace the former with the latter
in the single MFTC0 instruction with implicit 'sel' == 0 encoding where
it's used, matching the encoding with explicit 'sel' as well as other
instructions.
We print MFTR and MTTR instructions' thread context register operand in
disassembly using the ABI name the register number would correspond to
should the targeted register be a general-purpose register.
However in most cases it is wrong, because general-purpose registers are
only referred when the 'u' and 'sel' operands are 1 and 0 respectively.
And even in these cases the MFGPR and MTGPR aliases take precedence over
the corresponding generic instruction encodings, so you won't see the
valid case to normally trigger.
Conversely decoding the thread context register operand numerically is
always valid, so switch to using it. Adjust test coverage accordingly.
As from commit ab90248154 ("Add structures to describe MIPS
operands"), <https://sourceware.org/ml/binutils/2013-07/msg00135.html>,
the use of numerous regular MIPS and microMIPS OP_SH and OP_MASK macros
has been removed.
Similarly as from commit c3c0747817 ("Use operand structures for
MIPS16"), <https://sourceware.org/ml/binutils/2013-07/msg00136.html>,
the use of numerous MIPS16 OP_SH and OP_MASK macros has been removed.
And as from commit 9e12b7a2b0 ("Rewrite main mips_ip parsing loop"),
<https://sourceware.org/ml/binutils/2013-07/msg00139.html>, none of the
OP_OP macros are used anymore.
Discard all the unused macros then and only keep the small subset that
is still referred. This simplifies maintenance and removes the need to
keep the artificial arrangement where some regular MIPS and microMIPS
macros expand to 0 and are kept for compatibility with the opposite ISA
mode only, as it used to be required before the commit referred.
The "-t", "-u", "-v", and "-w" operand types refer 'rt' operand, which
is the target register rather than the source register. Additionally
the "-x" and "-y" R6 operand types refer 'rs' rather than 'rt' operand
of the BOVC/BNVC and the BEQC/BNEC instructions respectively. Also the
"-x" operand type does not permit 'rs' to be the same as 'rt'.
Correct inline documentation in opcode/mips.h accordingly.
