Previous code included the full $srcdir pathnames in the individual
subtest PASS/FAIL names, which makes it difficult to compute
comparisons or regressions between test runs on different machines.
This version switches to the basename only, which are common.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
This failed to properly byteswap its return value.
The ctf_archive format predates the idea of "just write natively and
flip on open", and byteswaps all over the place. It's too easy to
forget one. The next revision of the archive format (not versioned,
so we just tweak the magic number instead) should be native-endianned
like the dicts inside it are.
libctf/
* ctf-archive.c (ctf_archive_count): Byteswap return value.
If you asprintf something and then use it only as input to another asprintf,
it helps to free it afterwards.
libctf/
* ctf-dump.c (ctf_dump_header): Free the flagstr after use.
(ctf_dump): Make a NULL return slightly clearer.
A bug in ctf_dtd_delete led to refs in the string table to the
names of non-root-visible types not being removed when the DTD
was. This seems harmless, but actually it would lead to a write
down a pointer into freed memory if such a type was ctf_rollback()ed
over and then the dict was serialized (updating all the refs as the
strtab was serialized in turn).
Bug introduced in commit fe4c2d5563
("libctf: create: non-root-visible types should not appear in name tables")
which is included in binutils 2.35.
libctf/
* ctf-create.c (ctf_dtd_delete): Remove refs for all types
with names, not just root-visible ones.
The dict and archive opening code in libctf is somewhat unusual, because
unlike everything else, it cannot report errors by setting an error on the
dict, because in case of error there isn't one. They get passed an error
integer pointer that is set on error instead.
Inside ctf_bufopen this is implemented by calling ctf_set_open_errno and
passing it a positive error value. In turn this means that most things it
calls (including init_static_types) return zero on success and a *positive*
ECTF_* or errno value on error.
This trickles down to ctf_dynhash_insert_type, which is used by
init_static_types to add newly-detected types to the name tables. This was
returning the error value it received from a variety of functions without
alteration. ctf_dynhash_insert conformed to this contract by returning a
positive value on error (usually OOM), which is unfortunate for multiple
reasons:
- ctf_dynset_insert returns a *negative* value
- ctf_dynhash_insert and ctf_dynset_insert don't take an fp, so the value
they return is turned into the errno, so it had better be right, callers
don't just check for != 0 here
- more or less every single caller of ctf_dyn*_insert in libctf other than
ctf_dynhash_insert_type (and there are a *lot*, mostly in the
deduplicator) assumes that ctf_dynhash_insert returns a negative value
on error, even though it doesn't. In practice the only possible error is
OOM, but if OOM does happen we end up with a nonsense error value.
The simplest fix for this seems to be to make ctf_dynhash_insert and
ctf_dynset_insert conform to the usual interface contract: negative
values are errors. This in turn means that ctf_dynhash_insert_type
needs to change: let's make it consistent too, returning a negative
value on error, putting the error on the fp in non-negated form.
init_static_types_internal adapts to this by negating the error return from
ctf_dynhash_insert_type, so the value handed back to ctf_bufopen is still
positive: the new call site in ctf_track_enumerator does not need to change.
(The existing tests for this reliably detect when I get it wrong.
I know, because they did.)
libctf/
* ctf-hash.c (ctf_dynhash_insert): Negate return value.
(ctf_dynhash_insert_type): Set de-negated error on the dict:
return negated error.
* ctf-open.c (init_static_types_internal): Adapt to this change.
The recent change to detect duplicate enum values and return ECTF_DUPLICATE
when found turns out to perturb a great many callers. In particular, the
pahole-created kernel BTF has the same problem we historically did, and
gleefully emits duplicated enum constants in profusion. Handling the
resulting duplicate errors from BTF -> CTF converters reasonably is
unreasonably difficult (it amounts to forcing them to skip some types or
reimplement the deduplicator).
So let's step back a bit. What we care about mostly is that the
deduplicator treat enums with conflicting enumeration constants as
conflicting types: programs that want to look up enumeration constant ->
value mappings using the new APIs to do so might well want the same checks
to apply to any ctf_add_* operations they carry out (and since they're
*using* the new APIs, added at the same time as this restriction was
imposed, there is likely to be no negative consequence of this).
So we want some way to allow processes that know about duplicate detection
to opt into it, while allowing everyone else to stay clear of it: but we
want ctf_link to get this behaviour even if its caller has opted out.
So add a new concept to the API: dict-wide CTF flags, set via
ctf_dict_set_flag, obtained via ctf_dict_get_flag. They are not bitflags
but simple arbitrary integers and an on/off value, stored in an unspecified
manner (the one current flag, we translate into an LCTF_* flag value in the
internal ctf_dict ctf_flags word). If you pass in an invalid flag or value
you get a new ECTF_BADFLAG error, so the caller can easily tell whether
flags added in future are valid with a particular libctf or not.
We check this flag in ctf_add_enumerator, and set it around the link
(including on child per-CU dicts). The newish enumerator-iteration test is
souped up to check the semantics of the flag as well.
The fact that the flag can be set and unset at any time has curious
consequences. You can unset the flag, insert a pile of duplicates, then set
it and expect the new duplicates to be detected, not only by
ctf_add_enumerator but also by ctf_lookup_enumerator. This means we now
have to maintain the ctf_names and conflicting_enums enum-duplication
tracking as new enums are added, not purely as the dict is opened.
Move that code out of init_static_types_internal and into a new
ctf_track_enumerator function that addition can also call.
(None of this affects the file format or serialization machinery, which has
to be able to handle duplicate enumeration constants no matter what.)
include/
* ctf-api.h (CTF_ERRORS) [ECTF_BADFLAG]: New.
(ECTF_NERR): Update.
(CTF_STRICT_NO_DUP_ENUMERATORS): New flag.
(ctf_dict_set_flag): New function.
(ctf_dict_get_flag): Likewise.
libctf/
* ctf-impl.h (LCTF_STRICT_NO_DUP_ENUMERATORS): New flag.
(ctf_track_enumerator): Declare.
* ctf-dedup.c (ctf_dedup_emit_type): Set it.
* ctf-link.c (ctf_create_per_cu): Likewise.
(ctf_link_deduplicating_per_cu): Likewise.
(ctf_link): Likewise.
(ctf_link_write): Likewise.
* ctf-subr.c (ctf_dict_set_flag): New function.
(ctf_dict_get_flag): New function.
* ctf-open.c (init_static_types_internal): Move enum tracking to...
* ctf-create.c (ctf_track_enumerator): ... this new function.
(ctf_add_enumerator): Call it.
* libctf.ver: Add the new functions.
* testsuite/libctf-lookup/enumerator-iteration.c: Test them.
We set this flag at the top of ctf_link_write (to tell ctf_serialize, way
down under the archive file writing functions, to do the various link- time
serialization things like symbol filtering and the like), but we never
remember to clear it except on error. This is probably bad if you want to
serialize the dict yourself directly in the future after linking it (which
is... definitely a *possible* use of the API, if rather strange).
libctf/
* ctf-link.c (ctf_link_write): Clear LCTF_LINKING before exit.
libctf's dynsets are a straight wrapper around libiberty hashtab, storing
the key directly in the hashtab slot. However, we'd often like to be able
to store 0 and 1 (HTAB_EMPTY_ENTRY and HTAB_DELETED_ENTRY) in there, so we
move them out of the way and replace them with huge unlikely values
instead. Unfortunately we failed to do this replacement in one place, so
insertion of 0 or 1 ended up misinforming the hashtab machinery that an
entry was empty or deleted when it wasn't.
libctf/
* ctf-hash.c (ctf_dynset_insert): Call key_to_internal properly.
Drop an unnecessary variable, and fix a buggy comment.
No effect on generated code.
libctf/
* ctf-dedup.c (ctf_dedup_detect_name_ambiguity): Drop unnecessary
variable.
(ctf_dedup_rwalk_output_mapping): Fix comment.
Commit 483546ce4f ("libctf: make ctf_serialize() actually serialize")
accidentally broke dict compression. There were two bugs:
- ctf_arc_write_one_ctf was still making its own decision about
whether to compress the dict via direct ctf_size comparison, which is
unfortunate because now that it no longer calls ctf_serialize itself,
ctf_size is always zero when it does this: it should let the writing
functions decide on the threshold, which they contain code to do which is
simply not used for lack of one trivial wrapper to write to an fd and
also provide a compression threshold
- ctf_write_mem, the function underlying all writing as of the commit
above, was calling zlib's compressBound and avoiding compression if this
returned a value larger than the input. Unfortunately compressBound does
not do a trial compression and determine whether the result is
compressible: it just adds zlib header sizes to the value passed in, so
our test would *always* have concluded that the value was incompressible!
Avoid by simply always compressing if the raw size is larger than the
threshold: zlib is quite clever enough to avoid actually compressing
if the data is incompressible.
Add a testcase for this.
libctf/
* ctf-impl.h (ctf_write_thresholded): New...
* ctf-serialize.c (ctf_write_thresholded): ... defined here,
a wrapper around...
(ctf_write_mem): ... this. Don't check compressibility.
(ctf_compress_write): Reimplement as a ctf_write_thresholded
wrapper.
(ctf_write): Likewise.
* ctf-archive.c (arc_write_one_ctf): Just call
ctf_write_thresholded rather than trying to work out whether
to compress.
* testsuite/libctf-writable/ctf-compressed.*: New test.
If you deduplicate non-root-visible types, the resulting type should still
be non-root-visible! We were promoting all such types to root-visible, and
re-demoting them only if their names collided (which might happen on
cu-mapped links if multiple compilation units with conflicting types are
fused into one child dict).
This "worked" before now, in that linking at least didn't fail (if you don't
mind having your non-root flag value destroyed if you're adding
non-root-visible types), but now that conflicting enumerators cause their
containing enums to become conflicted (enums which might have *different
names*), this caused the linker to crash when it hit two enumerators with
conflicting values.
Not testable in ld because cu-mapped links are not exposed to ld, but can be
tested via direct creation of libraries and calls to ctf_link directly.
(This also tests the ctf_dump non-root type printout, which before now
was untested.)
libctf/
* ctf-dedup.c (ctf_dedup_emit_type): Non-root-visible input types
should be emitted as non-root-visible output types.
* testsuite/libctf-writable/ctf-nonroot-linking.c: New test.
* testsuite/libctf-writable/ctf-nonroot-linking.lk: New test.
The flag test when dumping non-root-visible tyeps was doubly wrong: the
flags word is a *bitfield* containing CTF_ADD_ROOT as one possible
value, so needs | and & testing, not just ==, and CTF_ADD_NONROOT is 0,
so cannot be tested for this way: one must check for the non-presence of
CTF_ADD_ROOT.
libctf/
* ctf-dump.c (ctf_dump_format_type): Fix non-root flag test.
In commit 149ce5c263 we introduced the concept of "movable" refs,
which are refs that can be moved in batches, to let us maintain valid ref
lists even when adding refs to blocks of memory that can be realloced (which
is any type containing a vlen which can expand, like names contained within
enum or struct members). Movable refs need a backpointer to the movable
refs dynhash for this dict; since non-movable refs are very common, we tried
to save memory by having a slightly bigger struct for moveable refs with a
backpointer in it, and casting appropriately, indicating which sort of ref
we were dealing with via a flag on the atom.
Unfortunately this doesn't work reliably, because you can perfectly well
have a string ("foo", say) which has both non-movable refs (say, an external
symbol and a variable name) and movable refs (say, a structure member name)
to the same atom. Indicate which struct we're dealing with with an atom
flag and suddenly you're casting a ctf_str_atom_ref to a
ctf_str_atom_ref_movable (which is bigger) and dereferencing random memory
off the end of it and interpreting it as a backpointer to the movable refs
dynhash. This is unlikely to work well.
So bite the bullet and split refs into two separate lists, one for movable
refs, one for immovable refs. It means some annoying code duplication, but
there's not very much of it, and it means we can keep the movable refs
hashtab (which in turn means we don't have to do linear searches to find all
relevant refs when moving refs, which in turn means that
structure/union/enum member additions remain amortized O(n) time, not
O(n^2).
Callers can now purge movable and non-movable refs independently of each
other. We don't use this yet, but a use is coming.
libctf/
* ctf-impl.h (CTF_STR_ATOM_MOVABLE): Delete.
(struct ctf_str_atom) [csa_movable_refs]: New.
(struct ctf_dict): Adjust comment.
(ctf_str_purge_refs): Add MOVABLE arg.
* ctf-string.c (ctf_str_purge_movable_atom_refs): Split out of...
(ctf_str_purge_atom_refs): ... this.
(ctf_str_free_atom): Call it.
(ctf_str_purge_one_atom_refs): Likewise.
(aref_create): Adjust accordingly.
(ctf_str_move_refs): Likewise.
(ctf_str_remove_ref): Remove movable refs too, including
deleting the ref from ctf_str_movable_refs.
(ctf_str_purge_refs): Add MOVABLE arg.
(ctf_str_update_refs): Update movable refs.
(ctf_str_write_strtab): Check, and purge, movable refs.
The PARENTS arg is carefully passed down through all the layers of hash
functions and then never used for anything. (In the distant past it was
used for cycle detection, but the algorithm eventually committed doesn't
need to do cycle detection...)
The PARENTS arg is still used by ctf_dedup_emit(), but even there we can
loosen the requirements and state that you can just leave entries
corresponding to dicts with no parents at zero (which will be useful
in an upcoming commit).
libctf/
* ctf-dedup.c (ctf_dedup_hash_type): Drop PARENTS arg.
(ctf_dedup_rhash_type): Likewise.
(ctf_dedup): Likewise.
(ctf_dedup_emit_struct_members): Mention what you can do to
PARENTS entries for parent dicts.
* ctf-impl.h (ctf_dedup): Adjust accordingly.
* ctf-link.c (ctf_link_deduplicating_per_cu): Likewise.
(ctf_link_deduplicating): Likewise.
The worry that caused this to not be supported was because we don't
bother endian-flipping version-related fields before checking them.
But they're all unsigned chars anyway, and don't need any flipping at
all.
This should be supported and should already work. Enable it.
libctf/
* ctf-open.c (ctf_bufopen): Don't prohibit foreign-endian
upgrades.
Most of these are harmless, but some of the type confusions and especially
a missing ctf_strerror() on an error path were actual bugs that could
have resulted in test failures crashing rather than printing an error
message.
libctf/
* testsuite/libctf-lookup/enumerator-iteration.c: Fix type
confusion, signedness confusion and a missing ctf_errmsg().
* testsuite/libctf-regression/libctf-repeat-cu-main.c: Return 0 from
the test function.
* testsuite/libctf-regression/open-error-free.c: Fix signedness
confusion.
* testsuite/libctf-regression/zrewrite.c: Remove unused label.
Three new functions for looking up the enum type containing a given
enumeration constant, and optionally that constant's value.
The simplest, ctf_lookup_enumerator, looks up a root-visible enumerator by
name in one dict: if the dict contains multiple such constants (which is
possible for dicts created by older versions of the libctf deduplicator),
ECTF_DUPLICATE is returned.
The next simplest, ctf_lookup_enumerator_next, is an iterator which returns
all enumerators with a given name in a given dict, whether root-visible or
not.
The most elaborate, ctf_arc_lookup_enumerator_next, finds all
enumerators with a given name across all dicts in an entire CTF archive,
whether root-visible or not, starting looking in the shared parent dict;
opened dicts are cached (as with all other ctf_arc_*lookup functions) so
that repeated use does not incur repeated opening costs.
All three of these return enumerator values as int64_t: unfortunately, API
compatibility concerns prevent us from doing the same with the other older
enum-related functions, which all return enumerator constant values as ints.
We may be forced to add symbol-versioning compatibility aliases that fix the
other functions in due course, bumping the soname for platforms that do not
support such things.
ctf_arc_lookup_enumerator_next is implemented as a nested ctf_archive_next
iterator, and inside that, a nested ctf_lookup_enumerator_next iterator
within each dict. To aid in this, add support to ctf_next_t iterators for
iterators that are implemented in terms of two simultaneous nested iterators
at once. (It has always been possible for callers to use as many nested or
semi-overlapping ctf_next_t iterators as they need, which is one of the
advantages of this style over the _iter style that calls a function for each
thing iterated over: the iterator change here permits *ctf_next_t iterators
themselves* to be implemented by iterating using multiple other iterators as
part of their internal operation, transparently to the caller.)
Also add a testcase that tests all these functions (which is fairly easy
because ctf_arc_lookup_enumerator_next is implemented in terms of
ctf_lookup_enumerator_next) in addition to enumeration addition in
ctf_open()ed dicts, ctf_add_enumerator duplicate enumerator addition, and
conflicting enumerator constant deduplication.
include/
* ctf-api.h (ctf_lookup_enumerator): New.
(ctf_lookup_enumerator_next): Likewise.
(ctf_arc_lookup_enumerator_next): Likewise.
libctf/
* libctf.ver: Add them.
* ctf-impl.h (ctf_next_t) <ctn_next_inner>: New.
* ctf-util.c (ctf_next_copy): Copy it.
(ctf_next_destroy): Destroy it.
* ctf-lookup.c (ctf_lookup_enumerator): New.
(ctf_lookup_enumerator_next): New.
* ctf-archive.c (ctf_arc_lookup_enumerator_next): New.
* testsuite/libctf-lookup/enumerator-iteration.*: New test.
* testsuite/libctf-lookup/enum-ctf-2.c: New test CTF, used by the
above.
This was always an error, because the ctn_fp routinely has errors set on it,
which is not something you can (or should) do to a const object.
libctf/
* ctf-impl.h (ctf_next_) <cu.ctn_fp>: Make non-const.
libctf has long prohibited addition of enums with overlapping constants in a
single enum, but now that we are properly considering enums with overlapping
constants to be conflciting types, we can go further and prohibit addition
of enumeration constants to a dict if they already exist in any enum in that
dict: the same rules as C itself.
We do this in a fashion vaguely similar to what we just did in the
deduplicator, by considering enumeration constants as identifiers and adding
them to the core type/identifier namespace, ctf_dict_t.ctf_names. This is a
little fiddly, because we do not want to prohibit opening of existing dicts
into which the deduplicator has stuffed enums with overlapping constants!
We just want to prohibit the addition of *new* enumerators that violate that
rule. Even then, it's fine to add overlapping enumerator constants as long
as at least one of them is in a non-root type. (This is essential for
proper deduplicator operation in cu-mapped mode, where multiple compilation
units can be smashed into one dict, with conflicting types marked as
hidden: these types may well contain overlapping enumerators.)
So, at open time, keep track of all enums observed, then do a third pass
through the enums alone, adding each enumerator either to the ctf_names
table as a mapping from the enumerator name to the enum it is part of (if
not already present), or to a new ctf_conflicting_enums hashtable that
tracks observed duplicates. (The latter is not used yet, but will be soon.)
(We need to do a third pass because it's quite possible to have an enum
containing an enumerator FOO followed by a type FOO: since they're processed
in order, the enumerator would be processed before the type, and at that
stage it seems nonconflicting. The easiest fix is to run through the
enumerators after all type names are interned.)
At ctf_add_enumerator time, if the enumerator to which we are adding a type
is root-visible, check for an already-present name and error out if found,
then intern the new name in the ctf_names table as is done at open time.
(We retain the existing code which scans the enum itself for duplicates
because it is still an error to add an enumerator twice to a
non-root-visible enum type; but we only need to do this if the enum is
non-root-visible, so the cost of enum addition is reduced.)
Tested in an upcoming commit.
libctf/
* ctf-impl.h (ctf_dict_t) <ctf_names>: Augment comment.
<ctf_conflicting_enums>: New.
(ctf_dynset_elements): New.
* ctf-hash.c (ctf_dynset_elements): Implement it.
* ctf-open.c (init_static_types): Split body into...
(init_static_types_internal): ... here. Count enumerators;
keep track of observed enums in pass 2; populate ctf_names and
ctf_conflicting_enums with enumerators in a third pass.
(ctf_dict_close): Free ctf_conflicting_enums.
* ctf-create.c (ctf_add_enumerator): Prohibit addition of duplicate
enumerators in root-visible enum types.
include/
* ctf-api.h (CTF_ADD_NONROOT): Describe what non-rootness
means for enumeration constants.
(ctf_add_enumerator): The name is not a misnomer.
We now require that enumerators have unique names.
Document the non-rootness of enumerators.
The libctf-regression/open-error-free.c test works by interposing malloc
and counting mallocs and frees across libctf operations. This only
works under suitably-interposable mallocs on systems supporting
dlsym (RTLD_NEXT, ...), so its operation is restricted to glibc
systems for now, but also it interacts badly with valgrind, which
interposes malloc itself. Detect a running valgrind and skip the test.
Add new facilities allowing libctf lookup tests to declare themselves
unsupported, by printing "UNSUPPORTED: " and then some meaningful
message instead of their normal output.
libctf/
* configure.ac: Check for <valgrind/valgrind.h>.
* config.h.in: Regenerate.
* configure: Likewise.
* testsuite/lib/ctf-lib.exp (run_lookup_test): Add support for
UNSUPPORTED tests.
* testsuite/libctf-regression/open-error-free.c: When running
under valgrind, this test is unsupported.
If a lookup fails for a reason unrelated to a lack of type data for this
symbol, we return with an error; but we fail to close the dict we opened
most recently, which is leaked.
libctf/
* ctf-archive.c (ctf_arc_lookup_sym_or_name): Close dict.
If ctf_add_type failed in the middle of enumerator addition, the
destination would end up containing the source enum type and some
but not all of its enumerator constants.
Use snapshots to roll back the enum addition as a whole if this happens.
Before now, it's been pretty unlikely, but in an upcoming commit we will ban
addition of enumerators that already exist in a given dict, making failure
of ctf_add_enumerator and thus of this part of ctf_add_type much more
likely.
libctf/
* ctf-create.c (ctf_add_type_internal): Roll back if enum or
enumerator addition fails.
The CTF deduplicator was not considering enumerators inside enum types to be
things that caused type conflicts, so if the following two TUs were linked
together, you would end up with the following in the resulting dict:
1.c:
enum foo { A, B };
2.c:
enum bar { A, B };
linked:
enum foo { A, B };
enum bar { A, B };
This does work -- but it's not something that's valid C, and the general
point of the shared dict is that it is something that you could potentially
get from any valid C TU.
So consider such types to be conflicting, but obviously don't consider
actually identical enums to be conflicting, even though they too have (all)
their identifiers in common. This involves surprisingly little code. The
deduplicator detects conflicting types by counting types in a hash table of
hash tables:
decorated identifier -> (type hash -> count)
where the COUNT is the number of times a given hash has been observed: any
name with more than one hash associated with it is considered conflicting
(the count is used to identify the most common such name for promotion to
the shared dict).
Before now, those identifiers were all the identifiers of types (possibly
decorated with their namespace on the front for enumerator identifiers), but
we can equally well put *enumeration constant names* in there, undecorated
like the identifiers of types in the global namespace, with the type hash
being the hash of each enum containing that enumerator. The existing
conflicting-type-detection code will then accurately identify distinct enums
with enumeration constants in common. The enum that contains the most
commonly-appearing enumerators will be promoted to the shared dict.
libctf/
* ctf-impl.h (ctf_dedup_t) <cd_name_counts>: Extend comment.
* ctf-dedup.c (ctf_dedup_count_name): New, split out of...
(ctf_dedup_populate_mappings): ... here. Call it for all
* enumeration constants in an enum as well as types.
ld/
* testsuite/ld-ctf/enum-3.c: New test CTF.
* testsuite/ld-ctf/enum-4.c: Likewise.
* testsuite/ld-ctf/overlapping-enums.d: New test.
* testsuite/ld-ctf/overlapping-enums-2.d: Likewise.
ctf_str_add_ref and ctf_str_add_movable_ref take a string and are supposed
to return a strtab offset. These offsets are "provisional": the ref
mechanism records the address of the location in which the ref is stored and
modifies it when the strtab is finally written out. Provisional refs in new
dicts start at 0 and go up via strlen() as new refs are added: this is fine,
because the strtab is empty and none of these values will overlap any
existing string offsets (since there are none). Unfortunately, when a dict
is ctf_open()ed, we fail to set the initial provisional strtab offset to a
higher value than any existing string offset: it starts at zero again!
It's a shame that we already *have* strings at those offsets...
This is all fixed up once the string is reserialized, but if you look up
newly-added strings before serialization, you get corrupted partial string
results from the existing ctf_open()ed dict.
Observed (and thus regtested) by an upcoming test (in this patch series).
Exposed by the recently-introduced series that permits modification of
ctf_open()ed dicts, which has not been released anywhere. Before that,
any attempt to do such things would fail with ECTF_RDONLY.
libctf/
* ctf-string.c (ctf_str_create_atoms): Initialize
ctf_str_prov_offset.
Ever since commit 1fa7a0c24e ("libctf: sort out potential refcount
loops") ctf_dict_close has only freed anything if the refcount on entry
to the function is precisely 1. >1 obviously just decrements the
refcount, but the linker machinery can sometimes cause freeing to recurse
from a dict to another dict and then back to the first dict again, so
we interpret a refcount of 0 as an indication that this is a recursive call
and we should just return, because a caller is already freeing this dict.
Unfortunately there is one situation in which this is not true: the bad:
codepath in ctf_bufopen entered when opening fails. Because the refcount is
bumped only at the very end of ctf_bufopen, any failure causes
ctf_dict_close to be entered with a refcount of zero, and it frees nothing
and we leak the entire dict.
The solution is to bump the refcount to 1 right before freeing... but this
codepath is clearly delicate enough that we need to properly validate it,
so we add a test that uses malloc interposition to count allocations and
frees, creates a dict, writes it out, intentionally corrupts it (by setting
a bunch of bytes after the header to a value high enough that it is
definitely not a valid CTF type kind), then tries to open it again and
counts the malloc/free pairs to make sure they're matched. (Test run only
on *-linux-gnu, because malloc interposition is not a thing you can rely
upon working everywhere, and this test is not arch-dependent so if it
passes on one arch it can be assumed to pass on all of them.)
libctf/
* ctf-open.c (ctf_bufopen): Bump the refcount on failure.
* testsuite/libctf-regression/open-error-free.*: New test.
This .lk option lets you run the lookup program via a wrapper executable.
For example, to run under valgrind and check for leaks (albeit noisily
because of the libtool shell script wrapper):
libctf/
* testsuite/lib/ctf-lib.exp (run_lookup_test): Add wrapper.
This .lk option lets you execute particular tests only on specific host
architectures.
libctf/
* testsuite/lib/ctf-lib.exp (run_lookup_test): Add host.
This .lk option lets you link the lookup program with extra libraries
in addition to -lctf.
libctf/
* testsuite/lib/ctf-lib.exp (run_lookup_test): Add lookup_link.
If iteration fails because opening a dict has failed, ctf_archive_next does
not destroy the iterator, so the caller can keep going and try to open other
dicts further into the archive. ctf_archive_iter just returns, though, so
it should free the iterator rather than leaking it.
libctf/
* ctf-archive.c (ctf_archive_iter): Don't leak the iterator on
failure.
CTF archive member opening (via ctf_arc_open_by_name, ctf_archive_iter, et
al) attempts to be helpful and auto-open and import any needed parent dict
in the same archive. But if this fails, the error is not reported but
simply discarded, and you silently get back a dict with no parent, that
*you* suddenly have to remember to import.
This is not helpful behaviour: if the parent is corrupted or we run out of
memory or something, the caller is going to want to know! Split it in two:
if the dict cites a parent that doesn't exist at all (a lot of historic
dicts name "PARENT" as their parent, even when they're not even children, or
perhaps the parent dict is stored separately and you plan to manually
associate it), we skip it as now, but if the import fails with an actual
error other than ECTF_ARNNAME, return the error and fail the open.
libctf/
* ctf-archive.c (ctf_arc_import_parent): Return failure if
parent opening fails for reasons other thnn nonexistence.
(ctf_dict_open_sections): Adjust.
Seen on 64-bit targets.
ERROR: compilation of lookup program .../libctf-regression/gzrewrite.c failed
* testsuite/libctf-regression/gzrewrite.c (main): Use %zu to
print size_t values.
* testsuite/libctf-regression/zrewrite.c (main): Likewise.
libctf's version script is applied to two libraries: libctf.so,
and libctf-nobfd.so. The latter library is a subset of the former
which does not link to libbfd and does not include a few public
entry points that use it (found in libctf-open-bfd.c). This means
that some of the symbols in this version script only exist in one
of the libraries it's applied to.
A number of linkers dislike this: before now, only Solaris's linker
caused serious problems, introducing NOTYPE-typed symbols when such
things were found, but now LLD has started to complain as well:
ld: error: version script assignment of 'LIBCTF_1.0' to symbol 'ctf_arc_open' failed: symbol not defined
ld: error: version script assignment of 'LIBCTF_1.0' to symbol 'ctf_fdopen' failed: symbol not defined
ld: error: version script assignment of 'LIBCTF_1.0' to symbol 'ctf_open' failed: symbol not defined
ld: error: version script assignment of 'LIBCTF_1.0' to symbol 'ctf_bfdopen' failed: symbol not defined
ld: error: version script assignment of 'LIBCTF_1.0' to symbol 'ctf_bfdopen_ctfsect' failed: symbol not defined
Rather than adding more and more whack-a-mole fixes for every
linker we encounter that does this, simply exclude such symbols
unconditionally, using the same trick we used to use for Solaris.
(Well, unconditionally if we can use version scripts with this
linker at all, which is not always the case.)
Thanks to Nicholas Vinson for the original report and a fix very
similar to this one (but not quite identical).
libctf/
* configure.ac: Always exclude libctf symbols from
libctf-nobfd's version script.
* configure: Regenerated.
Starting with ld.lld-17, ld.lld is invoked with the option
--no-undefined-version enabled by default. Furthermore, The functions
ctf_label_set() and ctf_label_get() are not defined. Their inclusion in
libctf/libctf.ver causes ld.lld-17 to fail emitting the following error
messages:
ld.lld: error: version script assignment of 'LIBCTF_1.0' to symbol 'ctf_label_set' failed: symbol not defined
ld.lld: error: version script assignment of 'LIBCTF_1.0' to symbol 'ctf_label_get' failed: symbol not defined
This patch fixes the issue by removing the symbol names from
libctf/libctf.ver.
[nca: fused in later commit that marked ctf_arc_open as libctf
only as well. Added ChangeLog entry.]
Signed-off-by: Nicholas Vinson <nvinson234@gmail.com>
libctf/
* libctf.ver: drop nonexistent label functions: mark
ctf_arc_open as libctf-only.
The libctf-internal warning function ctf_err_warn() can be passed a libctf
errno as a parameter, and will add its textual errmsg form to the passed-in
error message. But if there is an error on the fp already, and this is
specifically an error and not a warning, ctf_err_warn() will print the error
out regardless: there's no need to pass in anything but 0.
There are still a lot of places where we do
ctf_err_warn (fp, 0, EFOO, ...);
return ctf_set_errno (fp, 0, EFOO);
I've left all of those alone, because fixing it makes the code a bit longer:
but fixing the cases where no return is involved and the error has just been
set on the fp itself costs nothing and reduces redundancy a bit.
libctf/
* ctf-dedup.c (ctf_dedup_walk_output_mapping): Drop the errno arg.
(ctf_dedup_emit): Likewise.
(ctf_dedup_type_mapping): Likewise.
* ctf-link.c (ctf_create_per_cu): Likewise.
(ctf_link_deduplicating_close_inputs): Likewise.
(ctf_link_deduplicating_one_symtypetab): Likewise.
(ctf_link_deduplicating_per_cu): Likewise.
* ctf-lookup.c (ctf_lookup_symbol_idx): Likewise.
* ctf-subr.c (ctf_assert_fail_internal): Likewise.
This purely serves to make it easier to interpret valgrind output.
No functional effect.
libctf/
* testsuite/libctf-lookup/conflicting-type-syms.c: Free everything.
Now there's a chance of it actually working, we can add more tests for
the long-broken dict read-and-rewrite cases. This is the first ever
test for the (rarely-used, unpleasant, and until recently completely
broken) ctf_gzwrite function.
libctf/
* testsuite/libctf-regression/gzrewrite*: New test.
* testsuite/libctf-regression/zrewrite*: Likewise.
