Testing of the first code to generate CTF_K_SLICEs on big-endian
revealed a bunch of new problems in this area. Most importantly, the
trick we did earlier to avoid wasting two bytes on padding in the
ctf_slice_t is best avoided: because it leads to the whole file after
that point no longer being naturally aligned, all multibyte accesses
from then on must use memmove() to avoid unaligned access on platforms
where that is fatal. In future, this is planned, but for now we are
still doing direct access in many places, so we must revert to making
ctf_slice_t properly aligned for storage in an array.
Rather than wasting bytes on padding, we boost the size of cts_offset
and cts_bits. This is still a waste of space (we cannot have offsets or
bits in bitfields > 256) but it cannot be avoided for now, and slices
are not so common that this will be a serious problem.
A possibly-worse endianness problem fixed at the same time involves
a codepath used only for foreign-endian, uncompressed CTF files, where
we were not copying the actual CTF data into the buffer, leading to
libctf reading only zeroes (or, possibly, uninitialized garbage).
Finally, when we read in a CTF file, we copy the header and work from
the copy. We were flipping the endianness of the header copy, and of
the body of the file buffer, but not of the header in the file buffer
itself: so if we write the file back out again we end up with an
unreadable frankenfile with header and body of different endiannesses.
Fix by flipping both copies of the header.
include/
* ctf.h (ctf_slice_t): Make cts_offset and cts_bits unsigned
short, so following structures are properly aligned.
libctf/
* ctf-open.c (get_vbytes_common): Return the new slice size.
(ctf_bufopen): Flip the endianness of the CTF-section header copy.
Remember to copy in the CTF data when opening an uncompressed
foreign-endian CTF file. Prune useless variable manipulation.
This stops the file format from depending on the size of the host int.
(It does mean that we cannot encode enums with a value > 2^32 on
platforms with an int > 2^32: this will be fixed in the next format
revision.)
include/
* ctf.h (ctf_enum.cte_value): Fix type to int32_t.
If you need to store a large number of CTF containers somewhere, this
provides a dedicated facility for doing so: an mmappable archive format
like a very simple tar or ar without all the system-dependent format
horrors or need for heavy file copying, with built-in compression of
files above a particular size threshold.
libctf automatically mmap()s uncompressed elements of these archives, or
uncompresses them, as needed. (If the platform does not support mmap(),
copying into dynamically-allocated buffers is used.)
Archive iteration operations are partitioned into raw and non-raw
forms. Raw operations pass thhe raw archive contents to the callback:
non-raw forms open each member with ctf_bufopen() and pass the resulting
ctf_file_t to the iterator instead. This lets you manipulate the raw
data in the archive, or the contents interpreted as a CTF file, as
needed.
It is not yet known whether we will store CTF archives in a linked ELF
object in one of these (akin to debugdata) or whether they'll get one
section per TU plus one parent container for types shared between them.
(In the case of ELF objects with very large numbers of TUs, an archive
of all of them would seem preferable, so we might just use an archive,
and add lzma support so you can assume that .gnu_debugdata and .ctf are
compressed using the same algorithm if both are present.)
To make usage easier, the ctf_archive_t is not the on-disk
representation but an abstraction over both ctf_file_t's and archives of
many ctf_file_t's: users see both CTF archives and raw CTF files as
ctf_archive_t's upon opening, the only difference being that a raw CTF
file has only a single "archive member", named ".ctf" (the default if a
null pointer is passed in as the name). The next commit will make use
of this facility, in addition to providing the public interface to
actually open archives. (In the future, it should be possible to have
all CTF sections in an ELF file appear as an "archive" in the same
fashion.)
This machinery is also used to allow library-internal creators of
ctf_archive_t's (such as the next commit) to stash away an ELF string
and symbol table, so that all opens of members in a given archive will
use them. This lets CTF archives exploit the ELF string and symbol
table just like raw CTF files can.
(All this leads to somewhat confusing type naming. The ctf_archive_t is
a typedef for the opaque internal type, struct ctf_archive_internal: the
non-internal "struct ctf_archive" is the on-disk structure meant for
other libraries manipulating CTF files. It is probably clearest to use
the struct name for struct ctf_archive_internal inside the program, and
the typedef names outside.)
libctf/
* ctf-archive.c: New.
* ctf-impl.h (ctf_archive_internal): New type.
(ctf_arc_open_internal): New declaration.
(ctf_arc_bufopen): Likewise.
(ctf_arc_close_internal): Likewise.
include/
* ctf.h (CTFA_MAGIC): New.
(struct ctf_archive): New.
(struct ctf_archive_modent): Likewise.
* ctf-api.h (ctf_archive_member_f): New.
(ctf_archive_raw_member_f): Likewise.
(ctf_arc_write): Likewise.
(ctf_arc_close): Likewise.
(ctf_arc_open_by_name): Likewise.
(ctf_archive_iter): Likewise.
(ctf_archive_raw_iter): Likewise.
(ctf_get_arc): Likewise.
The data structures and macros in this header can be used, if desired,
to access or create CTF files directly, without going through libctf,
though this should rarely be necessary in practice.
libctf relies on this header as its description of the CTF file format.
include/
* ctf.h: New file.
