The object_p (and archive_p, core_file_p) functions are not supposed
to have any target specific malloc'd memory attached to the bfd on
their return. This should be obvious on a failure return, but it's
also true for a successful return. The reason is that even though the
object_p recognises the file, that particular target may not be used
and thus the bfd won't be closed calling close_and_cleanup for the
target that allocated the memory.
It turns out that the object_p bfd_target* return value isn't needed.
In all cases except ld/plugin.c the target is abfd->xvec and with
ld/plugin.c the target isn't used. So this patch returns a cleanup
function from object_p instead, called in bfd_check_format_matches to
tidy the bfd before trying a different target match. The only cleanup
that does anything at this stage is the alpha-vms one.
bfd/
* targets.c (bfd_cleanup): New typedef.
(struct bfd <_bfd_check_format>): Return a bfd_cleanup.
* libbfd-in.h (_bfd_no_cleanup): Define.
* format.c (bfd_reinit): Add cleanup parameter, call it.
(bfd_check_format_matches): Set cleanup from _bfd_check_format
call and pass to bfd_reinit. Delete temp, use abfd->xvec instead.
* aout-target.h (callback, object_p): Return bfd_cleanup.
* aout-tic30.c (tic30_aout_callback, tic30_aout_object_p): Likewise.
* archive.c (bfd_generic_archive_p): Likewise.
* binary.c (binary_object_p): Likewise.
* coff-alpha.c (alpha_ecoff_object_p): Likewise.
* coff-ia64.c (ia64coff_object_p): Likewise.
* coff-rs6000.c (_bfd_xcoff_archive_p, rs6000coff_core_p): Likewise.
* coff-sh.c (coff_small_object_p): Likewise.
* coff-stgo32.c (go32_check_format): Likewise.
* coff64-rs6000.c (xcoff64_archive_p, rs6000coff_core_p),
(xcoff64_core_p): Likewise.
* coffgen.c (coff_real_object_p, coff_object_p): Likewise.
* elf-bfd.h (bfd_elf32_object_p, bfd_elf32_core_file_p),
(bfd_elf64_object_p, bfd_elf64_core_file_p): Likewise.
* elfcode.h (elf_object_p): Likewise.
* elfcore.h (elf_core_file_p): Likewise.
* i386msdos.c (msdos_object_p): Likewise.
* ihex.c (ihex_object_p): Likewise.
* libaout.h (some_aout_object_p): Likewise.
* libbfd-in.h (bfd_generic_archive_p, _bfd_dummy_target),
(_bfd_vms_lib_alpha_archive_p, _bfd_vms_lib_ia64_archive_p): Likewise.
* libbfd.c (_bfd_dummy_target): Likewise.
* libcoff-in.h (coff_object_p): Likewise.
* mach-o-aarch64.c (bfd_mach_o_arm64_object_p),
(bfd_mach_o_arm64_core_p): Likewise.
* mach-o-arm.c (bfd_mach_o_arm_object_p),
(bfd_mach_o_arm_core_p): Likewise.
* mach-o-i386.c (bfd_mach_o_i386_object_p),
(bfd_mach_o_i386_core_p): Likewise.
* mach-o-x86-64.c (bfd_mach_o_x86_64_object_p),
(bfd_mach_o_x86_64_core_p): Likewise.
* mach-o.c (bfd_mach_o_header_p, bfd_mach_o_gen_object_p),
(bfd_mach_o_gen_core_p, bfd_mach_o_fat_archive_p): Likewise.
* mach-o.h (bfd_mach_o_object_p, bfd_mach_o_core_p),
(bfd_mach_o_fat_archive_p, bfd_mach_o_header_p): Likewise.
* mmo.c (mmo_object_p): Likewise.
* pef.c (bfd_pef_object_p, bfd_pef_xlib_object_p): Likewise.
* peicode.h (coff_real_object_p, pe_ILF_object_p),
(pe_bfd_object_p): Likewise.
* plugin.c (ld_plugin_object_p, bfd_plugin_object_p): Likewise.
* ppcboot.c (ppcboot_object_p): Likewise.
* rs6000-core.c (rs6000coff_core_p): Likewise.
* som.c (som_object_setup, som_object_p): Likewise.
* srec.c (srec_object_p, symbolsrec_object_p): Likewise.
* tekhex.c (tekhex_object_p): Likewise.
* vms-alpha.c (alpha_vms_object_p): Likewise.
* vms-lib.c (_bfd_vms_lib_archive_p, _bfd_vms_lib_alpha_archive_p),
(_bfd_vms_lib_ia64_archive_p, _bfd_vms_lib_txt_archive_p): Likewise.
* wasm-module.c (wasm_object_p): Likewise.
* xsym.c (bfd_sym_object_p): Likewise.
* xsym.h (bfd_sym_object_p): Likewise.
* aoutx.h (some_aout_object_p): Likewise, and callback parameter
return type.
* pdp11.c (some_aout_object_p): Likewise.
* plugin.c (register_ld_plugin_object_p): Update object_p
parameter type.
* plugin.h (register_ld_plugin_object_p): Likewise.
* bfd-in2.h: Regenerate.
* libbfd.h: Regenerate.
* libcoff.h: Regenerate.
ld/
* plugin.c (plugin_object_p): Return a bfd_cleanup.
(plugin_cleanup): New function.
See email thread starting here: https://www.sourceware.org/ml/binutils/2018-01/msg00001.html
include * coff/msdos.h: New header.
* coff/pe.h: Move common defines to msdos.h.
* coff/powerpc.h: Likewise.
bfd * i386msdos.c (msdos_mkobject); New function.
(msdos_object_p): New function.
(i386_msdos_vec): Use msdos_object_p as the check_format
function.
* peicode.h: Rename external_PEI_DOS_hdr, DOSMAGIC, and
NT_SIGNATURE to external_DOS_hdr, IMAGE_DOS_SIGNATURE, and
IMAGE_NT_SIGNATURE.
* peXXigen.c: Likewise.
* coff-ia64.c: Likewise.
PR binutils/17512
* dwarf.c (display_block): Do nothing if the block starts after
the end of the buffer.
(read_and_display_attr_value): Add range checks.
(struct Frame_Chunk): Make the ncols and ra fields unsigned.
(frame_need_space): Test for an ncols of zero.
(read_cie): Fail if the augmentation data extends off the end of
the buffer.
(display_debug_frames): Add checks for read_cie failing. Add
range checks.
* coff-h8300.c (rtype2howto): Replace abort with returning a NULL
value.
* coff-h8500.c (rtype2howto): Likewise.
* coff-tic30.c (rtype2howto): Likewise.
* coff-z80.c (rtype2howto): Likewise.
* coff-z8k.c (rtype2howto): Likewise.
* coff-ia64.c (RTYPE2HOWTO): Always return a valid howto.
* coff-m68k.c (m68k_rtype2howto): Return a NULL howto if none
could be found.
* coff-mcore.c (RTYPE2HOWTO): Add range checking.
* coff-w65.c (rtype2howto): Likewise.
* coff-we32k.c (RTYPE2HOWTO): Likewise.
* pe-mips.c (RTYPE2HOWTO): Likewise.
* coff-x86_64.c (coff_amd64_reloc): Likewise. Replace abort with
an error return.
* coffcode.h (coff_slurp_reloc_table): Allow the rel parameter to
be unused.
* coffgen.c (make_a_section_from_file): Check the length of a
section name before testing to see if it is a debug section name.
(coff_object_p): Zero out any uninitialised bytes in the opt
header.
* ecoff.c (_bfd_ecoff_slurp_symbolic_info): Test for the raw
source being empty when there are values to be processed.
(_bfd_ecoff_slurp_symbol_table): Add range check.
* mach-o.c (bfd_mach_o_canonicalize_one_reloc): Likewise.
(bfd_mach_o_mangle_sections): Move test for too many sections to
before the allocation of the section table.
(bfd_mach_o_read_symtab_strtab): If the read fails, free the
memory and nullify the symbol pointer.
* reloc.c (bfd_generic_get_relocated_section_contents): Add
handling of a bfd_reloc_notsupported return value.
* versados.c (EDATA): Add range checking.
(get_record): Likewise.
(process_otr): Check for contents being available before updating
them.
(versados_canonicalize_reloc): Add range check.
o bfd_read and bfd_write lose an unnecessary param and become
bfd_bread and bfd_bwrite.
o bfd_*alloc now all take a bfd_size_type arg, and will error if
size_t is too small. eg. 32 bit host, 64 bit bfd, verrry big files
or bugs in linker scripts etc.
o file_ptr becomes a bfd_signed_vma. Besides matching sizes with
various other types involved in handling sections, this should make
it easier for bfd to support a 64 bit off_t on 32 bit hosts that
provide it.
o I've made the H_GET_* and H_PUT_* macros (which invoke bfd_h_{get,put}_*)
generally available. They now cast their args to bfd_vma and
bfd_byte * as appropriate, which removes a swag of casts from the
source.
o Bug fixes to bfd_get8, aix386_core_vec, elf32_h8_relax_section, and
aout-encap.c.
o Zillions of formatting and -Wconversion fixes.
as _bfd_XXi_only_swap_filehdr_out.
(pe_mkobject) [PEI_FORCE_MINIMUM_ALIGNMENT]: Set
pe->force_minimum_alignment to TRUE.
(pe_mkobject) [PEI_TARGET_SUBSYSTEM]: Set pe->target_subsystem to
PEI_TARGET_SUBSYSTEM.
(pe_print_private_bfd_data): Call
_bfd_XX_print_private_bfd_data_common() instead of
_bfd_pe_print_private_bfd_data_common().
(pe_bfd_copy_private_bfd_data): Call
_bfd_XX_bfd_copy_private_bfd_data_common() instead of
_bfd_pe_bfd_copy_private_bfd_data_common().
(coff_bfd_copy_private_section_data): Define as
_bfd_XX_bfd_copy_private_section_data instead of
_bfd_pe_bfd_copy_private_section_data.
(coff_get_symbol_info): Define as _bfd_XX_get_symbol_info instead
of a _bfd_pe_get_symbol_info.
* peigen.c: Delete.
* peXXigen.c: Renamed from peigen.c.
(COFF_WITH_XX): Define this macro (will get expanded into
COFF_WITH_pep or COFF_WITH_pe, depending on whether this is being
compiled as peigen.c or pepigen.c.
[COFF_WITH_pep]: Include "coff/ia64.h" instead of "coff/i386.h" to
define the canonical PEP structures and definitions.
(_bfd_XXi_swap_aouthdr_out): If pe->force_minimum_alignment is in
effect, enforce minimum file and section alignments. If
extra->Subsystem is IMAGE_SUBSYSTEM_UNKNOWN, set it to
pe->target_subsystem (this defaults to IMAGE_SUBSYSTEM_UNKNOWN,
so, by default, this is a no-op).
* libpei.h: Rename COFF_WITH_PEP to COFF_WITH_pep.
(_bfd_XX_bfd_copy_private_bfd_data_common): Add macros to map
_bfd_XXfoo to _bfd_pepfoo if COFF_WIT_PEP is defined and to
_bfd_pefoo if it's not defined. Use these macros to define
coff swap macros.
* libcoff.h (pe_tdata): Add members target_subsystem and
force_minimum_alignment.
* efi-app-ia64.c (COFF_WITH_pep): Rename COFF_WITH_PEP to
COFF_WITH_pep.
(PEI_TARGET_SUBSYSTEM): Rename from PEI_DEFAULT_TARGET_SUBSYSTEM.
* configure.in (bfd_efi_app_ia64_vec): Use pepigen.lo instead of
peigen.lo.
* coff-ia64.c: Rename COFF_WITH_PEP to COFF_WITH_pep.
(AOUTSZ): Rename PEP64AOUTSZ and PEP64AOUTHDR to PEPAOUTSZ and
PEPAOUTHDR.
* Makefile.in (BFD64_BACKENDS): Mention pepigen.lo.
(BFD64_BACKENDS_CFILES): Mention pepigen.c
(peigen.c): Add rule to generate from peXXigen.c.
(pepigen.c): Ditto.
(pepigen.lo): List dependencies for pepigen.lo.
* unwind-ia64.c (unw_decode_x1): Declare code arg with
ATTRIBUTE_UNUSED.
(unw_decode_x2): Ditto.
(unw_decode_x3): Ditto.
(unw_decode_x4): Ditto.
* pe.h (PEPAOUTSZ): Rename from PEP64AOUTSZ.
Rename from PEPAOUTHDR.
