At present we sometimes see problems in gitlab where the environment has
0x80 characters or sequences which are not valid UTF-8.
Avoid this by using bytes for the environment, both internal to buildman
and when writing out the 'env' file. Add a test to make sure this works
as expected.
Reported-by: Marek Vasut <marex@denx.de>
Fixes: e5fc79ea71 ("buildman: Write the environment out to an 'env' file")
Signed-off-by: Simon Glass <sjg@chromium.org>
There have been at least a few cases where an exception has occurred in a
thread and resulted in buildman hanging: running out of disk space and
getting a unicode error.
Handle these by collecting a list of exceptions, printing them out and
reporting failure if any are found. Add a test for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
When used with hierarchical images, use the Chromium OS convention of
adding a section before all the subentries it contains.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use an interator in two of the fmap tests so it is easier to add new
items. Also check the name first since that is the first indication
that something is wrong. Use a variable for the expected size of the
fmap to avoid repeating the code.
Signed-off-by: Simon Glass <sjg@chromium.org>
Parse each empty-line-delimited message separately. This saves having to
deal with all the different line content styles, we only care about the
header ERROR | WARNING | NOTE...
Also make checkpatch print line information for a uboot specific
warning.
Signed-off-by: Evan Benn <evanbenn@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Given that we have tests that require pygit2 and it can be installed
like any other python module, fail much more loudly if it is missing.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
At present each invocation of run_steps() updates OUTPUT_FILES_COMMON,
since it does not make a copy of the dict. This is fine for a single
invocation, but for tests, run_steps() is invoked many times.
As a result it may include unwanted items from the previous run, if it
happens that a test runs twice on the same CPU. The problem has not been
noticied previously, as there are few enough tests and enough CPUs that
is is rare for the 'wrong' combination of tests to run together.
Fix this by making a copy of the dict, before updating it. Update the
tests to suit, taking account of the files that are no-longer generated.
With this fix, we no-longer generate files which are not needed for a
particular state of OF_PLATDATA_INST, so the check_instantiate() function
is not needed anymore. It has become dead code and so fails the
code-coverage test (dtoc -T). Remove it.
Signed-off-by: Simon Glass <sjg@chromium.org>
These two tests require an ELF image so that symbol information can be
written into the SPL/TPL binary. At present they rely on other tests
having set it up first, but every test must run independently. This can
cause occasional errors in CI.
Fix this by setting up the required files, as other tests do.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
It's not always desirable to use 'keydir' and some ad-hoc heuristics
to get the filename of the signing key. More often, just passing the
filename is the simpler, easier, and logical thing to do.
Since mkimage doesn't use long options, we're slowly running out of
letters. I've chosen '-G' because it was available.
Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
mkimage supports rsa2048, and rsa4096 signatures. With newer silicon
now supporting hardware-accelerated ECDSA, it makes sense to expand
signing support to elliptic curves.
Implement host-side ECDSA signing and verification with libcrypto.
Device-side implementation of signature verification is beyond the
scope of this patch.
Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
fdt_add_bignum() is useful for algorithms other than just RSA. To
allow its use for ECDSA, move it to a common file under lib/.
The new file is suffixed with '-libcrypto' because it has a direct
dependency on openssl. This is due to the use of the "BIGNUM *" type.
Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
rsa-checksum.c sontains the hash_calculate() implementations. Despite
the "rsa-" file prefix, this function is useful for other algorithms.
To prevent confusion, move this file to lib/, and rename it to
hash-checksum.c, to give it a more "generic" feel.
Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Skip the processing of *.aml and *.dat files while iterating through the
source in order to process header files.
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
strn(cat|cpy) has a bad habit of not nul-terminating the destination,
resulting in constructions like
strncpy(foo, bar, sizeof(foo) - 1);
foo[sizeof(foo) - 1] = '\0';
However, it is very easy to forget about this behavior and accidentally
leave a string unterminated. This has shown up in some recent coverity
scans [1, 2] (including code recently touched by yours truly).
Fortunately, the guys at OpenBSD came up with strl(cat|cpy), which always
nul-terminate strings. These functions are already in U-Boot, so we should
encourage new code to use them instead of strn(cat|cpy).
[1] https://lists.denx.de/pipermail/u-boot/2021-March/442888.html
[2] https://lists.denx.de/pipermail/u-boot/2021-January/438073.html
Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Show short arguments along with long arguments in online help:
$ tools/mkeficapsule -h
Usage: mkeficapsule [options] <output file>
Options:
-f, --fit <fit image> new FIT image file
-r, --raw <raw image> new raw image file
-i, --index <index> update image index
-I, --instance <instance> update hardware instance
-K, --public-key <key file> public key esl file
-D, --dtb <dtb file> dtb file
-O, --overlay the dtb file is an overlay
-h, --help print a help message
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Bitmap files should not be executable.
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Michael Trimarchi <michael@amarulasolutions.com>
Fix the warning by set the variable zero to uint64_t
"warning: ‘write’ reading 5 bytes from a region of size 4"
Signed-off-by: Peng Fan <peng.fan@nxp.com>
Integrate the Dockerfile from
https://source.denx.de/u-boot/gitlab-ci-runner.git as of
commit bc6130d572f1 ("Dockerfile: Remove high UID/GID") and introduce a
short rST on how to build the container.
Cc: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Tom Rini <trini@konsulko.com>
This existing code assumes that a reg property is larger than one cell,
but this is not always the case. Fix this assumption.
Also if a node's parent is missing the #address-cells and #size-cells
properties we use 2 as a default for each. But this should not happen in
practice. More likely the properties were removed for SPL due to there
being no 'u-boot,dm-pre-reloc' property, or similar. Add a warning for
this as the failure can be very confusing.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present an empty size is considered to be a 64-bit value. This does not
seem useful and wastes space. Limit the 64-bit detection to where one or
both of the addr/size is two cells or more.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present warnings are shown as soon as they are discovered in the
source scannner. But the function that detects them may be called multiple
times.
Collect all the warnings and show them at the end.
Signed-off-by: Simon Glass <sjg@chromium.org>
The environment may contain some unicode characters. At least that is what
seemed to happen on one commit:
Building current source for 1 boards (0 threads, 64 jobs per thread)
0 0 0 /1 -1 (starting)
Traceback (most recent call last):
File ".../tools/buildman/buildman", line 64, in <module>
ret_code = control.DoBuildman(options, args)
File "tools/buildman/control.py", line 372, in DoBuildman
options.keep_outputs, options.verbose)
File ".../tools/buildman/builder.py", line 1704, in BuildBoards
results = self._single_builder.RunJob(job)
File ".../tools/buildman/builderthread.py", line 526, in RunJob
self._WriteResult(result, job.keep_outputs, job.work_in_output)
File ".../tools//buildman/builderthread.py", line 349, in _WriteResult
print('%s="%s"' % (var, env[var]), file=fd)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
311-312: ordinal not in range(128)
The problem defies repetition with any change at all to buildman. But
let's set an encoding in any case.
Signed-off-by: Simon Glass <sjg@chromium.org>
While we cannot know which commit the warning relates to, this should not
be fatal. Print the warning and carry on.
Signed-off-by: Simon Glass <sjg@chromium.org>
On Arm-based Macs, -no_pie is ignored and gives a linker warning.
Moreover, the build falls over with:
ld: Absolute addressing not allowed in arm64 code but used in '_image_type_ptr_aisimage' referencing '_image_type_aisimage'
for dumpimage and mkimage, since we put data structs in text sections
not data sections and so cannot have dynamic relocations. Instead, move
the sections to __DATA and drop disabling PIE.
Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Add a few more internal checks to make sure offsets are correct, before
updating the dtb.
To make this easier, update the functions which add a property to return
that property,.
Signed-off-by: Simon Glass <sjg@chromium.org>
So far we have only needed to add subnodes to empty notds, so have not
had to deal with ordering. However this feature is needed for binman's
expanded nodes, since there may be another node in the same section.
While libfdt adds new properties after existing properties, it adds new
subnodes before existing subnodes. This means that we must reorder the
nodes in the cached version, so that the ordering remains consistent.
Update the sync implementation to sync existing subnodes first, then
add new ones, then tidy up the ordering in the cached version. Update the
test to cover this behaviour.
Also improve the comment about property syncing while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a new test that adds a subnode alongside an existing one, as well as
adding properties to a subnode. This will expand to adding multiple
subnodes in future patches. Put a node after the one we are adding to so
we can check that things sync correctly.
The testAddNode() test should be in the TestNode class since it is a node
test, so move it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Once the tree has been synced, thus potentially moving things around in the
fdt, we set _cached_offsets to False so that a refresh will happen next
time a property is accessed.
This 'lazy' refresh doesn't really save much time, since refresh is a very
fast operation, just a single walk of the tree. Also, having the refresh
happen in the bowels of property access it makes it harder to figure out
what is going on.
Simplify the code by always doing a refresh before and after a sync. Set
_cached_offsets to True immediately after this, in the Refresh() function,
since this makes more sense than doing it in the caller.
Signed-off-by: Simon Glass <sjg@chromium.org>
If a property does not yet have an offset, then that means it exists in
the cache'd fdt but has not yet been synced back to the flat tree. Use
the dirty flag for this so we don't need to check the offset too. Improve
the comments for Prop and Node to make it clear what an offset of None
means.
Also clear the dirty flag after the property is synced.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful to specify the default alignment for all entries
in a section, such as when word-alignment is necessary, for example. It
is tedious and error-prone to specify this individually for each section.
Add a property to control this for a section.
Signed-off-by: Simon Glass <sjg@chromium.org>
Generally the content of sections is not built until the final assembly
of the image. This is partly to avoid wasting time, since the entries
within sections may change multiple times as binman works through its
various stages. This works quite well since sections exist in a strict
hierarchy, so they can be processed in a depth-first manner.
However the 'collection' entry type does not have this luxury. If it
contains a section within its 'content' list, then it must produce the
section contents, if available. That section is typically a sibling
node, i.e. not part oc the collection's hierarchy.
Add a new 'required' argument to section.GetData() to support this. When
required is True, any referenced sections are immediately built. If this
is not possible (because one of the subentries does not have its data yet)
then an error is produced.
The test for this uses a 'collection' entry type, referencing a section as
its first member. This forces a call to _BuildSectionData() with required
set to False, at first, then True later, when the image is assembled.
Signed-off-by: Simon Glass <sjg@chromium.org>
The vblock entry type includes code to collect the data from a number of
other entries (not necessarily subentries) and concatenating it. This is
a useful feature for other entry types.
Make it a base class, so that vblock can use it, along with other entry
types.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present there is a command-line flag to disable substitution of expanded
entries. Add an option to the entry node as well, so it can be controlled
at the node level.
Add a test to cover this. Fix up the comment to the checkSymbols() function
it uses, while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a link to binman's documentation and adjust the files so that it is
accessible. Use the name README.rst so it is easy to discover when binman
is installed without U-Boot.
Signed-off-by: Simon Glass <sjg@chromium.org>
When creating an entry, check for an expanded version of that entry, then
use it instead. This allows, for example use of:
u-boot {
};
instead of having to write out in full:
u-boot {
type = "section";
u-boot-nodtb {
};
u-boot-dtb {
};
};
Add an implementaion of this and associated documentation.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for this feature in the control, image and section modules, so
that expanded entries will be selected by default. So far there are no
expanded entry types, so this is a nop.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a new command-line option to disable expanded entries. This is needed
for most tests, since it is much easier to 'factor out' this function into
a separate test and keep the existing packing tests simple.
Add the option and select it by default from tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
As the first step in supporting expanded entries, add a way for binman to
automatically select an 'expanded' version of an entry type, if requested.
This is controlled by a class method.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present, before any entry expansion is done (such as a 'files' entry
expanding out to individual entries for each file it contains), we check
the binman definition (i.e. '/binman' node) to find out what devicetree
files are used in the images.
This is a pain, since the definition may change during expansion. For
example if there is no u-boot-spl-dtb entry in the definition at the start,
we assume that the SPL devicetree is not used. But if an entry later
expands to include this, then we don't notice.
In fact the flexibility provided by the current approach of checking the
definition is not really useful. We know that we can have SPL and TPL
devicetrees. We know the pathname to each, so we can simply check if the
files are present. If they are present, we can prepare them and update
them regardless of whether they are actually used. If they are not present,
we cannot prepare/update them anyway, i.e. an error will be generated.
Simplify state.Prepare() so it uses a hard-coded list of devicetree files.
Note that state.PrepareFromLoadedData() is left untouched, since in that
case we have a complete definition from the loaded file, but cannot of
course rely on the devicetree files that created it still being present.
So in that case we still check the image defitions.
Signed-off-by: Simon Glass <sjg@chromium.org>
