Commit Graph

78686 Commits

Author SHA1 Message Date
Thomas Huth
100a5efbbc travis.yml: Test also the other targets on s390x
s390x is our only big endian host in our CI, so building and testing QEMU
there is quite valuable. Thus let's also test the other targets with
additional jobs (also using different sets of pre-installed libraries to
get a better coverage of the things that we test).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200608114049.4693-1-thuth@redhat.com>
2020-07-11 15:53:00 +01:00
Alex Bennée
6e3ede39e2 shippable: pull images from registry instead of building
Now we have a source for registry images pull from there rather than
re-building every time on shippable.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-41-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
a6eeac3bf5 testing: add check-build target
If we want to continue to split build and check phase it seems like a
good idea to allow building of the tests during our multi-threaded
build phase.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200701135652.1366-40-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
c8e6793903 containers.yml: build with docker.py tooling
Instead of building the docker files directly use the same docker.py
scripting as we do for building locally. This should help ensure we
use the exact same steps and allow us to cache properly when building
locally.

To get this working you have to have a fairly recent docker binary
otherwise you will see the error message:

 => ERROR importing cache manifest from registry.gitlab....

So far docker 19.03.12 works (from the docker apt repos) but 18.09.1,
build 4c52b90 which is packaged in Debian Buster fails.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
f2c5466c78 gitlab: limit re-builds of the containers
Most of the time we are just rebuilding the same things. We can skip
this although currently there is no mechanism for picking up new
distro releases.

Rather than try to be too fine grained allow any change to trigger all
the images being rebuilt.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-38-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Daniel P. Berrangé
3e7b80f84d tests: improve performance of device-introspect-test
Total execution time with "-m slow" and x86_64 QEMU, drops from 3
minutes 15 seconds, down to 54 seconds.

Individual tests drop from 17-20 seconds, down to 3-4 seconds.

The cost of this change is that any QOM bugs resulting in the test
failure will not be directly associated with the device that caused
the failure. The test case is not frequently identifying such bugs
though, and the cause is likely easily visible in the patch series
that causes the failure. So overall the shorter running time is
considered the more important factor.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[thuth: Add the tree check to test_device_intro_none() and
 test_device_intro_abstract(), too, just to be sure...]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200710060719.22386-1-thuth@redhat.com>
2020-07-11 15:53:00 +01:00
Alex Bennée
6957fd98dc gitlab: add avocado asset caching
These can be quite big so lets cache them. I couldn't find any nots on
ccache in the gitlab docs so I've just ignored it for now.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-36-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
62c902e03b gitlab: enable check-tcg for linux-user tests
Switch to building in the new debian-all-test-cross image which has
most of the cross compilers inline.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-35-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
c1f6ad798c linux-user/elfload: use MAP_FIXED_NOREPLACE in pgb_reserved_va
Given we assert the requested address matches what we asked we should
also make that clear in the mmap flags. Otherwise we see failures in
the GitLab environment for some currently unknown but allowable
reason. We use MAP_FIXED_NOREPLACE if we can so we don't just clobber
an existing mapping. Also include the strerror string for a bit more
info on failure.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20200701135652.1366-34-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
27ebeda0c0 tests/docker: add a linux-user testing focused image
We happily use all the cross images for both cross-building QEMU as
well as building the linux-user tests. However calling docker from
within docker seems not to work. As we can build in Debian anyway why
not include an image that has all the compilers available for
non-docker invocation.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-33-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
634ef789f8 tests/tcg: add more default compilers to configure.sh
We were missing a bunch of compilers which we could use if they were
locally installed. The defaults are based on Debian as they seem to be
the best distro for well distributed cross-build compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-32-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
c5008c76ee gitlab: add acceptance testing to system builds
As part of migrating things from Travis to GitLab add the acceptance
tests. To do this:

  - rename system1 to system-ubuntu-main
  - rename system2 to system-fedora-misc
  - split into build/check/acceptance
  - remove -j from check stages
  - use artifacts to save build stage
  - add post acceptance template and use

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-31-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
3c1de3af46 tests/acceptance: skip LinuxInitrd 2gib with v4.16 on GitLab
This fails on GitLab but not when run locally on the same container
image. It's very confusing.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200701135652.1366-30-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
61bbce96fe tests/acceptance: skip multicore mips_malta tests on GitLab
For some reason these tests fail all the time on GitLab. I can
re-create the hang around 3% of the time locally but it doesn't seem
to be MTTCG related. For now skipIf on GITLAB_CI.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <20200701135652.1366-29-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
fcae60436f tests/acceptance: fix dtb path for machine_rx_gdbsim
The old path doesn't exist but the rx-virt.dtb file has the same
checksum so lets use that.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200701135652.1366-28-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
0f26d94ec9 tests/acceptance: skip s390x_ccw_vrtio_tcg on GitLab
Currently the test takes more the 900 seconds on GitLab and then times
out. Running on Travis seems to be OK.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-27-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
cdc61815c7 tests/docker: add packages needed for check-acceptance
We need additional python packages to run check-acceptance. Add them
to the docker images we will be using later.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-26-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
e6f1306b10 tests/docker: add --registry support to tooling
This allows us to point the tools towards a registry from which they
can grab pre-built layers instead of doing everything from scratch
each time. To enable this we need to be using the DOCKER_BUILDKIT
engine.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-25-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
8a8a50a957 gitlab: build containers with buildkit and metadata
According to the documentation to be able to use --cache-from for
remote registries you need to enable both buildkit and inline the
metadata. We want to do this to support pulling from gitlab when users
build their local docker images.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-24-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Daniel P. Berrangé
c962864f1d gitlab: convert jobs to use custom built containers
Now that we're building standard container images from
dockerfiles in tests/docker/dockerfiles, we can convert
the build jobs to use them. The key benefit of this is
that a contributor can now more easily replicate the CI
environment on their local machine. The container images
are cached too, so we are not spending time waiting for
the apt-get/dnf package installs to complete.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200622153318.751107-4-berrange@redhat.com>
[AJB: tweak naming convention]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-23-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Daniel P. Berrangé
fa821f23e1 gitlab: build all container images during CI
We have a number of container images in tests/docker/dockerfiles
that are intended to provide well defined environments for doing
test builds. We want our CI system to use these containers too.

This introduces builds of all of them as the first stage in the
CI, so that the built containers are available for later build
jobs. The containers are setup to use the GitLab container
registry as the cache, so we only pay the penalty of the full
build when the dockerfiles change. The main qemu-project/qemu
repo is used as a second cache, so that users forking QEMU will
see a fast turnaround time on their CI jobs.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200622153318.751107-3-berrange@redhat.com>
[AJB: tweak the tag format]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Daniel P. Berrangé
d0caa0a881 gitlab: introduce explicit "container" and "build" stages
If no stage is listed, jobs get put in an implicit "test" stage.
Some jobs which create container images to be used by later stages
are currently listed as in a "build" stages.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200622153318.751107-2-berrange@redhat.com>
Message-Id: <20200701135652.1366-21-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Thomas Huth
5117ba25d2 gitlab-ci: Fix the change rules after moving the YML files
The edk2.yml and opensbi.yml files have recently been moved/renamed,
but the change has not been reflected in the rules in the YML files
yet.

Fixes: 922febe2af ("Move edk2 and opensbi YAML files to .gitlab-ci.d folder")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20200625151627.24986-1-thuth@redhat.com>
Message-Id: <20200701135652.1366-20-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
91f6ff72b6 .gitignore: un-ignore .gitlab-ci.d
The sooner we deprecate in-tree builds the sooner this mess of regexes
can be thrown away.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-19-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
767b6bd22b tests/docker: change tag naming scheme of our images
We've been misusing the tag naming scheme for some time by overloading
the post : section with the image type. Really it should be saved for
the revision of that particular build. Move the details to the other
side so we have:

  qemu/image-name

with the implied :latest version added by the tooling.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-18-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
10c927dcc5 tests/docker: check for an parameters not empty string
Reported-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-17-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
b09539444a tests/vm: allow us to take advantage of MTTCG
We currently limit TCG guests to -smp 1 but now we have added some
aarch64 guests we can do better when running on x86_64 hardware.
Raise the limit for TCG guests when it is safe to do so.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-16-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
2fea3a125d tests/vm: switch from optsparse to argparse
optparse has been deprecated since version 3.2 and argparse is the
blessed replacement. Take the opportunity to enhance our help output
showing defaults when called.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Robert Foley <robert.foley@linaro.org>

Message-Id: <20200701135652.1366-15-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Robert Foley
ff14ab0c13 tests/vm: Add workaround to consume console
This adds support to basevm.py so that we always
drain the console chars.  This makes use of
support added in an earlier commit that allows
QEMUMachine to use the ConsoleSocket.

This is a workaround we found was needed since
there is a known issue where QEMU will hang waiting
for console characters to be consumed.

We also added the option of logging the console to a file.
LOG_CONSOLE=1 will now log the output to a file.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-10-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-14-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Robert Foley
0fc8f660c5 python/qemu: Add ConsoleSocket for optional use in QEMUMachine
We add the ConsoleSocket object, which has a socket interface
and which will consume all arriving characters on the
socket, placing them into an in memory buffer.
This will also provide those chars via recv() as
would a regular socket.
ConsoleSocket also has the option of dumping
the console bytes to a log file.

We also give QEMUMachine the option of using ConsoleSocket
to drain and to use for logging console to a file.
By default QEMUMachine does not use ConsoleSocket.

This is added in preparation for use by basevm.py in a later commit.
This is a workaround we found was needed for basevm.py since
there is a known issue where QEMU will hang waiting
for console characters to be consumed.

Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-9-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-13-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Robert Foley
df00168039 tests/vm: change scripts to use self._config
This change converts existing scripts to using for example self.ROOT_PASS,
to self._config['root_pass'].
We made similar changes for GUEST_USER, and GUEST_PASS.
This allows us also to remove the change in basevm.py,
which adds __getattr__ for backwards compatibility.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200601211421.1277-8-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-12-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
Robert Foley
d322fe2daf tests/vm: Added a new script for centos.aarch64.
centos.aarch64 creates a CentOS 8 image.
Also added a new kickstart script used to build the centos.aarch64 image.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-7-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-11-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
Robert Foley
13336606a5 tests/vm: Added a new script for ubuntu.aarch64.
ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM.
Another new file is also added aarch64vm.py, which is a module with
common methods used by aarch64 VMs, such as how to create the
flash images.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-6-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-10-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
Robert Foley
e56833b48b tests/vm: Add common Ubuntu python module
Add a common Ubuntu python module and make use of
it with the ubuntu.i386 script.
This is preparation for adding an Ubuntu script
ubuntu.aarch64.  Splitting out the common
logic such as build_image() will reduce duplication.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-5-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-9-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
Robert Foley
3f1e8137f2 tests/vm: Added configuration file support
Changes to tests/vm/basevm.py to allow accepting a configuration file
as a parameter. Allows for specifying VM options such as
cpu, machine, memory, and arbitrary qemu arguments for specifying options
such as NUMA configuration.
Also added an example conf_example_aarch64.yml and conf_example_x86.yml.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-4-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-8-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
Robert Foley
5d676197eb tests/vm: Add configuration to basevm.py
Added use of a configuration to tests/vm/basevm.py.
The configuration provides parameters used to configure a VM.
This allows for providing alternate configurations to the VM being
created/launched. cpu, machine, memory, and NUMA configuration are all
examples of configuration which we might want to vary on the VM being created
or launched.
This will for example allow for creating an aarch64 vm.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-3-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-7-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
Robert Foley
1f335d18e5 tests/vm: pass args through to BaseVM's __init__
Adding the args parameter to BaseVM's __init__.
We will shortly need to pass more parameters to the class
so let's just pass args rather than growing the parameter list.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-2-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-6-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
Robert Foley
995f5c3c32 util/coroutine: Cleanup start_switch_fiber_ for TSAN.
This is a cleanup patch to follow-up the patch which introduced TSAN.
This patch makes separate start_switch_fiber_ functions for TSAN and ASAN.

This does two things:
1. Unrelated ASAN and TSAN code is separate and each function only
   has arguments that are actually needed.
2. The co->tsan_caller_fiber and co->tsan_co_fiber fields are only
   access from within #ifdef CONFIG_TSAN.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200626170001.27017-1-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-5-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
David Edmondson
b418d26561 crypto/linux_keyring: fix 'secret_keyring' configure test
The configure test for 'secret_keyring' incorrectly checked the
'have_keyring' variable.

Fixes: 54e7aac056
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200618092636.71832-1-david.edmondson@oracle.com>
Message-Id: <20200701135652.1366-4-alex.bennee@linaro.org>
2020-07-11 15:52:59 +01:00
Peter Maydell
827937158b xen patches
Fixes following harden checks in qdev.
 -----BEGIN PGP SIGNATURE-----
 
 iQFOBAABCgA4FiEE+AwAYwjiLP2KkueYDPVXL9f7Va8FAl8IZ6oaHGFudGhvbnku
 cGVyYXJkQGNpdHJpeC5jb20ACgkQDPVXL9f7Va/uhwgAhYVil9DePHVNADAk8nAN
 eRPsw9ukzxUYwKNx88YQA1W1BbEXdy+cBjsf4yxQ6gbfYSzZZKcXJieDVR3gz2uM
 Ll377KCE2QcqC0n+aAJp6fEYJ/CkL/dME/yeqSU6TIq3E0NKfqso3bXcKRHYEQ+z
 2UUsDGgis3P1AZcKuxbj+9p2YZUSwwt/TYo5mtosBxH/YM6vylEmN/1jlsCoOTMh
 6AsZxO3bo+qiZem8qLjh/Cv6nQOKM7o10dXLWteg9D/pRlQwtqeP6PlxXGWNS2eM
 5J+Ee3ho64HvebU3p6Z7Sbb/EZgjKOtg2aHnNFneEDG9MvWb6gMuKczNXFnXrKhu
 Xg==
 =pD/o
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20200710' into staging

xen patches

Fixes following harden checks in qdev.

# gpg: Signature made Fri 10 Jul 2020 14:05:46 BST
# gpg:                using RSA key F80C006308E22CFD8A92E7980CF5572FD7FB55AF
# gpg:                issuer "anthony.perard@citrix.com"
# gpg: Good signature from "Anthony PERARD <anthony.perard@gmail.com>" [marginal]
# gpg:                 aka "Anthony PERARD <anthony.perard@citrix.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 5379 2F71 024C 600F 778A  7161 D8D5 7199 DF83 42C8
#      Subkey fingerprint: F80C 0063 08E2 2CFD 8A92  E798 0CF5 572F D7FB 55AF

* remotes/aperard/tags/pull-xen-20200710:
  xen: cleanup unrealized flash devices
  xen: Fix xen-legacy-backend qdev types

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-11 13:56:03 +01:00
Peter Maydell
0fb7ea4515 virtio: bugfix
fixes vdpa on non-kvm platforms
 
 Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCAAtFiEEXQn9CHHI+FuUyooNKB8NuNKNVGkFAl8FnUwPHG1zdEByZWRo
 YXQuY29tAAoJECgfDbjSjVRp/HAIAK85sNj6fAGwEESnUsiY9M8yy3fX8z9jyXxu
 RcwHSwot6+NK6tLOj/879TV/9cLlHTNaAkpfFSFqvQEiaaP5RjutMC7RB/HiDNrs
 Q0114x0blKYtmChWxTDiXONVY8m8lCm36dWs2hMFL2JPhubCCJJaJaiBaFCvM7H1
 zv2yFLri+To01gyfqPf8YYzgXzVRneJze0KfIQIVI7WxAm7HyhxhofZ4kmX2VHOQ
 PIaMMv/bUhb54ZkFFDOawkw+hfJMQUtmwpW1OewA1bQ8PX3Pv8LN5sCWz2oZmiA9
 3qVEwbI1jBwUgvniqki2KwFn6jDRzJWsrhfLHxuyDkn1nQENh0c=
 =vIBD
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

virtio: bugfix

fixes vdpa on non-kvm platforms

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Wed 08 Jul 2020 11:17:48 BST
# gpg:                using RSA key 5D09FD0871C8F85B94CA8A0D281F0DB8D28D5469
# gpg:                issuer "mst@redhat.com"
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream:
  vhost-vdpa: fix the compile issue without kvm

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-10 19:29:07 +01:00
Peter Maydell
45db94cc90 qemu-openbios queue
-----BEGIN PGP SIGNATURE-----
 
 iQFSBAABCgA8FiEEzGIauY6CIA2RXMnEW8LFb64PMh8FAl8E4cEeHG1hcmsuY2F2
 ZS1heWxhbmRAaWxhbmRlLmNvLnVrAAoJEFvCxW+uDzIf+WgH/jTpXtNJ05r0e/la
 aec/P/xytNBZNVcAKBSdH4FiFC8q704l3D7kgy2LVqlF+gQeN1uOsr615PuHRY7e
 aDJFQXok7v1i2FBYr7V7zULzGhP46DkgGJvs/ROWVlqrjSKFEbl0t2tgF/DqJIDf
 iPEMXx+oGhVYMXaAz1GQw9dJX716yfqRgg3syZ+FSplooYI/aohvzGhesm3H5Aee
 VxEoDZvtoRGn0b/nK7THB0MazrfC+XQIWeRyFXfOtOIw7CbZRsuoJShBRTK1phUZ
 nn0ld9pRSj0ksb+SNgxqfGf4K3pZ4eyqk3WhWuAqn5V471y8g8vbr4e2n52R5gkB
 hcpzUH4=
 =Ktap
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mcayland/tags/qemu-openbios-20200707' into staging

qemu-openbios queue

# gpg: Signature made Tue 07 Jul 2020 21:57:37 BST
# gpg:                using RSA key CC621AB98E82200D915CC9C45BC2C56FAE0F321F
# gpg:                issuer "mark.cave-ayland@ilande.co.uk"
# gpg: Good signature from "Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>" [full]
# Primary key fingerprint: CC62 1AB9 8E82 200D 915C  C9C4 5BC2 C56F AE0F 321F

* remotes/mcayland/tags/qemu-openbios-20200707:
  Update OpenBIOS images to 75fbb41d built from submodule.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-10 16:43:40 +01:00
Peter Maydell
f2a1cf9180 Error reporting patches patches for 2020-07-07
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAl8IbBoSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTeBsP+waQcWmLHkRWSW4zRbCqc3JhavrD8m6h
 nBk8J4tvMMaKIUNNpmFLl7xxB17FBpuUGJ6l/LBcW8XuAXmN6utT47UQqxlmsQ45
 qtqcZ0GEfCqD0OVnrs2Mxun8ZpRSW/I+wrKVxaOFsF+Klq4lduixiRgFk+tpMtf2
 eYxX1GlxnQK0JDjRrQe+ssx+q5nwbamxx1dVsFSjObLKXg+GJElQBfU0+HblXCGi
 N5hiZ2rw5a2+wPGQ3UVu1EWDuv7y67PZBidpmDX4t16atMCEGMgqG0Q+Rm5Tqwak
 NS84fn7RF7Q6zbXj/BdVwhZUjBcbfnYyWCR8mU1h/VKtvedlLp9Q3NsYjK3ilyjT
 NtygbLai9IEeKUcAYTfHyeK/pcfo//5joasFc5Q4FPTtLzK1K9ZFBUOcNyxOTIFD
 XVcM5fmnxkVFPGfV3nfwuXxhDPfCwk9FyB7hpVpHqtAPYWjYrqmnThZ2ineVcslQ
 mU/EBBcpb/hovlPyNZxWqfH3mU+8DJ1ojZwOgU6Z6NlIrm3L4mDG8ZOSm+wp6Q+s
 VIeXPVnvDTYlBuehPsV/B8eEwwYxfl43EDRw5ZuScj2QM4fyKuNXYjbDfTj9O+By
 eGs4PLKLi/9Sa0fUSRYqFAk8+XSrlkM7VlJ/xdBxMu7MeJq8896xINcSkLkewGml
 wkzB78QRZ6k4
 =P9Td
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2020-07-07-v2' into staging

Error reporting patches patches for 2020-07-07

# gpg: Signature made Fri 10 Jul 2020 14:24:42 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-error-2020-07-07-v2: (53 commits)
  xen: Use ERRP_GUARD()
  nbd: Use ERRP_GUARD()
  virtio-9p: Use ERRP_GUARD()
  fw_cfg: Use ERRP_GUARD()
  pflash: Use ERRP_GUARD()
  sd: Use ERRP_GUARD()
  scripts: Coccinelle script to use ERRP_GUARD()
  error: New macro ERRP_GUARD()
  hmp: Ignore Error objects where the return value suffices
  qdev: Ignore Error objects where the return value suffices
  qemu-img: Ignore Error objects where the return value suffices
  error: Avoid error_propagate() after migrate_add_blocker()
  qapi: Purge error_propagate() from QAPI core
  qapi: Smooth visitor error checking in generated code
  qapi: Smooth another visitor error checking pattern
  block/parallels: Simplify parallels_open() after previous commit
  error: Reduce unnecessary error propagation
  error: Eliminate error_propagate() manually
  error: Eliminate error_propagate() with Coccinelle, part 2
  error: Eliminate error_propagate() with Coccinelle, part 1
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-10 14:41:23 +01:00
Vladimir Sementsov-Ogievskiy
1de7096d83 xen: Use ERRP_GUARD()
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
the ERRP_GUARD() macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_GUARD() leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_GUARD() macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  No such cases are being fixed here.

This commit is generated by command

    sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/errp-guard.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200707165037.1026246-9-armbru@redhat.com>
[ERRP_AUTO_PROPAGATE() renamed to ERRP_GUARD(), and
auto-propagated-errp.cocci to errp-guard.cocci.  Commit message
tweaked again.]
2020-07-10 15:18:09 +02:00
Vladimir Sementsov-Ogievskiy
795d946d07 nbd: Use ERRP_GUARD()
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
the ERRP_GUARD() macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_GUARD() leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_GUARD() macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix several such cases, e.g. in nbd_read().

This commit is generated by command

    sed -n '/^Network Block Device (NBD)$/,/^$/{s/^F: //p}' \
        MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/errp-guard.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200707165037.1026246-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[ERRP_AUTO_PROPAGATE() renamed to ERRP_GUARD(), and
auto-propagated-errp.cocci to errp-guard.cocci.  Commit message
tweaked again.]
2020-07-10 15:18:09 +02:00
Vladimir Sementsov-Ogievskiy
92c451222c virtio-9p: Use ERRP_GUARD()
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
the ERRP_GUARD() macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_GUARD() leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_GUARD() macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix such a case in
v9fs_device_realize_common().

This commit is generated by command

    sed -n '/^virtio-9p$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/errp-guard.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200707165037.1026246-7-armbru@redhat.com>
[ERRP_AUTO_PROPAGATE() renamed to ERRP_GUARD(), and
auto-propagated-errp.cocci to errp-guard.cocci.  Commit message
tweaked again.]
2020-07-10 15:18:09 +02:00
Vladimir Sementsov-Ogievskiy
8b4b52759a fw_cfg: Use ERRP_GUARD()
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
the ERRP_GUARD() macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_GUARD() leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_GUARD() macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  No such cases are being fixed here.

This commit is generated by command

    sed -n '/^Firmware configuration (fw_cfg)$/,/^$/{s/^F: //p}' \
        MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/errp-guard.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200707165037.1026246-6-armbru@redhat.com>
[ERRP_AUTO_PROPAGATE() renamed to ERRP_GUARD(), and
auto-propagated-errp.cocci to errp-guard.cocci.  Commit message
tweaked again.  Coccinelle script rerun for commit 3203148917
"hw/nvram/fw_cfg: Add the FW_CFG_DATA_GENERATOR interface"]
2020-07-10 15:18:09 +02:00
Vladimir Sementsov-Ogievskiy
76612456aa pflash: Use ERRP_GUARD()
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
the ERRP_GUARD() macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_GUARD() leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_GUARD() macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  No such cases are being fixed here.

This commit is generated by command

    sed -n '/^Parallel NOR Flash devices$/,/^$/{s/^F: //p}' \
        MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/errp-guard.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200707165037.1026246-5-armbru@redhat.com>
[ERRP_AUTO_PROPAGATE() renamed to ERRP_GUARD(), and
auto-propagated-errp.cocci to errp-guard.cocci.  Commit message
tweaked again.]
2020-07-10 15:18:09 +02:00
Vladimir Sementsov-Ogievskiy
de1b3800b7 sd: Use ERRP_GUARD()
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
the ERRP_GUARD() macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_GUARD() leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_GUARD() macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  No such cases are being fixed here.

This commit is generated by command

    sed -n '/^SD (Secure Card)$/,/^$/{s/^F: //p}' \
        MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/errp-guard.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200707165037.1026246-4-armbru@redhat.com>
[ERRP_AUTO_PROPAGATE() renamed to ERRP_GUARD(), and
auto-propagated-errp.cocci to errp-guard.cocci.  Commit message
tweaked again.]
2020-07-10 15:18:09 +02:00
Vladimir Sementsov-Ogievskiy
8220f3ac74 scripts: Coccinelle script to use ERRP_GUARD()
Script adds ERRP_GUARD() macro invocations where appropriate and
does corresponding changes in code (look for details in
include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/errp-guard.cocci \
 --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
 --max-width 80 FILES...

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200707165037.1026246-3-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[ERRP_AUTO_PROPAGATE() renamed to ERRP_GUARD(), and
auto-propagated-errp.cocci to errp-guard.cocci]
2020-07-10 15:18:09 +02:00