Commit Graph

67708 Commits

Author SHA1 Message Date
Kevin Wolf
65a3d073e9 qcow2: Extend spec for external data files
This adds external data file to the qcow2 spec as a new incompatible
feature.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Kevin Wolf
c5e86ebc11 qcow2: Simplify preallocation code
Image creation already involves a bdrv_co_truncate() call, which allows
to specify a preallocation mode. Just pass the right mode there and
remove the code that is made redundant by this.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Kevin Wolf
97f94cb4f8 qemu-iotests: Test qcow2 preallocation modes
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-08 12:26:45 +01:00
Philippe Mathieu-Daudé
bde36af1ab qemu-iotests: Ensure GNU sed is used
Various sed regexp from common.filter use sed GNU extensions.
Instead of spending time to write these regex to be POSIX compliant,
verify the GNU sed is available and use it.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Philippe Mathieu-Daudé
11a82d1429 qemu-iotests: Improve portability by searching bash in the $PATH
Bash is not always installed as /bin/bash. In particular on OpenBSD,
the package installs it in /usr/local/bin.
Use the 'env' shebang to search bash in the $PATH.

Patch created mechanically by running:

  $ git grep -lE '#! ?/bin/bash' -- tests/qemu-iotests \
    | while read f; do \
      sed -i 's|^#!.\?/bin/bash$|#!/usr/bin/env bash|' $f; \
    done

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Philippe Mathieu-Daudé
d68375816e tests/bios-tables: Improve portability by searching bash in the $PATH
Bash is not always installed as /bin/bash. In particular on OpenBSD,
the package installs it in /usr/local/bin.
Use the 'env' shebang to search bash in the $PATH.

Reviewed-by: Kamil Rytarowski <n54@gmx.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Philippe Mathieu-Daudé
a03700fd3e tests/multiboot: Improve portability by searching bash in the $PATH
Bash is not always installed as /bin/bash. In particular on OpenBSD,
the package installs it in /usr/local/bin.
Use the 'env' shebang to search bash in the $PATH.

Reviewed-by: Kamil Rytarowski <n54@gmx.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Andrey Shinkevich
d9df28e7b0 iotests: check whitelisted formats
Some test cases require specific formats. The method decorator
skip_if_unsupported() checks if requested formats are whitelisted.
The test #139 was selected for a sample output, after running
$ ./check -qcow2 131-140

137 3s ...
138 0s ...
139 2s ...
    [case not run] testBlkDebug (__main__.TestBlockdevDel): formats ['blkdebug'] are not whitelisted
    [case not run] testBlkVerify (__main__.TestBlockdevDel): formats ['blkverify'] are not whitelisted
    [case not run] testQuorum (__main__.TestBlockdevDel): formats ['quorum'] are not whitelisted
140 0s ...
Not run: 131 135 136
Some cases not run in: 139
Passed all 7 tests

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Andrey Shinkevich
57ed557f03 iotests: ask QEMU for supported formats
Supported formats listed by 'qemu' may differ from those listed by
'qemu-img' due to whitelists. Some test cases require specific formats
that may be used with qemu. They can be inquired directly by running
'qemu -drive format=help'. The response takes whitelists into account.
The method supported_formats() serves for that. The method decorator
skip_if_unsupported() checks if all requested formats are whitelisted.
If not, the test case will be skipped. That has been implemented in
the 'check' file in the way similar to the 'test notrun' mechanism.

Suggested-by: Roman Kagan <rkagan@virtuozzo.com>
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Andrey Shinkevich
9ac404c523 block: iterate_format with account of whitelisting
bdrv_iterate_format (which is currently only used for printing out the
formats supported by the block layer) doesn't take format whitelisting
into account.

This creates a problem for tests: they enumerate supported formats to
decide which tests to enable, but then discover that QEMU doesn't let
them actually use some of those formats.

To avoid that, exclude formats that are not whitelisted from
enumeration, if whitelisting is in use.  Since we have separate
whitelists for r/w and r/o, take this a parameter to
bdrv_iterate_format, and print two lists of supported formats (r/w and
r/o) in main qemu.

Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Andrey Shinkevich
ce090f656c iotests: open notrun files in text mode
Replace the binary mode with the default text one when *.notrun
files are opened for skipped tests. That change is made for the
compatibility with Python 3 which returns error otherwise.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Alberto Garcia
af39bd0d9a qcow2: Default to 4KB for the qcow2 cache entry size
QEMU 2.12 (commit 1221fe6f63) introduced
a new setting called l2-cache-entry-size that allows making entries on
the qcow2 L2 cache smaller than the cluster size.

I have been performing several tests with different cluster and entry
sizes and all of them show that reducing the entry size (aka L2 slice)
consistently improves I/O performance, notably during random I/O (all
tests done with sequential I/O show similar results). This is to be
expected because loading and evicting an L2 slice is more expensive
the larger the slice is.

Here are some numbers on fully populated 40GB qcow2 images. The
rightmost column represents the maximum L2 cache size in both cases.

   Cluster size = 64 KB
   |-------------+--------------+--------------+--------------|
   |             | 1MB L2 cache | 3MB L2 cache | 5MB L2 cache |
   |-------------+--------------+--------------+--------------|
   |  4KB slices |    6545 IOPS |   12045 IOPS |   55680 IOPS |
   | 16KB slices |    5177 IOPS |    9798 IOPS |   56278 IOPS |
   | 64KB slices |    2718 IOPS |    5326 IOPS |   57355 IOPS |
   |-------------+--------------+--------------+--------------|

   Cluster size = 256 KB
   |--------------+----------------+--------------+-----------------|
   |              | 512KB L2 cache | 1MB L2 cache | 1280KB L2 cache |
   |--------------+----------------+--------------+-----------------|
   |   4KB slices |      8539 IOPS |   21071 IOPS |      55417 IOPS |
   |  64KB slices |      3598 IOPS |    9772 IOPS |      57687 IOPS |
   | 256KB slices |      1415 IOPS |    4120 IOPS |      58001 IOPS |
   |--------------+----------------+--------------+-----------------|

As can be seen in the numbers, the only exception to the rule is when
the cache is large enough to hold all L2 tables. This is also to be
expected because in this case no cache entry is ever evicted so
reducing its size doesn't bring any benefit.

This patch sets the default L2 cache entry size to 4KB except when the
cache is large enough for the whole disk.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Stefan Hajnoczi
b74b1adef0 iotests: use iotests.VM in 238
Test 238 does not require the kvm accelerator.  Using the qtest
accelerator allows the test to run in both non-kvm and non-tcg
environments.

iotests.VM implicitly uses the qtest accelerator and is really the class
that this test should be using.  Switch to that instead of
qemu.QEMUMachine.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Peter Maydell
c4e0780ed1 - Update netlink types to linux v5.0
- fix accept4/getpeername/getsockname/recvfrom/recvmsg/read
 - add/fix ELF_PLATFORM ofr aarch64 and arm
 - fix "may be used uninitialized" warnings
 - Fix breakpoint support in Nios
 - Nicer strace output of chroot() syscall
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJcgO0YAAoJEPMMOL0/L748HU8P/3IknbXS9/WGLdSlKAbW4o3+
 aCq8a/l6RpvD/4JVQIuPjs5v6j6PsWZwlB0//SeTv0so0yWH6Lw+9JkTNx3TqjgF
 +qhSZxvDd2gmsZhhMpX3XDsGJujPYRGhJGgeuJ1uGalwZO5Q8pt6ta4Jx1NRxtzt
 B38gIAO3D8Rqz49omFvXNv8154ojMIQlLqDjfwtwj7e/O7fKRD8ROv4+HqfJCzHG
 jE/tInZeY5NoRCXhUeaoI8NaTMWGFs7IVgryHWanVKaixYMVb26LIYfaw3aK2CZB
 qnf2u+S5G9fE4ba121teaLlEm31V716cFhPV6axH+qesxeLFC5uACaTI7Z/OwLPQ
 IEnDbuZ5cAP49dBD6bEkS7h4V3OQ/yt+C8v+MH5ObDWzrLZuH6iPVgQka/0swYqc
 7c4uhV948uO84UJIOVfB4xhMdMtigFtTDQHpRAzNwRkER6t8zJIU6GRpg+MbGbwt
 8ivPgTknf+CjyDpw1CFUVkr2PAThFNqce+247EpHLtEtjidW5NjRTrdO5EjFs//p
 E8gGaoxb7NbwNheM/rtk99rLqYybObgV9lFdslyW5JaZ6VnhUkxjdDBMTetegNam
 BDyTf49p93ohom64S8pYz5iQPBm7X0vc7nfK7wCk6cHjGhbedjCOyMssz/2xCh9Y
 I5DK8q28jnJukr/6EgqI
 =Dqxx
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-4.0-pull-request' into staging

- Update netlink types to linux v5.0
- fix accept4/getpeername/getsockname/recvfrom/recvmsg/read
- add/fix ELF_PLATFORM ofr aarch64 and arm
- fix "may be used uninitialized" warnings
- Fix breakpoint support in Nios
- Nicer strace output of chroot() syscall

# gpg: Signature made Thu 07 Mar 2019 10:06:16 GMT
# gpg:                using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-4.0-pull-request:
  linux-user: add new netlink types
  linux-user: Nicer strace output of chroot() syscall
  linux-user: fix "may be used uninitialized" warnings
  linux-user: don't short-circuit read with zero length
  Fix breakpoint support in Nios II user-mode emulation.
  linux-user: fix emulation of accept4/getpeername/getsockname/recvfrom syscalls
  linux-user: Fix ELF_PLATFORM for aarch64_be-linux-user
  linux-user: Add ELF_PLATFORM for arm
  linux-user: fix recvmsg emulation

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-07 18:40:43 +00:00
Peter Maydell
6cb4f6db4f Python queue, 2019-02-22
Python:
 * introduce "python" directory with module namespace
 * log QEMU launch command line on qemu.QEMUMachine
 
 Acceptance Tests:
 * initrd 4GiB+ test
 * migration test
 * multi vm support in test class
 * bump Avocado version and drop "🥑 enable"
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJccE9jAAoJEGV+jTOl8gnzb1cP/j99kGbgfQJA4CftO9eRXdIm
 FKms4Z42n7KPus+/DphgfOXGYaHzPcqJQNguQYHuPlWaM3DWNU0rcFfAi/QdcZC1
 3iYMyQwiRubjnCMN0Ab4k+GhpCPW6fea6GTzyvqha4jNRhCIhx7v54GTDfxWESQp
 nqW40gAONGSG98DdFgubxg1YYqt7zlI9EVogGixe1gO9SVDkMEe7uH8tPCl9mt2m
 VjN7AeP/NTDmidiwu+2LwSpDC0UmpDAsFnxGI6rDcNx8NOnjSHkSHmtxNJ8j2uZz
 9P0ncGui+LfivdQh/yiBgrjTWXEXAx/oHKQCz7r8uJ8f60eYLFtjTHm//2G7lG48
 luLSnNKq/niM4k/vNhBQr0ByqoHHlpmqAjbmYqw7wdvImBbkXN2Gh9kjNs55S8VZ
 Z7wTceC0G7pyM3LCdFnikyCXKoRxLZ3AXQ3YXFN0PgX/IsyHVuBWBGPFkPkLwcRa
 JW3DEmwx/oeTg2MKp7iA3dGTUIarbsjp+R04erMznlLvE+NgmB8ENY8T+qZ6c+NM
 ZNyp1MH2nuTJsYxY3CkVKwPUqNSoaTLkMxvoZW5rKQdtvNinCYZpaeHuBchaHJed
 E63r0+1n9vAMH3PHDrypW5qjcjSDBOHS+8ajhr0jr2r+6grLQKYEP8q+PwubUaMq
 BsS5jOb8gLGC8ESfZxx/
 =dwff
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/cleber/tags/python-next-pull-request' into staging

Python queue, 2019-02-22

Python:
* introduce "python" directory with module namespace
* log QEMU launch command line on qemu.QEMUMachine

Acceptance Tests:
* initrd 4GiB+ test
* migration test
* multi vm support in test class
* bump Avocado version and drop "🥑 enable"

# gpg: Signature made Fri 22 Feb 2019 19:37:07 GMT
# gpg:                using RSA key 657E8D33A5F209F3
# gpg: Good signature from "Cleber Rosa <crosa@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3

* remotes/cleber/tags/python-next-pull-request:
  Acceptance tests: expect boot to extract 2GiB+ initrd with linux-v4.16
  Acceptance tests: use linux-3.6 and set vm memory to 4GiB
  tests.acceptance: adds simple migration test
  tests.acceptance: adds multi vm capability for acceptance tests
  scripts/qemu.py: log QEMU launch command line
  Introduce a Python module structure
  Acceptance tests: drop usage of "🥑 enable"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-07 16:16:02 +00:00
Peter Maydell
21afe115a4 Enable building and installing rST docs with Sphinx
-----BEGIN PGP SIGNATURE-----
 
 iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAlyBMygZHHBldGVyLm1h
 eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3hjeD/4nSnAT/dXIAQAbfJMDyKdd
 pywFyUb6ltmEwApw6NheM0C0Wxcr19QEFfLTSNDTjp6S3HSOF1eY5vPOt8sVThnB
 AQoPLt0pky2LVrLcb3Dereeh52fMxNnfwnFwvkz5ycxhFlIrTDaH9Gzu/2t/tX8r
 uTG5gSHUz0geA6LH1whn1R1dfcVhYNByC3Yn8+RwIckIPj10v6JC0YIeoMqaPAw/
 ETr5Cbat0i47GQzcGvHE0nnyaCoXxWeOy+DUMZDncWmQcF3q4dSagniLgS3aIbKj
 AqhhO9QxaDkzo6yQ1Cl39hJWj2njANBrGQnUm8IPiUnGclIjhtq2x7OskxW2YYII
 SsZOyrBg7t3hmdaQMpb5/PG2wzApirRUo8pxlVUGVFGc2SR+3wS7a4blNHozjR53
 w9doFoo3UeSqlFhN1Y359S+0qqZOMyOVotEbx48ElDBtKjV5IaouzickzbAfRpQh
 VrXSDhqtLwLDDeVrEpbl5uq/5P8KgVGlIc+Ll2XnH3OXsbfjH/juC1SdpsBtSASJ
 Ia2wi9wiuS+0kLQJwqfj/RtAmycEPGQWo1gxdZNyFA1FkhPTs7Mo+YMmwa96UGlC
 wzhh9vcdEfmZE3IEYU0+hf1VC9PXR69vqU66zIO4LllOXWmyea3q2DHRyxOaj0Ls
 5XSKwN4lNX9hoVhEbJGfdg==
 =iyAU
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-sphinx-20190307' into staging

Enable building and installing rST docs with Sphinx

# gpg: Signature made Thu 07 Mar 2019 15:05:12 GMT
# gpg:                using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg:                issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate]
# gpg:                 aka "Peter Maydell <pmaydell@gmail.com>" [ultimate]
# gpg:                 aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-sphinx-20190307:
  MAINTAINERS: Add entry for Sphinx documentation infrastructure
  docs/conf.py: Don't hard-code QEMU version
  Makefile: Abstract out "identify the pkgversion" code
  Makefile, configure: Support building rST documentation
  docs: Provide separate conf.py for each manual we want
  docs/conf.py: Disable option warnings
  docs/conf.py: Don't include rST sources in HTML build
  docs/conf.py: Configure the 'alabaster' theme
  docs/conf.py: Disable unused _static directory
  docs: Commit initial files from sphinx-quickstart
  docs: Convert memory.txt to rst format
  docs/cpu-hotplug.rst: Fix rST markup issues

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-07 15:21:54 +00:00
Peter Maydell
c10e01b996 MAINTAINERS: Add entry for Sphinx documentation infrastructure
Add a MAINTAINERS entry for Sphinx documentation infrastructure:
this doesn't cover actual content, only the machinery we use to
build the docs.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-13-peter.maydell@linaro.org
2019-03-07 14:26:47 +00:00
Peter Maydell
6038f5fca5 docs/conf.py: Don't hard-code QEMU version
Don't hard-code the QEMU version number into conf.py. Instead
we either pass it to sphinx-build on the command line, or
(if doing a standalone Sphinx run in a readthedocs.org setup)
extract it from the VERSION file.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-12-peter.maydell@linaro.org
Message-id: 20190228145624.24885-12-peter.maydell@linaro.org
2019-03-07 14:26:47 +00:00
Peter Maydell
57b49737ae Makefile: Abstract out "identify the pkgversion" code
Abstract out the "identify the pkgversion" code from the
rule for creating qemu-version.h, so it sets makefile
variables for QEMU_PKGVERSION and QEMU_FULL_VERSION.
(We will want to use these when building the Sphinx docs.)

NB: As we abstract this out, we use -e to check for .git
rather than -d, since in some situations .git may be a file
rather than a directory.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-11-peter.maydell@linaro.org
Message-id: 20190228145624.24885-11-peter.maydell@linaro.org
2019-03-07 14:26:47 +00:00
Peter Maydell
5f71eac06e Makefile, configure: Support building rST documentation
Add support to our configure and makefile machinery for building
our rST docs into HTML files.

Building the documentation now requires that sphinx-build is
available; this seems better than allowing half the docs to
be built if it is not present but having half of them missing.
(In particular it means that assuming that distros configured with
--enable-docs they'll get a helpful error from configure telling
them the new build dependency.)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-10-peter.maydell@linaro.org
Message-id: 20190228145624.24885-10-peter.maydell@linaro.org
2019-03-07 14:26:46 +00:00
Peter Maydell
f8cf7147f1 docs: Provide separate conf.py for each manual we want
By default Sphinx wants to build a single manual at once.
For QEMU, this doesn't suit us, because we want to have
separate manuals for "Developer's Guide", "User Manual",
and so on, and we don't want to ship the Developer's Guide
to end-users. However, we don't want to completely duplicate
conf.py for each manual, and we'd like to continue to
support "build all docs in one run" for third-party sites
like readthedocs.org.

Make the top-level conf.py support two usage forms:
 (1) as a common config file which is included by the conf.py
 for each of QEMU's manuals: in this case sphinx-build is run
 multiple times, once per subdirectory.
 (2) as a top level conf file which will result in building all
 the manuals into a single document: in this case sphinx-build is
 run once, on the top-level docs directory.

Provide per-manual conf.py files and top level pages for
our first two manuals:
 * QEMU Developer's Guide (docs/devel)
 * QEMU System Emulation Management and Interoperability Guide
   (docs/interop)

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-id: 20190305172139.32662-9-peter.maydell@linaro.org
Message-id: 20190228145624.24885-9-peter.maydell@linaro.org
2019-03-07 14:26:46 +00:00
Peter Maydell
e250e867dc docs/conf.py: Disable option warnings
sphinx-build complains about using :option: to mark up option
flags that it doesn't know about (because they were not defined
using the "option::" directive):
docs/pr-manager.rst:68: WARNING: unknown option: -d

Suppress these warnings. This way we get the semantic markup
of the option flag but no cross-referencing hyperlink.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-8-peter.maydell@linaro.org
Message-id: 20190228145624.24885-8-peter.maydell@linaro.org
2019-03-07 14:26:46 +00:00
Peter Maydell
479fb8a511 docs/conf.py: Don't include rST sources in HTML build
Sphinx defaults to including all the rST source files
in the HTML build and making each HTML page link to the
source file. Disable that.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-7-peter.maydell@linaro.org
Message-id: 20190228145624.24885-7-peter.maydell@linaro.org
2019-03-07 14:26:46 +00:00
Peter Maydell
4fad3864a8 docs/conf.py: Configure the 'alabaster' theme
Add the 'navigation' bar to the sidebar, which for some
reason is not enabled by default. Remove 'relations', which
is effectively disabled anyway and isn't useful for us.

This requires that we mandate having at least Sphinx 1.3,
where the theme was added.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-id: 20190305172139.32662-6-peter.maydell@linaro.org
Message-id: 20190228145624.24885-6-peter.maydell@linaro.org
2019-03-07 14:26:45 +00:00
Peter Maydell
07fd6563ae docs/conf.py: Disable unused _static directory
We don't yet have any custom static files, so disable this
config file setting to avoid a warning from sphinx about
not being able to find the directory.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-5-peter.maydell@linaro.org
Message-id: 20190228145624.24885-5-peter.maydell@linaro.org
2019-03-07 14:26:45 +00:00
Peter Maydell
5329da6a4f docs: Commit initial files from sphinx-quickstart
Commit the initial Sphinx conf.py and skeleton index.rst as
generated with sphinx-quickstart. We'll update these to
add QEMU-specific tweaks in subsequent commits.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-4-peter.maydell@linaro.org
Message-id: 20190228145624.24885-4-peter.maydell@linaro.org
2019-03-07 14:26:45 +00:00
Peter Maydell
859cdc01a0 docs: Convert memory.txt to rst format
Convert the memory API documentation from plain text
to restructured text format.

This is a very minimal conversion: all I had to change
was to mark up the ASCII art parts as Sphinx expects
for 'literal blocks', and fix up the bulleted lists
(Sphinx expects no leading space before the bullet, and
wants a blank line before after any list).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20190305172139.32662-3-peter.maydell@linaro.org
Message-id: 20190228145624.24885-3-peter.maydell@linaro.org
2019-03-07 14:26:44 +00:00
Peter Maydell
0defa86ca3 docs/cpu-hotplug.rst: Fix rST markup issues
sphinx-build complains:

docs/cpu-hotplug.rst:67: ERROR: Unexpected indentation.
docs/cpu-hotplug.rst:69: ERROR: Unexpected indentation.
docs/cpu-hotplug.rst:74: WARNING: Block quote ends without a blank line; unexpected unindent.
docs/cpu-hotplug.rst:75: WARNING: Block quote ends without a blank line; unexpected unindent.
docs/cpu-hotplug.rst:76: SEVERE: Unexpected section title.

}
{
docs/cpu-hotplug.rst:78: WARNING: Block quote ends without a blank line; unexpected unindent.

These are the result of not indicating one of the literal
blocks by finishing the preceding paragraph with the "::" marker.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20190305172139.32662-2-peter.maydell@linaro.org
Message-id: 20190228145624.24885-2-peter.maydell@linaro.org
2019-03-07 14:26:44 +00:00
Peter Maydell
3a75ef6a0f usb: mtp fixes, guest-reset switch for usb-host.
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABAgAGBQJcgOozAAoJEEy22O7T6HE4EzEP/3oPKSSMhxdk8gy///7aoF8V
 grBlDQPca+RBUnJyqPg7iBFO0uO+uxNVd5JYUnNuMYfTBgzJXNtLmWIZY0uHHT/u
 8oYju5ayYK+n4nwcmDDYO5gfNq6zIVqfn/Ur6GurdDlrJQQ0ERU7e4M2WUpB2lgo
 2Gar52fSovjV37ql5gBYeLlxADKNOxQqGeUS7k6fkxktu/hPrnmjBzzdLbntHP7B
 lzytUcEPej76Pm1SVQZgUewSnjpjVUAFNmWH79NpSk23VD8Q2ANE2SJFF43lxA/S
 vm7iklbpjrJVHOCNSQGmGKfXnO+39UMfzVmJwOmX1DMuO8zb4K6bSxUX578P6BPa
 Jm3qA3zE26ZkB2PlcA/XNkw2YoCBX6v+U8M7sjfRqfFLDLgvI54ZreGovM4Ai/65
 H0KENMRvn+5Kb4p5wQ/z57aSbdXrXFCJIiKoPr6JrOOMpq7t3sSUY6Am418NilJE
 9g+ahZrdF4NlQIQE4EnLw9vuMUvYspsYvs6qHkUe0dDPLh0elcplCVQgprsTncA1
 YFLCwZMR7c1W/OzETSFcx9ph9n0UejXwGXDKR7/DoLXJsX/gqaPDExfBbMO6wYBN
 oXXV5/jiLPrsKREP1xm2nIREZrxmwlpg55jmtaYJ1FSa1I7hzFl7e9/bccGFWPah
 BN4CB8cPdGNkyDNktMfK
 =UGuj
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kraxel/tags/usb-20190307-pull-request' into staging

usb: mtp fixes, guest-reset switch for usb-host.

# gpg: Signature made Thu 07 Mar 2019 09:53:55 GMT
# gpg:                using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/usb-20190307-pull-request:
  Introduce new "no_guest_reset" parameter for usb-host device
  usb-mtp: prevent null dereference while deleting objects
  usb-mtp: fix some usb_mtp_write_data return paths
  usb-mtp: return incomplete transfer on a lstat failure

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-07 14:06:42 +00:00
Laurent Vivier
61b463fbf6 linux-user: add new netlink types
Add QEMU_IFLA_BR_VLAN_STATS_PER_PORT (from linux v4.20),
    QEMU_IFLA_BR_MULTI_BOOLOPT (from linux v5.0).

The first new entry fixes the following error:

  Unknown QEMU_IFLA_BR type 45

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190306200925.17605-1-laurent@vivier.eu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 11:02:33 +01:00
Helge Deller
1b7695fe5f linux-user: Nicer strace output of chroot() syscall
Signed-off-by: Helge Deller <deller@gmx.de>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>

Message-Id: <20190227191115.GA20163@ls3530.dellerweb.de>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 10:57:29 +01:00
Marc-André Lureau
b78c522ab9 linux-user: fix "may be used uninitialized" warnings
Fixes:

/home/elmarco/src/qemu/linux-user/syscall.c: In function ‘do_ioctl_rt’:
/home/elmarco/src/qemu/linux-user/syscall.c:4773:9: error: ‘host_rt_dev_ptr’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
     if (*host_rt_dev_ptr != 0) {
         ^~~~~~~~~~~~~~~~
/home/elmarco/src/qemu/linux-user/syscall.c:4774:9: error: ‘target_rt_dev_ptr’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
         unlock_user((void *)*host_rt_dev_ptr,
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                     *target_rt_dev_ptr, 0);
                     ~~~~~~~~~~~~~~~~~~~~~~

Based on previous discussion from patch "linux-users/syscall: make
do_ioctl_rt safer" by Alex Bennée.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20190305151500.25038-1-marcandre.lureau@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 10:57:29 +01:00
Andreas Schwab
ba584f1de3 linux-user: don't short-circuit read with zero length
A zero-length read still needs to do the usual checks, thus it may return
errors like EBADF.  This makes the read syscall emulation consistent with
the pread64 syscall emulation.

Signed-off-by: Andreas Schwab <schwab@suse.de>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <mvm5zsxz2we.fsf@suse.de>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 10:57:29 +01:00
Sandra Loosemore
14c8a3a157 Fix breakpoint support in Nios II user-mode emulation.
Nios II user-mode emulation was missing handling for EXCP_DEBUG,
making the gdb stub essentially useless.  This patch adds the missing
piece.  The new code was copied from the existing EXCP_TRAP handling
and is also similar to what other targets (e.g., arm) do with EXCP_DEBUG.

Signed-off-by: Sandra Loosemore <sandra@codesourcery.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <1550076626-7202-1-git-send-email-sandra@codesourcery.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 10:57:29 +01:00
Andreas Schwab
cd8133679f linux-user: fix emulation of accept4/getpeername/getsockname/recvfrom syscalls
System calls that return a socket address do so by writing the (possibly
truncated) address into the provided buffer space, but setting the
addrlen parameter to the actual size of the address.  To determine how
much to copy back to the target memory the emulation needs to remember
the incoming value of the addrlen parameter, so that it doesn't write
past the buffer limits.

Signed-off-by: Andreas Schwab <schwab@suse.de>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <mvmimxmppcj.fsf_-_@suse.de>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 10:57:29 +01:00
Richard Henderson
e20e3ec928 linux-user: Fix ELF_PLATFORM for aarch64_be-linux-user
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20190212074840.13542-3-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 10:57:29 +01:00
Richard Henderson
13ec4ec3e1 linux-user: Add ELF_PLATFORM for arm
The 32-bit kernel has strings for v4, v5, v6, v7, v7m.
The 64-bit kernel, in compat mode, has strings for v8.

Fixes: https://bugs.launchpad.net/bugs/1813034
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20190212074840.13542-2-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 10:57:29 +01:00
Andreas Schwab
24894f39c6 linux-user: fix recvmsg emulation
Set msg_flags in the returned struct msghdr.

Signed-off-by: Andreas Schwab <schwab@suse.de>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <mvmimxprmn8.fsf@suse.de>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-03-07 10:57:28 +01:00
Alexander Kappner
ba4c735b4f Introduce new "no_guest_reset" parameter for usb-host device
With certain USB devices passed through via usb-host, a guest attempting to
reset a usb-host device can trigger a reset loop that renders the USB device
unusable. In my use case, the device was an iPhone XR that was passed through to
a Mac OS X Mojave guest. Upon connecting the device, the following happens:

1) Guest recognizes new device, sends reset to emulated USB host
2) QEMU's USB host sends reset to host kernel
3) Host kernel resets device
4) After reset, host kernel determines that some part of the device descriptor
has changed ("device firmware changed" in dmesg), so host kernel decides to
re-enumerate the device.
5) Re-enumeration causes QEMU to disconnect and reconnect the device in the
guest.
6) goto 1)

Here's from the host kernel (note the "device firmware changed" lines")

[3677704.473050] usb 1-1.3: new high-speed USB device number 53 using ehci-pci
[3677704.555594] usb 1-1.3: New USB device found, idVendor=05ac, idProduct=12a8, bcdDevice=11.08
[3677704.555599] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[3677704.555602] usb 1-1.3: Product: iPhone
[3677704.555605] usb 1-1.3: Manufacturer: Apple Inc.
[3677704.555607] usb 1-1.3: SerialNumber: [[removed]]
[3677709.401040] usb 1-1.3: reset high-speed USB device number 53 using ehci-pci
[3677709.479486] usb 1-1.3: device firmware changed
[3677709.479842] usb 1-1.3: USB disconnect, device number 53
[3677709.546039] usb 1-1.3: new high-speed USB device number 54 using ehci-pci
[3677709.627471] usb 1-1.3: New USB device found, idVendor=05ac, idProduct=12a8, bcdDevice=11.08
[3677709.627476] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[3677709.627479] usb 1-1.3: Product: iPhone
[3677709.627481] usb 1-1.3: Manufacturer: Apple Inc.
[3677709.627483] usb 1-1.3: SerialNumber: [[removed]]
[3677762.320044] usb 1-1.3: reset high-speed USB device number 54 using ehci-pci
[3677762.615630] usb 1-1.3: USB disconnect, device number 54
[3677762.787043] usb 1-1.3: new high-speed USB device number 55 using ehci-pci
[3677762.869016] usb 1-1.3: New USB device found, idVendor=05ac, idProduct=12a8, bcdDevice=11.08
[3677762.869024] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[3677762.869028] usb 1-1.3: Product: iPhone
[3677762.869032] usb 1-1.3: Manufacturer: Apple Inc.
[3677762.869035] usb 1-1.3: SerialNumber: [[removed]]
[3677815.662036] usb 1-1.3: reset high-speed USB device number 55 using ehci-pci

Here's from QEMU:

libusb: error [_get_usbfs_fd] libusb couldn't open USB device /dev/bus/usb/005/022: No such file or directory
libusb: error [udev_hotplug_event] ignoring udev action bind
libusb: error [udev_hotplug_event] ignoring udev action bind
libusb: error [_open_sysfs_attr] open /sys/bus/usb/devices/5-1/bConfigurationValue failed ret=-1 errno=2
libusb: error [_get_usbfs_fd] File doesn't exist, wait 10 ms and try again

libusb: error [_get_usbfs_fd] libusb couldn't open USB device /dev/bus/usb/005/024: No such file or directory
libusb: error [udev_hotplug_event] ignoring udev action bind
libusb: error [udev_hotplug_event] ignoring udev action bind
libusb: error [_open_sysfs_attr] open /sys/bus/usb/devices/5-1/bConfigurationValue failed ret=-1 errno=2
libusb: error [_get_usbfs_fd] File doesn't exist, wait 10 ms and try again

libusb: error [_get_usbfs_fd] libusb couldn't open USB device /dev/bus/usb/005/026: No such file or directory

The result of this is that the device remains permanently unusable in the guest.
The same problem has been previously reported for an iPad:
https://stackoverflow.com/questions/52617634/how-do-i-get-qemu-usb-passthrough-to-work-for-ipad-iphone

This problem can be elegantly solved by interrupting step 2) above. Instead of
passing through the reset, QEMU simply ignores it. To allow this to be
configured on a per-device level,  a new parameter "no_guest_reset" is
introduced for the usb-host device. I can confirm that the configuration
described above (iPhone XS + Mojave guest) works flawlessly with
no_guest_reset=True specified.

Working command line for my scenario:
device_add usb-host,vendorid=0x05ac,productid=0x12a8,no_guest_reset=True,id=iphone

Best regards
Alexander

Signed-off-by: Alexander Kappner <agk@godking.net>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20190128140027.9448-1-kraxel@redhat.com

[ kraxel: rename parameter to "guest-reset" ]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2019-03-07 10:03:54 +01:00
Bandan Das
7ddf837465 usb-mtp: prevent null dereference while deleting objects
Spotted by Coverity: CID 1399144

Signed-off-by: Bandan Das <bsd@redhat.com>
Message-id: 20190306210409.14842-4-bsd@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2019-03-07 10:02:48 +01:00
Bandan Das
298ac63c44 usb-mtp: fix some usb_mtp_write_data return paths
During a write, free up the "path" before getting more data.
Also, while we at it, remove the confusing usage of d->fd for
storing mkdir status

Spotted by Coverity: CID 1398642

Signed-off-by: Bandan Das <bsd@redhat.com>
Message-id: 20190306210409.14842-3-bsd@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2019-03-07 10:02:48 +01:00
Bandan Das
c5ead51f90 usb-mtp: return incomplete transfer on a lstat failure
MTP writes objects in small chunks and at the end gets the
real file size to update the object metadata. If this fails for
any reason, return an INCOMPLETE_TRANSFER to the initiator

Spotted by Coverity: CID 1398651

Signed-off-by: Bandan Das <bsd@redhat.com>
Message-id: 20190306210409.14842-2-bsd@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2019-03-07 10:02:48 +01:00
Peter Maydell
32694e98b8 Machine queue, 2019-03-06
* qdev: Hotplug handler chaining (David Hildenbrand)
 * qdev: fix qbus_is_full() (Tony Krowiak)
 * hostmem: fix crash when querying empty host-nodes property via
   QMP (Igor Mammedov)
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJcgBPhAAoJECgHk2+YTcWmWF0P/jyKA7fmdgTTqmgIvJ4nR2On
 d57GZobD+w3XqPUKFXJaayiwRiGrvZqBE1wUF9lURIVqG1vQVD5AT2hLQfC3PMv5
 DbmDkWlVSdZ3fQLlvXI7kNXFdYt7ae2Xr/HDKJiPe8LXL8D0EW8puUT0ywu/ihz9
 dKGo7erjASKZb3481mUyNcsRjUHYXxzyb8PHWciAmzm/xIedwsCcyZOA/ulIYl3t
 sqC/KewaxcZkExuvEUUpI+sPMHxGbu4Xq6QbhDMbxxT0b1MFa6zuoivNLnEvRE1q
 ixsSkKP1n9WqpdWqz+dmG7W6YTmlEw2F7E2MIPsLKij8V7pOnryECmd9FF1l95Cc
 1Ul/c0WE1LY/dN0G/fKND1TjFuvXN27kFl+X3bi5Kno3FPPu6ajMWjaiAbW+QSbV
 ODyMGWvDjVj0lRCmIwxIWQIqKp2f9NOB8xbdY1qdPXgQJm2zryMyGAcUJTBWFcXB
 7udErPIszhm2qwsjPnSPFYdn8MZbvzUauoGVJqxH0sbQmN237h3/DyV/+vZ/ZiRx
 1a2G3oKckIS1AccEusFuVRIvx2wBTfsD/9beEw/io2eKxknbxyCyj6i/qcivqSUY
 POUc9+Iwk8dC0tw6VKnENXdzHSmo7ZuWnO0jcuu+k37AHHDGLu2PgcP3Hm+SyGIY
 QjeumpJafu5Ek1GJ6sKz
 =rUEC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine queue, 2019-03-06

* qdev: Hotplug handler chaining (David Hildenbrand)
* qdev: fix qbus_is_full() (Tony Krowiak)
* hostmem: fix crash when querying empty host-nodes property via
  QMP (Igor Mammedov)

# gpg: Signature made Wed 06 Mar 2019 18:39:29 GMT
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/machine-next-pull-request:
  qdev: Provide qdev_get_bus_hotplug_handler()
  qdev: Let machine hotplug handler to override bus hotplug handler
  qdev: Let the hotplug_handler_unplug() caller delete the device
  hostmem: fix crash when querying empty host-nodes property via QMP
  qdev/core: fix qbus_is_full()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-06 18:52:19 +00:00
David Hildenbrand
14405c274e qdev: Provide qdev_get_bus_hotplug_handler()
Let's use a wrapper instead of looking it up manually. This function can
than be reused when we explicitly want to have the bus hotplug handler
(e.g. when the bus hotplug handler was overwritten by the machine
hotplug handler).

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-4-david@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-03-06 11:51:08 -03:00
Igor Mammedov
17cc0128da qdev: Let machine hotplug handler to override bus hotplug handler
it will allow to return another hotplug handler than the default
one for a specific bus based device type. Which is needed to handle
non trivial plug/unplug sequences that need the access to resources
configured outside of bus where device is attached.

That will allow for returned hotplug handler to orchestrate wiring
in arbitrary order, by chaining other hotplug handlers when
it's needed.

PS:
It could be used for hybrid virtio-mem and virtio-pmem devices
where it will return machine as hotplug handler which will do
necessary wiring at machine level and then pass control down
the chain to bus specific hotplug handler.

Example of top level hotplug handler override and custom plug sequence:

  some_machine_get_hotplug_handler(machine){
      if (object_dynamic_cast(OBJECT(dev), TYPE_SOME_BUS_DEVICE)) {
          return HOTPLUG_HANDLER(machine);
      }
      return NULL;
  }

  some_machine_device_plug(hotplug_dev, dev) {
      if (object_dynamic_cast(OBJECT(dev), TYPE_SOME_BUS_DEVICE)) {
          /* do machine specific initialization */
          some_machine_init_special_device(dev)

          /* pass control to bus specific handler */
          hotplug_handler_plug(dev->parent_bus->hotplug_handler, dev)
      }
  }

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-3-david@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-03-06 11:51:08 -03:00
David Hildenbrand
07578b0ad6 qdev: Let the hotplug_handler_unplug() caller delete the device
When unplugging a device, at one point the device will be destroyed
via object_unparent(). This will, one the one hand, unrealize the
removed device hierarchy, and on the other hand, destroy/free the
device hierarchy.

When chaining hotplug handlers, we want to overwrite a bus hotplug
handler by the machine hotplug handler, to be able to perform
some part of the plug/unplug and to forward the calls to the bus hotplug
handler.

For now, the bus hotplug handler would trigger an object_unparent(), not
allowing us to perform some unplug action on a device after we forwarded
the call to the bus hotplug handler. The device would be gone at that
point.

machine_unplug_handler(dev)
    /* eventually do unplug stuff */
    bus_unplug_handler(dev)
    /* dev is gone, we can't do more unplug stuff */

So move the object_unparent() to the original caller of the unplug. For
now, keep the unrealize() at the original places of the
object_unparent(). For implicitly chained hotplug handlers (e.g. pc
code calling acpi hotplug handlers), the object_unparent() has to be
done by the outermost caller. So when calling hotplug_handler_unplug()
from inside an unplug handler, nothing is to be done.

hotplug_handler_unplug(dev) -> calls machine_unplug_handler()
    machine_unplug_handler(dev) {
        /* eventually do unplug stuff */
        bus_unplug_handler(dev) -> calls unrealize(dev)
        /* we can do more unplug stuff but device already unrealized */
    }
object_unparent(dev)

In the long run, every unplug action should be factored out of the
unrealize() function into the unplug handler (especially for PCI). Then
we can get rid of the additonal unrealize() calls and object_unparent()
will properly unrealize the device hierarchy after the device has been
unplugged.

hotplug_handler_unplug(dev) -> calls machine_unplug_handler()
    machine_unplug_handler(dev) {
        /* eventually do unplug stuff */
        bus_unplug_handler(dev) -> only unplugs, does not unrealize
        /* we can do more unplug stuff */
    }
object_unparent(dev) -> will unrealize

The original approach was suggested by Igor Mammedov for the PCI
part, but I extended it to all hotplug handlers. I consider this one
step into the right direction.

To summarize:
- object_unparent() on synchronous unplugs is done by common code
-- "Caller of hotplug_handler_unplug"
- object_unparent() on asynchronous unplugs ("unplug requests") has to
  be done manually
-- "Caller of hotplug_handler_unplug"

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-2-david@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-03-06 11:51:08 -03:00
Igor Mammedov
15160ab72c hostmem: fix crash when querying empty host-nodes property via QMP
QEMU will crashes with
 qapi/qobject-output-visitor.c:210: qobject_output_complete: Assertion `qov->root && ((&qov->stack)->slh_first == ((void *)0))' failed
when trying to get value of not set hostmem's "host-nodes"
property, HostMemoryBackend::host_nodes bitmap doesn't have
any bits set in it, which leads to find_first_bit() returning
MAX_NODES and consequently to an early return from
host_memory_backend_get_host_nodes() without calling visitor.

Fix it by calling visitor even if "host-nodes" property wasn't
set before exiting from property getter to return valid empty
list.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20190214105733.25643-1-imammedo@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-03-06 11:51:08 -03:00
Tony Krowiak
12b2e9f30f qdev/core: fix qbus_is_full()
The qbus_is_full(BusState *bus) function (qdev_monitor.c) compares the max_index
value of the BusState structure with the max_dev value of the BusClass structure
to determine whether the maximum number of children has been reached for the
bus. The problem is, the max_index field of the BusState structure does not
necessarily reflect the number of devices that have been plugged into
the bus.

Whenever a child device is plugged into the bus, the bus's max_index value is
assigned to the child device and then incremented. If the child is subsequently
unplugged, the value of the max_index does not change and no longer reflects the
number of children.

When the bus's max_index value reaches the maximum number of devices
allowed for the bus (i.e., the max_dev field in the BusClass structure),
attempts to plug another device will be rejected claiming that the bus is
full -- even if the bus is actually empty.

To resolve the problem, a new 'num_children' field is being added to the
BusState structure to keep track of the number of children plugged into the
bus. It will be incremented when a child is plugged, and decremented when a
child is unplugged.

Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com>
Reviewed-by: Pierre Morel<pmorel@linux.ibm.com>
Reviewed-by: Halil Pasic <pasic@linux.ibm.com>
Message-Id: <1545062250-7573-1-git-send-email-akrowiak@linux.ibm.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-03-06 11:51:08 -03:00
Peter Maydell
c557a8c7b7 Migation pull 2019-03-06
(This replaces the pull sent yesterday)
 
    a) 4 small fixes including the cancel problem
      that caused the ahci migration test to fail
      intermittently
    b) Yury's ignore-shared feature
    c) Juan's extra tests
    d) Wei Wang's free page hinting
    e) Some Colo fixes from Zhang Chen
 
 Diff from yesterdays pull:
   1) A missing fix of mine (cleanup during exit)
   2) Changes from Eric/Markus on 'Create socket-address parameter'
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJcf7GJAAoJEAUWMx68W/3n+lMP/Rl/d7hpi0Ve2fm3VEwoFJea
 IRiqo7Yk6heyTCIutFq15pD2ef49AXHpLeGBp9gFNb4bdFTQzHmwOPxeJWig8YXV
 m+j5sGRaM9sV8XX24DsZM7yFhpVJmWky8ivMSv3LeEmjx251B9CNL13dc/qVUQHv
 lYP6ewnOmtjvR+x+z9Q/+vafnpLWJSxup1G0pZWdQfLpl71E2sMf7FY/G5EroVnf
 AXmJb1sjdFXF7n968myfcgYETHsnY0SUa89Bcnd+i40DXvSfa4njXSdE4FOhyIim
 n8c4SyRA/Ah2EUl+UGxn8TQ78C4RA3dUS+uXJDmjL1e4ACvqq//nhsfIqTJ9AbgF
 Jhx5ArwqrGf7D+/PM5ivDocNplT5JFcCB4OCmZO96Kn0/F6M3UHuL1+IvpQcFMm8
 1Ar1REB7BZ6f+QLfY8KKuzVrVRzUBi0DbqFHj5TNIStizOkuUEMMRpcWImBMzslG
 531YgTnsSeFfFr13ZJlXDscZSZ5i+fJMjNbH9QpTNy8qmLJoZzbKqpmP4pZmHVI2
 w3g1pCHpFejuQtUTNMR3+9mVH5hO+MNrANsTH0yfAXYDNToJ6NkY1nnILHp4P7t1
 tqHYN7AO2ZXTTTMSnfyv1+2wh3HZRFB/y7uF6uEowBfuZTRHBHnkaVQp5WbVVSJu
 4ovMmHDkcX2bM7VWwTHS
 =dk89
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20190306a' into staging

Migation pull 2019-03-06

(This replaces the pull sent yesterday)

   a) 4 small fixes including the cancel problem
     that caused the ahci migration test to fail
     intermittently
   b) Yury's ignore-shared feature
   c) Juan's extra tests
   d) Wei Wang's free page hinting
   e) Some Colo fixes from Zhang Chen

Diff from yesterdays pull:
  1) A missing fix of mine (cleanup during exit)
  2) Changes from Eric/Markus on 'Create socket-address parameter'

# gpg: Signature made Wed 06 Mar 2019 11:39:53 GMT
# gpg:                using RSA key 0516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20190306a: (22 commits)
  qapi/migration.json: Remove a variable that doesn't exist in example
  Migration/colo.c: Make COLO node running after failover
  Migration/colo.c: Fix double close bug when occur COLO failover
  virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
  migration/ram.c: add the free page optimization enable flag
  migration/ram.c: add a notifier chain for precopy
  migration: API to clear bits of guest free pages from the dirty bitmap
  migration: use bitmap_mutex in migration_bitmap_clear_dirty
  bitmap: bitmap_count_one_with_offset
  bitmap: fix bitmap_count_one
  tests: Add basic migration precopy tcp test
  migration: Create socket-address parameter
  tests: Add migration xbzrle test
  migration: Add capabilities validation
  tests/migration-test: Add a test for ignore-shared capability
  migration: Add an ability to ignore shared RAM blocks
  migration: Introduce ignore-shared capability
  exec: Change RAMBlockIterFunc definition
  migration/rdma: clang compilation fix
  migration: Cleanup during exit
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-06 14:50:33 +00:00
Peter Maydell
9b748c5e06 trivial patches pull request (20190206)
- acpi: remove unused functions/variables
 - tests: remove useless architecture checks
 - some typo fixes and documentation update
 - flash_cfi02: fix memory leak
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJcf6loAAoJEPMMOL0/L748mXUP/1m4TMqyu1RqxBLVGYJTXs/t
 LcrIvIJmhtLndO2XgAUUxWC9zLsTWhLDJ93yY4QmkYy6N1WhTbhMmKj0VwKNDq+9
 87nbY/VQMSp8Tjx/rGO8LSA0F7F3gemhdl5UFySfY3sg8AkNV94qo4XZ+KNnkz7p
 ZFzaN/3T14AUXSLlaEhXCZyR0d5D0R4UjeOaqrJLal+IqfaUrmCGR6+RGAHKkKss
 lzEwj9S6WG96YvRxfKJ4ozQ4TpaqPwN9svVCdd2kQN5zLjjTIXB9J1i6NudQm5n3
 AXn3uaHX77kEkoptydaaoWgEDcotoEfokaywE/R+FvrWArUK9IyekuBCgq8b9i1V
 MQLc07+kQBhdEtip04d7G/QNPpk4L4zzXgr+iQSuVclkW2tTT1xid+Tdf7S4e3xL
 tBc9WVdqbEd5JhDIurrSeJcquUCOCMFBCe8xqDm0aOdHowRn7c9nemDT+3r0Q4cc
 iVzzqTKbgUlSla/TzLcaaCoAtZfCvsvt9Slv5yRkMLnyN+WfXJt0fNXwPEUs08hP
 C7fhk8xSWJk8TRwaY0X4jlG0myq3RueaP8xNcOLj1xld/VTKQM1h09chbfDmLESz
 vylJlwTqL8MXF6/2Yi8byuIJw9mxkma7CfTK6mEAPfpyHzWfvmBWK26j1+V1LnFo
 Ay/tOt4QtyMare7K0d+b
 =ODNS
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging

trivial patches pull request (20190206)

- acpi: remove unused functions/variables
- tests: remove useless architecture checks
- some typo fixes and documentation update
- flash_cfi02: fix memory leak

# gpg: Signature made Wed 06 Mar 2019 11:05:12 GMT
# gpg:                using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-pull-request:
  thunk: fix of malloc to g_new
  hostmem-file: simplify ifdef-s in file_backend_memory_alloc()
  build: Correct explanation of unnest-vars example
  bswap: Fix accessors syntax in comment
  doc: fix typos for documents in tree
  block/pflash_cfi02: Fix memory leak and potential use-after-free
  hw/acpi: remove unnecessary variable acpi_table_builtin
  hw/acpi: remove unused function acpi_table_add_builtin()
  hw/i386/pc.c: remove unused function pc_acpi_init()
  tests: Remove (mostly) useless architecture checks

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-06 11:58:10 +00:00