Go to file
Peter Xu 8f8bfffcf1 migration: Maintain postcopy faulted addresses
Maintain a list of faulted addresses on the destination host for which we're
waiting on.  This is implemented using a GTree rather than a real list to make
sure even there're plenty of vCPUs/threads that are faulting, the lookup will
still be fast with O(log(N)) (because we'll do that after placing each page).
It should bring a slight overhead, but ideally that shouldn't be a big problem
simply because in most cases the requested page list will be short.

Actually we did similar things for postcopy blocktime measurements.  This patch
didn't use that simply because:

  (1) blocktime measurement is towards vcpu threads only, but here we need to
      record all faulted addresses, including main thread and external
      thread (like, DPDK via vhost-user).

  (2) blocktime measurement will require UFFD_FEATURE_THREAD_ID, but here we
      don't want to add that extra dependency on the kernel version since not
      necessary.  E.g., we don't need to know which thread faulted on which
      page, we also don't care about multiple threads faulting on the same
      page.  But we only care about what addresses are faulted so waiting for a
      page copying from src.

  (3) blocktime measurement is not enabled by default.  However we need this by
      default especially for postcopy recover.

Another thing to mention is that this patch introduced a new mutex to serialize
the receivedmap and the page_requested tree, however that serialization does
not cover other procedures like UFFDIO_COPY.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-4-peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-26 16:15:04 +00:00
.github .github: Enable repo-lockdown bot to refuse GitHub pull requests 2020-04-07 16:19:18 +01:00
.gitlab-ci.d gitlab: add a CI job to validate the DCO sign off 2020-10-13 12:48:17 +02:00
accel accel: Add xen CpusAccel using dummy-cpus 2020-10-24 07:23:19 +02:00
audio spice: move add_interface() to QemuSpiceOps. 2020-10-21 15:46:14 +02:00
authz authz-list-file: Fix crash when filename is not set 2020-10-13 15:56:30 -04:00
backends vhost-user: Register "chardev" as class property 2020-10-13 15:56:30 -04:00
block block/io: fix bdrv_is_allocated_above 2020-10-23 13:42:16 +01:00
bsd-user meson: bsd-user 2020-08-21 06:30:38 -04:00
capstone@f8b1b83301 capstone: Update to upstream "next" branch 2020-10-03 04:23:14 -05:00
chardev spice: move add_interface() to QemuSpiceOps. 2020-10-21 15:46:14 +02:00
contrib util/vhost-user-server: use static library in meson.build 2020-10-23 13:42:16 +01:00
crypto crypto: Move the creation of the library to the main meson.build 2020-10-12 11:50:20 -04:00
default-configs hw/arm: Restrict APEI tables generation to the 'virt' machine 2020-10-20 16:12:00 +01:00
disas disas/capstone: Add skipdata hook for s390x 2020-10-03 04:25:14 -05:00
docs fuzz: Add instructions for using generic-fuzz 2020-10-26 09:53:53 +01:00
dtc@85e5d83984 Makefile: dtc: update, build the libfdt target 2020-06-16 14:49:05 +01:00
dump monitor: Use getter/setter functions for cur_mon 2020-10-09 07:08:19 +02:00
fpu softfloat: Define misc operations for bfloat16 2020-08-28 10:48:07 -07:00
fsdev Pull request trivial patches 20200919 2020-09-22 15:42:23 +01:00
gdb-xml target/avr: CPU class: Add GDB support 2020-07-10 17:58:32 +02:00
hw some s390x fixes 2020-10-26 14:50:03 +00:00
include migration: Drop unused VMSTATE_FLOAT64 support 2020-10-26 16:15:04 +00:00
io io: Move the creation of the library to the main meson.build 2020-10-12 11:50:20 -04:00
libdecnumber qemu/: fix some comment spelling errors 2020-09-17 20:35:43 +02:00
linux-headers linux headers: sync to 5.9-rc7 2020-10-08 15:24:32 +01:00
linux-user linux-user/microblaze: Remove non-rt signal frames 2020-10-14 21:19:56 -07:00
meson@776acd2a80 submodules: bump meson to 0.55.3 2020-10-17 10:45:42 -04:00
migration migration: Maintain postcopy faulted addresses 2020-10-26 16:15:04 +00:00
monitor spice: move display_add_client() to QemuSpiceOps. 2020-10-21 15:46:14 +02:00
nbd block/export: add iothread and fixed-iothread options 2020-10-23 13:42:16 +01:00
net can-host-socketcan: Fix crash when 'if' option is not set 2020-10-13 15:56:30 -04:00
pc-bios Remove superfluous .gitignore files 2020-10-13 12:48:17 +02:00
plugins Remove superfluous .gitignore files 2020-10-13 12:48:17 +02:00
po meson: convert po/ 2020-08-21 06:30:45 -04:00
python python/qemu/qmp.py: Fix settimeout operation 2020-10-20 15:00:06 -04:00
qapi block/export: add vhost-user-blk multi-queue support 2020-10-23 13:42:16 +01:00
qga qmp: Call monitor_set_cur() only in qmp_dispatch() 2020-10-09 07:08:19 +02:00
qobject qemu/: fix some comment spelling errors 2020-09-17 20:35:43 +02:00
qom qom: Add user_creatable_print_help_from_qdict() 2020-10-15 16:06:27 +02:00
replay replay: create temporary snapshot at debugger connection 2020-10-06 08:34:49 +02:00
roms seabios: add microvm config, update build rules 2020-09-17 14:16:19 +02:00
scripts scripts/oss-fuzz: ignore the generic-fuzz target 2020-10-26 09:53:54 +01:00
scsi qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
slirp@ce94eba204 slirp: update to latest stable-4.2 branch 2020-07-28 18:27:59 +04:00
softmmu * qtest fixes (e.g. memory leaks) 2020-10-26 13:16:29 +00:00
storage-daemon qemu-storage-daemon: avoid compiling blockdev_ss twice 2020-10-23 13:42:16 +01:00
stubs block: move block exports to libblockdev 2020-10-23 13:42:16 +01:00
target some s390x fixes 2020-10-26 14:50:03 +00:00
tcg tcg: Remove TCG_TARGET_HAS_cmp_vec 2020-10-08 05:57:32 -05:00
tests * qtest fixes (e.g. memory leaks) 2020-10-26 13:16:29 +00:00
tools libvhost-user: Allow vu_message_read to be replaced 2020-10-23 13:42:16 +01:00
trace monitor: Use getter/setter functions for cur_mon 2020-10-09 07:08:19 +02:00
ui modules: build spice and opengl as module. 2020-10-22 12:33:21 +01:00
util util/vhost-user-server: use static library in meson.build 2020-10-23 13:42:16 +01:00
.cirrus.yml ci: include configure and meson logs in all jobs if configure fails 2020-10-17 10:45:53 -04:00
.dir-locals.el Add .dir-locals.el file to configure emacs coding style 2015-10-08 19:46:01 +03:00
.editorconfig meson: rename included C source files to .c.inc 2020-08-21 06:18:30 -04:00
.exrc qemu: add .exrc 2012-09-07 09:02:44 +03:00
.gdbinit .gdbinit: load QEMU sub-commands when gdb starts 2017-06-07 14:38:45 +01:00
.gitignore Add *.pyc back to the .gitignore file 2020-09-23 19:09:56 +02:00
.gitlab-ci.yml tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI 2020-10-26 09:54:16 +01:00
.gitmodules gitmodules: add qemu.org vbootrom submodule 2020-09-23 13:44:59 +01:00
.gitpublish Add a git-publish configuration file 2018-03-05 09:03:17 +00:00
.mailmap .mailmap: Fix more contributor entries 2020-10-09 17:27:55 +01:00
.patchew.yml ci: store Patchew configuration in the tree 2019-06-03 14:03:02 +02:00
.readthedocs.yml readthedocs: build with Python 3.6 2020-10-05 16:30:45 +01:00
.shippable.yml shippable.yml: Remove the Debian9-based MinGW cross-compiler tests 2020-10-02 12:29:11 +01:00
.travis.yml ci: include configure and meson logs in all jobs if configure fails 2020-10-17 10:45:53 -04:00
block.c block: Add bdrv_lock()/unlock() 2020-10-09 07:08:20 +02:00
blockdev-nbd.c nbd: Merge nbd_export_new() and nbd_export_create() 2020-10-02 15:46:40 +02:00
blockdev.c block: Convert 'block_resize' to coroutine 2020-10-09 07:08:20 +02:00
blockjob.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
Changelog qemu/: fix some comment spelling errors 2020-09-17 20:35:43 +02:00
CODING_STYLE.rst CODING_STYLE.rst: flesh out our naming conventions. 2020-09-10 10:34:58 +01:00
configure xen: xenguest is not used so is not needed 2020-10-19 16:22:58 +01:00
COPYING
COPYING.LIB COPYING.LIB: Synchronize the LGPL 2.1 with the version from gnu.org 2019-01-30 11:01:22 +01:00
cpu.c exec: split out non-softmmu-specific parts 2020-10-12 11:50:22 -04:00
cpus-common.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
disas.c disas: Enable capstone disassembly for s390x 2020-10-03 04:25:14 -05:00
exec-vary.c exec: Cache TARGET_PAGE_MASK for TARGET_PAGE_BITS_VARY 2019-10-28 10:35:20 +01:00
gdbstub.c replay: create temporary snapshot at debugger connection 2020-10-06 08:34:49 +02:00
gitdm.config contrib: gitdm: add a mapping for Janus Technologies 2019-03-12 19:31:29 +00:00
hmp-commands-info.hx replay: introduce info hmp/qmp command 2020-10-06 08:34:49 +02:00
hmp-commands.hx block: Convert 'block_resize' to coroutine 2020-10-09 07:08:20 +02:00
iothread.c qemu/atomic.h: rename atomic_ to qatomic_ 2020-09-23 16:07:44 +01:00
job-qmp.c trace: switch position of headers to what Meson requires 2020-08-21 06:18:24 -04:00
job.c trace: switch position of headers to what Meson requires 2020-08-21 06:18:24 -04:00
Kconfig Makefile: simplify MINIKCONF rules 2020-07-10 18:02:21 -04:00
Kconfig.host configure: Fix build dependencies with vhost-vdpa. 2020-09-29 02:15:24 -04:00
LICENSE tcg/LICENSE: Remove out of date claim about TCG subdirectory licensing 2019-11-11 15:11:21 +01:00
MAINTAINERS util/vhost-user-server: move header to include/ 2020-10-23 13:42:16 +01:00
Makefile build: replace ninjatool with ninja 2020-10-17 10:45:51 -04:00
memory_ldst.c.inc fuzz: Add fuzzer callbacks to DMA-read functions 2020-10-26 09:53:53 +01:00
meson_options.txt meson: Move the detection logic for sphinx to meson 2020-10-17 10:45:53 -04:00
meson.build block: move block exports to libblockdev 2020-10-23 13:42:16 +01:00
module-common.c all: Clean up includes 2016-02-04 17:41:30 +00:00
os-posix.c vl: relocate paths to data directories 2020-09-30 19:11:36 +02:00
os-win32.c vl: relocate paths to data directories 2020-09-30 19:11:36 +02:00
qemu-bridge-helper.c qemu-bridge-helper: relocate path to default ACL 2020-09-30 19:11:36 +02:00
qemu-edid.c qemu-edid: drop cast 2020-10-14 06:05:56 +02:00
qemu-img-cmds.hx qemu-img-cmds.hx: Update comment that mentions Texinfo 2020-10-12 23:11:32 +02:00
qemu-img.c bitmaps patches for 2020-09-21 2020-09-23 15:11:38 +01:00
qemu-io-cmds.c qemu-io-cmds: Simplify help_oneline 2020-10-02 15:46:40 +02:00
qemu-io.c block: add missing socket_init() calls to tools 2020-09-02 16:47:39 -05:00
qemu-keymap.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
qemu-nbd.c block: move block exports to libblockdev 2020-10-23 13:42:16 +01:00
qemu-options-wrapper.h qemu-img: remove references to GEN_DOCS 2018-05-20 08:35:54 +03:00
qemu-options.h Clean up ill-advised or unusual header guards 2016-07-12 16:20:46 +02:00
qemu-options.hx docs: add 'io_uring' option to 'aio' param in qemu-options.hx 2020-10-05 10:59:42 +01:00
qemu.nsi qemu.nsi: Install Sphinx documentation 2020-03-09 16:45:00 +00:00
qemu.sasl Default to GSSAPI (Kerberos) instead of DIGEST-MD5 for SASL 2017-05-09 14:41:47 +01:00
README.rst docs: merge HACKING.rst contents into CODING_STYLE.rst 2019-09-05 14:27:06 +01:00
replication.c replication: Introduce new APIs to do replication operation 2016-09-13 11:00:56 +01:00
replication.h Include qemu/module.h where needed, drop it from qemu-common.h 2019-06-12 13:18:33 +02:00
thunk.c linux-user: Add strace support for printing arguments for ioctls used for terminals and serial lines 2020-08-27 12:29:50 +02:00
trace-events trace-events: Fix attribution of trace points to source 2020-09-09 17:17:58 +01:00
VERSION Open 5.2 development tree 2020-08-18 13:44:04 +01:00
version.rc Use HTTPS for qemu.org and other domains 2017-11-21 13:34:13 +00:00
version.texi.in meson: build texi doc 2020-08-21 06:30:42 -04:00

===========
QEMU README
===========

QEMU is a generic and open source machine & userspace emulator and
virtualizer.

QEMU is capable of emulating a complete machine in software without any
need for hardware virtualization support. By using dynamic translation,
it achieves very good performance. QEMU can also integrate with the Xen
and KVM hypervisors to provide emulated hardware while allowing the
hypervisor to manage the CPU. With hypervisor support, QEMU can achieve
near native performance for CPUs. When QEMU emulates CPUs directly it is
capable of running operating systems made for one machine (e.g. an ARMv7
board) on a different machine (e.g. an x86_64 PC board).

QEMU is also capable of providing userspace API virtualization for Linux
and BSD kernel interfaces. This allows binaries compiled against one
architecture ABI (e.g. the Linux PPC64 ABI) to be run on a host using a
different architecture ABI (e.g. the Linux x86_64 ABI). This does not
involve any hardware emulation, simply CPU and syscall emulation.

QEMU aims to fit into a variety of use cases. It can be invoked directly
by users wishing to have full control over its behaviour and settings.
It also aims to facilitate integration into higher level management
layers, by providing a stable command line interface and monitor API.
It is commonly invoked indirectly via the libvirt library when using
open source applications such as oVirt, OpenStack and virt-manager.

QEMU as a whole is released under the GNU General Public License,
version 2. For full licensing details, consult the LICENSE file.


Building
========

QEMU is multi-platform software intended to be buildable on all modern
Linux platforms, OS-X, Win32 (via the Mingw64 toolchain) and a variety
of other UNIX targets. The simple steps to build QEMU are:


.. code-block:: shell

  mkdir build
  cd build
  ../configure
  make

Additional information can also be found online via the QEMU website:

* `<https://qemu.org/Hosts/Linux>`_
* `<https://qemu.org/Hosts/Mac>`_
* `<https://qemu.org/Hosts/W32>`_


Submitting patches
==================

The QEMU source code is maintained under the GIT version control system.

.. code-block:: shell

   git clone https://git.qemu.org/git/qemu.git

When submitting patches, one common approach is to use 'git
format-patch' and/or 'git send-email' to format & send the mail to the
qemu-devel@nongnu.org mailing list. All patches submitted must contain
a 'Signed-off-by' line from the author. Patches should follow the
guidelines set out in the CODING_STYLE.rst file.

Additional information on submitting patches can be found online via
the QEMU website

* `<https://qemu.org/Contribute/SubmitAPatch>`_
* `<https://qemu.org/Contribute/TrivialPatches>`_

The QEMU website is also maintained under source control.

.. code-block:: shell

  git clone https://git.qemu.org/git/qemu-web.git

* `<https://www.qemu.org/2017/02/04/the-new-qemu-website-is-up/>`_

A 'git-publish' utility was created to make above process less
cumbersome, and is highly recommended for making regular contributions,
or even just for sending consecutive patch series revisions. It also
requires a working 'git send-email' setup, and by default doesn't
automate everything, so you may want to go through the above steps
manually for once.

For installation instructions, please go to

*  `<https://github.com/stefanha/git-publish>`_

The workflow with 'git-publish' is:

.. code-block:: shell

  $ git checkout master -b my-feature
  $ # work on new commits, add your 'Signed-off-by' lines to each
  $ git publish

Your patch series will be sent and tagged as my-feature-v1 if you need to refer
back to it in the future.

Sending v2:

.. code-block:: shell

  $ git checkout my-feature # same topic branch
  $ # making changes to the commits (using 'git rebase', for example)
  $ git publish

Your patch series will be sent with 'v2' tag in the subject and the git tip
will be tagged as my-feature-v2.

Bug reporting
=============

The QEMU project uses Launchpad as its primary upstream bug tracker. Bugs
found when running code built from QEMU git or upstream released sources
should be reported via:

* `<https://bugs.launchpad.net/qemu/>`_

If using QEMU via an operating system vendor pre-built binary package, it
is preferable to report bugs to the vendor's own bug tracker first. If
the bug is also known to affect latest upstream code, it can also be
reported via launchpad.

For additional information on bug reporting consult:

* `<https://qemu.org/Contribute/ReportABug>`_


Contact
=======

The QEMU community can be contacted in a number of ways, with the two
main methods being email and IRC

* `<mailto:qemu-devel@nongnu.org>`_
* `<https://lists.nongnu.org/mailman/listinfo/qemu-devel>`_
* #qemu on irc.oftc.net

Information on additional methods of contacting the community can be
found online via the QEMU website:

* `<https://qemu.org/Contribute/StartHere>`_