qemu/qapi
David Hildenbrand 910b25766b virtio-mem: Paravirtualized memory hot(un)plug
This is the very basic/initial version of virtio-mem. An introduction to
virtio-mem can be found in the Linux kernel driver [1]. While it can be
used in the current state for hotplug of a smaller amount of memory, it
will heavily benefit from resizeable memory regions in the future.

Each virtio-mem device manages a memory region (provided via a memory
backend). After requested by the hypervisor ("requested-size"), the
guest can try to plug/unplug blocks of memory within that region, in order
to reach the requested size. Initially, and after a reboot, all memory is
unplugged (except in special cases - reboot during postcopy).

The guest may only try to plug/unplug blocks of memory within the usable
region size. The usable region size is a little bigger than the
requested size, to give the device driver some flexibility. The usable
region size will only grow, except on reboots or when all memory is
requested to get unplugged. The guest can never plug more memory than
requested. Unplugged memory will get zapped/discarded, similar to in a
balloon device.

The block size is variable, however, it is always chosen in a way such that
THP splits are avoided (e.g., 2MB). The state of each block
(plugged/unplugged) is tracked in a bitmap.

As virtio-mem devices (e.g., virtio-mem-pci) will be memory devices, we now
expose "VirtioMEMDeviceInfo" via "query-memory-devices".

--------------------------------------------------------------------------

There are two important follow-up items that are in the works:
1. Resizeable memory regions: Use resizeable allocations/RAM blocks to
   grow/shrink along with the usable region size. This avoids creating
   initially very big VMAs, RAM blocks, and KVM slots.
2. Protection of unplugged memory: Make sure the gust cannot actually
   make use of unplugged memory.

Other follow-up items that are in the works:
1. Exclude unplugged memory during migration (via precopy notifier).
2. Handle remapping of memory.
3. Support for other architectures.

--------------------------------------------------------------------------

Example usage (virtio-mem-pci is introduced in follow-up patches):

Start QEMU with two virtio-mem devices (one per NUMA node):
 $ qemu-system-x86_64 -m 4G,maxmem=20G \
  -smp sockets=2,cores=2 \
  -numa node,nodeid=0,cpus=0-1 -numa node,nodeid=1,cpus=2-3 \
  [...]
  -object memory-backend-ram,id=mem0,size=8G \
  -device virtio-mem-pci,id=vm0,memdev=mem0,node=0,requested-size=0M \
  -object memory-backend-ram,id=mem1,size=8G \
  -device virtio-mem-pci,id=vm1,memdev=mem1,node=1,requested-size=1G

Query the configuration:
 (qemu) info memory-devices
 Memory device [virtio-mem]: "vm0"
   memaddr: 0x140000000
   node: 0
   requested-size: 0
   size: 0
   max-size: 8589934592
   block-size: 2097152
   memdev: /objects/mem0
 Memory device [virtio-mem]: "vm1"
   memaddr: 0x340000000
   node: 1
   requested-size: 1073741824
   size: 1073741824
   max-size: 8589934592
   block-size: 2097152
   memdev: /objects/mem1

Add some memory to node 0:
 (qemu) qom-set vm0 requested-size 500M

Remove some memory from node 1:
 (qemu) qom-set vm1 requested-size 200M

Query the configuration again:
 (qemu) info memory-devices
 Memory device [virtio-mem]: "vm0"
   memaddr: 0x140000000
   node: 0
   requested-size: 524288000
   size: 524288000
   max-size: 8589934592
   block-size: 2097152
   memdev: /objects/mem0
 Memory device [virtio-mem]: "vm1"
   memaddr: 0x340000000
   node: 1
   requested-size: 209715200
   size: 209715200
   max-size: 8589934592
   block-size: 2097152
   memdev: /objects/mem1

[1] https://lkml.kernel.org/r/20200311171422.10484-1-david@redhat.com

Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eric Blake <eblake@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20200626072248.78761-11-david@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-07-03 07:57:04 -04:00
..
audio.json audio/jack: add JACK client audiodev 2020-05-25 11:30:03 +02:00
authz.json authz: add QAuthZList object type for an access control list 2019-02-26 15:32:18 +00:00
block-core.json qcow2: Expose bitmaps' size during measure 2020-05-28 13:16:16 -05:00
block.json qapi: Mark deprecated QMP parts with feature 'deprecated' 2020-03-17 21:43:12 +01:00
char.json qapi: Belatedly update doc comment for @wait deprecation 2020-03-17 19:58:34 +01:00
common.json qapi: Move query-target from misc.json to machine.json 2019-08-21 13:24:01 +02:00
control.json qapi: Mark deprecated QMP parts with feature 'deprecated' 2020-03-17 21:43:12 +01:00
crypto.json crypto: Fix defaults in QCryptoBlockCreateOptionsLUKS 2018-12-12 11:16:49 +00:00
dump.json qapi: Fix indent level on doc comments in json files 2020-02-15 11:41:50 +01:00
error.json qapi: Split error.json off common.json 2019-08-16 13:31:51 +02:00
introspect.json qapi: Add feature flags to struct members 2020-03-17 21:25:47 +01:00
job.json qapi: Fix indent level on doc comments in json files 2020-02-15 11:41:50 +01:00
machine-target.json qapi: Fix indent level on doc comments in json files 2020-02-15 11:41:50 +01:00
machine.json Add rx-softmmu 2020-03-19 17:58:05 +01:00
Makefile.objs qemu-storage-daemon: Add --monitor option 2020-03-06 17:21:28 +01:00
migration.json migration/xbzrle: add encoding rate 2020-05-07 17:40:24 +01:00
misc-target.json qapi: Fix indent level on doc comments in json files 2020-02-15 11:41:50 +01:00
misc.json virtio-mem: Paravirtualized memory hot(un)plug 2020-07-03 07:57:04 -04:00
net.json net: Drop the NetLegacy structure, always use Netdev instead 2020-06-18 21:05:52 +08:00
opts-visitor.c make check-unit: use after free in test-opts-visitor 2019-09-24 11:03:44 +02:00
pragma.json qapi: Create 'pragma' module 2020-03-06 17:21:28 +01:00
qapi-clone-visitor.c qapi/qnull: Add own header 2017-11-17 18:21:30 +01:00
qapi-dealloc-visitor.c qapi: Fix Visitor contract for start_alternate() 2020-04-30 06:51:15 +02:00
qapi-schema.json qapi: Create 'pragma' module 2020-03-06 17:21:28 +01:00
qapi-util.c qemu-common: Move qemu_isalnum() etc. to qemu/ctype.h 2019-06-11 20:22:09 +02:00
qapi-visit-core.c qapi: Assert non-input visitors see only valid narrow integers 2020-04-30 07:26:40 +02:00
qdev.json qapi: Fix indent level on doc comments in json files 2020-02-15 11:41:50 +01:00
qmp-dispatch.c qmp: fix leak on callbacks that return both value and error 2020-04-02 14:55:46 -04:00
qmp-event.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
qmp-registry.c qmp: constify QmpCommand and list 2020-03-17 21:43:12 +01:00
qobject-input-visitor.c qobject: Eliminate qdict_iter(), use qdict_first(), qdict_next() 2020-04-30 06:51:15 +02:00
qobject-output-visitor.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
qom.json qapi: Flatten object-add 2020-03-06 17:21:27 +01:00
rdma.json qapi: Define new QMP message for pvrdma 2018-12-22 11:09:56 +02:00
rocker.json qapi: Fix indent level on doc comments in json files 2020-02-15 11:41:50 +01:00
run-state.json qapi: Fix indent level on doc comments in json files 2020-02-15 11:41:50 +01:00
sockets.json qemu-sockets: add abstract UNIX domain socket support 2020-05-20 10:34:40 +01:00
string-input-visitor.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
string-output-visitor.c cutils: Move size_to_str() from "qemu-common.h" to "qemu/cutils.h" 2019-09-19 11:57:34 +02:00
tpm.json qapi: Use explicit bulleted lists 2020-02-15 11:41:50 +01:00
trace-events trace-events: Shorten file names in comments 2019-03-22 16:18:07 +00:00
trace.json qapi: Add blank lines before bulleted lists 2020-02-15 11:41:50 +01:00
transaction.json qemu-storage-daemon: Add --monitor option 2020-03-06 17:21:28 +01:00
ui.json qapi: Delete all the "foo: dropped in n.n" notes 2020-02-15 11:41:50 +01:00