Commit Graph

93182 Commits

Author SHA1 Message Date
Juan Quintela
468fcb5dd0 multifd: Remove send_write() method
Everything use now iov's.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela
0a818b89eb multifd: Make zstd use iov's
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela
48a4a44c1c multifd: Make zlib use iov's
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela
226468ba3d multifd: Move iov from pages to params
This will allow us to reduce the number of system calls on the next patch.

Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela
fc6705229c multifd: Use proper maximum compression values
It happens that there are functions to calculate the worst possible
compression size for a packet.  Use them.

Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela
47fe16ff66 migration: Move ram_release_pages() call to save_zero_page_to_file()
We always need to call it when we find a zero page, so put it in a
single place.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela
e7f2e190e5 migration: simplify do_compress_ram_page
The goto is not needed at all.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela
20d549cb0b migration: Remove masking for compression
Remove the mask in the call to ram_release_pages().  Nothing else does
it, and if the offset has that bits set, we have a lot of trouble.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela
0189c72291 migration: ram_release_pages() always receive 1 page as argument
Remove the pages argument. And s/pages/page/

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>

---

- Use 1LL instead of casts (philmd)
- Change the whole 1ULL for TARGET_PAGE_SIZE
2022-01-28 15:38:22 +01:00
Juan Quintela
05931ec561 migration: We only need last_stage in two places
We only need last_stage in two places and we are passing it all
around.  Just add a field to RAMState that passes it.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>

---

Repeat subject (philmd suggestion)
2022-01-28 15:38:22 +01:00
Juan Quintela
04e1140494 migration: All this fields are unsigned
So printing it as %d is wrong.  Notice that for the channel id, that
is an uint8_t, but I changed it anyways for consistency.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
2022-01-28 15:38:22 +01:00
Peter Maydell
b367db4812 Xen patches
- bug fixes for mapcache and ioreq handling
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEE+AwAYwjiLP2KkueYDPVXL9f7Va8FAmHyt4IACgkQDPVXL9f7
 Va8SVgf+N6uOYoCVooWFk5H6MpXPRGLNw7YB1L5bR1T8DmXef+149p3NaqYuD3Kr
 Fz+6Kc1xpRivtfz7psy6ME+VVLQaCL5pYDYvoWBW/EPDJaBIGe6oednY4N0KEPZH
 4AqyuLwNZ+xPhv78YMnjmvF6foIiuO712yAwlG7oi+E+wp25SlKi6DaWHlY2U4/F
 hRC4CS3fJKUONn+s511EdSYhjZVqC0+Dp5pyAPGShyp7AMFQUV7mKl+lC2E3/gWo
 Qudq/VqX/k4dlLt1zudh/qQTUxJA+nddM1a7grvXmgZL0nngdvKyXQHTOv60HYgF
 yFC5OkHu9XohxYaY8gQYNreYiqCxeQ==
 =p6C5
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20220127' into staging

Xen patches

- bug fixes for mapcache and ioreq handling

# gpg: Signature made Thu 27 Jan 2022 15:17:22 GMT
# gpg:                using RSA key F80C006308E22CFD8A92E7980CF5572FD7FB55AF
# gpg: Good signature from "Anthony PERARD <anthony.perard@gmail.com>" [unknown]
# gpg:                 aka "Anthony PERARD <anthony.perard@citrix.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 5379 2F71 024C 600F 778A  7161 D8D5 7199 DF83 42C8
#      Subkey fingerprint: F80C 0063 08E2 2CFD 8A92  E798 0CF5 572F D7FB 55AF

* remotes/aperard/tags/pull-xen-20220127:
  xen-mapcache: Avoid entry->lock overflow
  xen-hvm: Allow disabling buffer_io_timer

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-28 11:05:29 +00:00
Peter Maydell
cfe63e46be QAPI patches patches for 2022-01-27
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmHyqgcSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZT3A0P/3FpDflF8zwLlKEuTaLdprUCowEjouTq
 8lIKE1La0NK+cvuH0lDMSVLitqdk57WzopLvoSU7sSUXtvb9oJuJNnBPiCXVU8Ou
 uapzUnYxSM0KF3ehRtH+eOefyQ9MWpdzfzImmR6DDFjf1Ff03g4dNW9qrNUbMl9D
 Oy0ZUMsTms3OtXqkKeg9giAbDOARYgHcHcQEkm41xp28ORMUJvORiP6HV7CTWF4Q
 M2fLNT2cOtKZcby+YBU1HUYARFZuAII5718OD4YDWcv+ADMpwp0yCl7BRtLHDDNu
 cSm+6KNF+r8SfvwyLVHIvA1W5jP32lzJ/VpgK+2N5mWw8wKAgGzvIZHL8i5rE+b+
 NHXv3bFburrBAM/nzHntbFxzLCl5ysgWvQb32Uc/sDd6KKOwHqf1NqoBsn1POn3D
 AeQCBaXGlgBpKmeHeU99GnmgV5UKputHahAQqTzslSy8OKkEK9P0GTyEwl9KUmgJ
 g8rY2fhsSP8j+dwSRIhHE6B3WDVJ+bxUMUg0Io3BZjAP7SQosoUUYZYhOubTn4B4
 6usz2+QVoJePokC7GFl+LdyO5NOIfxvuRkCH9nw3q5eU5WYskvrh/vBvQ8H9UAvP
 NBeFHdKwTm82ZGTKz18w2/9imyy5NaAZGte0S3isDOLdGdUqmDxUC0hsHuibrSq2
 hjsQSYrsGS5C
 =gEIj
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2022-01-27-v2' into staging

QAPI patches patches for 2022-01-27

# gpg: Signature made Thu 27 Jan 2022 14:19:51 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2022-01-27-v2:
  qapi: generate trace events by default
  meson: document why we don't generate trace events for tests/ and qga/
  docs/qapi-code-gen: update to cover trace events code generation
  meson: generate trace events for qmp commands
  qapi/commands: Optionally generate trace for QMP commands
  qapi/commands: refactor error handling code
  qapi/gen: Add FOO.trace-events output module
  schemas: add missing vim modeline

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-27 15:45:13 +00:00
Ross Lagerwall
a021a2dd8b xen-mapcache: Avoid entry->lock overflow
In some cases, a particular mapcache entry may be mapped 256 times
causing the lock field to wrap to 0. For example, this may happen when
using emulated NVME and the guest submits a large scatter-gather write.
At this point, the entry map be remapped causing QEMU to write the wrong
data or crash (since remap is not atomic).

Avoid this overflow by increasing the lock field to a uint32_t and also
detect it and abort rather than continuing regardless.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Message-Id: <20220124104450.152481-1-ross.lagerwall@citrix.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
2022-01-27 15:14:21 +00:00
Jason Andryuk
9288e803e6 xen-hvm: Allow disabling buffer_io_timer
commit f37f29d314 "xen: slightly simplify bufioreq handling" hard
coded setting req.count = 1 during initial field setup before the main
loop.  This missed a subtlety that an early exit from the loop when
there are no ioreqs to process, would have req.count == 0 for the return
value.  handle_buffered_io() would then remove state->buffered_io_timer.
Instead handle_buffered_iopage() is basically always returning true and
handle_buffered_io() always re-setting the timer.

Restore the disabling of the timer by introducing a new handled_ioreq
boolean and use as the return value.  The named variable will more
clearly show the intent of the code.

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Message-Id: <20211210193434.75566-1-jandryuk@gmail.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
2022-01-27 15:14:11 +00:00
Vladimir Sementsov-Ogievskiy
761a1a488e qapi: generate trace events by default
We don't generate trace events for tests/ and qga/ because that it is
not simple and not necessary. We have corresponding comments in both
tests/meson.build and qga/meson.build.

Still to not miss possible future qapi code generation call, and not to
forget to enable trace events generation, let's enable it by default.
So, turn option --gen-trace into opposite --no-trace-events and use new
option only in tests/ and qga/ where we already have good comments why
we don't generate trace events code.

Note that this commit enables trace-events generation for qapi-gen.py
call from tests/qapi-schema/meson.build and storage-daemon/meson.build.
Still, both are kind of noop: tests/qapi-schema/ doesn't seem to
generate any QMP command code and no .trace-events files anyway,
storage-daemon/ uses common QMP command implementations and just
generate empty .trace-events

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-8-vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 15:17:35 +01:00
Vladimir Sementsov-Ogievskiy
378dfa482d meson: document why we don't generate trace events for tests/ and qga/
Making trace generation work for tests/ and qga/ would involve some
Meson hackery to ensure we generate the trace-events files before
trace-tool uses them. Since we don't actually support tracing there
anyway, we bypass that problem.

Let's add corresponding comments.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220126161130.3240892-7-vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Pasto fixed, commit message punctuation tidied up]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 15:17:35 +01:00
Vladimir Sementsov-Ogievskiy
ff8e4827ad docs/qapi-code-gen: update to cover trace events code generation
Previous commits enabled trace events generation for most of QAPI
generated code (except for tests/ and qga/). Let's update documentation
to illustrate it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220126161130.3240892-6-vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 15:16:28 +01:00
Vladimir Sementsov-Ogievskiy
b83a80e831 meson: generate trace events for qmp commands
1. Use --gen-trace when generate qmp commands
2. Add corresponding .trace-events files as outputs in qapi_files
   custom target
3. Define global qapi_trace_events list of .trace-events file targets,
   to fill in trace/qapi.build and to use in trace/meson.build
4. In trace/meson.build use the new array as an additional source of
   .trace_events files to be processed

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-5-vsementsov@virtuozzo.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:26:40 +01:00
Vladimir Sementsov-Ogievskiy
bd2017bc41 qapi/commands: Optionally generate trace for QMP commands
Add trace generation disabled by default and new option --gen-trace to
enable it.  The next commit will enable it for qapi/, but not for qga/
and tests/.  Making it work for the latter two would involve some Meson
hackery to ensure we generate the trace-events files before trace-tool
uses them.  Since we don't actually support tracing there, we'll bypass
that problem.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-4-vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Superfluous #include dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:25:52 +01:00
Vladimir Sementsov-Ogievskiy
167d913f34 qapi/commands: refactor error handling code
Move error_propagate() to if (err) and make "if (err)" block mandatory.
This is to simplify further commit, which will bring trace events
generation for QMP commands.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-3-vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:25:09 +01:00
Vladimir Sementsov-Ogievskiy
4e86df1732 qapi/gen: Add FOO.trace-events output module
We are going to generate trace events for QMP commands. We should
generate both trace_*() function calls and trace-events files listing
events for trace generator.

So, add an output module FOO.trace-events for each FOO schema module.

Since we're going to add trace events only to command marshallers,
make the trace-events output optional, so we don't generate so many
useless empty files.

Currently nobody set add_trace_events to True, so new functionality is
disabled. It will be enabled for QAPISchemaGenCommandVisitor
in a further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-2-vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:24:18 +01:00
Victor Toso
7e7237cd2b schemas: add missing vim modeline
Similar to f7160f3218 "schemas: Add vim modeline"

Signed-off-by: Victor Toso <victortoso@redhat.com>
Message-Id: <20211220145624.52801-1-victortoso@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:20:50 +01:00
Peter Maydell
48302d4eb6 virtiofsd: Security fix
Fixes: CVE-2022-0358
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAmHxJpQACgkQBRYzHrxb
 /ecd/xAAl4o5rqmLHIpuzUbCbfYliFkBqn9X1zsZc3VJ1T3e3Vxq36h4kUQGY63g
 4KPRghjjO3v3+YU9a3Tmlp704TaalNWhOMCTWnFT28Pet2u6492zDjGUz3h5scms
 18NZmyjmFHREumeSFM5s7rgouhhsa8dv8pqrkvIezBAmp6BQDhZThkd4dlx1OTAq
 zdiOwCmEY/1z55bPFBeU6Tz1BMGgWP+0kqoY2YLFms+kRgWrmacsNsUfdAXSuLr1
 jUosr3f96inwts9dlTVR0jdMs1/Dh0eLSuasWa1VgQmak4vYdx3ytiqjf1eaLyjc
 r41GQOvqyqRy+BqzjXLHkMOXdwMKGTtj9pNBnnKMnEcUA4Kja0d4i37Gf9T3tWT7
 WVcA1It3kdYuXPrxY1++toasCv4BId8dofJWnjgULf8gsEgKdLm/Aee9EiUYsBHv
 OPdX/iQB1B3eyc4g6vZHwnQNx7r2+yvh6eSho9NiRD57u8FZwuHWXudVTrZ9QFwZ
 AQ7p6b0qf9Un5n7fQ48t2hJKmHLKGZmiLp4zVCqSHP0DF5wImepwgOIHaqxaToN/
 iu/Y6tc9UZarNFKA3rSKSlRrerbRVPEL36+1KyYZHOq3685q+xIsK64iVD6Z0FeX
 9deb0lOCYQkG/fhetv1HTsCLlqAUK+M4Vk4g+4pd176gjOUBRhw=
 =7mdJ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/dgilbert-gitlab/tags/pull-virtiofs-20220126' into staging

virtiofsd: Security fix

Fixes: CVE-2022-0358

# gpg: Signature made Wed 26 Jan 2022 10:46:44 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert-gitlab/tags/pull-virtiofs-20220126:
  virtiofsd: Drop membership of all supplementary groups (CVE-2022-0358)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-26 10:59:50 +00:00
Vivek Goyal
449e8171f9 virtiofsd: Drop membership of all supplementary groups (CVE-2022-0358)
At the start, drop membership of all supplementary groups. This is
not required.

If we have membership of "root" supplementary group and when we switch
uid/gid using setresuid/setsgid, we still retain membership of existing
supplemntary groups. And that can allow some operations which are not
normally allowed.

For example, if root in guest creates a dir as follows.

$ mkdir -m 03777 test_dir

This sets SGID on dir as well as allows unprivileged users to write into
this dir.

And now as unprivileged user open file as follows.

$ su test
$ fd = open("test_dir/priviledge_id", O_RDWR|O_CREAT|O_EXCL, 02755);

This will create SGID set executable in test_dir/.

And that's a problem because now an unpriviliged user can execute it,
get egid=0 and get access to resources owned by "root" group. This is
privilege escalation.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2044863
Fixes: CVE-2022-0358
Reported-by: JIETAO XIAO <shawtao1125@gmail.com>
Suggested-by: Miklos Szeredi <mszeredi@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Message-Id: <YfBGoriS38eBQrAb@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Fixed missing {}'s style nit
2022-01-26 10:32:05 +00:00
Peter Maydell
aeb0ae95b7 Python patches
A few fixes to the Python CI tests, a few fixes to the (async) QMP
 library, and a set of patches that begin to shift us towards using the
 new qmp lib.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmHrSt4ACgkQfe+BBqr8
 OQ4BLg/5AdhcWsAtKq+xZj/nz2DNAbvXmrGJRuVln1yofrj22w2MYUTGfpQ0m8JO
 Ezu+LYtSTPQAaQw54WByaliD5h2ucYl5W8H13cDc8NPZkbsX+dD7G99u4XkSIY4I
 sSCYDMKi4j/P+4YR2MN1Iol4362fWbi3O3rsRX6PqNymIAaaklDrH3QZCGMiBMjl
 2OAcgba31uguyXnMuM3WY8XAnnRsib3wZ/a+a3WWcEsEm1HAUC0pb8VmrRzH5Rv9
 CpR3EBYoVc3of96jd/qLjucnoUL0+K2RVN2qPeie3+o7yEM2VYj5o+cG2H8pEG5p
 Fk/J7kqs0XWBOeX3A3IlGqKEXFPGDjEJZpcjwd8+IhgA2Y/MByTqRr1EvrOSO+bg
 q3njEg5DsORQS/xgZrnAidk5fdgLj7Cv39LfsxMnv77RBnlLubEAet7pT1XtprAv
 DI7STKknVpPu0VtYI8ALVjVhpeCkIt95DXACMtPZiSJ5X1NdoY5qubV1y8/vsExI
 RMDMepcS2A75Un2DA1bkStHTPN2PSUfM15fmUCebxbHp53FlJCh44gxAAsfj9j41
 xUmwSz1c81bCU4m+jsMBdNrbtkpPz/gX/3ZS8KqGoZmWN0wDkh3vEYFj5Y/310HY
 xmzug6o+tR7OD3bBGxZ73k9rn86X3+1PsYxOZjvYM0wiJIisPk4=
 =D2kj
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python patches

A few fixes to the Python CI tests, a few fixes to the (async) QMP
library, and a set of patches that begin to shift us towards using the
new qmp lib.

# gpg: Signature made Sat 22 Jan 2022 00:07:58 GMT
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jsnow-gitlab/tags/python-pull-request:
  scripts/render-block-graph: switch to AQMP
  scripts/cpu-x86-uarch-abi: switch to AQMP
  scripts/cpu-x86-uarch-abi: fix CLI parsing
  python: move qmp-shell under the AQMP package
  python: move qmp utilities to python/qemu/utils
  python/qmp: switch qmp-shell to AQMP
  python/qmp: switch qom tools to AQMP
  python/qmp: switch qemu-ga-client to AQMP
  python/qemu-ga-client: don't use deprecated CLI syntax in usage comment
  python/aqmp: rename AQMPError to QMPError
  python/aqmp: add SocketAddrT to package root
  python/aqmp: copy type definitions from qmp
  python/aqmp: handle asyncio.TimeoutError on execute()
  python/aqmp: add __del__ method to legacy interface
  python/aqmp: fix docstring typo
  python: use avocado's "new" runner
  python: pin setuptools below v60.0.0

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-22 12:03:22 +00:00
John Snow
0590860242 scripts/render-block-graph: switch to AQMP
Creating an instance of qemu.aqmp.ExecuteError is too involved here, so
just drop the specificity down to a generic QMPError.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
0665410dcf scripts/cpu-x86-uarch-abi: switch to AQMP
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
9922125615 scripts/cpu-x86-uarch-abi: fix CLI parsing
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2022-01-21 16:01:31 -05:00
John Snow
fd9c3a6219 python: move qmp-shell under the AQMP package
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
0347c4c4cf python: move qmp utilities to python/qemu/utils
In order to upload a QMP package to PyPI, I want to remove any scripts
that I am not 100% confident I want to support upstream, beyond our
castle walls.

Move most of our QMP utilities into the utils package so we can split
them out from the PyPI upload.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
f3efd12930 python/qmp: switch qmp-shell to AQMP
We have a replacement for async QMP, but it doesn't have feature parity
yet. For now, then, port the old tool onto the new backend.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2022-01-21 16:01:31 -05:00
John Snow
8d6cdc5118 python/qmp: switch qom tools to AQMP
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
26db07516f python/qmp: switch qemu-ga-client to AQMP
Async QMP always raises a "ConnectError" on any connection error which
houses the cause in a second exception. We can check if this root cause
was python's ConnectionError to determine a fairly similar condition to
the original error check here.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
7017f3853a python/qemu-ga-client: don't use deprecated CLI syntax in usage comment
Cleanup related to commit ccd3b3b811, "qemu-option: warn for
short-form boolean options".

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
6e7751dc38 python/aqmp: rename AQMPError to QMPError
This is in preparation for renaming qemu.aqmp to qemu.qmp. I should have
done this from this from the very beginning, but it's a convenient time
to make sure this churn is taken care of.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2022-01-21 16:01:31 -05:00
John Snow
728dcac5e3 python/aqmp: add SocketAddrT to package root
It's a commonly needed definition, it can be re-exported by the root.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
0e6bfd8b96 python/aqmp: copy type definitions from qmp
Copy the remaining type definitions from QMP into the qemu.aqmp.legacy
module. Now, users that require the legacy interface don't need to
import anything else but qemu.aqmp.legacy wrapper.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
3b5bf136f5 python/aqmp: handle asyncio.TimeoutError on execute()
This exception can be injected into any await statement. If we are
canceled via timeout, we want to clear the pending execution record on
our way out.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2022-01-21 16:01:31 -05:00
John Snow
3bc72e3aed python/aqmp: add __del__ method to legacy interface
asyncio can complain *very* loudly if you forget to back out of things
gracefully before the garbage collector starts destroying objects that
contain live references to asyncio Tasks.

The usual fix is just to remember to call aqmp.disconnect(), but for the
sake of the legacy wrapper and quick, one-off scripts where a graceful
shutdown is not necessarily of paramount imporance, add a courtesy
cleanup that will trigger prior to seeing screenfuls of confusing
asyncio tracebacks.

Note that we can't *always* save you from yourself; depending on when
the GC runs, you might just seriously be out of luck. The best we can do
in this case is to gently remind you to clean up after yourself.

(Still much better than multiple pages of incomprehensible python
warnings for the crime of forgetting to put your toys away.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
dc6877bd2e python/aqmp: fix docstring typo
Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
57a6b4478c python: use avocado's "new" runner
The old legacy runner no longer seems to work with output logging, so we
can't see failure logs when a test case fails. The new runner doesn't
(seem to) support Coverage.py yet, but seeing error output is a more
important feature.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Message-id: 20220119193916.4138217-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-01-21 16:01:13 -05:00
John Snow
1e4d8b31be python: pin setuptools below v60.0.0
setuptools is a package that replaces the python stdlib 'distutils'. It
is generally installed by all venv-creating tools "by default". It isn't
actually needed at runtime for the qemu package, so our own setup.cfg
does not mention it as a dependency.

However, tox will create virtual environments that include it, and will
upgrade it to the very latest version. the 'venv' tool will also include
whichever version your host system happens to have.

Unfortunately, setuptools version 60.0.0 and above include a hack to
forcibly overwrite python's built-in distutils. The pylint tool that we
use to run code analysis checks on this package relies on distutils and
suffers regressions when setuptools >= 60.0.0 is present at all, see
https://github.com/PyCQA/pylint/issues/5704

Instruct tox and the 'check-dev' targets to avoid setuptools packages
that are too new, for now. Pipenv is unaffected, because setuptools 60
does not offer Python 3.6 support, and our pipenv config is pinned
against Python 3.6.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20220121005221.142236-1-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-01-21 16:01:09 -05:00
Peter Maydell
5e9d14f2be Third RISC-V PR for QEMU 7.0
* Fixes for OpenTitan timer
  * Correction of OpenTitan PLIC stride length
  * RISC-V KVM support
  * Device tree code cleanup
  * Support for the Zve64f and Zve32f extensions
  * OpenSBI binary loading support for the Spike machine
  * Removal of OpenSBI ELFs
  * Support for the UXL field in xstatus
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEE9sSsRtSTSGjTuM6PIeENKd+XcFQFAmHqSzUACgkQIeENKd+X
 cFRN+Qf+LZPtuu7Axn0g4S1cVil7XEmSCOGe12FUIklYJTJUMf6GIni5/DyRCET5
 PUBbOh3BKGsswaPH8qFlDr5gxX59hucffxaLnuag4/MogNJTB6EAF+sc9FdllIkV
 Tn4X6FAa3z5Q/PMyG6umjlXlKhtIM9Tt0sCVACmi105l8MPWYUzkFeXKpT6qIXU0
 vqUqeyG1IoMWU54a8IBJsuA+Rioxa4Y1j30Uqy2uZG22dkAhxNSBwb+N18JwTUzC
 WCS7LluWwAKgDfHTKPSMZh2CCGHszA4cL1W6mJOkXowzUTJzz9G+Wxm+aq8hzku4
 ONeU0z2GwMtZtf4YI2ahorlyv/gktg==
 =JmZZ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20220121-1' into staging

Third RISC-V PR for QEMU 7.0

 * Fixes for OpenTitan timer
 * Correction of OpenTitan PLIC stride length
 * RISC-V KVM support
 * Device tree code cleanup
 * Support for the Zve64f and Zve32f extensions
 * OpenSBI binary loading support for the Spike machine
 * Removal of OpenSBI ELFs
 * Support for the UXL field in xstatus

# gpg: Signature made Fri 21 Jan 2022 05:57:09 GMT
# gpg:                using RSA key F6C4AC46D4934868D3B8CE8F21E10D29DF977054
# gpg: Good signature from "Alistair Francis <alistair@alistair23.me>" [full]
# Primary key fingerprint: F6C4 AC46 D493 4868 D3B8  CE8F 21E1 0D29 DF97 7054

* remotes/alistair/tags/pull-riscv-to-apply-20220121-1: (61 commits)
  target/riscv: Relax UXL field for debugging
  target/riscv: Enable uxl field write
  target/riscv: Set default XLEN for hypervisor
  target/riscv: Adjust scalar reg in vector with XLEN
  target/riscv: Adjust vector address with mask
  target/riscv: Fix check range for first fault only
  target/riscv: Remove VILL field in VTYPE
  target/riscv: Adjust vsetvl according to XLEN
  target/riscv: Split out the vill from vtype
  target/riscv: Split pm_enabled into mask and base
  target/riscv: Calculate address according to XLEN
  target/riscv: Alloc tcg global for cur_pm[mask|base]
  target/riscv: Create current pm fields in env
  target/riscv: Adjust csr write mask with XLEN
  target/riscv: Relax debug check for pm write
  target/riscv: Use gdb xml according to max mxlen
  target/riscv: Extend pc for runtime pc write
  target/riscv: Ignore the pc bits above XLEN
  target/riscv: Create xl field in env
  target/riscv: Sign extend pc for different XLEN
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-21 10:31:25 +00:00
LIU Zhiwei
f297245f6a target/riscv: Relax UXL field for debugging
Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220120122050.41546-24-zhiwei_liu@c-sky.com
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
2022-01-21 15:52:57 +10:00
LIU Zhiwei
f310df58bd target/riscv: Enable uxl field write
Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220120122050.41546-23-zhiwei_liu@c-sky.com
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
2022-01-21 15:52:57 +10:00
LIU Zhiwei
5a2ae2350e target/riscv: Set default XLEN for hypervisor
When swap regs for hypervisor, the value of vsstatus or mstatus_hs
should have the right XLEN. Otherwise, it will propagate to mstatus.

Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220120122050.41546-22-zhiwei_liu@c-sky.com
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
2022-01-21 15:52:57 +10:00
LIU Zhiwei
d8c40c24fd target/riscv: Adjust scalar reg in vector with XLEN
When sew <= 32bits, not need to extend scalar reg.
When sew > 32bits, if xlen is less that sew, we should sign extend
the scalar register, except explicitly specified by the spec.

Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220120122050.41546-21-zhiwei_liu@c-sky.com
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
2022-01-21 15:52:57 +10:00
LIU Zhiwei
d6b9d93023 target/riscv: Adjust vector address with mask
The mask comes from the pointer masking extension, or the max value
corresponding to XLEN bits.

Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220120122050.41546-20-zhiwei_liu@c-sky.com
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
2022-01-21 15:52:57 +10:00
LIU Zhiwei
01d09525da target/riscv: Fix check range for first fault only
Only check the range that has passed the address translation.

Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220120122050.41546-19-zhiwei_liu@c-sky.com
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
2022-01-21 15:52:57 +10:00