mirrors/qemu
0
0
Fork 0
mirror of https://github.com/qemu/qemu.git synced 2024-11-23 10:53:37 +08:00
Code Issues Packages Projects Releases Wiki Activity

tests: Use separate virtual environment for avocado

Browse Source 
This reverts commits eea2d14117 ("Makefile: remove $(TESTS_PYTHON)",
2023-05-26) and 9c6692db55 ("tests: Use configure-provided pyvenv for
tests", 2023-05-18).

Right now, there is a conflict between wanting a ">=" constraint when
using a distro-provided package and wanting a "==" constraint when
installing Avocado from PyPI; this would provide the best of both worlds
in terms of resiliency for both distros that have required packages and
distros that don't.

The conflict is visible also for meson, where we would like to install
the latest 0.63.x version but also accept a distro 1.1.x version.
But it is worse for avocado, for two reasons:

1) we cannot use an "==" constraint to install avocado if the venv
includes a system avocado.  The distro will package plugins that have
"==" constraints on the version that is included in the distro, and, using
"pip install avocado==88.1" on a venv that includes system packages will
result in this error:

   ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
   avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
   avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
   make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'

2) we cannot use ">=" either if the venv does _not_ include a system
avocado, because that would result in the installation of v101.0 which
is the one we've just reverted.

So the idea is to encode the dependencies as an (acceptable, locked)
tuple, like this hypothetical TOML that would be committed inside
python/ and used by mkvenv.py:

  [meson]
  meson = { minimum = "0.63.0", install = "0.63.3", canary = "meson" }

  [docs]
  # 6.0 drops support for Python 3.7
  sphinx = { minimum = "1.6", install = "<6.0", canary = "sphinx-build" }
  sphinx_rtd_theme = { minimum = "0.5" }

  [avocado]
  avocado-framework = { minimum = "88.1", install = "88.1", canary = "avocado" }

Once this is implemented, it would also be possible to install avocado in
pyvenv/ using "mkvenv.py ensure", thus using the distro package on Fedora
and CentOS Stream (the only distros where it's available).  But until
this is implemented, keep avocado in a separate venv.  There is still the
benefit of using a single python for meson custom_targets and for sphinx.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This commit is contained in:
Paolo Bonzini 2023-06-05 17:30:38 +02:00
parent eaf245becd
commit e8e4298fea
8 changed files with 28 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.gitlab-ci.d/buildtest.yml
View File

@ -103,7 +103,7 @@ crash-test-debian:
script:
- cd build
- make NINJA=":" check-venv
- pyvenv/bin/python3 scripts/device-crash-test -q --tcg-only ./qemu-system-i386
- tests/venv/bin/python3 scripts/device-crash-test -q --tcg-only ./qemu-system-i386
build-system-fedora:
extends:
@ -146,8 +146,8 @@ crash-test-fedora:
script:
- cd build
- make NINJA=":" check-venv
- pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc
- pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32
- tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc
- tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32
build-system-centos:
extends:

6
docs/devel/acpi-bits.rst
View File

@ -61,19 +61,19 @@ Under ``tests/avocado/`` as the root we have:
::
$ make check-venv (needed only the first time to create the venv)
$ ./pyvenv/bin/avocado run -t acpi tests/avocado
$ ./tests/venv/bin/avocado run -t acpi tests/avocado
The above will run all acpi avocado tests including this one.
In order to run the individual tests, perform the following:
::
$ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py --tap -
$ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py --tap -
The above will produce output in tap format. You can omit "--tap -" in the
end and it will produce output like the following:
::
$ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py
$ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py
Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
JOB ID : eab225724da7b64c012c65705dc2fa14ab1defef
JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log

14
docs/devel/testing.rst
View File

@ -888,9 +888,9 @@ You can run the avocado tests simply by executing:
make check-avocado
This involves the automatic installation, from PyPI, of all the
necessary avocado-framework dependencies into the QEMU venv within the
build tree (at ``./pyvenv``). Test results are also saved within the
This involves the automatic creation of Python virtual environment
within the build tree (at ``tests/venv``) which will have all the
right dependencies, and will save tests results also within the
build tree (at ``tests/results``).
Note: the build environment must be using a Python 3 stack, and have
@ -947,7 +947,7 @@ may be invoked by running:
.. code::
pyvenv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/
tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/
Note that if ``make check-avocado`` was not executed before, it is
possible to create the Python virtual environment with the dependencies
@ -962,20 +962,20 @@ a test file. To run tests from a single file within the build tree, use:
.. code::
pyvenv/bin/avocado run tests/avocado/$TESTFILE
tests/venv/bin/avocado run tests/avocado/$TESTFILE
To run a single test within a test file, use:
.. code::
pyvenv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME
tests/venv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME
Valid test names are visible in the output from any previous execution
of Avocado or ``make check-avocado``, and can also be queried using:
.. code::
pyvenv/bin/avocado list tests/avocado
tests/venv/bin/avocado list tests/avocado
Manual Installation
~~~~~~~~~~~~~~~~~~~

4
scripts/ci/org.centos/stream/8/x86_64/test-avocado
View File

@ -4,7 +4,7 @@
# KVM and x86_64, or tests that are generic enough to be valid for all
# targets. Such a test list can be generated with:
#
# ./pyvenv/bin/avocado list --filter-by-tags-include-empty \
# ./tests/venv/bin/avocado list --filter-by-tags-include-empty \
# --filter-by-tags-include-empty-key -t accel:kvm,arch:x86_64 \
# tests/avocado/
#
@ -22,7 +22,7 @@
# - tests/avocado/virtio_check_params.py:VirtioMaxSegSettingsCheck.test_machine_types
#
make get-vm-images
./pyvenv/bin/avocado run \
./tests/venv/bin/avocado run \
--job-results-dir=tests/results/ \
tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_i440fx_kvm \
tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_q35_kvm \

2
scripts/device-crash-test
View File

@ -43,7 +43,7 @@ except ModuleNotFoundError as exc:
print(f"Module '{exc.name}' not found.")
print(" Try 'make check-venv' from your build directory,")
print(" and then one way to run this script is like so:")
print(f' > $builddir/pyvenv/bin/python3 "{path}"')
print(f' > $builddir/tests/venv/bin/python3 "{path}"')
sys.exit(1)
logger = logging.getLogger('device-crash-test')

16
tests/Makefile.include
View File

@ -89,9 +89,10 @@ distclean-tcg: $(DISTCLEAN_TCG_TARGET_RULES)
# Build up our target list from the filtered list of ninja targets
TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
TESTS_VENV_TOKEN=$(BUILD_DIR)/pyvenv/tests.group
TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
TESTS_PYTHON=$(TESTS_VENV_DIR)/bin/python3
ifndef AVOCADO_TESTS
AVOCADO_TESTS=tests/avocado
endif
@ -107,10 +108,11 @@ else
endif
quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
$(PYTHON) -m pip -q --disable-pip-version-check $1, \
$(TESTS_PYTHON) -m pip -q --disable-pip-version-check $1, \
"VENVPIP","$1")
$(TESTS_VENV_TOKEN): $(TESTS_VENV_REQ)
$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
$(call quiet-command, $(PYTHON) -m venv $@, VENV, $@)
$(call quiet-venv-pip,install -e "$(SRC_PATH)/python/")
$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
$(call quiet-command, touch $@)
@ -119,7 +121,7 @@ $(TESTS_RESULTS_DIR):
$(call quiet-command, mkdir -p $@, \
MKDIR, $@)
check-venv: $(TESTS_VENV_TOKEN)
check-venv: $(TESTS_VENV_DIR)
FEDORA_31_ARCHES_TARGETS=$(patsubst %-softmmu,%, $(filter %-softmmu,$(TARGETS)))
FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(FEDORA_31_ARCHES_TARGETS))
@ -129,7 +131,7 @@ FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES))
# download one specific Fedora 31 image
get-vm-image-fedora-31-%: check-venv
$(call quiet-command, \
$(PYTHON) -m avocado vmimage get \
$(TESTS_PYTHON) -m avocado vmimage get \
--distro=fedora --distro-version=31 --arch=$*, \
"AVOCADO", "Downloading avocado tests VM image for $*")
@ -138,7 +140,7 @@ get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOW
check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
$(call quiet-command, \
$(PYTHON) -m avocado \
$(TESTS_PYTHON) -m avocado \
--show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
$(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
--filter-by-tags-include-empty-key) \
@ -161,7 +163,7 @@ check:
check-build: run-ninja
check-clean:
rm -rf $(TESTS_RESULTS_DIR)
rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR)
clean: check-clean clean-tcg
distclean: distclean-tcg

7
tests/requirements.txt
View File

@ -1,9 +1,6 @@
# Add Python module requirements, one per line, to be installed
# in the qemu build_dir/pyvenv Python virtual environment. For more info,
# in the tests/venv Python virtual environment. For more info,
# refer to: https://pip.pypa.io/en/stable/user_guide/#id1
#
# Note that qemu.git/python/ is implicitly installed to this venv when
# 'make check-venv' is run, and will persist until configure is run
# again.
# Note that qemu.git/python/ is always implicitly installed.
avocado-framework==88.1
pycdlib==1.11.0

2
tests/vm/Makefile.include
View File

@ -5,7 +5,7 @@ ifeq ($(realpath $(SRC_PATH)),$(realpath .))
VM_PYTHON = PYTHONPATH=$(SRC_PATH)/python /usr/bin/env python3
VM_VENV =
else
VM_PYTHON = $(PYTHON)
VM_PYTHON = $(TESTS_PYTHON)
VM_VENV = check-venv
endif