With the addition of multiple callback-flood kthreads, the maximum number
of callbacks from any one of those kthreads is reported in the rcutorture
run summary. This commit changes this to report the sum of each kthread's
maximum number of callbacks in a given callback-flooding episode.
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit modifies the TASKS01 scenario to use four callback queues
and the TRACE01 scenario to use two queues, thus providing testing of
multiple queues by default.
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The echo commands following initialization of the "oldrun" variable need
to be "tee"d to $oldrun/remote-log. This commit fixes several stragglers.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The "exit 4" in kvm-remote.sh is pointlessly redirected, so this commit
removes the redirection.
Fixes: 0092eae4cb ("torture: Add kvm-remote.sh script for distributed rcutorture test runs")
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adjusts the TRACE02 scenario to use a pair of callback-flood
kthreads. This in turn forces lock contention on the single RCU Tasks
Trace callback queue, which forces use of all CPUs' queues, thus testing
this transition. (No, there is not yet any way to transition back.
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit enables two callback-flood kthreads for the TREE02 scenario
and 28 for the TREE10 scenario.
Cc: Neeraj Upadhyay <neeraj.iitr10@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, a transient network error can kill a run if it happens while
downloading the tarball to one of the target systems. This commit
therefore does a 60-second wait and then a retry. If further experience
indicates, a more elaborate mechanism might be used later.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit makes kvm-find-errors.sh check for and report undefined
symbols that are detected at link time.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit brings the kvm.sh script's help text up to date with recently
(and some not-so-recently) added parameters.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
All of the uses of CONFIG_RCU_FAST_NO_HZ=y that I have seen involve
systems with RCU callbacks offloaded. In this situation, all that this
Kconfig option does is slow down idle entry/exit with an additional
allways-taken early exit. If this is the only use case, then this
Kconfig option nothing but an attractive nuisance that needs to go away.
This commit therefore removes the RCU_FAST_NO_HZ Kconfig option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
All of the rcu scenarios that mentioning CONFIG_RCU_FAST_NO_HZ disable it.
But this Kconfig option is disabled by default, so this commit removes
the pointless "CONFIG_RCU_FAST_NO_HZ=n" lines from these scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
All of the rcuscale and refscale scenarios that mention the Kconfig option
CONFIG_RCU_FAST_NO_HZ disable it. But this Kconfig option is disabled by
default, so this commit removes the pointless "CONFIG_RCU_FAST_NO_HZ=n"
lines from these scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
With CONFIG_PREEMPT_DYNAMIC=y, the kernel builds with CONFIG_PREEMPTION=y
because preemption can be enabled at runtime. This prevents any tests
of Tiny RCU or Tiny SRCU from running correctly. This commit therefore
explicitly sets CONFIG_PREEMPT_DYNAMIC=n for those scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes kvm-remote.sh to print the size of the tarball that
is downloaded to each of the remote systems. This size can help with
performance projections and analysis.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
By default, torture.sh allots 512M of memory for each guest OS. However,
when running scftorture with KASAN, 1G is needed. This commit therefore
causes torture.sh to provide the required 1G.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Compressing gigabyte vmlinux files can take some time, and it can be a
bit annoying to not know many more batches of compression there will be.
This commit therefore makes torture.sh print the number of files to be
compressed just before starting compression and just after compression
completes.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the --kcsan argument to kvm.sh applies a laundry list of
Kconfig options. Now that KCSAN provides the CONFIG_KCSAN_STRICT Kconfig
option, this commit reduces the laundry list to this one option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
It turns out that certain types of early boot bugs can result in reboot
loops, even within a guest OS running under qemu/KVM. This commit
therefore upgrades the kvm-test-1-run-qemu.sh script's hang-detection
heuristics to detect such situations and to terminate the run when
they occur.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-test-1-run-qemu.sh script logs the torture-test start time and
also when it starts getting impatient for the test to finish. However, it
does not timestamp these log messages, which can make debugging needlessly
challenging. This commit therefore adds timestamps to these messages.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
There was a time long ago when the "test" command's documentation
claimed that the "-a" and "-o" arguments did something useful.
But this documentation now suggests letting the shell execute
these boolean operators, so this commit applies that suggestion to
kvm-test-1-run-qemu.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes kvm-test-1-run-batch.sh to use the new
kvm-assign-cpus.sh and kvm-get-cpus-script.sh scripts to create a
TORTURE_AFFINITY environment variable containing either an empty string
(for no affinity) or a list of CPUs to pin the scenario's vCPUs to.
The additional change to kvm-test-1-run.sh places the per-scenario
number-of-CPUs information where it can easily be found.
If there is some reason why affinity cannot be supplied, this commit
prints and logs the reason via changes to kvm-again.sh.
Finally, this commit updates the kvm-remote.sh script to copy the
qemu-affinity output files back to the host system.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
There is "qemu-affinity", "qemu-cmd", "qemu-retval", but also "qemu_pid".
This is hard to remember, not so good for bash tab completion, and just
plain inconsistent. This commit therefore renames the "qemu_pid" file to
"qemu-pid". A couple of the scripts must deal with old runs, and thus
must handle both "qemu_pid" and "qemu-pid", but new runs will produce
"qemu-pid".
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The jitter.sh script has some entertaining awk code to generate a
hex mask from a randomly selected CPU number, which is handed to the
"taskset" command. Except that this command has a "-c" parameter to
take a comma/dash-separated list of CPU numbers. This commit therefore
saves a few lines of awk by switching to a single-number CPU list.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
There is no way to place the vCPUs in a two-CPU rcutorture scenario to
get variable memory latency. This commit therefore upgrades the current
two-CPU rcutorture scenarios to four CPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes the kvm-test-1-run-qemu.sh script to check the
TORTURE_AFFINITY environment variable and to add "taskset" commands to
the qemu-cmd file. The first "taskset" command is applied only if the
TORTURE_AFFINITY environment variable is a non-empty string, and this
command pins the current scenario's guest OS to the specified CPUs.
The second "taskset" command reports the guest OS's affinity in a new
"qemu-affinity" file.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-test-1-run-qemu.sh applies redirection to each and every
line of each qemu-cmd script. Only the first line (the only one that
is not a bash comment) needs to be redirected. Although redirecting
the comments is currently harmless, just adding to the comment, it is
an accident waiting to happen. This commit therefore adjusts the "sed"
command to redirect only the qemu-system* command itself.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes kvm.sh to use the new kvm-assign-cpus.sh and
kvm-get-cpus-script.sh scripts to create a TORTURE_AFFINITY environment
variable containing either an empty string (for no affinity) or a list
of CPUs to pin the scenario's vCPUs to. A later commit will make
use of this information to actually pin the vCPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit is a first step towards pinning guest-OS vCPUs so as
to force latency differences, especially on multi-socket systems.
The kvm.sh script puts its batch-creation awk script into a temporary
file so that later commits can add the awk code needed to dole out CPUs
so as to maximize latency differences. This awk code will be used by
multiple scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The last line of kvm-test-1-run-qemu.sh invokes parse-console.sh, but
kvm-test-1-run-qemu.sh is unaware of the PATH containing this script
and does not have the job title handy. This commit therefore moves
the invocation of parse-console.sh to kvm-test-1-run.sh, which has
PATH and title at hand. This commit does not add an invocation of
parse-console.sh to kvm-test-1-run-batch.sh because this latter script
is run in the background, and the information will be gathered at the
end of the full run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-recheck.sh attempts to create a kcsan.sum file even for
build-only runs. This results in false-positive bash errors due to
there being no console.log files in that case. This commit therefore
makes kvm-recheck.sh skip creating the kcsan.sum file for build-only runs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm-remote.sh script places the datestamped directory containing
all the build artifacts in the destination systems' /tmp directories,
where they accumulate runtime artifacts such as console.log. This works,
but some systems have a habit of removing files in /tmp that have not
been recently accessed. This commit therefore runs a simple script that
periodically accesses all files in the datestamped directory.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The qemu-cmd file can contain comments that are not relevant to the
operation of kvm-recheck-lock.sh. This commit therefore strips these
comments before looking for timing information.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The qemu-cmd file can contain comments that are not relevant to the
operation of kvm-recheck-scf.sh. This commit therefore strips these
comments before looking for timing information.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, each -kcsan run in a torture.sh group of runs has its own
kcsan.sum summary. This works, but there is usually a lot of duplication
between the runs. This commit therefore also creates an overall kcsan.sum
file for the entire torture.sh run, if there was at least one -kcsan run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kcsan-collapse.sh script assumes that it is being run over the output
of a single kvm.sh run, which is less than helpful for torture.sh runs.
This commit therefore changes the kcsan-collapse.sh script's "ls" pattern
with a "find" command to enable a KCSAN summary across all the -kcsan
runs in a full torture.sh run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, torture.sh accepts --doall on the one hand and --do-none
on the other, which is a bit inconsistent. This commit therefore adds
--do-all and --donone so that a fully consistent test may be used.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds three short tests of the clocksource-watchdog capability
to the torture.sh script, all to avoid otherwise-inevitable bitrot.
While in the area, fix an obsolete comment.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Fix ~12 single-word typos in RCU code comments.
[ paulmck: Apply feedback from Randy Dunlap. ]
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, if a torture scenario requires more CPUs than are present
on the build system, kvm.sh and friends limit the CPUs available to
that scenario. This makes total sense when the build system and the
system running the scenarios are one and the same, but not so much when
remote systems might well have more CPUs.
This commit therefore introduces a --remote flag to kvm.sh that suppresses
this CPU-limiting behavior, and causes kvm-remote.sh to use this flag.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In a long-duration kvm-remote.sh run, almost all of the remote accesses will
be simple file-existence checks. These are thus the most likely to be caught
out by network failures, which do happen from time to time.
This commit therefore takes a first step towards tolerating temporary
network outages by making the file-existence checks repeat in the face of
such an outage. They also print a message every minute during a outage,
allowing the user to take appropriate action.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the BUSTED-BOOST rcutorture scenario, which can be
used to test rcutorture's ability to test RCU priority boosting.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Some of the code invoked directly and indirectly from kvm.sh parses
the output of commands. This parsing assumes English, which can cause
failures if the user has set some other language. In a few cases,
there are language-independent commands available, but this is not
always the case. Therefore, as an alternative to polyglot parsing,
this commit sets the LANG environment variable to en_US.UTF-8.
Reported-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Grepping for "CPU" on lscpu output isn't always successful, depending
on the local language setting. As a result, the build can be aborted
early with:
"make: the '-j' option requires a positive integer argument"
This commit therefore uses the human-language-independent approach
available via the getconf command, both in kvm-build.sh and in
kvm-remote.sh.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-find-errors.sh assumes that if "--buildonly" appears in
the log file, then the run did builds but ran no kernels. This breaks
with kvm-remote.sh, which uses kvm.sh to do a build, then kvm-again.sh
to run the kernels built on remote systems. This commit therefore adds
a check for a kvm-remote.sh run.
While in the area, this commit checks for "--build-only" as well as
"--build-only".
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Given remote rcutorture runs, it is quite possible that the build system
will have fewer CPUs than the system(s) running the actual test scenarios.
In such cases, using the number of CPUs on the test systems can overload
the build system, slowing down the build or, worse, OOMing the build
system. This commit therefore uses the build system's CPU count to set
N in "make -jN", and by tradition sets "N" to double the CPU count.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit reduces duplicate code by making kvm.sh use the new
kvm-end-run-stats.sh script rather than taking its historical approach
of open-coding it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit abstractst the end-of-run summary from kvm-again.sh, and,
while in the area, brings its format into line with that of kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
