binutils-gdb/gdbserver
Kevin Buettner b6d8d612d3 gdbserver: Reinstall software single-step breakpoints in resume_stopped_resumed_lwps
At the moment, while performing a software single-step, gdbserver fails
to reinsert software single-step breakpoints for a LWP when
interrupted by a signal in another thread.  This commit fixes this
problem by reinstalling software single-step breakpoints in
linux_process_target::resume_stopped_resumed_lwps in
gdbserver/linux-low.cc.

This bug was discovered due to a failing assert in maybe_hw_step()
in gdbserver/linux-low.cc.  Looking at the backtrace revealed
that the caller was linux_process_target::resume_stopped_resumed_lwps.
I was uncertain whether the assert should still be valid when called
from that method, so I tried hoisting the assert from maybe_hw_step
to all callers except resume_stopped_resumed_lwps.  But running the
new test case, described below, showed that merely eliminating the
assert for this case was NOT a good fix - a study of the log file for
the test showed that the single-step operation failed to occur.
Instead GDB (via gdbserver) stopped at the next breakpoint that was
hit.

Zhiyong Yan had proposed a fix which resinserted software single-step
breakpoints, albeit at a different location in linux-low.cc.  Testing
revealed that, while running gdb.threads/pending-fork-event-detach,
the executable associated with that test would die due to a SIGTRAP
after the test program was detached.  Examination of the core file(s)
showed that a breakpoint instruction had been left in program memory.
Test results were otherwise very good, so Zhiyong was definitely on
the right track!

This commit causes software single-step breakpoint(s) to be inserted
before the call to maybe_hw_step in resume_stopped_resumed_lwps.  This
will cause 'has_single_step_breakpoints (thread)' to be true, so that
the assert in maybe_hw_step...

      /* GDBserver must insert single-step breakpoint for software
	 single step.  */
      gdb_assert (has_single_step_breakpoints (thread));

...will no longer fail.  And better still, the single-step breakpoints
are reinstalled, so that stepping will actually work, even when
interrupted.

The C code for the test case was loosely adapted from the reproducer
provided in Zhiyong's bug report for this problem.  The .exp file was
copied from next-fork-other-thread.exp and then tweaked slightly.  As
noted in a comment in next-fork-exec-other-thread.exp, I had to remove
"on" from the loop for non-stop as it was failing on all architectures
(including x86-64) that I tested.  I have a feeling that it ought to
work, but this can be investigated separately and (re)enabled once it
works.  I also increased the number of iterations for the loop running
the "next" commands.  I've had some test runs which don't show the bug
until the loop counter exceeded 100 iterations.  The C file for the
new test uses shorter delays than next-fork-other-thread.c though, so
it doesn't take overly long (IMO) to run this new test.

Running the new test on a Raspberry Pi w/ a 32-bit (Arm) kernel and
userland using a gdbserver build without the fix in this commit shows
the following results:

FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=fork: target-non-stop=auto: non-stop=off: displaced-stepping=auto: i=12: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=fork: target-non-stop=auto: non-stop=off: displaced-stepping=on: i=9: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=fork: target-non-stop=auto: non-stop=off: displaced-stepping=off: i=18: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=fork: target-non-stop=off: non-stop=off: displaced-stepping=auto: i=3: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=fork: target-non-stop=off: non-stop=off: displaced-stepping=on: i=11: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=fork: target-non-stop=off: non-stop=off: displaced-stepping=off: i=1: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=auto: non-stop=off: displaced-stepping=auto: i=1: next to break here
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=auto: non-stop=off: displaced-stepping=on: i=3: next to break here
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=auto: non-stop=off: displaced-stepping=off: i=1: next to break here
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=on: non-stop=off: displaced-stepping=auto: i=47: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=on: non-stop=off: displaced-stepping=on: i=57: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=off: non-stop=off: displaced-stepping=auto: i=1: next to break here
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=off: non-stop=off: displaced-stepping=on: i=10: next to break here
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=off: non-stop=off: displaced-stepping=off: i=1: next to break here

		=== gdb Summary ===

 # of unexpected core files	12
 # of expected passes		3011
 # of unexpected failures	14

Each of the 12 core files were caused by the failed assertion in
maybe_hw_step in linux-low.c.  These correspond to 12 of the
unexpected failures.

When the tests are run using a gdbserver build which includes the fix
in this commit, the results are significantly better, but not perfect:

FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=on: non-stop=off: displaced-stepping=auto: i=143: next to other line
FAIL: gdb.threads/next-fork-exec-other-thread.exp: fork_func=vfork: target-non-stop=on: non-stop=off: displaced-stepping=on: i=25: next to other line

		=== gdb Summary ===

 # of expected passes		10178
 # of unexpected failures	2

I think that the two remaining failures are due to some different
problem.  They are also racy - I've seen runs with no failures or only
one failure, but never more than two.  Also, those runs were conducted
with the loop count in next-fork-exec-other-thread.exp set to 200.
During his testing of this fix and the new test case, Luis Machado
found that this test was taking a long time and asked about ways to
speed it up.  I then conducted additional tests in which I gradually
reduced the loop count, timing each one, also noting the number of
failures.  With the loop count set to 30, I found that I could still
reliably reproduce the failures that Zhiyong reported (in which, with
the proper settings, core files are created).  But, with the loop
count set to 30, the other failures noted above were much less likely
to show up.  Anyone wishing to investigate those other failures should
set the loop count back up to 200.

Running the new test on x86-64 and aarch64, both native and
native-gdbserver shows no failures.

Also, I see no regressions when running the entire test suite for
armv7l-unknown-linux-gnueabihf (i.e.  the Raspberry Pi w/ 32-bit
kernel+userland) with --target_board=native-gdbserver.  Additionally,
using --target_board=native-gdbserver, I also see no regressions for
the entire test suite for x86-64 and aarch64 running Fedora 38.

Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30387
Co-Authored-By: Zhiyong Yan <zhiyong.yan@windriver.com>
Tested-By: Zhiyong Yan <zhiyong.yan@windriver.com>
Tested-By: Luis Machado <luis.machado@arm.com>
2023-08-11 20:54:09 -07:00
..
.dir-locals.el Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
.gitattributes gdb, gdbserver, gdbsupport: add .gitattributes files 2020-03-05 15:59:22 +01:00
.gitignore
acinclude.m4 gdb: don't use -Wmissing-prototypes with g++ 2022-01-13 10:25:45 +00:00
aclocal.m4 gdb/gdbserver: switch to AC_CONFIG_MACRO_DIRS 2021-06-19 23:06:01 -04:00
ax-result-types.def gdbserver: handle all eval_result_type values in tracepoint.cc 2023-07-10 14:42:54 +01:00
ax.cc [gdb] Fix typos 2023-06-03 22:43:57 +02:00
ax.h gdbserver: handle all eval_result_type values in tracepoint.cc 2023-07-10 14:42:54 +01:00
ChangeLog-2002-2021 gdb: move remaining ChangeLogs to legacy files 2021-07-26 12:20:33 +01:00
config.in gdbsupport: move libxxhash configure check to gdbsupport 2023-01-05 14:38:51 -05:00
configure Set _WIN32_WINNT in common.m4 configure check 2023-01-11 11:00:42 -07:00
configure.ac Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
configure.srv gdbserver/csky add csky gdbserver support 2022-09-13 11:20:54 +08:00
debug.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
debug.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
dll.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
dll.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
fork-child.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
gdb_proc_service.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
gdbreplay.cc Update copyright year in help message of gdb, gdbserver, gdbreplay 2023-01-01 17:01:15 +04:00
gdbthread.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
hostio.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
hostio.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
i387-fp.cc gdbserver: Clear upper ZMM registers in the right location. 2023-05-08 10:39:52 -07:00
i387-fp.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
inferiors.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
inferiors.h [gdb] Fix typos 2023-06-03 22:43:57 +02:00
linux-aarch32-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-aarch32-low.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-aarch32-tdesc.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-aarch32-tdesc.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-aarch64-ipa.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-aarch64-low.cc [gdbserver] Fix typo in debug message 2023-06-03 22:43:57 +02:00
linux-aarch64-tdesc.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-aarch64-tdesc.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-amd64-ipa.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-arc-low.cc [gdb] Fix typos 2023-06-03 22:43:57 +02:00
linux-arm-low.cc gdbserver: Add PID parameter to linux_get_auxv and linux_get_hwcap 2023-02-01 20:42:50 +00:00
linux-arm-tdesc.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-arm-tdesc.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-csky-low.cc [gdb] Fix typos 2023-06-03 22:43:57 +02:00
linux-i386-ipa.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-ia64-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-loongarch-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-low.cc gdbserver: Reinstall software single-step breakpoints in resume_stopped_resumed_lwps 2023-08-11 20:54:09 -07:00
linux-low.h Don't use struct buffer in handle_qxfer_btrace 2023-02-24 11:52:48 -07:00
linux-m68k-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-mips-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-nios2-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-or1k-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-ppc-ipa.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-ppc-low.cc [gdb] Fix typos 2023-06-03 22:43:57 +02:00
linux-ppc-tdesc-init.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-riscv-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-s390-ipa.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-s390-low.cc gdbserver: Add PID parameter to linux_get_auxv and linux_get_hwcap 2023-02-01 20:42:50 +00:00
linux-s390-tdesc.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-sh-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-sparc-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-tic6x-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-x86-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-x86-tdesc.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-x86-tdesc.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
linux-xtensa-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
Makefile.in Fix 'make TAGS' in gdbserver 2023-01-18 11:12:20 -07:00
mem-break.cc [gdb] Fix typos 2023-06-03 22:43:57 +02:00
mem-break.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
netbsd-aarch64-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
netbsd-amd64-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
netbsd-i386-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
netbsd-low.cc gdbserver: Add PID parameter to linux_get_auxv and linux_get_hwcap 2023-02-01 20:42:50 +00:00
netbsd-low.h gdbserver: Add PID parameter to linux_get_auxv and linux_get_hwcap 2023-02-01 20:42:50 +00:00
notif.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
notif.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
proc-service.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
proc-service.list Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
README gdbserver: small cleanup of README file 2020-06-12 16:01:35 -04:00
regcache.cc gdbserver: Add assert in find_register_by_number 2023-02-01 19:48:07 +00:00
regcache.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
remote-utils.cc gdbserver: use target_waitstatus::to_string in 'prepare_resume_reply' 2023-06-21 08:47:13 +02:00
remote-utils.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
server.cc [gdb] Fix typos 2023-06-03 22:43:57 +02:00
server.h [gdb] Fix typos 2023-06-03 22:43:57 +02:00
symbol.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
target.cc Don't use struct buffer in handle_qxfer_btrace 2023-02-24 11:52:48 -07:00
target.h Don't use struct buffer in handle_qxfer_btrace 2023-02-24 11:52:48 -07:00
tdesc.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
tdesc.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
thread-db.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
tracepoint.cc gdbserver: handle all eval_result_type values in tracepoint.cc 2023-07-10 14:42:54 +01:00
tracepoint.h gdbserver: allows agent_mem_read to return an error code 2023-04-03 14:46:32 +01:00
utils.cc Move implementation of perror_with_name to gdbsupport 2023-02-10 21:04:45 -05:00
utils.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
win32-i386-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
win32-low.cc [gdb] Fix typos 2023-06-03 22:43:57 +02:00
win32-low.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
x86-low.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
x86-low.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
x86-tdesc.h Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00
xtensa-xtregs.cc Update copyright year range in header of all files managed by GDB 2023-01-01 17:01:16 +04:00

		   README for GDBserver & GDBreplay
		    by Stu Grossman and Fred Fish

Introduction:

This is GDBserver, a remote server for Un*x-like systems.  It can be used to
control the execution of a program on a target system from a GDB on a different
host.  GDB and GDBserver communicate using the standard remote serial protocol.
They communicate via either a serial line or a TCP connection.

For more information about GDBserver, see the GDB manual:

    https://sourceware.org/gdb/current/onlinedocs/gdb/Remote-Protocol.html

Usage (server (target) side):

First, you need to have a copy of the program you want to debug put onto
the target system.  The program can be stripped to save space if needed, as
GDBserver doesn't care about symbols.  All symbol handling is taken care of by
the GDB running on the host system.

To use the server, you log on to the target system, and run the `gdbserver'
program.  You must tell it (a) how to communicate with GDB, (b) the name of
your program, and (c) its arguments.  The general syntax is:

	target> gdbserver COMM PROGRAM [ARGS ...]

For example, using a serial port, you might say:

	target> gdbserver /dev/com1 emacs foo.txt

This tells GDBserver to debug emacs with an argument of foo.txt, and to
communicate with GDB via /dev/com1.  GDBserver now waits patiently for the
host GDB to communicate with it.

To use a TCP connection, you could say:

	target> gdbserver host:2345 emacs foo.txt

This says pretty much the same thing as the last example, except that we are
going to communicate with the host GDB via TCP.  The `host:2345' argument means
that we are expecting to see a TCP connection to local TCP port 2345.
(Currently, the `host' part is ignored.)  You can choose any number you want for
the port number as long as it does not conflict with any existing TCP ports on
the target system.  This same port number must be used in the host GDB's
`target remote' command, which will be described shortly. Note that if you chose
a port number that conflicts with another service, GDBserver will print an error
message and exit.

On some targets, GDBserver can also attach to running programs.  This is
accomplished via the --attach argument.  The syntax is:

	target> gdbserver --attach COMM PID

PID is the process ID of a currently running process.  It isn't necessary
to point GDBserver at a binary for the running process.

Usage (host side):

You need an unstripped copy of the target program on your host system, since
GDB needs to examine it's symbol tables and such.  Start up GDB as you normally
would, with the target program as the first argument.  (You may need to use the
--baud option if the serial line is running at anything except 9600 baud.)
Ie: `gdb TARGET-PROG', or `gdb --baud BAUD TARGET-PROG'.  After that, the only
new command you need to know about is `target remote'.  It's argument is either
a device name (usually a serial device, like `/dev/ttyb'), or a HOST:PORT
descriptor.  For example:

	(gdb) target remote /dev/ttyb

communicates with the server via serial line /dev/ttyb, and:

	(gdb) target remote the-target:2345

communicates via a TCP connection to port 2345 on host `the-target', where
you previously started up GDBserver with the same port number.  Note that for
TCP connections, you must start up GDBserver prior to using the `target remote'
command, otherwise you may get an error that looks something like
`Connection refused'.

Building GDBserver:

See the `configure.srv` file for the list of host triplets you can build
GDBserver for.

Building GDBserver for your host is very straightforward.  If you build
GDB natively on a host which GDBserver supports, it will be built
automatically when you build GDB.  You can also build just GDBserver:

	% mkdir obj
	% cd obj
	% path-to-toplevel-sources/configure --disable-gdb
	% make all-gdbserver

(If you have a combined binutils+gdb tree, you may want to also
disable other directories when configuring, e.g., binutils, gas, gold,
gprof, and ld.)

If you prefer to cross-compile to your target, then you can also build
GDBserver that way.  For example:

	% export CC=your-cross-compiler
	% path-to-topevel-sources/configure --disable-gdb
	% make all-gdbserver

Using GDBreplay:

A special hacked down version of GDBserver can be used to replay remote
debug log files created by GDB.  Before using the GDB "target" command to
initiate a remote debug session, use "set remotelogfile <filename>" to tell
GDB that you want to make a recording of the serial or tcp session.  Note
that when replaying the session, GDB communicates with GDBreplay via tcp,
regardless of whether the original session was via a serial link or tcp.

Once you are done with the remote debug session, start GDBreplay and
tell it the name of the log file and the host and port number that GDB
should connect to (typically the same as the host running GDB):

	$ gdbreplay logfile host:port

Then start GDB (preferably in a different screen or window) and use the
"target" command to connect to GDBreplay:

	(gdb) target remote host:port

Repeat the same sequence of user commands to GDB that you gave in the
original debug session.  GDB should not be able to tell that it is talking
to GDBreplay rather than a real target, all other things being equal.  Note
that GDBreplay echos the command lines to stderr, as well as the contents of
the packets it sends and receives.  The last command echoed by GDBreplay is
the next command that needs to be typed to GDB to continue the session in
sync with the original session.