mirror of
https://sourceware.org/git/binutils-gdb.git
synced 2024-12-13 20:33:33 +08:00
Fix stack smashing error during gdb_mpq_write_fixed_point selftest
When building GDB using Ubuntu 20.04's system libgmp and compiler, running the "maintenance selftest" command triggers the following error: | Running selftest gdb_mpq_write_fixed_point. | *** stack smashing detected ***: terminated | [1] 1092790 abort (core dumped) ./gdb gdb This happens while trying to construct an mpq_t object (a rational) from two integers representing the numerator and denominator. In our test, the numerator is -8, and the denominator is 1. The problem was that the rational was constructed using the wrong function. This is what we were doing prior to this patch: mpq_set_ui (v.val, numerator, denominator); The 'u' in "ui" stands for *unsigned*, which is wrong because numerator and denominator's type is "int". As a result of the above, instead of getting a rational value of -8, we get a rational with a very large positive value (gmp_printf says "18446744073709551608"). From there, the test performs an operation which is expected to write this value into a buffer which was not dimensioned to fit such a number, thus leading GMP into a buffer overflow. This was verified by applying the formula that GMP's documentation gives for the required memory buffer size needed during export: | When an application is allocating space itself the required size can | be determined with a calculation like the following. Since | mpz_sizeinbase always returns at least 1, count here will be at | least one, which avoids any portability problems with malloc(0), | though if z is zero no space at all is actually needed (or written). | | numb = 8*size - nail; | count = (mpz_sizeinbase (z, 2) + numb-1) / numb; | p = malloc (count * size); With the very large number, mpz_sizeinbase returns 66 and thus the malloc size becomes 16 bytes instead of the 8 we allocated. This patch fixes the issue by using the correct "set" function. gdb/ChangeLog: * unittests/gmp-utils-selftests.c (write_fp_test): Use mpq_set_si instead of mpq_set_ui to initialize our GMP rational.
This commit is contained in:
parent
fc23d4728d
commit
4fbb7ccebe
@ -1,3 +1,8 @@
|
||||
2020-11-24 Joel Brobecker <brobecker@adacore.com>
|
||||
|
||||
* unittests/gmp-utils-selftests.c (write_fp_test): Use mpq_set_si
|
||||
instead of mpq_set_ui to initialize our GMP rational.
|
||||
|
||||
2020-11-23 Tom de Vries <tdevries@suse.de>
|
||||
|
||||
* debuginfod-support.c (debuginfod_source_query)
|
||||
|
@ -400,7 +400,7 @@ write_fp_test (int numerator, unsigned int denominator,
|
||||
memset (buf, 0, len);
|
||||
|
||||
gdb_mpq v;
|
||||
mpq_set_ui (v.val, numerator, denominator);
|
||||
mpq_set_si (v.val, numerator, denominator);
|
||||
mpq_canonicalize (v.val);
|
||||
v.write_fixed_point (buf, len, byte_order, 0, scaling_factor);
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user