2016-10-15 18:09:20 +08:00
|
|
|
Running the BoringSSL test suite with OpenSSL
|
|
|
|
|
=============================================
|
|
|
|
|
|
|
|
|
|
It is possible to integrate external test suites into OpenSSL's "make test". At
|
|
|
|
|
the current time the only supported external suite is the one used by
|
|
|
|
|
BoringSSL.
|
|
|
|
|
|
|
|
|
|
This capability is considered a developer option and may not work on all
|
|
|
|
|
platforms.
|
|
|
|
|
|
|
|
|
|
In order to run the BoringSSL tests with OpenSSL, first checkout the BoringSSL
|
|
|
|
|
source code into an appropriate directory:
|
|
|
|
|
|
2016-10-15 18:24:13 +08:00
|
|
|
$ git clone https://boringssl.googlesource.com/boringssl boringssl
|
2016-10-15 18:09:20 +08:00
|
|
|
|
|
|
|
|
The BoringSSL tests are only confirmed to work at a specific commit in the
|
|
|
|
|
BoringSSL repository. Later commits may or may not pass the test suite:
|
|
|
|
|
|
|
|
|
|
$ cd boringssl
|
|
|
|
|
$ git checkout 490469f850e
|
|
|
|
|
|
|
|
|
|
From the OpenSSL source code configure to use the external tests:
|
|
|
|
|
|
|
|
|
|
$ cd ../openssl
|
|
|
|
|
$ ./config enable-ssl3 enable-ssl3-method enable-weak-ssl-ciphers \
|
|
|
|
|
enable-external-tests
|
|
|
|
|
|
|
|
|
|
Note that using other config option than those given above may cause the tests
|
|
|
|
|
to fail.
|
|
|
|
|
|
|
|
|
|
Run the OpenSSL tests by providing the path to the BoringSSL test runner in the
|
|
|
|
|
BORING_RUNNER_DIR environment variable:
|
|
|
|
|
|
|
|
|
|
$ BORING_RUNNER_DIR=/path/to/boringssl/ssl/test/runner make test
|
|
|
|
|
|
|
|
|
|
Note that the test suite may change directory while running so the path provided
|
|
|
|
|
should be absolute and not relative to the current working directory.
|
|
|
|
|
|
|
|
|
|
To see more detailed output you can run just the BoringSSL tests with the
|
|
|
|
|
verbose option:
|
|
|
|
|
|
|
|
|
|
$ VERBOSE=1 BORING_RUNNER_DIR=/path/to/boringssl/ssl/test/runner make \
|
|
|
|
|
TESTS="test_external" test
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Test failures and suppressions
|
|
|
|
|
==============================
|
|
|
|
|
|
|
|
|
|
A large number of the BoringSSL tests are known to fail. A test could fail
|
|
|
|
|
because of many possible reasons. For example:
|
|
|
|
|
|
|
|
|
|
- A bug in OpenSSL
|
|
|
|
|
- Different interpretations of standards
|
|
|
|
|
- Assumptions about the way BoringSSL works that do not apply to OpenSSL
|
|
|
|
|
- The test uses APIs added to BoringSSL that are not present in OpenSSL
|
|
|
|
|
- etc
|
|
|
|
|
|
|
|
|
|
In order to provide a "clean" baseline run with all the tests passing a config
|
|
|
|
|
file has been provided that suppresses the running of tests that are known to
|
|
|
|
|
fail. These suppressions are held in the file "test/ossl_shim/ossl_config.json"
|
|
|
|
|
within the OpenSSL source code.
|
|
|
|
|
|
|
|
|
|
The community is encouraged to contribute patches which reduce the number of
|
|
|
|
|
suppressions that are currently present.
|
|
|
|
|
|
