openssh/regress
Darren Tucker 6e2fe81f92 - dtucker@cvs.openbsd.org 2013/11/09 05:41:34
[regress/test-exec.sh regress/rekey.sh]
     Use smaller test data files to speed up tests.  Grow test datafiles
     where necessary for a specific test.
2013-11-09 16:55:03 +11:00
..
.cvsignore - (djm) [regress/.cvsignore] Ignore Makefile here 2006-03-15 13:13:27 +11:00
addrmatch.sh - dtucker@cvs.openbsd.org 2012/05/13 01:42:32 2012-06-30 15:01:22 +10:00
agent-getpeereid.sh - dtucker@cvs.openbsd.org 2013/05/17 10:33:09 2013-05-17 20:46:00 +10:00
agent-pkcs11.sh - markus@cvs.openbsd.org 2010/02/08 10:52:47 2010-02-24 17:31:20 +11:00
agent-ptrace.sh - (djm) [regress/agent-ptrace.sh] Fix false failure on OS X by adding 2011-01-17 11:20:18 +11:00
agent-timeout.sh - dtucker@cvs.openbsd.org 2013/05/17 01:16:09 2013-05-17 14:46:20 +10:00
agent.sh - dtucker@cvs.openbsd.org 2013/05/17 00:37:40 2013-05-17 13:28:36 +10:00
banner.sh - dtucker@cvs.openbsd.org 2003/10/11 11:49:49 2004-01-30 12:58:51 +11:00
broken-pipe.sh - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
brokenkeys.sh - djm@cvs.openbsd.org 2004/10/29 23:59:22 2004-12-06 23:07:37 +11:00
cert-hostkey.sh - dtucker@cvs.openbsd.org 2013/05/17 00:37:40 2013-05-17 13:28:36 +10:00
cert-userkey.sh - dtucker@cvs.openbsd.org 2013/05/17 00:37:40 2013-05-17 13:28:36 +10:00
cfgmatch.sh - (dtucker) [regress/cfgmatch.sh] Remove unneeded sleep renderd obsolete by 2013-05-17 20:12:57 +10:00
cipher-speed.sh - dtucker@cvs.openbsd.org 2013/11/07 02:48:38 2013-11-07 15:21:19 +11:00
conch-ciphers.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
connect-privsep.sh - dtucker@cvs.openbsd.org 2012/07/02 14:37:06 2012-07-03 00:53:18 +10:00
connect.sh - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
dsa_ssh2.prv - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
dsa_ssh2.pub - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
dynamic-forward.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
envpass.sh - djm@cvs.openbsd.org 2005/03/04 08:48:46 2005-03-07 18:35:34 +11:00
exit-status.sh - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
forcecommand.sh sync missing ID 2013-05-17 14:44:53 +10:00
forward-control.sh - (dtucker) [regress/forward-control.sh] Wait longer for the forwarding 2013-03-06 14:06:48 +11:00
forwarding.sh - (tim) [regress/forwarding.sh] Fix for building outside read only source tree. 2013-07-25 16:03:16 -07:00
host-expand.sh - (tim) [regress/host-expand.sh] Fix for building outside of read only 2011-01-10 12:56:26 -08:00
integrity.sh - dtucker@cvs.openbsd.org 2013/11/07 02:48:38 2013-11-07 15:21:19 +11:00
kextype.sh - dtucker@cvs.openbsd.org 2013/11/07 04:26:56 2013-11-07 15:32:37 +11:00
key-options.sh - djm@cvs.openbsd.org 2008/06/30 08:07:34 2008-07-04 17:08:58 +10:00
keygen-change.sh - (djm) Sync regress with OpenBSD -current 2003-01-22 17:53:16 +11:00
keygen-convert.sh - dtucker@cvs.openbsd.org 2009/11/09 04:20:04 2010-02-24 17:33:30 +11:00
keys-command.sh - dtucker@cvs.openbsd.org 2012/12/06 06:06:54 2012-12-07 13:03:10 +11:00
keyscan.sh - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
keytype.sh - dtucker@cvs.openbsd.org 2013/05/17 00:37:40 2013-05-17 13:28:36 +10:00
krl.sh - (dtucker) [regress/integrity.sh regress/krl.sh regress/test-exec.sh] 2013-05-17 19:17:58 +10:00
localcommand.sh - dtucker@cvs.openbsd.org 2013/05/17 10:24:48 2013-05-17 20:42:05 +10:00
login-timeout.sh - dtucker@cvs.openbsd.org 2013/05/17 10:23:52 2013-05-17 20:41:07 +10:00
Makefile - djm@cvs.openbsd.org 2013/10/09 23:44:14 2013-11-07 14:50:09 +11:00
modpipe.c - (djm) [regress/modpipe.c regress/rekey.sh] Never intended to commit these 2013-11-07 12:03:26 +11:00
multiplex.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
portnum.sh - dtucker@cvs.openbsd.org 2013/05/17 10:34:30 2013-05-17 20:47:29 +10:00
proto-mismatch.sh - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
proto-version.sh - dtucker@cvs.openbsd.org 2013/05/17 00:37:40 2013-05-17 13:28:36 +10:00
proxy-connect.sh - djm@cvs.openbsd.org 2013/03/07 00:20:34 2013-05-17 09:07:12 +10:00
putty-ciphers.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
putty-kex.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
putty-transfer.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
README.regress - (djm) [Makefile.in WARNING.RNG aclocal.m4 buildpkg.sh.in configure.ac] 2011-05-05 13:48:37 +10:00
reconfigure.sh - (tim) [regress/login-timeout.sh regress/reconfigure.sh regress/reexec.sh 2010-08-12 09:43:13 -07:00
reexec.sh - dtucker@cvs.openbsd.org 2013/05/17 10:23:52 2013-05-17 20:41:07 +10:00
rekey.sh - dtucker@cvs.openbsd.org 2013/11/09 05:41:34 2013-11-09 16:55:03 +11:00
rsa_openssh.prv - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
rsa_openssh.pub - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
rsa_ssh2.prv - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
scp-ssh-wrapper.sh - dtucker@cvs.openbsd.org 2005/12/14 04:36:39 2006-01-31 22:03:11 +11:00
scp.sh - dtucker@cvs.openbsd.org 2013/05/17 10:35:43 2013-05-17 20:48:59 +10:00
sftp-badcmds.sh - dtucker@cvs.openbsd.org 2013/05/17 10:26:26 2013-05-17 20:43:13 +10:00
sftp-batch.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
sftp-chroot.sh - (tim) [regress/sftp-chroot.sh] skip if no sudo. ok dtucker 2013-06-02 15:13:09 -07:00
sftp-cmds.sh - djm@cvs.openbsd.org 2013/06/21 02:26:26 2013-07-25 12:11:18 +10:00
sftp-glob.sh - (djm) [configure.ac regress/agent-getpeereid.sh regress/multiplex.sh] 2011-01-17 16:17:09 +11:00
sftp-perm.sh - (tim) [regress/sftp-perm.sh] We need a shell that understands "! somecmd" 2013-10-24 12:22:49 -07:00
sftp.sh - dtucker@cvs.openbsd.org 2013/05/17 10:28:11 2013-05-17 20:44:09 +10:00
ssh2putty.sh fix id 2009-10-07 10:58:40 +11:00
ssh-com-client.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
ssh-com-keygen.sh - markus@cvs.openbsd.org 2004/02/24 17:06:52 2004-02-29 20:33:51 +11:00
ssh-com-sftp.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
ssh-com.sh - dtucker@cvs.openbsd.org 2013/05/17 00:37:40 2013-05-17 13:28:36 +10:00
sshd-log-wrapper.sh - dtucker@cvs.openbsd.org 2013/04/07 02:16:03 2013-05-17 09:31:39 +10:00
stderr-after-eof.sh - (dtucker) [regress/stderr-after-eof.sh regress/test-exec.sh] Move the md5 2013-05-17 19:53:25 +10:00
stderr-data.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
t4.ok - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
t5.ok - (djm) Import OpenBSD regression tests. Requires BSD make to run 2002-05-01 13:17:33 +10:00
test-exec.sh - dtucker@cvs.openbsd.org 2013/11/09 05:41:34 2013-11-09 16:55:03 +11:00
transfer.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
try-ciphers.sh - dtucker@cvs.openbsd.org 2013/11/07 02:48:38 2013-11-07 15:21:19 +11:00
yes-head.sh - (dtucker) [regress/yes-head.sh] Work around breakage caused by some 2005-11-28 16:41:03 +11:00

Overview.

$ ./configure && make tests

You'll see some progress info. A failure will cause either the make to
abort or the driver script to report a "FATAL" failure.

The test consists of 2 parts. The first is the file-based tests which is
driven by the Makefile, and the second is a set of network or proxycommand
based tests, which are driven by a driver script (test-exec.sh) which is
called multiple times by the Makefile.

Failures in the first part will cause the Makefile to return an error.
Failures in the second part will print a "FATAL" message for the failed
test and continue.

OpenBSD has a system-wide regression test suite. OpenSSH Portable's test
suite is based on OpenBSD's with modifications.


Environment variables.

SUDO: path to sudo command, if desired. Note that some systems (notably
	systems using PAM) require sudo to execute some tests.
TEST_SSH_TRACE: set to "yes" for verbose output from tests 
TEST_SSH_QUIET: set to "yes" to suppress non-fatal output.
TEST_SSH_x: path to "ssh" command under test, where x=SSH,SSHD,SSHAGENT,SSHADD
	SSHKEYGEN,SSHKEYSCAN,SFTP,SFTPSERVER
OBJ: used by test scripts to access build dir.
TEST_SHELL: shell used for running the test scripts.
TEST_SSH_PORT: TCP port to be used for the listening tests.
TEST_SSH_SSH_CONFOPTS: Configuration directives to be added to ssh_config
	before running each test.
TEST_SSH_SSHD_CONFOTPS: Configuration directives to be added to sshd_config
	before running each test.


Individual tests.

You can run an individual test from the top-level Makefile, eg:
$ make tests LTESTS=agent-timeout

If you need to manipulate the environment more you can invoke test-exec.sh
directly if you set up the path to find the binaries under test and the
test scripts themselves, for example:

$ cd regress
$ PATH=`pwd`/..:$PATH:. TEST_SHELL=/bin/sh sh test-exec.sh `pwd` \
    agent-timeout.sh
ok agent timeout test


Files.

test-exec.sh: the main test driver. Sets environment, creates config files
and keys and runs the specified test.

At the time of writing, the individual tests are:
agent-timeout.sh:	agent timeout test
agent.sh:		simple agent test
broken-pipe.sh:		broken pipe test
connect-privsep.sh:	proxy connect with privsep
connect.sh:		simple connect
exit-status.sh:		remote exit status
forwarding.sh:		local and remote forwarding
keygen-change.sh:	change passphrase for key
keyscan.sh:		keyscan
proto-mismatch.sh:	protocol version mismatch
proto-version.sh:	sshd version with different protocol combinations
proxy-connect.sh:	proxy connect
sftp.sh:		basic sftp put/get
ssh-com-client.sh:	connect with ssh.com client
ssh-com-keygen.sh:	ssh.com key import
ssh-com-sftp.sh:	basic sftp put/get with ssh.com server
ssh-com.sh:		connect to ssh.com server
stderr-after-eof.sh:	stderr data after eof
stderr-data.sh:		stderr data transfer
transfer.sh:		transfer data
try-ciphers.sh:		try ciphers
yes-head.sh:		yes pipe head


Problems?

Run the failing test with shell tracing (-x) turned on:
$ PATH=`pwd`/..:$PATH:. sh -x test-exec.sh `pwd` agent-timeout.sh

Failed tests can be difficult to diagnose. Suggestions:
- run the individual test via ./test-exec.sh `pwd` [testname]
- set LogLevel to VERBOSE in test-exec.sh and enable syslogging of
  auth.debug (eg to /var/log/authlog).


Known Issues.

- Similarly, if you do not have "scp" in your system's $PATH then the
  multiplex scp tests will fail (since the system's shell startup scripts
  will determine where the shell started by sshd will look for scp).

- Recent GNU coreutils deprecate "head -[n]": this will cause the yes-head
  test to fail.  The old behaviour can be restored by setting (and
  exporting) _POSIX2_VERSION=199209 before running the tests.

$Id: README.regress,v 1.12 2011/05/05 03:48:42 djm Exp $