Skip to content

Running Tests

Jump to bottom
smurugap edited this page Nov 25, 2014 · 9 revisions

Outlined below are the steps for running tests from Releases 2.0 onwards. It includes an updated contrail-test infra which allows for

  • Better organization of test suites and utilities for easier use
  • Ability to run tests in parallel as well as in serial mode
  • Enhanced reporting and logging
  • Results to be stored in result.xml in Junit Test report format to be later readable by tools like Jenkins
  • Test runner similar to that in Openstack Tempest

Steps

Please clone the test code from

https://github.com/Juniper/contrail-test

And fabric code from

git clone https://github.com/Juniper/contrail-fabric-utils/

The testbed details need to be populated in sanity_params.ini and sanity_testbed.json and necessary python packages need to be installed. fab setup_test_env automates this task

cd /opt/contrail/utils; fab setup_test_env

To run tests , you could do 'fab run_sanity'. To manually run it on the target node, refer to manual setup of test environment and login to the target Config node and run the following command

cd contrail-test; ./run_tests.sh -s -m -U

It is recommended to add the images before running parallel tests. Parallel download of images may give some trouble

fab add_basic_images

run_tests.sh options

root@nodea11:~/contrail-test# ./run_tests.sh --help
Usage: ./run_tests.sh [OPTION]...
Run Contrail test suite

  -V, --virtual-env        Always use virtualenv.  Install automatically if not present
  -N, --no-virtual-env     Don't use virtualenv.  Run tests in local environment
  -n, --no-site-packages   Isolate the virtualenv from the global Python environment
  -f, --force              Force a clean re-build of the virtual environment. Useful when dependencies have been added.
  -u, --update             Update the virtual environment with any newer package versions
  -U, --upload             Upload test logs
  -s, --sanity             Only run sanity tests
  -t, --parallel           Run testr in parallel
  -C, --config             Config file location
  -h, --help               Print this usage message
  -d, --debug              Run tests with testtools instead of testr. This allows you to use PDB
  -l, --logging            Enable logging
  -L, --logging-config     Logging config file location.  Default is logging.conf
  -r, --result-xml         Path of Junitxml report to be generated
  -m, --send-mail          Send the report at the end
  -F, --features           Only run tests from features listed
  -T, --tags               Only run tests taged with tags
  -c, --concurrency        Number of threads to be spawned
  -- [TESTROPTIONS]        After the first '--' you can pass arbitrary arguments to testr

Test logs will be saved at contrail-tests/logs directory. There will be multiple logs directory , one log file per test class ran

We can run sanity,ci_sanity,quick_sanity with fab as well.

git clone https://github.com/Juniper/contrail-fabric-utils/

fab run_sanity:sanity
fab run_sanity:ci_sanity
fab run_sanity:quick_sanity

These will run tests in parallel

By default run_tests.sh would run tests in serial. To run tests in parallel, add the option "-t"

cd contrail-test; ./run_tests.sh -s -m -U -t

To run ha_sanity tests along with sanity tests, use the below command line.

cd contrail-test; ./run_tests.sh -s -m -U -t -T ha_sanity

To run tests from a particular directory

cd contrail-test; ./run_tests.sh -F '<test_directory_name>'
For example
cd contrail-test; ./run_tests.sh -F 'vm_regression'

The above command will run only tests from 'vm_regression' in scripts/serial_scripts folder The results of the tests will also be saved in contrail-test/result.xml and Junit html report will be in contrail-test/report/junit-testreport.html

Manual setup of Test environment:

If you do not want to use fab to setup the test environment, refer to sanity_params.ini.sample and sanity_testbed.json.sample to manually populate the content.

Install the below python Packages

# Centos
pip install fixtures testtools==1.1.0 testresources discover unittest2 selenium pyvirtualdisplay testrepository junitxml pytun pexpect
yum install tcpdump ant patch ant-junit ant-trax

# Ubuntu
pip install fixtures testtools==1.1.0 testresources selenium pyvirtualdisplay testrepository junitxml pytun pexpect
apt-get install tcpdump ant patch ant-junit ant-trax

Related Stuff

Running Neutron API Tests

Test Discovery and running Individual tests

Clone this wiki locally