Test code organization

The test code is organized in two repos contrail-test-ci and contrail-test
All the test infrastructure code and CI tests are in contrail-test-ci repo
In both the repos, tests which qualify to be run in parallel are in scripts/ folder. Cases which need be run serially ONLY are in serial_scripts/ folder
Within contrail-test-ci, object CRUD operations are mostly implemented using fixtures.Fixture ( eg: VMFixture in contrail-test-ci/fixtures/vm_test.py). Most fixture classes can be called independently outside of a testcase as well

Common feature-specific methods which can be called from within a TestCase context would be in common//base.py
fixtures.contrailapi.ContrailVncApi provides a set of high-level commonly used APIs to interact with Contrail Vnc API
tcutils.agent.vna_introspect_utils.AgentInspect provides a handle to query contrail-vrouter-agent introspect
tcutils.agent.control.ControlNodeInspect provides a handle to query control-node introspect
tcutils.config.vnc_api_test.VNCApiInspect provides a handle to query vnc-api using REST API
tcutils.collector.opserver_introspect_utils.VerificationOpsSrv provides a handle to query analytics collector API
tcutils.kubernetes.Client is a kubernetes client class to interface with kubernetes-master
tcutils.traffic_utils has helper classes for generating hping3, iperf3, netcat, pktgen, scapy traffic patterns

All the tests derive from testtools.TestCase base class.
A typical inheritance pattern that you would notice is for example :

scripts.neutron.test_routers_basic.TestRoutersBasic 
|
common.neutron.base.BaseNeutronTest
|
common.base.GenericTestBase
|
test_v1.BaseTestCase_v1
|
test.BaseTestCase

All logging is done per test module file and are logged in contrail-test/logs folder
The orchestrators in use are openstack, vcenter, kubernetes

Connections to various client modules like nova-client, keystone, vnc-api, kube-manager, vcenter, contrail-control, opserver, etc. are managed in common.connections.ContrailConnections class

In openstack environments, a typical testcase when derived from BaseTestCase_v1 would have a project and its users already created and is deleted upon deletion of the testcase class (ref self.connections in BaseTestCase_v1)
Connections for admin user would be available in (self.admin_connections in BaseTestCase_v1)

Given a testbed.py, contrail-test-ci/tools/configure.py auto-generates sanity_params.ini and sanity_testbed.json common.contrail_test_init.ContrailTestInit reads testbed-specific info from sanity_params.ini and sanity_testbed.json, popularly referred to as "self.inputs" in the test code

tcutils.wrappers.preposttest_wrapper() is a decorator for each testcase which checks for test-case exceptions/crash-files/xmpp-flaps etc.

testrunner.sh loads the test docker container for a build with a entrypoint script which runs run_tests.sh and backs up the test folder in HOME/contrail-test-runs/ for later use
run_tests.sh runs the chosen set of testcases using testr(from testrepository pip package) and subunit module. The results would be captured in result_*.xml(in junitxml format) and and a html report would be generated from the same

Useful tools

tools.tor.contrail-ovs-tool.sh : Tool to bringup and manage openvswitch switches
tools.tor.setup_tor_agents.py  : Setup pre-created certs for contrail tor-agents
tools.tor.setup_tors.py        : Set up QFX for ovsdb interaction with tor-agent, establish handles to work with MX (using netconf), and Vcenter
tools.collect_bts              : Search python tracebacks on contrail clusters
tools.db.anydbm-manage.py      : Update testr db with run-times. By default, has runtimes for sanity cases captured