-
Notifications
You must be signed in to change notification settings - Fork 26
Home
This repo contain contrail ci tests and other common tools and libraries to run contrail-test.
A script provided (install.sh) to install both contrail-test-ci as well as contrail-test.
$ ./install.sh -h
Install or do docker build of Contrail-test and contrail-test-ci
Usage: ./install.sh (install|docker-build) [OPTIONS] (contrail-test|contrail-test-ci)
Subcommands:
install Install contrail-test/contrail-test-ci
docker-build Build docker container
Run ./install.sh <Subcommand> -h|--help to get subcommand specific help
$ ./install.sh install -h
Install Contrail-test or contrail-test-ci
Usage: ./install.sh install [OPTIONS] (contrail-test|contrail-test-ci)
-h|--help Print help message
--test-repo REPO Contrail-test git repo, Default: github.com/juniper/contrail-test-ci.git
--test-ref REF Contrail-test git reference - commit id, branch, tag, Default: master
--fab-repo FAB_REPO Contrail-fabric-utils git repo
--fab-ref FAB_REF Contrail-fabric-utils git reference (commit id, branch, or tag), Default: master
--ci-repo CI_REPO Contrail-test-ci git repo, Default: github.com/juniper/contrail-test.git
--ci-ref CI_REF Contrail-test-ci reference (commit id, branch, or tag), Default: master
--test-artifact ARTIFACT Contrail test tar file - this tar file will be used instead of git source in case provided
--ci-artifact CI_ARTICACT Contrail test ci tar file
--fab-artifact FAB_ARTIFACT Contrail-fabric-utils tar file
-u|--package-url PACKAGE_URL Contrail-install-packages deb package web url (http:// or https://) or scp path
(ssh://<server ip/name/< package path>), if url is provided, the
package will be installed and setup local repo.
In case of scp, user name and password will be read from environment variables
SSHUSER - user name to be used during scp
SSHPASS - user password to be used during scp
positional arguments
What to install Valid options are contrail-test, contrail-test-ci
Example:
./install.sh install --test-repo https://github.com/hkumarmk/contrail-test --test-ref working
--ci-repo https://$GITUSER:$GITPASS@github.com/juniper/contrail-test-ci -e /tmp/export2
-u http://nodei16/contrail-install-packages_2.21-105~juno_all.deb contrail-test
$ export SSHUSER=user1 SSHPASS=password
$ ./install.sh install --test-repo https://github.com/hkumarmk/contrail-test --test-ref working
--ci-repo https://$GITUSER:$GITPASS@github.com/juniper/contrail-test-ci -e /tmp/export2
-u ssh://nodei16/var/cache/artifacts/contrail-install-packages_2.21-105~juno_all.deb contrail-test-ci
- Install with artifacts
./install.sh install --ci-artifact /tmp/contrail-test-ci.tar.gz \
-u http://nodei16/contrail-install-packages_2.21-105~juno_all.deb \
--fab-artifact /tmp/contrail-fabric-utils.tar.gz contrail-test-ci # Just use contrail-test as last argument to install contrail-test
- combination of artifacts and git refs
./install.sh install --test-ref 1890b36 --ci-artifact /tmp/contrail-test-ci.tar.gz \
-u http://nodei16/contrail-install-packages_2.21-105~juno_all.deb \
--fab-artifact /tmp/contrail-fabric-utils.tar.gz contrail-test
Once it is installed, one should copy testbed.py to /opt/contrail/utils/fabfile/testbeds (or any custom fab path in which case, one have to provide --contrail-fab-path while running run_tests.sh). Now just go to /opt/contrail-test and run run_tests.sh as they have done before.
NOTE: No need to run any fab tasks like setup_testenv or anything. Running run_tests.sh will read testbed.py and build required files to run the tests (like sanity_params.ini, sanity_testbed.json).
Here is how I run contrail-test after Install it.
- Copy testbed.py to /opt/contrail/utils/fabfile/testbeds/
- cd /opt/contrail-test
- run ./run_tests.sh with appropriate parameters
$ cd /opt/contrail-test
$ ./run_tests.sh -h
Usage: ./run_tests.sh [OPTION]...
Run Contrail test suite
-V, --virtual-env Always use virtualenv. Install automatically if not present
-N, --no-virtual-env Don't use virtualenv. Run tests in local environment
-n, --no-site-packages Isolate the virtualenv from the global Python environment
-f, --force Force a clean re-build of the virtual environment. Useful when dependencies have been added.
-u, --update Update the virtual environment with any newer package versions
-U, --upload Upload test logs
-s, --sanity Only run sanity tests
-t, --parallel Run testr in parallel
-C, --config Config file location
-h, --help Print this usage message
-d, --debug Run tests with testtools instead of testr. This allows you to use PDB
-l, --logging Enable logging
-L, --logging-config Logging config file location. Default is logging.conf
-m, --send-mail Send the report at the end
-F, --features Only run tests from features listed
-T, --tags Only run tests taged with tags
-c, --concurrency Number of threads to be spawned
--contrail-fab-path Contrail fab path, default to /opt/contrail/utils
-- [TESTROPTIONS] After the first '--' you can pass arbitrary arguments to testr
$ ./run_tests.sh
[localhost] local: git branch
fatal: Not a git repository (or any of the parent directories): .git
2016-02-09 10:25:35:568148: Warning: local() encountered an error (return code 128) while executing 'git branch'
2016-02-09 10:25:35:568148:
2016-02-09 10:25:35:568148: 2016-02-09 10:25:35:561571: [root@10.204.217.88] run: cat /opt/contrail/contrail_packages/VERSION
2016-02-09 10:25:35:568437: [root@10.204.217.88] out: BUILDID=2709
2016-02-09 10:25:36:714451: [root@10.204.217.88] out:
2016-02-09 10:25:36:714859:
2016-02-09 10:25:36:719259: [localhost] local: git branch
fatal: Not a git repository (or any of the parent directories): .git
2016-02-09 10:25:36:726339: Warning: local() encountered an error (return code 128) while executing 'git branch'
2016-02-09 10:25:36:726339:
2016-02-09 10:25:36:726339: 2016-02-09 10:25:36:719675: [root@10.204.217.88] run: cat /opt/contrail/contrail_packages/VERSION
2016-02-09 10:25:36:726615: [root@10.204.217.88] out: BUILDID=2709
2016-02-09 10:25:36:777523: [root@10.204.217.88] out:
2016-02-09 10:25:36:777781:
2016-02-09 10:25:36:778033: [root@10.204.217.88] run: hostname
2016-02-09 10:25:36:778463: [root@10.204.217.88] out: harishku-vm1
2016-02-09 10:25:36:832563: [root@10.204.217.88] out:
2016-02-09 10:25:36:832820:
2016-02-09 10:25:36:833036: [root@10.204.217.88] run: hostname
2016-02-09 10:25:36:833272: [root@10.204.217.88] out: harishku-vm1
2016-02-09 10:25:36:884548: [root@10.204.217.88] out:
2016-02-09 10:25:36:884789:
2016-02-09 10:25:36:885041: [root@10.204.217.88] run: hostname
2016-02-09 10:25:36:885263: [root@10.204.217.88] out: harishku-vm1
2016-02-09 10:25:36:969527: [root@10.204.217.88] out:
2016-02-09 10:25:36:969768:
2016-02-09 10:25:36:974452: [root@10.204.217.90] run: hostname
2016-02-09 10:25:36:974680: [root@10.204.217.90] out: harishku-vm3
2016-02-09 10:25:37:708216: [root@10.204.217.90] out:
2016-02-09 10:25:37:708449:
2016-02-09 10:25:37:708709: [root@10.204.217.88] run: hostname
2016-02-09 10:25:37:708983: [root@10.204.217.88] out: harishku-vm1
2016-02-09 10:25:37:729606: [root@10.204.217.88] out:
2016-02-09 10:25:37:729832:
2016-02-09 10:25:37:730082: [root@10.204.217.89] run: hostname
2016-02-09 10:25:37:730289: [root@10.204.217.89] out: localhost
2016-02-09 10:25:38:520887: [root@10.204.217.89] out:
2016-02-09 10:25:38:521179:
2016-02-09 10:25:38:521440: [root@10.204.217.90] run: hostname
2016-02-09 10:25:38:521712: [root@10.204.217.90] out: harishku-vm3
2016-02-09 10:25:38:542269: [root@10.204.217.90] out:
2016-02-09 10:25:38:542559:
2016-02-09 10:25:38:542855: [root@10.204.217.88] run: hostname
2016-02-09 10:25:38:543112: [root@10.204.217.88] out: harishku-vm1
2016-02-09 10:25:38:589205: [root@10.204.217.88] out:
2016-02-09 10:25:38:589499:
2016-02-09 10:25:38:589721: [root@10.204.217.89] run: hostname
2016-02-09 10:25:38:590029: [root@10.204.217.89] out: localhost
2016-02-09 10:25:38:611151: [root@10.204.217.89] out:
2016-02-09 10:25:38:611428:
2016-02-09 10:25:38:616572: [root@10.204.217.90] run: hostname
2016-02-09 10:25:38:616852: [root@10.204.217.90] out: harishku-vm3
2016-02-09 10:25:38:639135: [root@10.204.217.90] out:
2016-02-09 10:25:38:639500:
2016-02-09 10:25:38:639774: [root@10.204.217.129] run: hostname
2016-02-09 10:25:38:640058: [root@10.204.217.129] out: nodei17
2016-02-09 10:25:38:965649: [root@10.204.217.129] out:
2016-02-09 10:25:38:965880:
2016-02-09 10:25:38:968799: Python 2.7.6
no match
Reversed (or previously applied) patch detected! Skipping patch.
1 out of 1 hunk ignored
+ echo 'Validating if test discovery passes in scripts/ and serial_scripts'
Validating if test discovery passes in scripts/ and serial_scripts
+ echo ''
+ GIVEN_TEST_PATH=
+ export PYTHONPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/contrail-test/scripts:/opt/contrail-test/fixtures
+ PYTHONPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/contrail-test/scripts:/opt/contrail-test/fixtures
+ export OS_TEST_PATH=./scripts
+ OS_TEST_PATH=./scripts
+ testr list-tests
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-6000} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./scripts} --list
scripts.analytics.test_analytics.AnalyticsTestSanity.test_bgprouter_uve_for_xmpp_and_bgp_peer_count
scripts.analytics.test_analytics.AnalyticsTestSanity.test_colector_uve_module_sates
scripts.analytics.test_analytics.AnalyticsTestSanity.test_config_node_uve_states
scripts.analytics.test_analytics.AnalyticsTestSanity.test_contrail_alarms[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity.test_contrail_status[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity.test_message_table
scripts.analytics.test_analytics.AnalyticsTestSanity.test_verify_hrefs
scripts.analytics.test_analytics.AnalyticsTestSanity1.test_stats_tables
scripts.analytics.test_analytics.AnalyticsTestSanity1.test_verify__bgp_router_uve_up_xmpp_and_bgp_count
scripts.analytics.test_analytics.AnalyticsTestSanity1.test_verify_bgp_peer_uve
scripts.analytics.test_analytics.AnalyticsTestSanity2.runTest
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_contrail_database_status[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_db_nodemgr_status[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_db_purge[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_verify_generator_collector_connections[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_verify_generator_connections_to_collector_node[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_verify_process_status_agent[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_verify_process_status_analytics_node[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_verify_process_status_config[sanity,vcenter]
scripts.analytics.test_analytics.AnalyticsTestSanity3.test_verify_process_status_control_node[sanity,vcenter]
scripts.analytics.test_analytics_basic.AnalyticsBasicTestSanity.test_verify_object_logs[ci_sanity,sanity,vcenter]
scripts.ceilometer_tests.test_ceilometer.CeilometerTest.test_resources_by_admin_tenant
scripts.ceilometer_tests.test_ceilometer.CeilometerTest.test_resources_by_user_tenant
scripts.ceilometer_tests.test_ceilometer.CeilometerTest.test_sample_floating_ip_transmit_packets[sanity]
scripts.discovery_regression.test_discovery.TestDiscovery.test_ApiServer_subscribed_to_collector_service
scripts.discovery_regression.test_discovery.TestDiscovery.test_Schema_subscribed_to_collector_service
scripts.discovery_regression.test_discovery.TestDiscovery.test_ServiceMonitor_subscribed_to_collector_service
scripts.discovery_regression.test_discovery.TestDiscovery.test_agents_connected_to_collector_service[sanity,vcenter]
scripts.discovery_regression.test_discovery.TestDiscovery.test_agents_connected_to_dns_service
scripts.discovery_regression.test_discovery.TestDiscovery.test_cleanup
scripts.discovery_regression.test_discovery.TestDiscovery.test_control_nodes_connected_to_collector_service
There is a script (install.sh) which build docker containers for both contrail-test-ci as well as contrail-test.
$ ./install.sh docker-build -h
Build Contrail-test and contrail-test-ci docker container
Usage: ./install.sh docker-build [OPTIONS] (contrail-test|contrail-test-ci)
-h|--help Print help message
--test-repo REPO Contrail-test git repo, Default: github.com/juniper/contrail-test-ci.git
--test-ref REF Contrail-test git reference - commit id, branch, tag, Default: master
--fab-repo FAB_REPO Contrail-fabric-utils git repo
--fab-ref FAB_REF Contrail-fabric-utils git reference (commit id, branch, or tag), Default: master
--ci-repo CI_REPO Contrail-test-ci git repo, Default: github.com/juniper/contrail-test.git
--ci-ref CI_REF Contrail-test-ci reference (commit id, branch, or tag), Default: master
--container-tag CONTAINER_TAG Docker container tag, default to contrail-test-ci-<openstack-release>:<contrail-version>
openstack-release and contrail-version is extracted from contrail-install-package name
e.g contrail-test-ci-juno:2.21-105
--test-artifact ARTIFACT Contrail test tar file - this tar file will be used instead of git source in case provided
--ci-artifact CI_ARTICACT Contrail test ci tar file
--fab-artifact FAB_ARTIFACT Contrail-fabric-utils tar file
-u|--package-url PACKAGE_URL Contrail-install-packages deb package web url (http:// or https://) or scp path
(ssh://<server ip/name/< package path>), if url is provided, the
package will be installed and setup local repo.
In case of scp, user name and password will be read from environment variables
SSHUSER - user name to be used during scp
SSHPASS - user password to be used during scp
-c|--use-cache Use docker cache for the build
-e|--export EXPORT_PATH Export Container image to the path provided
positional arguments
Type of build What to build, valid options are contrail-test, contrail-test-ci
Example:
$ ./install.sh docker-build --test-repo https://github.com/hkumarmk/contrail-test --test-ref working
--ci-repo https://$GITUSER:$GITPASS@github.com/juniper/contrail-test-ci -e /tmp/export2
-u http://nodei16/contrail-install-packages_2.21-105~juno_all.deb contrail-test
$ export SSHUSER=user1 SSHPASS=password
$ ./install.sh docker-build --test-repo https://github.com/hkumarmk/contrail-test --test-ref working
--ci-repo https://$GITUSER:$GITPASS@github.com/juniper/contrail-test-ci -e /tmp/export2
-u ssh://nodei16/var/cache/artifacts/contrail-install-packages_2.21-105~juno_all.deb contrail-test
- Docker build with artifacts
$ ./install.sh docker-build -c --ci-artifact /tmp/contrail-test-ci.tar.gz \
-u http://nodei16/contrail-install-packages_2.21-105~juno_all.deb \
--fab-artifact /tmp/contrail-fabric-utils.tar.gz contrail-test-ci
- Docker build with combination of artifacts and git refs
$ ./install.sh docker-build --test-ref 1890b36 --ci-artifact /tmp/contrail-test-ci.tar.gz \
-u http://nodei16/contrail-install-packages_2.21-105~juno_all.deb \
--fab-artifact /tmp/contrail-fabric-utils.tar.gz contrail-test-ci
- Build and export to /export/docker/contrail-test-ci/
./install.sh docker-build --test-ref 1890b36 --ci-artifact /tmp/contrail-test-ci.tar.gz \
-u http://nodei16/contrail-install-packages_2.21-105~juno_all.deb \
--fab-artifact /tmp/contrail-fabric-utils.tar.gz contrail-test-ci -e /export/docker/contrail-test-ci/
- Load docker image from /export/docker/contrail-test-ci/contrail-test-ci-juno-2.21-105.tar.gz
$ docker load < /export/docker/contrail-test-ci/contrail-test-ci-juno-2.21-105.tar.gz
-
Execute docker container
it run contrail-test ci tests and log the console. The console output may be captured at later point by running "docker logs [-f] ".
$ docker run -v /opt/contrail/utils/fabfile/testbeds/testbed.py:/opt/contrail/utils/fabfile/testbeds/testbed.py \
-t contrail-test-ci-juno:2.21-105
-
Execute the container with logs saved in specific location
The logs will be saved under /export/logs/contrail-test-ci/ on docker host.
$ docker run -v /opt/contrail/utils/fabfile/testbeds/testbed.py:/opt/contrail/utils/fabfile/testbeds/testbed.py \
-v /export/logs/contrail-test-ci/:/contrail-test/logs -t contrail-test-ci-juno:2.21-105
This is a helper script to manage (run/rebuild/list) contrail-test/contrail-test-ci container
This script can be used to cleanup tenant data. This can be run after testrun to make sure tenant is clean.
$ ./tenant_cleaner.py -h
usage: tenant_cleaner.py [-h] [--user USER] [--password PASSWORD]
[--auth-url AUTH_URL] [--auth-tenant AUTH_TENANT]
[--ip IP] [--port PORT]
tenant [tenant ...]
Cleanup provided tenants by removing all objects in it.It expect openstack
credentials from OS_ environment variables in openrc format.The credentials
can be provided as commandline arguments also
positional arguments:
tenant List of tenants to be cleaned up Note that, the user
must have access to the tenants listed
optional arguments:
-h, --help show this help message and exit
--user USER Openstack user, if not provided, try to get it from
environment variable "OS_USERNAME" with default user
as "admin"
--password PASSWORD Openstack password
--auth-url AUTH_URL Openstack auth url, by default try to get from
environment variableOS_AUTH_URL with default of
http://127.0.0.1:5000/v2.0/
--auth-tenant AUTH_TENANT
Openstack tenant to connect to
--ip IP IP Address of the controller
--port PORT Port of the controller
$ ./tenant_cleaner.py --ip 10.204.217.88 tenant1 tenant2 tenant3
No objects found in tenant tenant1
No objects found in tenant tenant2
Deleted: http://10.204.217.88:8082/security-group/f633163c-2c96-4722-94b2-3a125e981492
No nova objects found in tenant tenant3