-
Notifications
You must be signed in to change notification settings - Fork 67
Working with Google Test
We use Google Test to test the public interface of CARLsim. TODO: How to install on Windows.
Download the Google Test 1.7 zip file from [here] (https://code.google.com/p/googletest/). From the command line, navigate to the directory containing the zip file, unzip the file, and enter the unzipped Google Test directory:
$ unzip gtest-1.7.0.zip
$ cd gtest-1.7.0
Run 'configure' and 'make' to configure and build the Google Test libraries:
$ ./configure
$ make
This should result in Google Test libraries being compiled and built. Usually at this point one would type something like 'sudo make install' but the developers of Google Test have disabled this feature have called it dangerous. So you have to install it manually. You can install it in your home directory or somewhere accessible to all users like /opt or /usr/local. If you are going to install it somewhere globally, you will require admin/root privileges. Let's assume you want to install Google Test in /opt. To do this, create a directory called 'gtest' and create a folder named 'lib' in this folder.
$ sudo mkdir /opt/gtest
$ sudo mkdir /opt/gtest/lib
Then copy the 'include' folder to the new 'gtest' folder and then copy all files inside 'lib/.lib' to the 'gtest/lib'
$ sudo cp -r include /opt/gtest
$ sudo cp -r lib/.libs/* /opt/gtest/lib
Finally, add the following two lines to your .bashrc file that point CARLsim to the relevant Google Test files and enable colors in the Google Test output, respectively.
$ export GTEST_DIR=/opt/gtest
$ export GTEST_COLOR=1
Now the makefiles used by CARLsim will be able to find the Google Test source files and compile tests using them. Currently, the CARLsim test framework rebuilds the Google Test library locally. This isn't optimal but doesn't take very long either. There are plans to change the CARLsim test framework to use the Google Test libraries directly in the future.
Documentation for the Google Test framework can be found here.
The CARLsim Google Tests should be run in the CARLSIM_SRC_DIR/carlsim/test directory. The regression suite is compiled and run via:
$ make carlsim_tests
$ ./carlsim_tests
To see a list of all supported flags and their usage, run:
$ ./carlsim_tests --help
By default, Google Test runs all the tests the user has defined.
If you want to run only a subset of tests, you can use the GTEST_FILTER environment variable or simply use the --gtest_filter flag to filter a string.
Examples:
# run everything in the CUBA test case
$ ./carlsim_tests --gtest_filter=CUBA.*
# run any test whose full name contains either "Null" or "Constructor"
$ ./carlsim_tests --gtest_filter=*Null*:*Constructor*
# run all tests that do NOT have the string "Death" in its name
$ ./carlsim_tests --gtest_filter=-*Death*
By default, Google Test bases the random seed on the current time. However, you can also explicitly specify the seed using environment variable GTEST_RANDOM_SEED or the flag --gtest_random_seed, where the value must be an integer between 0 and 99999. Seed value 0 calculates the seed from the current time.
$ ./carlsim_tests --gtest_random_seed=42
You can repeatedly run certain tests by using the environment variable GTEST_REPEAT or the flag --gtest_repeat.
# repeat CUBA test case 10 times
$ ./carlsim_tests --gtest_filter=CUBA.* --gtest_repeat=10
# repeat all tests 1000 times, stopping at first failure
# if you run this in a debugger, gtest will drop into the debugger
# and you can inspect variables and stacks
$ ./carlsim_tests --gtest_repeat=1000 --gtest_break_on_failure
Both CARLsim (in testing mode) and Google Test redirect some output from its usual destination so it's recommended you use cerr or fprintf(stderr,...) for outputting things to the console/terminal.
To use cerr:
#include <iostream.h>
std::cerr << "Here is a test message." << std::endl;
To use fprintf(stderr,...):
#include <stdio.h>
int val=8;
fprintf(stderr,"Some var has value %d\n",val);
TODO: How to do this in Windows/MS Visual Studio
For the most part, Google Tests automatically detects the addition of a new test file to the Makefile. First navigate from CARLSIM_SRC_DIR to 'carlsim/test' where you will be running the tests. Assuming you are already in CARLSIM_SRC_DIR.
$ cd carlsim/test
Let's say you are testing a new feature you added called foo. Create a file named 'foo.cpp' that includes the gtest.h, carlsim_tests.h, and carlsim.h header files. Then create individual test classes (named 'FOO' in below example) with specific test cases (named 'FooTestX' in below example). Below is an example that may be used as an example. It will be instructive to look at other test .cpp files in the carlsim/test directory as well as they may take into account additional features (like compatibility with Windows).
#include "gtest/gtest.h"
#include "carlsim_tests.h"
#include <carlsim.h>
// tests the new foo functionality in CARLsim
TEST(FOO, FooTest1) {
EXPECT_TRUE(anImportantBoolean);
EXPECT_TRUE(anImportantBoolean);
EXPECT_EQ(anImportantComparison);
}
TEST(FOO, FooTest2) {
EXPECT_TRUE(anImportantBoolean);
EXPECT_TRUE(anImportantBoolean);
EXPECT_EQ(anImportantComparison);
}
TEST(FOO, FooTest3) {
EXPECT_TRUE(anImportantBoolean);
EXPECT_TRUE(anImportantBoolean);
EXPECT_EQ(anImportantComparison);
}
Finally, you need to add the name of your new test .cpp to the CARLSIM_SRC_DIR/carlsim/test/Makefile to the 'carlsim_tests_cpps' Makefile variable. To add foo.cpp to this environment variable, the modified entry would look like:
carlsim_tests_cpps := stdp stp cuba coba interface spike_mon conn_mon group_mon \
core connect carlsim_tests_common spike_gen spike_counter poiss_rate foo
After that you can run
$ make carlsim_tests
in the CARLSIM_SRC_DIR/carlsim/test directory to test all the tests (including your new test) or just run:
$ make carlsim_tests --gtest_filter=FOO.*
To run/test only your new class of tests for the foo feature.