figure out a way to run multiple playbooks against the same VM (sequentially)

[nusenu]

The idea is to
have a vagrant VM setup once and then run multiple playbooks against it sequentially:

• guard
• exit
• .
  to reduce required network traffic (package installation) and VM creation overhead.

This is probably against the design of how kitchen is supposed to be used but I think it is the only way to make ~20 new testcases feasible to run in a reasonable timeframe.

We will still have multiple test suites for things where the VM is different like multiple IPs, IPv6.

[Lirt]

Maybe there is a way how to not destroy VM after first test and use it afterwards. Yes this is against the testing design, since changes from last test will stay. In some cases it doesn't matter - eg. installation of tor package is OK, but another test case with same OS is installing tor again (as you mentioned). I will try it when I find a time.

However we could also test all cases simultaneously in some cloud provider if we find a sponsor. Test Kitchen has plugins for many cloud providers and this way we can even integrate it with Travis CI for example.

Local Vagrant tests are great for development purposes, you can run converge on one test case and don't need to destroy the machine all the time. But doing more than 10 tests is already really time consuming.

[nusenu]

thanks for picking this up!

I'm mostly using converge after an initial create, and I'm aiming to cover more test cases with a single converge run.