Replies: 4 comments 3 replies
-
What exactly prevents your tests from being independent?
and your test is absolutely independent from every other test. And if it's too wordy for you, just create Issue 1:
Do not agree. The test is responsible only for correct assertions. Of course, if the test has messed up the system, the system must be restored. But a dirty database it's just an artifact, that can be investigated after tests, it's not a mess. issue 2:Just as you said, it's the same :) Issue 3:
it looks like you haven't checked the code :)
Just use it deliberately) Issue 4Even though I like fixtures and I suggested using them, I don't agree that it's better. It's just a matter of taste and style, but I prefer to have a choice:) Example 1: Refactor clear_db fixtureSo, what is actual change? Instead of cleaning up before we'll do clean-up after the test. Neither from the user's point of view nor from a performance point it will improve the situation. But, if we'll do cleaning-up before the test(like we are doing right now), we can remove the Instead, I suggest using an extra fixture, Example 2: Writing some good practices for unit test different implementations+1
But as I currently working on speeding up CKAN tests (actually I'm waiting till #6335 merged, and then I'll be able to continue), I can suggest a better way in future |
Beta Was this translation helpful? Give feedback.
-
Hey! Thanks for answering :) Issue 1 & 2The design principle of Unit Testing approach is that each test should clean up the database after execution. As per pytest documentation:
But our current approach is: We both agree that we do not want our test-suite turns into a fragile structure where all the consequent tests will fail if one of them accidentally won't clean the database. It's just a matter of who is responsible for cleaning. pytest approach (and unit testing frameworks) assign that responsibility to the test creating/editing the environment (teardown methods). Which makes sense since: why I should introduce in my test B a cleanup for logic that other methods may have introduced or not? We do it in in fixtures like
And an example of their documentation: @pytest.fixture
def sending_user(mail_admin):
user = mail_admin.create_user()
yield user
admin_client.delete_user(user) Issue 3The code for Re-checking my examples I noticed that it calls Note: is it safe to do a refactor and only use The problem with using deliberately
Same applies if suddenly I add a new extension to my work and some tests now requires database migrations to work. Would I add Again, it is a matter on who clean but it is not a good software pattern (and it is not expected) to make classes/code deal with other's mess. (Unless you're writting a garbage collector of course ;)) New fixture: with_dbIIUC I like the overall idea, it's something similar to what Example 2Thanks for the tip on the DB migration, I will do some refactors in my code :) |
Beta Was this translation helpful? Give feedback.
-
I think we need more fixtures with different responsibilities. Like (of course, names can be better 😄 ):
Right now we have a lack of expressiveness and that makes tests more stiff and uncertain. |
Beta Was this translation helpful? Give feedback.
-
Clarification: Let's not take my Yes, databases are artifacts and I agree with some kind of feature to explicitly request for it (like As pytest documentation suggest, if a fixture creates a User (or a connection), the fixture should remove it after using it. What's the goal of creating Some example of what it's confusing to me: class TestTests:
def test_a(self):
factories.Sysadmin()
class TestTests2:
def test_b(self):
factories.Sysadmin() The previous example will:
To fix it I need to execute @pytest.mark.usefixtures('clean_db')
class TestTests:
def test_a(self):
factories.Sysadmin()
@pytest.mark.usefixtures('clean_db')
class TestTests2:
def test_b(self):
factories.Sysadmin() That would run okay all the times but it requires running a If we do something like: @pytest.mark.usefixtures('with_db')
@pytest.mark.usefixtures('do_not_leave_a_mess')
class TestTests:
def test_a(user_factory):
user = user_factory(name="xxx")
@pytest.mark.usefixtures('with_db')
@pytest.mark.usefixtures('do_not_leave_a_mess')
class TestTests2:
def (user_factory):
user = user_factory(name="xxx") We have a more expressive fixture but again, A more clean approach for me will be something like: @pytest.mark.usefixtures('with_db')
class TestTests:
def test_a(user_factory):
user = user_factory(name="xxx")
@pytest.mark.usefixtures('with_db')
class TestTests2:
def (user_factory):
user = user_factory(name="xxx")
@pytest.fixture
def user_factory():
_user = User(name="Susan")
yield _user
_user.delete() # or something to clean it from the database This way:
|
Beta Was this translation helpful? Give feedback.
-
Hey all! 👋
I have been spending a few days working on Unit Test for a project and I would like to open the floor for a discussion about our approaches and experiences. The goal is to discuss and maybe, if needed, come up with improvements to our tools to do Unit Testing.
The main issue I found is that our unit tests are not really units. When trying to write big suits of Unit Tests it gets hard to keep each test/class/module independent.
Issue 1:
Our main factories write to the database and don't clean it after execution. Even when we have
clean_db
fixture, each test is resposible for cleaning it's environment before executing in case other tests messed up with the database. In a Unit Testing approach each test should be responsible for cleaning their own mess.Issue 2:
Adding tables to the database have the same side effect than the previous issue. We have the fixture
with_plugins
to load them while testing but if we need to initialize the database to add tables and columns then we depend on other tests to callreset_db
so they have a clean DB before run.Issue 3:
Issue 1 and 2 cause an indiscriminate use of fixtures like
clear_db
that ends up in an overhead of the database. Also, withclean_db
we are deleting and initializing the database when most of the time we just need to cleanup the tables.Issue 4:
We are using fixtures only to write tools that allows us to clean/modify the environment and relying on factories for ORM objects. A better approach would be to use fixtures to create ORM objects as well. (something similar to django-pytest maybe?) Some work is being done in: #6335
Overall the experience to write tests it's okay but it gets messy when the implementations starts interacting with several extensions.
Possible solutions:
The approach for unit testing and fixture building should be to isolate each function and to cleanup the environment after each execution.
Example 1: Refactor
clear_db
fixtureThis fixture will delete all the data from the tables once the execution of the test finishes
Example 2: Writing some good practices for unit test different implementations
test.ini
file an initialize the database once before executing the test or create fixtures for each extension to set/clean the database? (Maybe each extensions can provide a fixtures to encapsulate implementation details to other users?)Let me know your thoughts!
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions