Geek asks and answers

Creating a Python unit test that never runs in parallel

tl; dr - I want to write a Python function unittestthat deletes a file, runs a test and restores the file. This causes race conditions, because it unittestperforms several tests in parallel, and deleting and creating a file for one test ruined the other tests that occur simultaneously.

Long concrete example:

I have a Python module named converter.pyand it has related tests in test_converter.py. If there is a file with a name config_custom.csvin the same directory as converter.py, then a custom configuration will be used. If there is no custom CSV configuration file, then the converter.pydefault configuration is built in.

I wrote a unit test using unittestfrom the Python 2.7 standard library to test this behavior. unit test will setUp()rename config_custom.csvto wrong_name.csv, then it will run the tests (hopefully using the default configuration), and then to tearDown()it will rename the file as it should be.

Problem: Python batch tests run in parallel, and I got terrible race conditions. The file config_custom.csvwill be renamed in the middle of other unit tests in a non-deterministic way. This will lead to at least one error or error in about 90% of the cases when I ran the entire test suite.

The ideal solution is to say unittest: DO NOT run this test in parallel with other tests, this test is special and needs to be completely isolated.

My job is to add an optional argument to a function that searches for configuration files. The argument is passed only by test case. It ignores the configuration file without deleting it. Actually deleting the test file is more elegant, here is what I really want to check.

+3
source share
3 answers

-, , parallelism, . , Python Fabric ( -P):

from fabric.api import *

def runs_in_parallel():
    pass

@serial
def runs_serially():
    pass

, , , , . , , .

, , .

, "converter.py" , , , , .

, , , / , , .

+1

, config_custom.csv . config_custom_<nonce>.csv, .

config_custom_*.csv, .

+1

The best testing strategy would be to make sure that you are testing disjoint datasets. This will circumvent any race conditions and simplify the code. I would also make fun of / openor if you use a context manager. This will allow you to fake the event so that the file does not exist.__enter____exit__

0
source

More articles:


All Articles