subprocess.Popen aped

subprocess.Popen aped

Published July 30, 2016 in Development, Tests and quality - Last update on July 30, 2016.

Lately, I worked on Django DBBackup, I was remaking the database command system: classes allowing launching of backup or restore commands through subprocess.Popen. There was no unit tests and major part of them are based on command line but contrary to what I though mock out Popen took me a some time.

What's the code to dry ?

    def run_command(self, command, stdin=None, env=None):
        cmd = shlex.split(command)
        stdout = SpooledTemporaryFile(max_size=10 * 1024 * 1024)
        stderr = SpooledTemporaryFile(max_size=10 * 1024 * 1024)
        full_env = self.env.copy()
        full_env.update(env or {})
            process = Popen(cmd, stdin=stdin, stdout=stdout, stderr=stderr,
            if process.poll():
                raise exceptions.CommandConnectorError(
                    "Error running: {}\n{}".format(command,
            return stdout, stderr
        except OSError as err:
            raise exceptions.CommandConnectorError(
                "Error running: {}\n{}".format(command, str(err)));

I want to pass all those lines and do not be affected by Popen's behavior, or in other words: make real unit test without to have the underlying system as a factor.

Are you a n00b ? There is true command...

true is Unix-Like and the OS whose we do not pronounce the name (Win****) may not have true/false/echo commands. More precisely, I must ape process.wait() and process.poll() behaviors. Below their documentation:


Wait for child process to terminate. Set and return returncode attribute. 


Check if child process has terminated. Set and return returncode attribute.

And the million of SO answers and blog post about ?

Most of reference about this problem have ways that I don't like:

  • Some configure Mock object manually inside tests (cf StackOverflow answer) and I want to configure only in decorator.
  • Other will use Popen.communicate(), I use Popen.poll(), the issue is relatively the same but they use autospec parameter.

My method is the following:

@patch('dbbackup.db.base.Popen', **{
    'return_value.wait.return_value': True,
    'return_value.poll.return_value': 0,
class PgDumpConnectorRunCommandTest(TestCase):
    def test_run_command(self, mock_popen):
        connector = PgDumpConnector()
        self.assertEqual(mock_popen.call_args[0][0][0], 'pg_dump');

I mock out Popen (only where I use it obviously) and with the keyword utility I define their attributes. The sublety is here, with classic form of argument (arg=value) I couldn't chain mock of attributes, the maxium would be 1 attribute and after we would set a MagicMock for customize subattributes.

Here we specify that the MagicMock returned by Popen.__init__ will have a wait (and poll) attribute, and this attribute will return True when launched.

Yes, even in Python I love One-Line.



No comments yet.

Post your comment

Comment as . Log out.