Factory Boy of poor

Factory Boy of poor

Published Aug. 13, 2015 in Development - Last update on Aug. 13, 2015.

In some projects when I write unit test, I sometime need to have fixtures: a set of objects ready to use. As a Djangonaute, I used to deal with Factory Boy and django fixtures system but sometimes I can't use neither and for small usage list comprehesion or dict comprehension make the job.

For those who are not aware, Python, since 2.6, has comprehension list written like this:

>>> people = ['Ji', 'Hai', 'Joe']
>>> [p for p in people if p.startswith('J')]
['Ji', 'Joe']

And since Python 2.7, there's also dictionnary comprehension:

>>> people = ['Ji', 'Hai', 'Joe']
>>> {i: p for i,p in enumerate(people)}
{0: 'Ji', 1: 'Hai', 2: 'Joe'}

I tried with fixture creation to take the maximum of this programming technique.

Fast random creation

Imagine you have the following class and want to create a lot instances quickly:

class People(object):
    def __init__(self, firstname, lastname):
        self.firstname = firstname
        self.lastname = lastname

    def __repr__(self):
        return '<People: %s>' % str(vars(self))

Create many people

I wanna create several ones, I will iterate over firstnames and choose randomly a lastname in comprehension list:

>>> firstnames = ['Jo', 'Roberto', 'Olivio', 'Ronaldo', 'Augusto']
>>> lastnames = ['Da Silva', 'Monthe', 'Dubois', 'Ben Allah', 'Zero']
>>> [People(firstname=f, lastname=random.choice(lastnames)) for f in firstnames]
[<People: {'lastname': 'Da Silva', 'firstname': 'Jo'}>,
 <People: {'lastname': 'Zero', 'firstname': 'Roberto'}>,
 <People: {'lastname': 'Da Silva', 'firstname': 'Olivio'}>,
 <People: {'lastname': 'Zero', 'firstname': 'Ronaldo'}>,
 <People: {'lastname': 'Ben Allah', 'firstname': 'Augusto'}>]

I want a unique ID for everyone

 Play with iterator is pretty simple in Python, just add enumerate allows to have an index for every first name.

>>> [People(firstname=f, lastname=random.choice(lastnames), id=i)
... for i,f in enumerate(firstnames)]
[<People: {'lastname': 'Monthe', 'id': 0, 'firstname': 'Jo'}>,
 <People: {'lastname': 'Monthe', 'id': 1, 'firstname': 'Roberto'}>,
 <People: {'lastname': 'Da Silva', 'id': 2, 'firstname': 'Olivio'}>,
 <People: {'lastname': 'Ben Allah', 'id': 3, 'firstname': 'Ronaldo'}>,
 <People: {'lastname': 'Zero', 'id': 4, 'firstname': 'Augusto'}>]

I want gender defined by name

I've update my first name list with female names and want to add a gender attribute to my People object. Gender will be define by firstname, the ones which ends with a 'o' will be male ('M') and others female ('F'). 

>>> firstnames = ['Jo', 'Roberto', 'Olivio', 'Ronaldo', 'Augusto',
...               'Julia', 'Angelica', 'Louisa', 'Laura']
>>> [People(firstname=f, lastname=random.choice(lastnames), id=i,
...         gender=('M' if f.endswith('o') else 'F'))
...  for i,f in enumerate(firstnames)]
[<People: {'lastname': 'Da Silva', 'id': 0, 'firstname': 'Jo', 'gender': 'M'}>,
 <People: {'lastname': 'Da Silva', 'id': 1, 'firstname': 'Roberto', 'gender': 'M'}>,
 <People: {'lastname': 'Dubois', 'id': 2, 'firstname': 'Olivio', 'gender': 'M'}>,
 <People: {'lastname': 'Zero', 'id': 3, 'firstname': 'Ronaldo', 'gender': 'M'}>,
 <People: {'lastname': 'Da Silva', 'id': 4, 'firstname': 'Augusto', 'gender': 'M'}>,
 <People: {'lastname': 'Da Silva', 'id': 5, 'firstname': 'Julia', 'gender': 'F'}>,
 <People: {'lastname': 'Monthe', 'id': 6, 'firstname': 'Angelica', 'gender': 'F'}>,
 <People: {'lastname': 'Ben Allah', 'id': 7, 'firstname': 'Louisa', 'gender': 'F'}>,
 <People: {'lastname': 'Dubois', 'id': 8, 'firstname': 'Laura', 'gender': 'F'}>]

 Most of above usages operate on each element of the iterator, we use elements to produce a more complex list. Next chapter we'll play with serveral list looping and conditions.

Create matrix from comprehension

The previous example was great but doesn't match with matrixes. Now we're going to create a list of dictionnary, each dict wll represent an object with color, form and more. It looks like this:

{'color': 'cyan', 'form' 'square'}

We create firstly lists of colors and forms:

>>> colors = ['red', 'green', 'yellow']
>>> forms = ['circle', 'square', 'triangle']

Create with every forms and every colors

With double list comprehension, we can create a list of all combinaison of forms and color:

>>> [{'color': c, 'form': f} for c in colors for f in forms]
[{'color': 'red', 'form': 'circle'},
 {'color': 'red', 'form': 'square'},
 {'color': 'red', 'form': 'triangle'},
 {'color': 'green', 'form': 'circle'},
 {'color': 'green', 'form': 'square'},
 {'color': 'green', 'form': 'triangle'},
 {'color': 'yellow', 'form': 'circle'},
 {'color': 'yellow', 'form': 'square'},
 {'color': 'yellow', 'form': 'triangle'}]
>>> 3*3 == len([{'color': c, 'form': f} for c in colors for f in forms])
True

We have 9 distinct objects.

Add a size parameter

We want the same thing but with a new parameter 'size'. Size is supposed to be an integer between 1 and 3. By simply add a for loop to enlarge matrix.

>>> [{'color': c, 'form': f, 'size': s}
...  for c in colors
...  for f in forms
...  for s in range(1,4)]
[{'color': 'red', 'form': 'circle', 'size': 1},
 {'color': 'red', 'form': 'circle', 'size': 2},
 {'color': 'red', 'form': 'circle', 'size': 3},
 {'color': 'red', 'form': 'square', 'size': 1},
 {'color': 'red', 'form': 'square', 'size': 2},
 {'color': 'red', 'form': 'square', 'size': 3},
 {'color': 'red', 'form': 'triangle', 'size': 1},
 {'color': 'red', 'form': 'triangle', 'size': 2},
 {'color': 'red', 'form': 'triangle', 'size': 3},
 {'color': 'green', 'form': 'circle', 'size': 1},
 {'color': 'green', 'form': 'circle', 'size': 2},
 {'color': 'green', 'form': 'circle', 'size': 3},
 {'color': 'green', 'form': 'square', 'size': 1},
 {'color': 'green', 'form': 'square', 'size': 2},
 {'color': 'green', 'form': 'square', 'size': 3},
 {'color': 'green', 'form': 'triangle', 'size': 1},
 {'color': 'green', 'form': 'triangle', 'size': 2},
 {'color': 'green', 'form': 'triangle', 'size': 3},
 {'color': 'yellow', 'form': 'circle', 'size': 1},
 {'color': 'yellow', 'form': 'circle', 'size': 2},
 {'color': 'yellow', 'form': 'circle', 'size': 3},
 {'color': 'yellow', 'form': 'square', 'size': 1},
 {'color': 'yellow', 'form': 'square', 'size': 2},
 {'color': 'yellow', 'form': 'square', 'size': 3},
 {'color': 'yellow', 'form': 'triangle', 'size': 1},
 {'color': 'yellow', 'form': 'triangle', 'size': 2},
 {'color': 'yellow', 'form': 'triangle', 'size': 3}]
>>> 3*3*3 == len([{'color': c, 'form': f, 'size': s} for c in colors for f in forms for s in range(1,4)])
True

All combinaisons of forms, colors and sizes, that make 27 objects.

Make exclusions

Exclusions could be made with an if statement, below we exclude all objects other than red. Like in classic for loop usage, it's better to test condition soon as possible, else false condition will be tested several times.

>>> [{'color': c, 'form': f, 'size': s}
...  for c in colors if c == 'red'
...  for f in forms
...  for s in range(1,4)]
[{'color': 'red', 'form': 'circle', 'size': 1},
 {'color': 'red', 'form': 'circle', 'size': 2},
 {'color': 'red', 'form': 'circle', 'size': 3},
 {'color': 'red', 'form': 'square', 'size': 1},
 {'color': 'red', 'form': 'square', 'size': 2},
 {'color': 'red', 'form': 'square', 'size': 3},
 {'color': 'red', 'form': 'triangle', 'size': 1},
 {'color': 'red', 'form': 'triangle', 'size': 2},
 {'color': 'red', 'form': 'triangle', 'size': 3}]

All forms and all size only in red.

Generate IDs

In peoples example, we generate IDs by enumerate firstnames. It is not possible here because there isn't any unique attribute, and if we try to apply on color for example, all objects with same color will have the same ID. ID will be generate during each dictionnary creation, so I write a simple Counter class which increment each time we call it:

>>> class Counter(object):
...    value = 0
...    def __call__(self):
...        self.value += 1
...        return self.value
...
>>> count = Counter()

>>> [{'color': c, 'form': f, 'size': s, 'id': count()}
...  for c in colors
...  for f in forms
...  for s in range(1,4)]
[{'color': 'red', 'form': 'circle', 'id': 1, 'size': 1},
{'color': 'red', 'form': 'circle', 'id': 2, 'size': 2},
{'color': 'red', 'form': 'circle', 'id': 3, 'size': 3},
...
{'color': 'yellow', 'form': 'triangle', 'id': 25, 'size': 1},
{'color': 'yellow', 'form': 'triangle', 'id': 26, 'size': 2},
{'color': 'yellow', 'form': 'triangle', 'id': 27, 'size': 3}] 

Generate Turkish Ikea names

For fun we'll add an unreadable name to objects. I create a word with random letter like Swedish:

>>> count = Counter()
>>> [{'color': c, 'form': f, 'size': s, 'id': count(),
...  'name': ''.join([chr(random.choice(range(97, 123))) for i in range(10)])}
...  for c in colors
...  for f in forms
...  for s in range(1,4)]
[{'color': 'red', 'form': 'circle', 'id': 1, 'name': 'lyzzhhidtw', 'size': 1},
{'color': 'red', 'form': 'circle', 'id': 2, 'name': 'idtvxrlwns', 'size': 2},
{'color': 'red', 'form': 'circle', 'id': 3, 'name': 'btfrcupckl', 'size': 3},
{'color': 'red', 'form': 'square', 'id': 4, 'name': 'swhtcrzyqg', 'size': 1},
{'color': 'red', 'form': 'square', 'id': 5, 'name': 'mauxdhvqiq', 'size': 2},
{'color': 'red', 'form': 'square', 'id': 6, 'name': 'swykchdvuc', 'size': 3},
...
{'color': 'yellow', 'form': 'triangle', 'id': 26, 'name': 'jbdfaansmt', 'size': 2},
{'color': 'yellow', 'form': 'triangle', 'id': 27, 'name': 'ungogbpmnv', 'size': 3}]

Sort all in a dictionnary

I want to sort my dictionnaries in a big other one. Keys will in a small human readable format like 'tg1' for 'triangle', 'green' and size '1'. The solution is only to use dictionnary comprehension:

>>> {'{}{}{}'.format(f[0],c[0],s):
...      {'color': c, 'form': f, 'size': s, 'id': count(), 'name': ''.join([chr(random.choice(range(97, 123))) for i in range(10)])}
...  for c in colors
...  for f in forms
...  for s in range(1,4)}
{'cg1': {'color': 'green', 'form': 'circle', 'id': 1, 'name': 'hwufaekchm', 'size': 1},
 'cg2': {'color': 'green','form': 'circle', 'id': 2, 'name': 'xlutvaovff', 'size': 2},
...
 'ty3': {'color': 'yellow', 'form': 'triangle', 'id': 27, 'name': 'eagtezswje', 'size': 3}}

I easily format dict's keys myself and add the previous dictionnaries as values.

Conclusion

List comprehension is one strenght of Python, it avoids to write nested for loop in many case. Being more efficient in performance and readability than map, filter an co I encourage everyone to use this syntax. Do not forget there are:

  • list compresion - [i for i in mylist]
  • dict comprehension - {k: v for k, v in myitems}
  • set comprehension - {i for i in mylist}

Reference and link

Tags :
Discussions : 1 comment

Previous entry

Related entries

Similar entries

Next entry

Comments

Post your comment

Comment as . Log out.

Made during geeking nights with Django Zinnia