00:00:00

pytest

recommendations, basic packages for testing in Python and Django

@avallbona

Notes

pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

whoami/whoarewe

  • Andreu Vallbona
  • Bachelor degree in computer science (UOC)
  • Professional programmer since 1995
  • Web developer at APSL, Mallorca
  • https://www.linkedin.com/in/andreu-vallbona-plazas-b0b58720/
  • What we do in APSL:
    • web development
    • systems engineering - devops
    • mobile apps
    • data science
    • consulting and formation

Notes

pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

advantages of testing

ensure the quality of the code

Peace of mind when making changes

facilitate Python and / or Django version upgrades

ease in database system changes (e.g.: from mysql to postgres)

Notes

- with the tests we get many tickets opened by the client (which uses an api) are not really bugs but errors on his part

- we have accomplished several database system changes without problem
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

basic recommendations 1

Do not overly sophisticate the tests

self-contained and independent tests

use fixtures

use parametrize

overwrite settings by default

Notes

- Do not over-sophisticate the logic of the tests, keep them as simple as possible. If they become too sophisticated they are difficult to return later by oneself or another developer. They are difficult to maintain. Try to test only one function at a time

- The tests have to be self-contained and independent of any other test, when they are run concurrently we can not know in which order the tests will be executed in each thread

- Avoid the copy / paste of values for the tests using fixtures, own or custom

- Avoid also copy / paste code using the parametrize, allows multiple calls with different values to the same test function

- Overwrite settings by default. E.g .: overwrite the email sending backend in the test environment

pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

basic recommendations 2

test database in local and / or memory

Be careful with the signals of Django

mocking external services

concurrent execution

review PEP8

Notes

- Importance of database in local and / or in a memory tablespace to accelerate the execution of tests

- Avoid execution of * signals * of Django in the test environment

- Mocking, it helps us to define the different types of response that an external service can give us. E.g .: stripe, without the need to connect to them

- Running tests concurrently is convenient when we have a lot of tests

- Beware of django-constance, in test environment change the backend

pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

fixture

fixtures are objects that we predefine and can then be used by our test functions

you can define different scopes::

  • function level: runs once per test
  • class level: runs once per test class
  • module level: runs once per per module
  • session level: runs once per session

example of fixture:

1 import smtplib
2 import pytest
3 
4 @pytest.fixture(scope="module")
5 def smtp():
6     smtp = smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
7     yield smtp  # provide the fixture value
8     print("teardown smtp")
9     smtp.close()

Notes

- key concept in pytest

- very useful to avoid code repetition

- new way to execute code from **tearUp** i **tearDown**
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-django

pytest plugin that gives us a whole series of * helpers * and * fixtures * very useful to test projects implemented in Django

among others we can highlight:

  • django_db - gives access to the database
  • rf - request factory
  • client - test client
  • admin_client - authenticated test client
  • admin_user - authenticated superuser
  • settings - access to the django settings
  • mailoutbox - mailbox where to test the sending of emails

Notes

- django_db - marker that give access to the database to a function or class, it is executed in a transaction and it restores the state of the database at the end of the test

- rf - useful for views tests (views)

- client i admin_client - useful for API tests

- scope of the fixtures, determines when a fixture is executed

- autoinjection of a fixture in each test with autouse

pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

model-mommy

plugin that helps us to easily create fixtures based on django models very easily

  • field values are generated automatically
  • content random of the fields but can be specified individually
  • You can create objects:
    • only in memory ( mommy.prepare ) useful for unit tests model methods
    • persistent ( mommy.make ) useful for integration tests
  • you can define relationships between objects (fk, m2m)
  • you can define recipes, which are like templates
  • sequence fields can be defined

Notes

* avoid having to maintain static fixtures

* automatically fill in the required fields with the corresponding data type, you can specify the values you want to specify or let them be generated randomly

* in foreign keys you can generate parent objects automatically. example Dog (child) <- Owner (father), generating a dog (child) automatically generates an owner (father)

* comment _ quantity per generate N elements d'un model

* recipes are like templates from which we can generate instances

* sequences can be generated in the recipes to avoid problems with the fields that their value has to be unique

* is similar to factory_boy but simpler and easier to use, in the end there is a link comparing the two, model-mommy is designed for Django, Factory_boy is more generic

pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

model-mommy (2)

given the following model

 1 class Suscripcion(models.Model):
 2     uuid = models.UUIDField(unique=True, default=uuid4, editable=False, verbose_name=_('Uuid'))
 3     administrador = models.ForeignKey(
 4         Administrador, verbose_name=_('Administrador'), related_name='suscripciones_administrador'
 5     )
 6     status = models.CharField(_('Estado suscripción'), max_length=9, default=TRIAL_STATUS, choices=STATUS_CHOICES)
 7     trial_end = models.DateTimeField(verbose_name=_('Fin periodo prueba'), blank=True, null=True)
 8     comunidades_contratadas = models.IntegerField(
 9         verbose_name=_('Comunidades contratadas'), default=1,
10     )
11     viviendas_contratadas = models.IntegerField(
12         verbose_name=_('Viviendas contratadas'), default=0, null=True
13     )
14     garajes_contratados = models.IntegerField(
15         verbose_name=_('Garajes contratados'), default=0, null=True
16     )
17     locales_contratados = models.IntegerField(
18         verbose_name=_('Locales contratados'), default=0, null=True
19     )
20     trasteros_contratados = models.IntegerField(
21         verbose_name=_('Trasteros contratados'), default=0, null=True
22     )
23 
24     class Meta:
25         verbose_name = _('Suscripción')
26         verbose_name_plural = _('Suscripciones')

Notes

- example of model with several fields
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

model-mommy (3)

we could generate an instance of it with:

1 @pytest.fixture()
2 def trial_subscription_obj(self, admin_obj, community_obj, precios):
3     return mommy.make(
4         Suscripcion,
5         administrador=admin_obj,
6         status=Suscripcion.TRIAL_STATUS,
7         trial_end=SubscriptionBO()._set_trial_end()
8     )

and then use it in a test:

1 def test_expire_trial_subscription(self, trial_subscription_obj):
2     sus = SubscriptionBO()
3     sus.expire_subscription(trial_subscription_obj)
4 
5     # check for subscription status
6     subs = Suscripcion.objects.get(pk=trial_subscription_obj.pk)
7     assert subs
8     assert subs.status == subs.EXPIRED_STATUS

Notes

- from the class of the object we have generated an instance only with the specific fields desired

- unspecified fields are generated with random content

- for the construction of the instance we can rely on other fixtures
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

model-mommy (4)

example of sequence fields

1 from model_mommy.recipe import seq
2 
3 admins = mommy.make(Administrador, nombre=seq('Test User '), _quantity=3)

that would generate 3 instances of the class Administrator and the value of the name field would be respectively

1 In [5]: for it in admins:
2    ...:     print(it.nombre)
3    ...:
4 Test User 1
5 Test User 2
6 Test User 3

Notes

- can help us to avoid problems of uniqueness of the type ** unique = True **
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

model-mommy (5)

custom fields

 1 import pytest
 2 from django.db import models
 3 from model_mommy import mommy
 4 from backoffice.models import Grupo
 5 
 6 def gen_func():
 7     return 'readability-counts'
 8 
 9 mommy.generators.add(models.SlugField, gen_func)
10 
11 @pytest.mark.django_db
12 def test_prova():
13     item = mommy.prepare(Grupo)
14     assert item.slug == 'readability-counts'

Notes

- we can define generators of values for a certain type of field

- comment on the case of GeoPositionField
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

model-mommy (6)

recipes

 1 from model_mommy.recipe import Recipe, related
 2 from api.models import Sale, SaleItem
 3 
 4 
 5 sale_item1 = Recipe(SaleItem)
 6 sale_item2 = Recipe(SaleItem)
 7 sale_item3 = Recipe(SaleItem)
 8 
 9 sale = Recipe(
10     Sale,
11     items=related('sale_item1', 'sale_item2', 'sale_item3'),
12 )
13 
14 @pytest.mark.django_db
15 def test_sale_detail(client):
16 
17     sale = mommy.make_recipe('api.sale')
18     build = mommy.make(BuildRequest, sale=sale)
19 
20     url = reverse('transaction-detail', kwargs={'order': sale.order})
21     response = client.get(url)
22 
23     assert response.status_code == 200
24     assert response.data.get('order')
25 
26     items = response.data.get('items', [])
27     assert items
28     assert len(items) == sale.items.count()

Notes

- the **recipes** can be defined as templates that serve to generate a set of data configured according to the needs we have without having to specify each attribute in each case we want to use it

- note that in the recipe **sale** it is not necessary to define a customer object since it does it by default **model-mommy**
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

model-mommy (7)

model-mommy vs factory boy

 1 class OwnerFactory(factory.django.DjangoModelFactory):
 2     class Meta:
 3         model = models.Owner
 4 
 5     first_name = 'bobby'
 6     last_name = 'D'
 7     # age will be somewhere between 25 and 42
 8     age = FuzzyInteger(25, 42)
 9     # email will use first_name and last_name (the default or the one you provide)
10     email = factory.LazyAttribute(lambda a: '{0}.{1}@example.com'.format(a.first_name, a.last_name).lower())
11 
12 class TeashopFactory(factory.django.DjangoModelFactory):
13     class Meta:
14         model = models.Teashop
15 
16     name = 'Tea-Bone'
17     # the first teashop will be 0 Downing street, the second 1 Downing Street etc
18     address = factory.Sequence(lambda n: '{0} Downing Street'.format(n))
19     owner = factory.SubFactory(OwnerFactory)
20 
21 OwnerFactory()                          # will save into database and return an instance of the model
22 OwnerFactory.create()                   # same as above
23 OwnerFactory.build()                    # will create the object but not save it in database, very cool for unit tests
24 
25 OwnerFactory(first_name='Malcom')       # override the default first name we defined in the factory
26 TeashopFactory()                        # will create both a teashop and an associated owner model and return the teashop
27 TeashopFactory(owner=my_owner_object)   # will use the Owner object provided instead of creating one

Notes

- way to define the factories of objects with Factory Boy

- more thought for generic projects in python
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

model-mommy (8)

model-mommy vs factory boy

1 mommy.make(Owner) # identical to .create() in factory boy
2 mommy.make(Teashop) # will create and save the teashop and the owner
3 mommy.prepare(Owner) # create but not save in the database
4 
5 mommy.make(Owner, first_name='Malcolm')
6 mommy.make(Teashop, owner__first_name='Malcom') # AWESOME
7 mommy.make(Teashop, _quantity=7) # creates and return 7 teashop objects, everyone gets a teashop!

Notes

- way to define the factories of objects with mommy

- package specifically for django
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-lazy-fixture

plugin that helps us to use the fixtures in lazy mode, which allows us, for example, to use the fixtures as parameters with the parametrize.

 1 @pytest.fixture()
 2 def owners_participations(self, owners, estates):
 3     """
 4     creates owners for each "inmueble"
 5     :param owners:
 6     :param estates:
 7     :return:
 8     """
 9     participaciones = [
10         {'inmueble': estates[0], 'propietario': owners[0], 'participacion': 100},
11         {'inmueble': estates[1], 'propietario': owners[1], 'participacion': 100},
12         {'inmueble': estates[2], 'propietario': owners[2], 'participacion': 100},
13         {'inmueble': estates[3], 'propietario': owners[3], 'participacion': 40},
14         {'inmueble': estates[3], 'propietario': owners[4], 'participacion': 60},
15         {'inmueble': estates[4], 'propietario': owners[5], 'participacion': 100},
16         {'inmueble': estates[5], 'propietario': owners[5], 'participacion': 100},
17         {'inmueble': estates[6], 'propietario': owners[6], 'participacion': 100},
18     ]
19     result = [mommy.make(ParticipacionPropietario, **participacion) for participacion in participaciones]
20     return result
21 
22 @pytest.fixture
23 def no_owners(self):
24     return []

Notes

- here we have two fixtures, one that represents the participation of the owners in a community

- and another that represents the case "without data", without owners (absurd case)
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-lazy-fixture (2)

 1 @pytest.fixture
 2 def remesa(self, community):
 3     remesa = mommy.make(
 4         Remesa, comunidad=community, fecha_cargo=datetime.now(),
 5         tipo='D', importe=10000, descripcion='remesa test',
 6         grupo_inmueble=community.get_default_group()
 7     )
 8     return remesa
 9 
10 @pytest.fixture(params=[
11     pytest.lazy_fixture('owners_participations'),
12     pytest.lazy_fixture('no_owners')
13 ])
14 def ownerset(self, request):
15     return request.param
16 
17 def test_generate_recibos(self, remesa, ownerset):
18     total_initial = remesa.importe
19     num_recibos = Recibo.generar_recibos(remesa)
20     assert num_recibos == len(ownerset)
21     calculated_total = Recibo.objects.aggregate(total=Sum('importe')).get('total') or 0.0
22     if len(ownerset) > 0:
23         assert float(total_initial) == float(calculated_total)
24     else:
25         assert float(calculated_total) == 0
26         assert float(total_initial) > 0

Notes

- here another fixture **ownerset** is created with the two previous fixtures as parameters for each value that returns us **ownerset** a test is executed thanks to the **parametrize**

- at first glance it may seem unhelpful, but as we progress with **testing** we can avoid quite a bit of code repetition
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-lazy-fixture (3)

 1 @pytest.mark.django_db
 2 class TestApi(object):
 3 
 4     @pytest.fixture()
 5     def mocked_build_response(self):
 6         mocked_response = {
 7             'build_operation_ref': '1234',
 8             'license_key': '5678',
 9             'package_file_url': 'https://www.google.com'
10         }
11         return mock.patch('api.models.Sale._request_build', return_value=mocked_response)
12 
13     @pytest.mark.parametrize('data,num_items,status', [
14         (pytest.lazy_fixture('one_product_order'), 1, 201),
15         (pytest.lazy_fixture('multi_product_order'), 3, 201),
16     ])
17     def test_new_order(self, client, mocked_build_response, data, num_items, status):
18         with mocked_build_response:
19             response = client.post(
20                 reverse('transaction-add'), data=json.dumps(data), content_type='application/json'
21             )
22             assert response.status_code == status
23 
24             assert Sale.objects.count() > 0
25             sale = Sale.objects.get(order=response.data.get('order'))
26 
27             assert sale
28             assert sale.items.count() == num_items
29 
30             assert sale.build.license_key
31             assert sale.build.package_file_url
32             assert sale.build.build_operation_ref

Notes

- example of parameterizing a test directly with two fixtures

- used in combination with other parameters
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

hypothesis

plugin that helps us generate data random for our specification or data model

helps us to test that our code works for any value within a range and not only for specific cases

 1 from hypothesis import given, settings, Verbosity
 2 from hypothesis.strategies import decimals, floats
 3 from mock import MagicMock
 4 ...
 5 @given(
 6     price=floats(min_value=1, max_value=50, allow_infinity=False, allow_nan=False),
 7     discount=floats(min_value=0.0, max_value=0.9, allow_infinity=False, allow_nan=False)
 8 )
 9 @settings(perform_health_check=False, max_examples=25, verbosity=Verbosity.verbose)
10 def test_cdai(self, price, discount):
11     order_item = self.order_item(price=price, discount=Decimal(discount))
12     order = order_item.order
13     order.coupon.is_instructor_coupon = MagicMock(return_value=True)
14 
15     CouponUsage.objects.create = MagicMock(return_value=None)
16     discount_amount = order.calculate_discount_amount()
17     assert discount_amount == order.discount_amount

Notes

- Here is an example of a test that we want the parameters **price** and **discount** to take values in different ranges of values

- price: between 5 and 50

- discount: between 0.1 and 0.9

- if the values are not delimited, you can try values such as Infiniy, NaN

- we set the maximum number of examples to 25, if you do not set the maximum, by default you execute about 200 examples. In the development phase it is better to reduce the number of examples and in the deployment phase to leave it by default.

- even if many examples are tried, is considered a single test

- this package can do many other things, but the philosophy that follows is: describe the parameters, describe the result, let the computer prove that my code fails, philosophy **Property-based testing**
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

hypothesis (2)

resultado

pytest-hypothesis

Notes

- in the image we see the values that values taking the parameters

- we also see the 25 times they have taken a value
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-mock

plugin that helps us to do patching of methods or functions that we use to test our logic. It provides us with a fixture mocker, which allows us to patch a certain function. E.g. Test calls to external services, such as an external API.

 1 # -*- encoding: utf-8 -*-
 2 
 3 import pytest
 4 from io import StringIO
 5 from django.core.management import call_command
 6 
 7 @pytest.mark.django_db()
 8 class TestCustomCommands(object):
 9 
10     def test_sync_products(self, supplier_activities_rates, mocker):
11         mocker.patch(
12             'backoffice.management.commands.sync_products.Command.get_api_data',
13             return_value=supplier_activities_rates
14         )
15         out = StringIO()
16         err = StringIO()
17         call_command('sync_products', action='dry-run', stdout=out, stderr=err)
18         assert err.getvalue() == ''
19         assert 'END PROCESS' in out.getvalue()

Notes

- with this technique we do not access the external service directly and we return an "example" of response

- the response example, in this case, is given by the fixture: **supplier_activities_rates**

- we avoid stressing the service and creating a lot of test data even in a test environment, e.g .: stripe
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-mock (2)

detail of the method that we mock, get_api_data

1 def get_api_data(self):
2     self.stdout.write('Accessing API data')
3     return requests.post(self.API_URL).json()

fixture example

 1 @pytest.fixture()
 2 def supplier_activities_rates():
 3     data = """{
 4         "id": "503618",
 5         "name": "This is a title sample",
 6         "isActive": true,
 7         "type": "ORGANIZED",
 8         "rates": {
 9             "rate": [
10                 {
11                     "ActivityDateFrom": "1/6/2017",
12                     "ActivityDateTo": "31/10/2017",
13                     "id": "4879",
14                     "name": "This is a sample name",
15                     "isActive": true
16                 }
17             ]
18         }
19     }"""
20     return json.loads(data)

Notes

- we call the get_api_data method, which is the one that actually accesses the external service

- the answer is "mocked"
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-checkipdb

plugin that helps us verify that we do not leave any breakpoint inserted in our code

1 def render_to_message(self, *args, **kwargs):
2     msg = super().render_to_message(*args, **kwargs)
3     if self.adj:
4         import ipdb; ipdb.set_trace()
5         msg.attach(filename=self.adj.name, content=self.adj.read(), mimetype=self.adj.content_type)
6     return msg

pytest-checkipdb

Notes

- analyze the syntactic code tree

- visit each call to a function and verify that they are not of the pdb or ipdb type

- plugin developed in APSL
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-cov

plugin that helps us detect which part of our code is not yet "covered" by the tests

allows us to generate reports to easily see untested parts

Notes

- the higher the percentage the better

- is the percentage that we usually see inserted as an icon on the top of the github repos
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-cov (2)

small case example without testing

Notes

- in the example we would fail to implement the reactivation test of a subscription that is already active, which is not canceled
pytest: recommendations, basic packages for testing in Python and Django · September 2017 · apsl.net · @avallbona

pytest-flake8

plugin that helps us to ensure that our code follows a style guide, for example, in the case of python, the PEP8