How to write parametrized tests using pytest

pytest has become pretty much the standard test runner in the Python ecosystem.

Although the base Python distribution ships with a unittest library that you can use to write tests without having to pip-install anything. At some point though, as more and more features make it in to your codebase, you the developer start wishing for more capabilities when writing tests that unittest just doesn't provide.

Enter pytest.

In this post, we'll look at how to ask pytest to run tests using arguments, in effect generating multiple tests from one test. This pattern can come in really handy in case you're looking to remove toil from your test suite.

Writing simple tests using pytest

pytest is a testing framework that makes it really easy to write and maintain tests.

It lets you use Python's in-built assert mechanism to write assertions within your test cases. And it gives you a lot of useful features (eg. fixtures, or parameterizing tests which happens to be the subject of this blog post) that make your life easier.

Here's a quick example of what a basic test looks like.

from my_package import calculate


def test_the_answer_to_life():
    assert calculate() == 42

Note that there's nothing in this code snippet that calls out "pytest". But that's the beauty of it. All this code is native Python code that does not use any extra libraries. But pytest can deduce from the name of this function (which starts with the test_ prefix) that this is a test, and can work with the assertion inside this function to run the test and finally generate a report.

Why introduce parameters to tests

Let's take a real-life example to illustrate where parametrizing tests can be helpful.

Imagine for a moment that you are a developer at a SaaS (Software as a Subscription) company and the backend software is responsible for tracking what subscription plan a user is on. A user might start out subscribed to the "Free" plan and then choose to upgrade to a "Basic" plan or a "Premium" plan. This information is tracked in our database where we store the rest of the user details. Furthermore, users can edit this information somewhere in the subscription settings section when they're logged in.

Now, we want to test that for free plan users, all the subscription plans our SaaS app supports are available for them to upgrade to in their subscription settings.

What would such a test case look like?

Basic test without parameters

Let's rephrase the problem statement: we want to write a test case that renders the "Subscription Settings" page from the web server, and for free users, asserts that all the paid plans are listed as an <option> in a HTML form.

Also note that this problem statement is an oversimplification and that you might choose to implement this feature differently but this framing helps us restrict the scope of this blog post.

Let's take an initial stab at this test case.

from http import HTTPStatus


def test_get_subscription_settings_page_shows_basic_plan(client, user_free_plan):
    response = client.get("/settings/subscriptions", as_user=user_free_plan)

    assert response.status_code == HTTPStatus.OK.value
    assert '<option value="basic">Basic</option>' in response.text


def test_get_subscription_settings_page_shows_premium_plan(client, user_free_plan):
    response = client.get("/settings/subscriptions", as_user=user_free_plan)

    assert response.status_code == HTTPStatus.OK.value
    assert '<option value="premium">Premium</option>' in response.text


def test_get_subscription_settings_page_shows_enterprise_plan(client, user_free_plan):
    response = client.get("/settings/subscriptions", as_user=user_free_plan)

    assert response.status_code == HTTPStatus.OK.value
    assert '<option value="enterprise">Enterprise</option>' in response.text

This solves our problem, but as you can see, there's a lot of repetition. The structure of each test is the same; just that the values of some variables are different.

One solution to this repetition would be to include all the plan names in a list, and iterate on that list to include everything in one test case. Something like the following:

from http import HTTPStatus


def test_get_subscription_settings_page_shows_all_plans(client, user_free_plan):
    response = client.get("/settings/subscriptions", as_user=user_free_plan)

    assert response.status_code == HTTPStatus.OK.value

    for plan_name in ("basic", "premium", "enterprise"):
        assert (
            f'<option value="{plan_name}">{plan_name.capitalize()}</option>'
            in response.text
        )

This looks much better than our first attempt. But the problem we're facing now is that this test case is mixing concerns. We'd like the test function to be a self-contained unit of work that makes sure that some invariant holds, given a user object and a plan name string.

As it turns out, pytest has something that would help us solve this problem much more elegantly.

pytest parametrize

Let's introduce our test suite to pytest.mark.parametrize.

pytest.mark.parametrize is a decorator that takes (mainly) two parameters: argument names and values. The argument names contain what you want to parametrize, and the value(s) contain all the possible values you want the names to take.

Given these two pieces of information, pytest will run the same test multiple times, each time setting the variable to one of each value until it has run through all the values.

Parametrizing one value

Let's convert our test code from before to use parametrize to see all this in action.

from http import HTTPStatus

import pytest


@pytest.mark.parametrize("plan_name", ("basic", "premium", "enterprise"))
def test_get_subscription_settings_page_shows_specific_plan(client, user_free_plan, plan_name):
    response = client.get("/settings/subscriptions", as_user=user_free_plan)

    assert response.status_code == HTTPStatus.OK.value
    assert (
        f'<option value="{plan_name}">{plan_name.capitalize()}</option>'
        in response.text
    )

Essentially, this tells pytest that the test_get_subscription_settings_page_shows_specific_plan test should run three times, once for each subscription plan we have available.

This is a bit of a variation over our list approach from before, except that now pytest takes care of iterating through all the values for us, which saves us some effort. The different is subtle, but this approach lets our test code be more focused on one logical chunk of work (to assert the behavior given a specific plan name). We can then ask pytest to repeat the process for all the different values we're interested in.

Parametrizing multiple values

You can also pass multiple names and multiple values for each of those names. For such parameter values, pytest will cycle through all the provided values.

Let's illustrate this approach by extracting our plan_name.capitalize() call from the earlier snippet to a second parameter.

from http import HTTPStatus

import pytest


@pytest.mark.parametrize(
    "plan_name,plan_name_capitalized",
    (
        ("basic", "Basic"),
        ("premium", "Premium"),
        ("enterprise", "Enterprise"),
    ),
)
def test_get_subscription_settings_page_shows_specific_plan(client, user_free_plan, plan_name, plan_name_capitalized):
    response = client.get("/settings/subscriptions", as_user=user_free_plan)

    assert response.status_code == HTTPStatus.OK.value
    assert (
        f'<option value="{plan_name}">{plan_name_capitalized}</option>'
        in response.text
    )

In this example, we're giving pytest two variables called plan_name and plan_name_capitalized, and a tuple of value tuples, each tuple representing a (plan_name, plan_name_capitalized) combination we're interested in.

Parametrizing fixtures

Let's do one last example! Parameters aren't limited to just these two use cases. You can also parametrize fixtures!

This is where the versatality of pytest comes in. You can parametrize a fixture, so that the tests depending on the parametrized fixtures run depending on the parameter values you've provided. All this happens without the test code knowing about any of all this.

In our example, let's convert the user_free_plan fixture to a plain user fixture cycling through all the available subscription plans.

from http import HTTPStatus

import pytest

@pytest.fixture(params=("basic", "premium", "enterprise"))
def user(request):
    return User(plan_name=request.param)


def test_get_subscription_settings_page_renders(client, user):
    response = client.get("/settings/subscriptions", as_user=user_free_plan)

    assert response.status_code == HTTPStatus.OK.value

In this snippet we've parametrized the user fixture to iterate through all the subscription plans. And having the test_get_subscription_settings_page_renders test depend on this fixture means that this test will be run three times, one for each kind of user that the original fixture returns.

You might have noticed that the nature of this test has been changed as we're asserting different things. But this has been done to restrict the scope of this blog post so we don't go off on a tangent. 🙃

Conclusion

I hope that this blog post gave you a quick taste of the versatality of pytest and all the amazing features available to you should you decide to use pytest. For further details on what all pytest has to offer, I highly recommend perusing their documentation.

Happy testing!

Photo by Ben Mullins on Unsplash