View source for Pytest

{{Short description|Software testing framework}}
{{Lowercase title}}
{{Infobox software
| title = Pytest
| name = Pytest
| logo = Pytest logo.svg
| logo size = 200px
| logo alt = 
| logo caption = <!-- Selenium logo -->
| screenshot = <!-- Image name is enough -->
| screenshot size = 
| screenshot alt = 
| caption = <!-- Demonstration of Selenium -->
| collapsible = <!-- yes -->
| author = Krekel et al.
| developer = 
| released = <!-- {{Start date and age|YYYY|MM|DD|df=yes/no}} -->
| discontinued = 
| latest release version = 7.1.1
| latest release date = 
| latest preview version = 
| latest preview date = <!-- {{Start date and age|YYYY|MM|DD|df=yes/no}} -->
| programming language = [[Python (programming language)|Python]]
| platform = [[macOS]], [[Microsoft Windows|Windows]], [[POSIX]]
| size = 
| language = 
| language count = <!-- DO NOT include this parameter unless you know what it does -->
| language footnote = 
| genre = [[Software testing]] [[Software framework|framework]].
| license = [[MIT License]]
| alexa = 
| standard = 
| AsOf = 
}}
'''Pytest''' is a software testing framework based on the [[Python (programming language)|Python]] programming language. It can be used to write various types of software tests, including [[unit tests]], [[integration tests]], [[System testing|end-to-end tests]], and [[functional tests]]. Its features include [[Parameter (computer programming)|parametrized]] testing, [[Test fixture|fixtures]], and [[Assertion (software development)|assert]] re-writing.

Pytest has been described as a [[Scalability|scalable]] framework with less [[boilerplate code]] than other [[List of unit testing frameworks|testing frameworks]]. Pytest fixtures provide the contexts for tests by passing in parameter names in test cases; its parametrization eliminates duplicate code for testing multiple sets of input and output; and its re-writed [[Assertion (software development)|assert statements]] provides detailed output for causes of failures.

Many technology projects have switched to pytest from other testing frameworks, including those of [[Mozilla]] and [[Dropbox]]. According to developer security platform Snyk, Pytest is classified as a key ecosystem project in Python due to its high degree of popularity.

==History==

Pytest had began as part of [[PyPy]], an alternative implementation of Python to [[CPython]]. It later separated from PyPy into its own project and was named pytest.<ref name="pytest-history">{{cite web |title=History |url=https://docs.pytest.org/en/latest/history.html |website=pytest |access-date=13 April 2022}}</ref>

Since early 2003 when PyPy emerged, it held a heavy emphasis on [[Software testing|testing]]. PyPy had unit tests for newly written code, regressions tests for bugs, and integration tests using CPython's test suite.<ref name="PyPy">{{cite web |last1=Bolz-Tereick |first1=Carl Friedrich |title=PyPy Status Blog |url=https://morepypy.blogspot.com/2018/09/the-first-15-years-of-pypy.html#continuous-integration |website=PyPy |access-date=12 May 2022}}</ref>
In mid 2004, a testing framework called utest emerged, and contributors to PyPy began converting existing [[test case|test cases]] to utest. Meanwhile, for EuroPython 2004, a complementary [[standard library]] for testing named std was created, which layed out the principles, such as assert rewriting, of what later became pytest. In late 2004, the std project was renamed to py and std.utest became py.test, afterwhich the py [[Library (computing)|library]] was separated from PyPy. In November 2010, pytest 2.0.0 was released as a package separate from py, being still called py.test until August 2016 with the release of pytest 3.0.0 when the recommended [[Command-line interface|command line]] [[entry point]] became pytest.<ref name="pytest-history"/>

==Notable features==
===Parametrized testing===
It is a common pattern in software testing to send values through test [[Subroutine|functions]] and check for correct output. In many cases, in order to thoroughly [[functional testing|test functionalities]], one needs to test multiple sets of input/outputs, and writing such cases separately would cause [[duplicate code]] as most of the actions would remain the same, only differing in input/output values. Pytest's parametrized testing feature eliminates such duplicate code by combining different iterations into one test case, then runs these iterations and displays each test's result separately.<ref name="okken">{{cite book |last1=Okken |first1=Brian |title=Python Testing with Pytest |date=September 2017 |publisher=The Pragmatic Bookshelf |isbn=9781680502404 |edition=1st |url=https://pragprog.com/titles/bopytest/python-testing-with-pytest/ |access-date=22 January 2022}}</ref>

This feature is preferred by some developers over unittest's parametrizing method where multiple test results are grouped into one, in which case if one test fails and the rest passes, unittest only displays a single failing result for the entire group, which contrasts with pytest's individual test reporting.<ref name="real-python">{{cite web |last1=Hillard |first1=Dave |title=Effective Python Testing With Pytest |url=https://realpython.com/pytest-python-testing/ |website=Real Python |access-date=12 February 2022}}</ref>

Parametrized tests in pytest are marked by the {{code|@pytest.mark.parametrize(argnames, argvalues)}} [[Python syntax and semantics#Decorators|decorator]], where the first [[Parameter (computer programming)|parameter]], {{code| argnames }}, is a string of comma-separated names, and {{code|argvalues}} is a list of values to pass into {{code| argnames }}. When there are multiple names in {{code| argnames }}, {{code|argvalues}} would be a list of tuples where values in each tuple corresponds to the names in {{code|argnames}} by index. The names in {{code|argnames}} are then passed into the test function makred by the decorator as parameters. When pytest runs such decorated tests, each pair of {{code| argnames }} and {{code|argvalues}} would constitute a separate run with its own test output and unique identifier. The identifier can then be used to run individual data pairs.<ref name="okken" />

===Assert rewriting===
When writing software tests, the [[Assertion (software development)|assert statement]] is a primary means for communicating test failure, where expected values are compared to actual values.<ref name="okken" /> While Python's builtin assert [[Python_syntax_and_semantics#Keywords|keyword]] would only raise AssertionError with no details in cases of failure, pytest rewrites Python's assert keyword and provides detailed output for the causes of failures, such as what expressions in the assert statement evaluate to. Its concision and [[Readability#Readability of source code|readability]] drive many to use pytest over other testing frameworks. For instance, a comparison can be made with unittest's assert statements:

{|class="wikitable"
! pytest
! unittest
|-
| {{code|lang=python|assert x}}
| {{code|assertTrue(x)}}
|-
| <syntaxhighlight lang="python" inline>assert x == y</syntaxhighlight>
| {{code|assertEqual(x, y)}}
|-
| <syntaxhighlight lang="python" inline>assert x >= y</syntaxhighlight>
| {{code|assertLessEqual(x, y)}}
|}

{{code| unittest }} adheres to a more verbose syntax because it is written with inspiration from [[JUnit]] in [[Java]], as are most unit testing libraries. However, being considered by some Python developers as a more pythonic approach, pytest takes advantage of Python's ability for user defined libraries to achieve what in other languages would require [[compiler]] modifications to display [[traceback]] that provides information on the exact reasons of failed assertions, hence this assert rewriting feature.<ref>{{cite web |title=Assertion rewriting in Pytest part 1: Why it’s needed |url=https://www.pythoninsight.com/2018/01/assertion-rewriting-in-pytest-part-1/ |website=Python Insight |access-date=12 February 2022}}</ref>

=== Pytest fixtures ===
Pytest [[Test fixture|fixtures]] provide the context for tests. They can be used to set a system in to known [[State (computer science)|state]] and to pass dataset into test functions. Fixtures practically constitute the ''arrange'' phase in the anatomy of a test (AAA, short for ''arrange'', ''act'', ''assert'').<ref>{{cite web |title=Unit test basics |url=https://docs.microsoft.com/en-us/visualstudio/test/unit-test-basics?view=vs-2022 |website=[[Microsoft]] |access-date=28 March 2022}}</ref> Pytest fixtures can run before test cases as setup or after test cases for clean up, though they are different from unittest and nose's [[Test_fixture#Setup|setup]] and [[Test_fixture#Examples|teardowns]]. Functions declared as pytest fixtures are marked by the {{code| @pytest.fixture() }} [[Python syntax and semantics#Decorators|decorator]], whose names can then be passed into test functions as parameters.<ref>{{cite web |title=About fixtures |url=https://docs.pytest.org/en/latest/explanation/fixtures.html |website=Pytest |access-date=7 February 2022}}</ref> When pytest finds the fixtures names in test functions parameters, it first searches in the same module for such fixtures, and if not found, it searches for such fixtures in the conftest.py [[Computer file|file]].<ref name="okken" />

For example:

<syntaxhighlight lang="python">
import pytest

@pytest.fixture()
def dataset():
    """Return some data to test functions"""
    return {'data1': 1, 'data2': 2}

def test_dataset(dataset):
    """test and confirm fixture value"""
    assert dataset == {'data1': 1, 'data2': 2}
</syntaxhighlight>

In the above example, pytest fixture {{code| dataset }} returns a dictionary, which is then passed into test function {{code| test_dataset }} for assertion. In addition to fixture detection within the same file as test cases, pytest fixtures can also be placed in the conftest.py file in the tests directory. There can be multiple conftest.py files, each placed within a tests directory for fixtures to be detected for each sebset of tests.<ref name="okken"/><ref name="real-python"/>

====Fixture scopes====

In pytest, fixure scopes let the user define when a fixure should be called. There are four fixture scopes: [[Function object|function]] scope, [[Class (computer programming)|class]] scope, [[Modular programming|module scope]], and session scope. Function-scoped fixtures are default for all pytest fixtures, which are called every time a function having the fixture as a parameter runs.  The goal of specifying a broader fixture scope is to eliminate repeated fixture calls, which could slow down test execution. Class-scoped fixtures are called once per test class, regardless of the number of times they are called, and the same logic goes for all other scopes. When changing fixture scope, one need only add the scope parameter to fixture decorators, for example, {{code|2=python|1=@pytest.fixture(scope="class")}}.<ref name="okken"/><ref name="Apress">{{cite book |last1=Ashwin |first1=Pajankar |title=Python Unit Test Automation: Practical Techniques for Python Developers and Testers |publisher=Apress |isbn=9781484226766 |url=https://www.oreilly.com/library/view/python-unit-test/9781484226766/ |access-date=7 March 2022}}</ref>

===Test filtering===
Another notable feature of pytest is its ability to filter through tests, where only desired tests are selected to run, or behave in a certain way as desired by the developer. With the "k" [[Command-line interface#Command-line option|option]] (e.g. {{kbd|pytest -k some_name}}), pytest would only runs tests whose names include {{code|some_name}}. The opposite is true, where one can run {{kbd|pytest -k "not some_name"}}, and pytest will run all tests whose names do not include {{code|some_name}}.<ref name="Molina">{{cite book |last1=Molina |first1=Alessandro |title=Crafting Test-Driven Software with Python |date=February 2021 |publisher=Publisher(s): Packt Publishing |isbn=9781838642655 |url=https://www.oreilly.com/library/view/crafting-test-driven-software/9781838642655/ |access-date=8 March 2022}}</ref>

Pytest's markers can, in addition to altering test behaviour, also filter tests. Pytest's markers are Python decorators starting with the {{code|2=python|@pytest.mark.<markername>}} syntax placed on top of test functions. With different arbitrarily named markers, running {{kbd|pytest -m <markername>}} on the command line will only run those tests decorated with such markers. All available markers can be listed by the {{kbd|pytest --markers}} along with their descriptions; custom markers can also be defined by users and registered in pytest.ini, in which case {{kbd|pytest --markers}} will also list those custom markers along with builtin markers.<ref name="klein">{{cite web |last1=Klein |first1=Bernd |title=Testing with Pytest |url=https://python-course.eu/advanced-python/pytest.php |website=python-course.eu |access-date=5 March 2022}}</ref><ref>{{cite web |title=PyTest Marks |url=https://developers.perfectomobile.com/display/TT/PyTest+Marks |website=developers.perfectomobile.com |access-date=5 March 2022}}</ref>

==Popularity==

Pytest was developed as part of an effort by [[Python Package Index|third-party packages]] to address [[Python (programming language)#Libraries|Python's built-in module]] unittest's shortcomings, and has been classified by developer security platform Snyk as one of the key ecosystem projects in Python due to its popularity. Some well-known cases of projects switching to pytest from unittest and nose (another testing package) include those of [[Mozilla]] and [[Dropbox]].<ref>{{cite web |title=Project examples |url=https://docs.pytest.org/en/6.2.x/projects.html |website=Pytest |access-date=1 February 2022}}</ref><ref>{{cite web |last1=Koorapati |first1=Nipunn |title=Open Sourcing Pytest Tools |url=https://dropbox.tech/application/open-sourcing-pytest-tools |website=[[Dropbox]] |access-date=1 February 2022}}</ref><ref name="Oliveira">{{cite book |last1=Oliveira |first1=Bruno |title=pytest Quick Start Guide |date=August 2018 |publisher=[[Packt Publishing]] |isbn=978-1-78934-756-2 |url=https://www.packtpub.com/product/pytest-quick-start-guide/9781789347562 |access-date=1 February 2022}}</ref><ref name="Snyk">{{cite web |title=pytest |url=https://snyk.io/advisor/python/pytest |website=Snyk |access-date=12 May 2022}}</ref>

==Installation and running tests==

One needs to run {{kbd| pip install pytest}} from the command line. After entering {{kbd| pytest}} on the command line, pytest then recursively detects and runs tests that have either {{code|test_}} or {{code|_test}} as function names in files within the working directory. Files containing tests are named according to the following format: test_*.py or *_test.py; Test classes are named such that they either begin or end with Test. This is the default naming convention pytest uses for detection, though it can be customized in pytest's configuration file pytest.ini.<ref name="real-python" /><ref name="okken" />

For instance, one can define a test in a file named test_pass.py as follows:
<syntaxhighlight lang="python">
def test_pass():
    assert 1 == 1
</syntaxhighlight>

Then, enter pytest test_pass.py on the command line within the project directory where the test function resides. The output is as follows:

<syntaxhighlight lang="output">
===================== test session starts ======================
collected 1 items
test_one.py .
=================== 1 passed in 0.01 seconds ===================
</syntaxhighlight>

Pytest uses a {{code|.}} beside the file under test to signify a passing test, and {{code|F}} for a failing test, along with various other test output behaviours which are all customizable.<ref name="okken" />

==See also==
{{Portal|Free and open-source software}}
* [[JUnit]], well-known software testing framework based on [[Java]]
* [[Doctest]], well-known testing framework in [[Python (programming language)|Python]] for [[Docstring|docstrings]]
* [[List of unit testing frameworks]]

== References ==
<!-- See https://en.wikipedia.org/wiki/Help:Referencing_for_beginners on how to create references. -->
{{Reflist}}

== External links ==
* [https://pypi.python.org/pypi/pytest https://pypi.python.org/pypi/pytest]
* [https://docs.pytest.org https://docs.pytest.org]

<!-- Categories -->

[[Category:Python (programming language) development tools]]