Specific use of assert assertion of pytest

Background

This article summarizes the assert assertions commonly used when writing automated tests using pytest.

Description

This article will summarize from the following points:

Assert for test results Add descriptive information for the result that the assertion fails Assert for expected exceptions Customize description information for failed assertions

Assert for test results

In terms of assertions, the pytest framework is simpler and easier to use than other similar frameworks (such as unittest), which I think is one of the reasons why I chose pytest as one of the automated test frameworks.
The assert assertion keyword of pytest supports the use of assert expressions built into python. It can be understood that the assertion of pytest is to directly use the assert keyword that comes with python.

Concept of python assert:

Python assert (Assertion) is used to determine an expression and triggers an exception when the expression condition is false.

We can add any expression that conforms to the python standard after assert. If the value of the expression is equal to False after bool conversion, it means that the assertion result is failure.

The following are examples of commonly used expressions:

# ./test_case/test_func.py
import pytest
from func import *

class TestFunc:
 
 def test_add_by_class(self):
  assert add(2,3) == 5

def test_add_by_func_aaa():

 assert 'a' in 'abc'
 assert 'a' not in 'bbc'
 something = True
 assert something
 something = False
 assert not something
 assert 1==1
 assert 1!=2
 assert 'a' is 'a'
 assert 'a' is not 'b'
 assert 1 < 2
 assert 2 > 1
 assert 1 <= 1
 assert 1 >= 1
 assert add(3,3) == 6

'''
#  All of the above are legal expressions and the values of the expressions are True, So the test result is pass 
============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-5.3.4, py-1.8.1, pluggy-0.13.1 -- D:\Python3.7\python.exe
cachedir: .pytest_cache
rootdir: D:\Python3.7\project\pytest, inifile: pytest.ini
plugins: allure-pytest-2.8.9, rerunfailures-8.0
collecting ... collected 2 items

test_case/test_func.py::TestFunc::test_add_by_class PASSED               [ 50%]
test_case/test_func.py::test_add_by_func_aaa PASSED                      [100%]

============================== 2 passed in 0.06s ==============================
[Finished in 1.8s]

'''

Add descriptive information for the result that the assertion fails

When writing tests, for ease of use, we want to know some explanatory information about the reasons for the failure when the assertion fails, and assert can also meet this function.
See the example:

# ./test_case/test_func.py
import pytest
from func import *

class TestFunc:
 def test_add_by_class(self):
  assert add(2,3) == 5


def test_add_by_func_aaa():
 assert add(3,3) == 5, "3+3 Should be equal to 6"

'''
============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-5.3.4, py-1.8.1, pluggy-0.13.1 -- D:\Python3.7\python.exe
cachedir: .pytest_cache
rootdir: D:\Python3.7\project\pytest, inifile: pytest.ini
plugins: allure-pytest-2.8.9, rerunfailures-8.0
collecting ... collected 2 items

test_case/test_func.py::TestFunc::test_add_by_class PASSED               [ 50%]
test_case/test_func.py::test_add_by_func_aaa FAILED                      [100%]

================================== FAILURES ===================================
____________________________ test_add_by_func_aaa _____________________________

    def test_add_by_func_aaa():
    
>    assert add(3,3) == 5, "3+3 Should be equal to 6"
E    AssertionError: 3+3 Should be equal to 6
E    assert 6 == 5
E      -6
E      +5

test_case\test_func.py:14: AssertionError
========================= 1 failed, 1 passed in 0.09s =========================
[Finished in 1.4s]
'''

Assert for expected exceptions

In some test cases, such as exception test cases, the result of the test must be a failure and should be an exception. At this time, the expected result of automated test cases is this exception. If the expected result is equal to the exception, the test case execution passes, otherwise the use case result is a failure. pytest provides a method for asserting an expected exception: pytest. raises (). 1 in conjunction with with context manager.

Use example:

# ./func.py
def add(a,b):
 if isinstance(a,int) and isinstance(b,int):
  return a+b
 else:
  raise NameError(' Wrong data type ')


# ./test_case/test_func.py
import pytest
from func import *

class TestFunc:

 #  Normal test cases 
 def test_add_by_class(self):
  assert add(2,3) == 5

#  Exception test case, the expected result is burst TypeError Anomaly 
def test_add_by_func_aaa():
 with pytest.raises(TypeError):
  add('3',4)
  

# ./run_test.py
import pytest

if __name__ == '__main__':
 pytest.main(['-v'])

'''
============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-5.3.4, py-1.8.1, pluggy-0.13.1 -- D:\Python3.7\python.exe
cachedir: .pytest_cache
rootdir: D:\Python3.7\project\pytest, inifile: pytest.ini
plugins: allure-pytest-2.8.9, rerunfailures-8.0
collecting ... collected 2 items

test_case/test_func.py::TestFunc::test_add_by_class PASSED               [ 50%]
test_case/test_func.py::test_add_by_func_aaa PASSED                      [100%]

============================== 2 passed in 0.06s ==============================
[Finished in 1.4s]
''' 

Let's take a look at examples where no expected anomalies broke out:

# ./func.py
def add(a,b):
 #  Specify an exception 
 raise NameError(" Heavenly anomaly ")
 if isinstance(a,int) and isinstance(b,int):
  return a+b
 else:
  raise NameError(' Wrong data type ')

# ./test_case/test_func.py
import pytest
from func import *
'''
class TestFunc:

 #  Normal test cases 
 def test_add_by_class(self):
  assert add(2,3) == 5
'''
#  Exception test case, the expected result is burst TypeError Anomaly 
def test_add_by_func_aaa():
 with pytest.raises(TypeError):
  add('3',4)
  
# ./run_test.py
import pytest

if __name__ == '__main__':
 pytest.main(['-v'])


'''
============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-5.3.4, py-1.8.1, pluggy-0.13.1 -- D:\Python3.7\python.exe
cachedir: .pytest_cache
rootdir: D:\Python3.7\project\pytest, inifile: pytest.ini
plugins: allure-pytest-2.8.9, rerunfailures-8.0
collecting ... collected 1 item

test_case/test_func.py::test_add_by_func_aaa FAILED                      [100%]

================================== FAILURES ===================================
____________________________ test_add_by_func_aaa _____________________________

    def test_add_by_func_aaa():
     with pytest.raises(TypeError):
>     add('3',4)

test_case\test_func.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = '3', b = 4

    def add(a,b):
     #  Specify an exception 
>    raise NameError(" Heavenly anomaly ")
E    NameError:  Heavenly anomaly 

func.py:4: NameError
============================== 1 failed in 0.09s ==============================
[Finished in 1.4s]
'''

Determine that the result of use case execution is failure.

Above, we just assert the type of exception. But sometimes we want to go one step further and assert the exception description information, and pytest can do the same. The execution of with pytest. raises () generates an instance object of ExceptionInfo. This object contains type, value, traceback properties. The value attribute is the exception description information we need.

See example:

# ./func.py
def add(a,b):
 if isinstance(a,int) and isinstance(b,int):
  return a+b
 else:
  raise TypeError(' Wrong data type ')
 
# ./test_case/test_func.py
import pytest
from func import *

class TestFunc:

 #  Normal test cases 
 def test_add_by_class(self):
  assert add(2,3) == 5

#  Exception test case, the expected result is burst TypeError Anomaly 
def test_add_by_func_aaa():
 with pytest.raises(TypeError) as E:
  add('3',4)
 print(E.type)
 print(E.value)
 print(E.traceback)
 #  Join this non-assertion to view stdout
 assert 1 == 2


# ./run_test.py
import pytest

if __name__ == '__main__':
 pytest.main(['-v'])

'''
============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-5.3.4, py-1.8.1, pluggy-0.13.1 -- D:\Python3.7\python.exe
cachedir: .pytest_cache
rootdir: D:\Python3.7\project\pytest, inifile: pytest.ini
plugins: allure-pytest-2.8.9, rerunfailures-8.0
collecting ... collected 2 items

test_case/test_func.py::TestFunc::test_add_by_class PASSED               [ 50%]
test_case/test_func.py::test_add_by_func_aaa FAILED                      [100%]

================================== FAILURES ===================================
____________________________ test_add_by_func_aaa _____________________________

    def test_add_by_func_aaa():
     with pytest.raises(TypeError) as E:
      add('3',4)
     print(E.type)
     print(E.value)
     print(E.traceback)
>    assert 1 == 2
E    assert 1 == 2
E      -1
E      +2

test_case\test_func.py:18: AssertionError
---------------------------- Captured stdout call -----------------------------
<class 'TypeError'>
 Wrong data type 
[<TracebackEntry D:\Python3.7\project\pytest\test_case\test_func.py:14>, <TracebackEntry D:\Python3.7\project\pytest\func.py:6>]
========================= 1 failed, 1 passed in 0.10s =========================
[Finished in 1.4s]
'''

The "Captured stdout call" output from the console is the exception information, including the type, exception description and exception tracking information.
This information can be asserted through assert.

You can also accomplish the assertion of E. value by passing the match keyword parameter to pytest. raises (), which applies the principle of regular expressions in python.

Example:

This example means that the assertion passes through

def test_add_by_func_aaa():
 with pytest.raises(TypeError, match=r'.* Type error $') as E:
  add('3',4)

This example means that the assertion failed:

#  Exception test case, the expected result is burst TypeError Anomaly 
def test_add_by_func_aaa():
 with pytest.raises(TypeError, match=r'.* Correct $') as E:
  add('3',4)
'''
During handling of the above exception, another exception occurred:

    def test_add_by_func_aaa():
     with pytest.raises(TypeError, match=r'.* Correct $') as E:
>     add('3',4)
E     AssertionError: Pattern '.* Correct $' not found in ' Wrong data type '

test_case\test_func.py:14: AssertionError
'''

If a test case may have different expected exceptions, as long as the burst exception is within a few expected exceptions, then how to assert. The solution is very simple, the principle and interface have not changed, only in pytest. raises () passed in the exception type parameter, from passing in 1 exception type, changed to passing in 1 exception type composed of tuple. Also just passing in 1 parameter.

Example:

# ./func.py
def add(a,b):
 raise NameError(' Wrong name ')
 if isinstance(a,int) and isinstance(b,int):
  return a+b
 else:
  raise TypeError(' Wrong data type ')
 
# ./test_case/test_func.py
import pytest
from func import *

'''
class TestFunc:

 #  Normal test cases 
 def test_add_by_class(self):
  assert add(2,3) == 5
'''

#  Exception test case, the expected result is burst TypeError Anomaly 
def test_add_by_func_aaa():
 with pytest.raises((TypeError,NameError),match=r'.* Wrong .*$') as E:
  add('3',4)
 
 
# ./run_test.py
import pytest

if __name__ == '__main__':
 pytest.main(['-v'])
 
'''
============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-5.3.4, py-1.8.1, pluggy-0.13.1 -- D:\Python3.7\python.exe
cachedir: .pytest_cache
rootdir: D:\Python3.7\project\pytest, inifile: pytest.ini
plugins: allure-pytest-2.8.9, rerunfailures-8.0
collecting ... collected 1 item

test_case/test_func.py::test_add_by_func_aaa PASSED                      [100%]

============================== 1 passed in 0.04s ==============================
[Finished in 1.4s]
'''

Customize description information for failed assertions

This behavior is equivalent to changing the operation mode of pytest, although it is only a icing on the cake. We change the behavior of pytest by writing hook function. hook function is provided by pytest, there are many, each hook function detailed definition should refer to pytest official documentation.
Customizing the description information for the failure assertion is done through the hook function pytest_assertrepr_compare.
First look at the default failure assertion description when the hook function pytest_assertrepr_compare is not written:

def test_add_by_func_aaa():
 assert 'aaa' == 'bbb'

'''
================================== FAILURES ===================================
____________________________ test_add_by_func_aaa _____________________________

    def test_add_by_func_aaa():
>    assert 'aaa' == 'bbb'
E    AssertionError: assert 'aaa' == 'bbb'
E      - aaa
E      + bbb

test_case\test_func.py:16: AssertionError
'''

Look again at writing the hook function pytest_assertrepr_compare:

# ./conftest.py

def pytest_assertrepr_compare(op, left, right):
    if isinstance(left, str) and isinstance(right, str) and op == "==":
        return [' Comparison of two strings :',
                '    Value : %s != %s' % (left, right)]


# ./test_case/test_func.py
import pytest
def test_add_by_func_aaa():
 assert 'aaa' == 'bbb'


'''
.F                                                                       [100%]
================================== FAILURES ===================================
____________________________ test_add_by_func_aaa _____________________________

    def test_add_by_func_aaa():
>    assert 'aaa' == 'bbb'
E    assert  Comparison of two strings :
E          Value : aaa != bbb

test_case\test_func.py:15: AssertionError
1 failed, 1 passed in 0.09s
[Finished in 1.5s]
'''

pytest also provides other hook functions, which are used to change the operation mode and operation effect of pytest. Therefore, writing a third-party plug-in 1 generally uses these hook functions.