Pytest approximately equal scalars and arrays

1 minute read

Pytest cleanly handles so many continuous integration issues that it is not worth fooling around with obsolete nose and legacy unittest modules. One of those many areas is in the constant need to compare equality of floating point numbers. In general computer representations of floating point numbers have finite precision, and so in general the fundamental arithmetic assumptions learned in elementary school about real numbers are broken, including associativity. A practical solution to this problem is to compare numbers (scalars or arrays) to within an absolute and relative tolerance. Widely known functions exist to compare “equality” of floating point numbers in Python and Fortran among other numerical languages.

pytest.approx provides a syntactically clean approach that may be more intuitive and readable for CI applications. It works with scalars and arrays of all sorts including the ubiquitous numpy.ndarray.

Example

This example shows how to replace numpy.testing.assert_allclose() with pytest.approx():

import numpy as np
from pytest import approx


def test_mynums():
    x = np.array([2.00000000001, 1.99999999999])
    
    assert x == approx(2.)

Whereas with Numpy, the last line would have been np.testing.assert_allclose(x, 2.). I find the Pytest syntax and appearance to be more readable.

NaN

NaN is an unrepresentable, undefined quantity as results from invalid computations like:

  • ∞ / ∞
  • 0 / 0

NaN are also useful as sentinel values to indicate a problem with a computation for example. Python sentinel values include NaN and None. In Julia, performance is approximately the same for nan vs. nothing sentinel values.

The option pytest.approx(nan_ok=True) allows one or more NaN’s in the test array.

Tolerance

Absolute and relative tolerance are important considerations for CI, particularly across OS and computer types, where order of operation and numerical CPU/GPU/APU/etc. tolerances and low level math routines (BLAS, AVX, etc.) differ. Especially (perhaps obviously) for single precision arithmetic vs. double precision arithmetic, tolerances will be larger. The pytest.approx() default tolerances are:

  • rel=1e-6
  • abs=1e-12

Absolute tolerance shouldn’t be 0 to avoid unexpected effects near 0.

Depending on the parameter, rel=1e-4 may be more appropriate for CI tests that need to work on a variety of systems.

Leave a comment