Python using NaN or None as sentinel

Comparing to None instead of NaN is:

  • 4..50 times faster in CPython
  • more than 1000 times faster in PyPy3 with Numpy, same speed with math

Benchmarks

Tests using Intel Haswell CPU on Linux with

ipython

Python 3.7.2 IPython 7.2.0 Numpy 1.16.0

or PyPy3 with IPython

pypy3-ipython

Python 3.5.3 (PyPy3 6.0) IPython 7.2.0 Numpy 1.16.0

Numpy NaN

Numpy is well known to be slower at scalar operations than pure Python. But many data science and STEM application using arrays are vastly faster and more convenient with Numpy than pure Python methods.

from numpy import isnan
%timeit isnan(0.)
  • CPython: 631 ns ± 7.5 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
  • PyPy3: 3.12 µs ± 195 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Python NaN

from math import isnan
%timeit isnan(0.)
  • CPython: 74.9 ns ± 0.783 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
  • PyPy3: 2.02 ns ± 0.0206 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)

Python None

%timeit 0. is not None
  • CPython: 17.1 ns ± 0.236 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)
  • PyPy3: 2.07 ns ± 0.00567 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)

Numba

using python-performance

./RunNoneVsNan.py Numba version 0.42.0 Python version 3.7.2 (default, Dec 29 2018, 06:19:36) Numpy version 1.16.0 –> Numba NaN sentinel: 2.400e-07 –> Numba None sentinel: 1.950e-07 –> CPython NaN sentinel: 2.840e-07 –> Numpy NaN sentinel: 2.108e-06 –> CPython None sentinel: 4.150e-07

Tags: , ,

Categories:

Written by Michael Hirsch, Ph.D. //

Comments