Majority of new Python work is Python 3

6 minute read

tl;dr: You should be requiring Python ≥ 3.5 or 3.6 for your Python project. Do not bother supporting Python < 3.5 in general.


Python 3 is used by a large and growing majority of new and active Python developers in science, engineering, medical research and education. Python 3 was released in December 2008. While initially there was pushback over backward incompatibilities, the performance, efficiencies and features of Python 3 have won out over Python 2. Python 2 is obsolete ~ April 2020, so it is unwise to continue to put effort into new Python 2 code in general.

Python 2 deprecation

  • Numpy freezes Python 2 support Jan 1, 2019.
  • Numpy 1.16 requires Python ≥ 3.5
  • Matplotlib 3.0 requires Python ≥ 3.5
  • Xarray requires Python ≥ 3.5

Major Python data analysis packages used in STEM industry, research and education have already abandoned Python 2, including IPython , AstroPy, SunPy Django and virtually every worthwhile maintained Python module.

The most popular Python packages have supported Python 3 for some time now, including Amazon AWS and Google Cloud Platform. Already in 2016, Python 3 was starting to take over in PyCharm IDE:

and by May 2017, the majority of PyCharm users were on Python 3:

More than 98% of our Python projects require at least Python 3.5, and > 95% require at least Python 3.6. PyPy supports Python 3.5 syntax and soon supports Python 3.6 syntax. In these exceptions, due to the shortness of the program and simplicity of code, there wasn’t any reason to lock Python 2 users out. PyMap3D is a key utility in 3-D coordinate transformation that parallels conversions available in Matlab for which I have allowed Python 2 compatibility.

In 2018, Python projects should generally be written for Python ≥ 3.6, due to the important number of features requiring Python 3.6.

The main holdouts in Python 2 code are of the same nature as those that hang on to COBOL systems. Those with static, unchanging requirements in large proprietary codebases that few people are familiar with. Some programmers thrive and make a decent living servicing those legacy COBOL and Python 2 environments. The majority of STEM coders, analysts and educators have been writing Python 3 code. The old Python 2-only and Python 2-first objections were mostly written before 2016 and almost all were before 2017. Some of their complaints were addressed in Python 3.6 (released December 23, 2016).

The main outstanding complaint I see over Python 3 is over the separation between bytes and strings. Our work in applications with IoT and embedded systems distinguishes between bytes and strings, so I appreciate the separation of bytes and strings in Python 3. For the global environment I write for, I appreciate that strings are Unicode in Python 3.

Efficiency

Python 3 efficiencies in terms of programmer time come in more efficient syntax. The Python 3 core itself is as much as 10% faster in several aspects, with some standard modules like re processing regular expressions as much as 20x faster. The modernize tool and six and __future__ modules smooth over most of these changes to make Python 2/3 compatible code. Some of the most enticing changes from Python 3.5, 3.6 and 3.7 are not easily backportable to Python 2.7, however. These features simplify syntax and make the code more self-documenting. Python 2 has many years of cruft bolted in, and that’s a big part of why Python 3 introduced backward incompatibilities, and they are growing with time.

The brief list below are just a few Python 3 features that I take advantage of. There are very many more than this that may apply for you.

Asynchronous execution

asyncio has been rapidly maturing since its genesis in Python 3.4. It brings to core Python features that used to require Tornado, twisted, etc. Asynchronous execution is required for applications that need to scale massively. Not just for Facebook, but also IoT applications where remote sensors report in are a perfect use case for asyncio. asyncio is a convenient , relatively safe way to thread, increasingly baked right into Python.

Since asyncio continues to develop at speed, you should use the newest version of Python suitable (at least Python 3.6) and ensure the documentation you’re reading is for the version of Python you’re using.

LRU caching

Least-recently used caching is enabled by a decorator to functions. For example, you have a computationally-expensive function where you sometimes use the same arguments, but don’t want to bother tracking the output by saving in a variable. LRU caching is as simple as:

from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

print([fib(n) for n in range(16)])

print(fib.cache_info())

[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]

CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)

Factory installed pip

Factory installed pip encourages proper use of setup.py and setuptools instead of one-off non-OS-independent setup scripts. In 2018, pip ≥ 10 can use the simpler pyproject.toml instead of or along with setup.py for Python projects.

Type hinting

Python 3.5 added type hinting, which some parsers use to give code warnings, while not actually enforcing strict typing (unless you want to).

import math

def 2sinxy(x:float, y:float) -> float:
    return 2*math.sin(x*y)

This function would not crash if you fed int in on most interpreters, but PyCharm IDE and other can emit warnings when conflicting variable types are passed in/out.

Argument unpacking

Python 3.5 added argument unpacking, where iterables can be fed into functions requiring multiple arguments, expanded with *iterable. Multiple iterables can be unpacked into a single function.

OO IPv4/v6 addressing

ipaddress is a useful standard library feature for manipulating IP addresses, used in the findssh program to scan for active servers in IP address ranges.

OO path/filename

pathlib is standard library. This gets rid of most awkward os.path functions.

F-strings (no ‘{}’.format)

f-strings from Python 3.6 allow f'This is {weight} kg for {price} dollars. instead of 'This is {} kg for {} dollars'.format(weight,price)

Python 3.7

Python 3.7 adds several more compelling reasons to upgrade.

Large Companies

Patreon transitioned from PHP → Python 3 in 2015. Key appeals for Patreon to switch to Python 3 included:

  • future-proofing
  • appeal to developer hiring, working with latest language
  • lower risk than porting to static typed language like Scala

Instagram runs fully on Python 3 as noted at the 2017 PyCon Keynote at the 13 minute mark:


Facebook is proud of its ongoing conversion to Python 3 as well, and uses this fact to help recruit top talent.

Linux Python 2 demotion

Starting in 2010, Arch Linux defaulted to Python 3. In 2017, Ubuntu 17.10 defaulted to Python 3, and Python 2 had to be manually installed. In 2018, Ubuntu 18.04 is cleaning out Python 2 from all programs in the main Ubuntu repository and defaults to Python 3.6. The goal is to demote Python from the main repository, requiring an extra step to install Python 2 from the “universe” repository.

Executable Python scripts should continue to have the first line

#!/usr/bin/env python

so that users can configure their desired Python version. Many users install a third party Python distribution such as Anaconda Python, PyCharm, Intel Python , etc. that have high performance math libraries such as PyCUDA.

Further Reading

  • Very detailed notes from Python Software Foundation Fellow Nick Coghlan on why, when, what, how, where of Python 3 transition with fascinating historical notes.
  • ActiveState saw majority of downloads being Python 3 since January 2017.

Categories:

Updated:

Leave a comment