We like to keep up to date with what’s going on in the Python ecosystem. Today, we’ll be sharing with you some of the most important Python news from the past few months.
New versions of Python semantics provide a lot of shortcuts, and learning about them can only make your code better. As the language evolves, it can do more and provide better performance. The sooner you learn how to update your code to take advantage of Python’s technological improvements, the better for you and your team.
In this article, we will analyze the essential changes and new features that come with Python 3.7, 3.8, and 3.9.
Before we get into it, we’ll also take a very quick look at changes in the Python ecosystem, because there have been several developments that affect the whole Python community.
We’ve included a quick recap of Python basics in the first part of this article. If you don’t need it, you can go straight to “Key news from the Python ecosystem”.
Table of contents
IV. Final thoughts
1. What is Python?
Python is an object-oriented programming language that was developed in 1991 by Guido van Rossum.
The governing principles of Python are encapsulated in the Zen of Python. You can, in fact, call them up with the ‘import this’ command, which is a little Easter egg. It has been included in Python since the PEP 20 - The Zen of Python. It contains 19 proverbs, and it’s essentially a list of principles for writing software that influenced the design patterns of Python.
The set contains general statements, for instance “complex is better than complicated”, and “special cases aren’t special enough to break the rules”. But there are also several very specific statements, such as “There should be one—and preferably only one—obvious way to do it.
Although that way may not be obvious at first unless you're Dutch”.
What are PEPs?
PEPs are Python Enhancement Proposals. There are three types of PEPs:
To propose changes to the Python code library.
To inform the Python community and aggregate important information.
To suggest broad changes in the Python ecosystem, e.g. how decisions are made, the development path, etc.
PEPs have to be co-authored by one of Python’s core developers. If you want to submit a PEP but don’t work with a Python core developer, you need to get one of them to become your ‘sponsor’. Their role is essentially to guide you through the complex process of PEP review.
The Steering Council, a select group of highest-ranking Python creators and experts, has the final authority to decide which PEPs are approved. More information about PEPs can be found in PEP 1.
One thing that might confuse you when you read the PEP 1 is that the primary audience for PEPs are “the core developers of the CPython reference interpreter and their elected Steering Council, as well as developers of other implementations of the Python language specification”. CPython?
2. What is CPython?
It’s confusing because when you think Python, you don’t think CPython. But it turns out that there is no Python without CPython. At least if you don’t consider other Python implementations.
Programming languages don’t just ‘exist’, they have to somehow be interpreted by the machine. The main interpreter for Python, the one that Guido van Rossum created using the C programming language, is the CPython implementation.
Other implementations include Jython (in Java) and PyPy (in RPython, which is a subset of Python). But CPython is the original implementation that comes with the Python download package on Python.org.
3. Who is Guido van Rossum?
A professional programmer from the Netherlands who had contributed a lot to the industry even before creating Python. His previous accomplishments include contributing a glob() routine to BSD Unix, and co-developing the programming language ABC.
Python was first published in February 1991, two years after Guido started working on it as a side project. In 2019, a year after renouncing the title of Python’s Benevolent Dictator For Life (BDFL), van Rossum retired from his career full of exceptional programming accomplishments.
4. Python: the origin of the name
The legend goes that Guido van Rossum was in a humorous mood, and as a fan of English comedy, he borrowed the name from Monty Python.
1. Python versions
One of the biggest changes in the Python verse was the long-awaited announcement of the demise of Python 2, and the deadline after which no new updates (including security updates) will be added by the core Python development team. The release of new updates for version 2 ended on January 1st, 2020. For more details, see PEP-0373.
This means that Python developers need to migrate to one of the Python 3 versions (such as 3.8 stable)—at least those who rely on upgrades from the official Python development team.
The Python team has been maintaining both Python 2 and Python 3 simultaneously since 2006, but this year marks the end of that era. It was pushed back till 2015, when official support for Python 2 was supposed to end.
Python 3 is not compatible with Python 2, and it was actually slower than Python 2 when it was first released. These complications stopped a lot of people from migrating in 2008, and a lot of them didn’t do so even in 2015. Now, if they want to benefit from official support for the language, they have no other choice than to migrate to Python 3.
2. Python’s BDFL steps down
After years of trying to convince developers to move to Python 3, all the while maintaining two versions of Python to satisfy everyone, the Python development team has had enough.
And let’s face it, at this point it was becoming kind of silly. The date when support for Python was to stop had already been postponed. Originally, it was planned for 2015.
The other big story, which broke out a year ago but might not be widely known, is that the creator of Python, Guido van Rossum, stepped down as Python’s Benevolent Dictator for Life.
Right now, the Steering Council has to make do without Guido for the first time in 30 years. But, as he said in his announcement, there are many excellent candidates to take his place.
Thank you for everything, Guido!
a) PEP 553—Built-in breakpoint()
breakpoint() function can be used to debug a running application or a script. It enters a debugger at the point where it’s placed.
Until 3.7, the standard Python debugger was pdb:
From 3.7 on, you can use the breakpoint() instead:
Who is this for? The thing is that you can already set it automatically in your IDE. So if you use, for example, PyCharm, this won’t be revolutionary to you.
However, the main benefit is that with
breakpoint(), the Python development team was able to address several disadvantages of pdb, the previous debugger. The new function is just better, and it simplifies things a lot.
b) PEP 564—Add new time functions with nanosecond resolution
To understand the impact of this improvement, we need to take a look at the problem of nanoseconds in Python.
time.time() relies on a 64-bit binary floating-point number for precision. But this has a serious disadvantage. After 104 days, the ‘float’ type starts to lose nanoseconds.
Take a look at the example from the PEP. Converting int nanoseconds into float seconds, and back to int nanoseconds, is a good way to visualize this problem:
To aid solving problems that might stem from this, six new functions have been added:
time.clock_settime_ns(clock_id, time: int)
The different functions enable greater precision for developers. They return a number of nanoseconds in the format of a Python int. With these alternatives, developers can avoid using the problematic time.time() function.
c) PEP 557—Data Classes
This PEP introduced a new class for data, which can be applied using the
@dataclass decorator, so the developer doesn’t need to change anything—unless it’s a case where special logic is required.
@dataclass is a sort of wrapper that contains other functions or classes. Here’s an example:
@dataclass decorator creates special “dunder” (double-under-double-wonder) functions for us, which are:
__init__ - constructor
__repr__ - string object representation
__eq__ - object comparison method using the == operator
__ne__- object comparison method using the != operator
__lt__- object comparison method using the < operator
__le__- object comparison method using the <= operator
__gt__- object comparison method using the > operator
__ge__- object comparison method using the >= operator
The cool part is that
@dataclasses compares fields by itself, it doesn’t need anything more.
3.7 brought about the introduction of the importlib.resources library. It is used as an easy way to open packages and read data from them. It makes it simple to extract data from non-Python files.
Take a look at this example from 3.6:
And now, Python 3.7 and importlib:
If you’re using Python 3.6 or lower, you can look for the importlib_resources library, which serves as a backport of this function.
e) PEP 563—Postponed Evaluation of Annotations
This introduces a special
_annotations_ module. Most importantly, the module is used to hold string representations of function and variable annotations. Instead of evaluating them while defining the function, they become strings stored in
Take a look at a similar function from 3.6:
In 3.7, it’s not a string representation anymore, but a class name that is evaluated within annotations:
From 3.7 and up, the first line (
from __future__ import annotations) is required. If you don’t add it at the top of your file, the compiler will raise a NameError exception, just as if the feature didn’t exist.
According to the PEP, this annotation will be a default in 4.0. Why should you need it?
One good example is using a style checker to verify function/variable output data with the data in the annotation. You can use MyPy for static code analysis like this. Also, this feature was designed with its precursors in mind (meaning PEP484 and PEP0526), and it enables those typing executions to perform up to seven times faster than before.
f) PEP 567—Context Variables
This is a new mechanism for managing context variables. The PEP explains this as an additional element that simplifies the management of variables in asynchronous programming (e.g., when you use asyncio).
Two new classes were introduced:
.Context, you can extract various data from the same context variable, which is adequate for data exchanged in asynchronous jobs.
To use context variables, the PEP suggests declaring an object of
ContextVar, that serves as a sort of key to your stored data in that particular context. Then you can use ‘get’ to receive the current value, and ‘set’ to modify the variable’s value.
Here’s a simple example that shows how using
contextVar differs from using a normal variable:
And for a complex example of
Context.run, take a look at this:
First, just like in this simple example, we create a context variable.
Then we define the main function, in which we change the context variable using ‘set’.
Next, we copy the context beyond the scope of the main function that hasn’t been called out yet. Any context copying in the asyncio environment translates into slightly different data, called a ‘weak reference’.
We run the main function in context. As you can see, it accepts new parameters, which are exchanged in run.
It’s not the only intriguing thing about Context, and if you’d like to know more about it, analyze the PEP.
a) F-Strings—self-documenting expressions (debug)
This enables you to print out the value of a variable. It was introduced in version 3.6 and enabled printing and formatting text (instead of
.format). In 3.8, F-strings became enriched with a debugging option.
As you can see, there’s also the
‘name’ to ‘name=’ change. It enables debug printing of the variable, as it prints the value and the name of the variable.
b) PEP 572—Assignment Expressions := (the infamous walrus)
This is the most controversial of all changes in Python 3.8. It introduced a new notation for assigning value, and it’s
It was nicknamed 'the walrus' because the symbol kind of resembles the animal’s fangs and eyes. This new notation makes it possible to assign value while creating a different expression, e.g. list-comprehension.
The walrus was actually one of the important reasons why Guido van Rossum resigned from the role of the BDFL of Python (if you don’t know what BDFL means, check out the Quick Recap—Python Basics section of this article).
You can declare a variable within the ‘if’ statement. You don’t need to declare it before the function.
c) PEP 570—Python Positional-Only Parameters
In Python there are positional parameters and keyword parameters. This causes confusion.
To help you avoid this confusion, this PEP introduces new syntax. It adds a slash ‘/’, which can be used to set parameters as positional-only.
The 3.9 version is still in alpha, and it’s test-only. This means you shouldn’t use it for your production codebase. There are plenty of small changes, and there are no new features yet, just optimizations and bug fixes. The following optimization is the most important:
parser module was used to extract data from Python files, essentially to parse Python with Python. Now it’s outdated, and the Python development team recommends against using
parser, which is going to be removed in newer versions. It’s now advisable to use
Ast took over
parser’s role about the time of version 3.2, but its role is increasing now. Thanks to ast, we can analyze Python code within Python code, and extract information such as the number of times you imported a certain module.
ast.dump function received a new parameter: ‘indent’. It comes with the option to add multiple lines when moving the contents of parsed Python code into the console.
When analyzing this feature, I came across a package called ASTPRETTY, which also enables multiple lines. It does the same thing as ast.dump, without the indent and tabulation.
The nomenclature is slightly different here:
c) Running the Python 3.9 alpha version
There are three main ways to start the alpha:
- Using Docker
Docker is a suite for virtualization and containerization, where you can run your app outside of your system libraries. To start Python 3.9, you need to get Docker:
docker pull python:3.9.0a4
And then run it:
docker run -it python:3.9.0a4 /bin/python
- Using PyENV
First, install PyENV according to the instructions on GitHub.
Next, you’ll be able to install 3.9 through PyENV with:
pyenv install 3.9-dev
But that’s not all. You need to add the plugin to be able to create a virtual environment:
git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv
Then you create virtualenv:
pyenv virtualenv 3.9-dev venv3.9
And finally, activate virtualenv:
pyenv activate venv3.9
- Using the Ubuntu/Debian PPA
The last option is the Ubuntu PPA.
You can install Python 3.9 using apt-get and add-apt-repository:
sudo apt-get install -y software-properties-common
sudo add-apt-repository ppa:deadsnakes/nightly
sudo apt update && sudo apt install python3.9
Python is evolving all the time, and the Python developer community has been faced with several big changes lately.
Will things be different now that the original creator has retired from his role as the almighty arbiter of all things Python?
Python has always been a strongly community-driven project, and it will likely continue in this spirit. Apart from Guido van Rossum stepping down, there are no changes to the process for implementing changes to the technology and the ecosystem.
Having said ‘goodbye’ to Python 2, it’s possible that the development of new Python versions will go faster. But this is just speculation, we’ll have to wait and see for ourselves.
In the meantime, if you’re hungry for more quality analysis, we have an extensive collection of other articles about Python on our blog. We thought you might find these ones particularly interesting:
- Python 2.7 to 3.X Migration Guide: How to Port from Python 2 to Python 3
- 5 Reasons Why You Should Migrate from Python 2 to Python 3
- The Best Python IDEs and Code Editors (According to Our Developers and the Python Community)
- How to Learn Python: Top Sites and Courses, from Beginner to Pro
Last but not least, if you have any questions, feel free to get in touch! We’d love to hear from you.