[Local meetup talk] Porting Python application from Python 2 to 3
For those of you interested, here are the full set of slides I used for the talk - Talk slides .
TL;DR :
- Why should you port your applications from 2 to 3?
- What does it mean to port your code base?
- How can you port your packages/apps?
Why?
Python 2 end of life (EoL) is 2020. After it's EoL, there will be no more feature additions, changes or bug fix releases on the 2 or 2.7.x version of Python. You might not care about changes to the underlying Python language as long as your code keeps working. FYI any new security vulnerability will not be fixed on 2.7.x. Maybe you don't care about that either. FYI the packages you use will probably stop supporting Python 2.7.x. Some already have. Use numpy or pandas in your code base? They already stopped supporting Python 2 and all new feature releases will be Python 3 only. See here and here for more info.
What?
There are roughly four types of changes between Python 2 and 3
- Syntax changes. One of the most popular example has to be that print is a statement on Python 2 vs a function on Python 3)
- Behavioral changes in builtins. For example, `dict.keys()` will return a lazy iterable on Python 3 whereas on Python 2, it will return a list. This difference in behavior - lazy computation on Python 3 vs greedy computation on Python 2 - is something you will notice in general, not just with dictionary method calls. Another example is the change in integer division i.e.`1/2` in Python 2 is 0 where as on Python 3 it is `0.5`.
- Changes in the standard library. For example, some of the functionality provided by the cStringIO and StringIO modules got moved to the io module in Python 3 so cStringIO/StringIO don't exist on Python 3.
- Changes in the API. For example, the open builtin on Python 3 accepts an encoding argument whereas on Python 2, it doesn't.
This is just a very small list of changes between Python 2 and Python 3. In fact, I was talking to one of the organizers of the local ChennaiPy meetup after my talk and he was talking about how Python 2 and Python 3 are practically two different languages and not just two versions of the same language. For more information on what the differences between Python 2 and Python 3 are, see here. Obviously, the best source of information is the changelog published with every release of Python.
How?
There are four utilities I found very useful when porting Python applications from 2 to 3.
- flake8
- modernize
- __future__
- six
flake8 can be used to check for trivial Python 3 syntax errors in your code base. modernize can be used to automate the process of converting your code to support Python 2 and 3 or only Python 3. __future__ can be used to make Python 2 code behave like Python 3. Finally, six can act as a compatibility module between 2 and 3.
Strategies
- It's highly recommended that you version control your code base. If your code base is already version controlled, you can tell modernize to automatically write all of the recommended changes to the files, instead of displaying the recommendations on the screen and you having to manually make the changes.
- Make incremental changes across multiple PRs. modernize will by default apply a large number of changes to your code, which will be hard for you to comb through and hard for a reviewer to check and approve. Small commits, each of which make only one type of change/transformation, are recommended.
- Make sure that the code base continues to work on Python 2, as you slowly update it to Python 3. Do NOT break your code on Python 2 to speedup the process of converting it to 3.
- It's highly recommended that you unit test your code. This will make sure that your package continues to behave as expected on Python 2. And it will be easy to identify if your code behaves differently on Python 3.
There's a lot missing from this article, both in terms of the full list of changes between Python 2 and 3, and in terms of how the tools can be used to change Python 2 code to Python 3.
But, I have hopefully provided enough information for you to start.
Until next time.
TL;DR :
- Why should you port your applications from 2 to 3?
- What does it mean to port your code base?
- How can you port your packages/apps?
Why?
Python 2 end of life (EoL) is 2020. After it's EoL, there will be no more feature additions, changes or bug fix releases on the 2 or 2.7.x version of Python. You might not care about changes to the underlying Python language as long as your code keeps working. FYI any new security vulnerability will not be fixed on 2.7.x. Maybe you don't care about that either. FYI the packages you use will probably stop supporting Python 2.7.x. Some already have. Use numpy or pandas in your code base? They already stopped supporting Python 2 and all new feature releases will be Python 3 only. See here and here for more info.
What?
There are roughly four types of changes between Python 2 and 3
- Syntax changes. One of the most popular example has to be that print is a statement on Python 2 vs a function on Python 3)
- Behavioral changes in builtins. For example, `dict.keys()` will return a lazy iterable on Python 3 whereas on Python 2, it will return a list. This difference in behavior - lazy computation on Python 3 vs greedy computation on Python 2 - is something you will notice in general, not just with dictionary method calls. Another example is the change in integer division i.e.`1/2` in Python 2 is 0 where as on Python 3 it is `0.5`.
- Changes in the standard library. For example, some of the functionality provided by the cStringIO and StringIO modules got moved to the io module in Python 3 so cStringIO/StringIO don't exist on Python 3.
- Changes in the API. For example, the open builtin on Python 3 accepts an encoding argument whereas on Python 2, it doesn't.
This is just a very small list of changes between Python 2 and Python 3. In fact, I was talking to one of the organizers of the local ChennaiPy meetup after my talk and he was talking about how Python 2 and Python 3 are practically two different languages and not just two versions of the same language. For more information on what the differences between Python 2 and Python 3 are, see here. Obviously, the best source of information is the changelog published with every release of Python.
How?
There are four utilities I found very useful when porting Python applications from 2 to 3.
- flake8
- modernize
- __future__
- six
flake8 can be used to check for trivial Python 3 syntax errors in your code base. modernize can be used to automate the process of converting your code to support Python 2 and 3 or only Python 3. __future__ can be used to make Python 2 code behave like Python 3. Finally, six can act as a compatibility module between 2 and 3.
Strategies
- It's highly recommended that you version control your code base. If your code base is already version controlled, you can tell modernize to automatically write all of the recommended changes to the files, instead of displaying the recommendations on the screen and you having to manually make the changes.
- Make incremental changes across multiple PRs. modernize will by default apply a large number of changes to your code, which will be hard for you to comb through and hard for a reviewer to check and approve. Small commits, each of which make only one type of change/transformation, are recommended.
- Make sure that the code base continues to work on Python 2, as you slowly update it to Python 3. Do NOT break your code on Python 2 to speedup the process of converting it to 3.
- It's highly recommended that you unit test your code. This will make sure that your package continues to behave as expected on Python 2. And it will be easy to identify if your code behaves differently on Python 3.
There's a lot missing from this article, both in terms of the full list of changes between Python 2 and 3, and in terms of how the tools can be used to change Python 2 code to Python 3.
But, I have hopefully provided enough information for you to start.
Until next time.