__future__ is mysterious

Let me jump the gun and go straight to what I find mysterious -

Running `from __future__ import *` raises a SyntaxError. Specifically, you will get the error -

>>> from __future__ import *
  File "<stdin>", line 1
SyntaxError: future feature * is not defined

Now, that's weird and mysterious.

For those of you who are not steeped in Python land, __future__ is part of Python's standard library. It is a library that can make the version of Python you use currently behave like a __future__ version of Python. For example, adding the line `from __future__ import print_function` on Python 2.7 will change print into a function (Python 3) instead of being a statement (Python 2).

Now, so far, there's not much that's mysterious about the __future__ module. Sure, it's an interesting way to make one version of a programming language behave like another version.

One of the __future__ module imports is barry_as_FLUFL , which is an easter egg. It was introduced in the PEP https://www.python.org/dev/peps/pep-0401/. To quote the PEP -

Recognized that the != inequality operator in Python 3.0 was a horrible, finger pain inducing mistake, the FLUFL reinstates the <> diamond operator as the sole spelling. This change is important enough to be implemented for, and released in Python 3.1. To help transition to this feature, a new future statement, from __future__ import barry_as_FLUFL has been added.

So, adding `from __future__ import barry_as_FLUFL` in your module will make the use of `!=` a SyntaxError and the only way to check for inequality in the Python code would be to use `<>` aka the diamond operator.

Now, here's the mystery. In Python, all names defined in a module can be imported from any other module by doing `from package.module import *`. So, if someone were to accidentally use `from __future__ import *`, they would implicitly be modifying the inequality operator and all hell would break loose.

But, this isn't what happens. Like I mentioned at the beginning, when you do `from __future__ import *`, you will get a SyntaxError, specifically 

>>> from __future__ import *
  File "<stdin>", line 1
SyntaxError: future feature * is not defined

Now, that's weird and mysterious.

Note that the behavior of `from package.module import *` can be overridden by the module to specify what exactly can be imported from the module. The module can define an `__all__` name, which is a list variable containing names that can be imported from the module. The `__future__` module defines one such list - https://github.com/python/cpython/blob/ff27f8145d7194fb3891b610443dee15be8f8f63/Lib/__future__.py#L50-L63

all_feature_names = [
    "nested_scopes",
    "generators",
    "division",
    "absolute_import",
    "with_statement",
    "print_function",
    "unicode_literals",
    "barry_as_FLUFL",
    "generator_stop",
    "annotations",
]

__all__ = ["all_feature_names"] + all_feature_names

Defining the __all__ in this way prevents the `CO_***` variables and the `_Feature` class defined in the module.

But, this still doesn't explain the `SyntaxError` we saw earlier when we tried importing `from __future__ import *`.

Looking for the error message in the Python code base, we find the future.c module - https://github.com/python/cpython/blob/808180c206fbde390d9dbdf24a8989fc8a6446ec/Python/future.c#L10 where the SyntaxError is raised with the appropriate message.

So, at the moment, it looks like the future.c module is overriding the standard behavior of `from package.module import *`.

Haven't had the time to dig deeper into this mystery. If you know the solution to this mystery, tell me!!! Or give me hints and see if I can figure it out. I'll hopefully have time to spend sometime later this week and figure this out.

Until then ...

Further reading :

If you're interested, here's the Python Enhancement Proposal (PEP) which proposed the behavioral change - https://www.python.org/dev/peps/pep-3105, here's the documentation - https://docs.python.org/3/library/__future__.html and here's the __future__ module source code in the Python standard library - https://github.com/python/cpython/blob/3.7/Lib/__future__.py .