Fix subclassing builtin protocols on older Python versions

[ilevkivskyi]

This was recently fixed in CPython repo (while adding Protocol there). This PR backports the fix to all the older versions including now Python 2 version in typing.

I also removed the unnecessary conditional import in Python 2 tests since I stumbled at it while adding the test.

[jivid]

I believe there's a bug here because _PROTO_WHITELIST contains ContextManager and AsyncContextManager but neither of those come from the collections.abc module. This makes code like the following fail:

In [24]: from typing import ContextManager ...: from typing_extensions import Protocol ...: ...: class Foobar(ContextManager, Protocol): ...: pass ...: --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-24-fa1f28400733> in <module> 1 from typing import ContextManager 2 ----> 3 class Foobar(ContextManager, Protocol): 4 pass 5 /mnt/xarfuse/uid-4861/0260e378-ns-4026531840/typing_extensions.py in __init__(cls, *args, **kwargs) 1201 base.__origin__ is Generic): 1202 raise TypeError('Protocols can only inherit from other' -> 1203 ' protocols, got %r' % base) 1204 1205 def _no_init(self, *args, **kwargs): TypeError: Protocols can only inherit from other protocols, got typing.ContextManager

Is there something I'm missing here, or is there genuinely an issue that needs to be fixed by maintaining a map of module -> class name for everything in the _PROTO_WHITELIST list?

[ilevkivskyi]

Does this happen on all Python versions? I think it may be relevant only for 3.5, if not however it makes sense to fix this by making whitelisting more precise.

[jivid]

I'll see if I can repro in 2.7 as well, my error is from 3.6.

If it is 3.5+ only, would you have an alternate suggestion apart from having a more explicit module -> class name whitelist?

[jivid]

I can repro this in 2.7 as well:

In [4]: from typing import ContextManager ...: from typing_extensions import Protocol ...: ...: class Foobar(ContextManager, Protocol): ...: pass ...: --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-4-b18f9414f140> in <module>() 2 from typing_extensions import Protocol 3 ----> 4 class Foobar(ContextManager, Protocol): 5 pass /data/users/divij/fbsource/fbcode/buck-out/dev/gen/experimental/divij/test_smc_db_types-ipython#link-tree/typing_extensions.py in __init__(cls, *args, **kwargs) 187 isinstance(base, GenericMeta) and base.__origin__ is Generic): 188 raise TypeError('Protocols can only inherit from other protocols,' --> 189 ' got %r' % base) 190 cls._callable_members_only = all(callable(getattr(cls, attr)) 191 for attr in cls._get_protocol_attrs()) TypeError: Protocols can only inherit from other protocols, got typing.ContextManager In [5]: import sys In [6]: sys.version Out[6]: '2.7.14 (default, May 13 2019, 13:45:19) \n[GCC 7.x 20190403 (Facebook) 8.x]' 

[ilevkivskyi]

It looks like there is a bigger problem. I can reproduce this also in Python 3.8 (beta) and Protocol imported from typing. I think we need to fix it there first, and then apply the fix to this backport. The simplest fix may be indeed to keep the mapping. Could you please open an issue on https://bugs.python.org?

[jivid]

Sure, I'll do that right now. Thanks for following up @ilevkivskyi!

[jivid]

Created https://bugs.python.org/issue38008, I'll move the discussion there

[ilevkivskyi]

@jivid Now that this is fixed upstream, would you like to work on a backport?

[jivid]

@ilevkivskyi yes! Is it enough for me to make a similar change to the original PR in this repo, or do I need to do something else for backporting?

[ilevkivskyi]

Is it enough for me to make a similar change to the original PR in this repo, or do I need to do something else for backporting?

I don't remember for sure TBH, but most likely yes. Note however that you might need to add this change to more than one place. There are four places that might need to be updated: typing_extensions/src_py2, typing_extensions/src_py3, src, and python2. Also you would need to add tests for all of these versions (even if some of them already work).