Python 3.10 tracking issue

[freundTech]

Here's an issue for tracking the current state of mypy on CPython 3.10

CPython version: 3.10.0
mypy version: latest master (e6b91bd)

Dependencies:

• typed_ast is currently broken. Fixed on master, but no release to pypi yet (Allow compilation on python 3.10.0a6 typed_ast#155)
• typed_ast again broken in 3.10.0a7. PR Fix compilation on python 3.10.0a7 typed_ast#158
• lxml is currently broken due to a cython bug. The bug has been fixed in cython 0.29.22, but lxml hasn't been built with the new version yet (https://bugs.launchpad.net/lxml/+bug/1904900)
• virtualenv uses distutils, which is deprecated and will be removed in python 3.12. Mypy test-depends on virtualenv<20, and it looks like only virtualenv >=20 is still in active development. (Virtualenv pinned to <20 #10407)
• pytest breaks on python 3.10. This is already fixed upstream, but mypy pins pytest to <6.2, because of python 3.5 support

Tests:

• Some tests from FineGrainedSuite, FineGrainedCacheSuite and PythonCmdlineSuite` are broken, because cpython parser error messages changed. (Fix a bunch of tests broken by python 3.10 #10404)
  • Some occurrences of error: invalid syntax are now error: expected ':'
  • Some occurrences of error: unexpected EOF while parsing are now error: '(' was never closed
• Tests from SamplesSuite break with mypy/typeshed/stdlib/types.pyi:334: error: Name '_NotImplementedType' is not defined (Small fixes for Python 3.10 typeshed#5044)
• Enable CI for python 3.10 (Enable CI to use Python 3.10 #11294)

Features

• Pattern matching
  • Pattern matching is not yet supported (Support for python 3.10 match statement #10191)
  • Exhaustiveness checking for pattern matching (Reachability/exhaustiveness checking for match statements #12010)
  • Pattern matching in mypyc (Compiling code with match statements fails with an exception mypyc/mypyc#911)
• Support explicit TypeAlias (Support PEP 613 #9404)
• Support PEP 604 syntax for runtime use cases (Support PEP 604 syntax for runtime use cases #9880)
  • PEP 604 for TypeVar bounds new union syntax (types.Union) doesn't work on generic bounds #10949
• Support for dataclasses' kw_only ([dataclasses plugin] Support kw_only=True #10867)
• Support PEP 612 Parameter Specification Variables (PEP 612 tracking issue #8645)

Breaking changes

• mypyc is broken, because _PyObject_HasAttrId has been removed from CPython (bpo-41991: Remove _PyObject_HasAttrId cpython#22629). PR Fix mypyc failing to compile on CPython 3.10.0a6 #10202.
• mypy/sitepkgs.py imports distutils, which is deprecated and will be removed in python 3.12 (Only import distutils if it is needed #10203).

Misc

• Build Python 3.10 wheels
• Add Python 3.10 classifier

[hauntsaninja]

Here's another important part to 3.10 support: #9880

[erikkemperman]

Not sure if this ought to be mentioned here, but test-requirements.txt pins the pytest dependency to <6.2.0, because they dropped support for Python 3.5 with their 6.2 release.

However, the tests currently fail with the Python 3.10 beta due to this bug. I've submitted a PR to backport the fix to pytest 6.1.x but it was rejected because they have a policy of not backporting to versions prior to the latest minor release.

So I guess, assuming dropping Python 3.5 support is still not an option for mypy, that the dependencies would somehow have to be split in such a way that pytest 6.1.x is used with Python 3.5 and the latest pytest can be used for Python 3.6 and up. This particular bug was fixed in pytest 6.2.4, for what it's worth.

[domdfcoding]

Could you use PEP 508 markers in the requirements file to specify the older pytest for 3.5 and a newer version for 3.6 and later?

Something like pytest<6.2.0; python_version < "3.6"

[erikkemperman]

Perhaps. I'm not sure what the best way to do this "split" would be, to be honest, I just wanted to signal here that something may have to be done about this.

[freundTech]

Mypy is in the process of dropping python 3.5 support (See #9950) and CI runs on python 3.5 have already been disabled (#10345), so if python 3.5 support is dropped before python 3.10 is released the pin on pytest could just be removed entirely.

CC: @JukkaL

[cdce8p]

Python 3.10 also adds TypeAlias (PEP 613): #9404

[erikkemperman]

@freundTech Ah, sorry, I had not seen that. Yes, in that case it can just be set to >= 6.2.4 (to include the fix for this particular issue). Thanks!

[cdce8p]

3.10 also adds KW_ONLY args to dataclasses: #10865, open PR: #10867
https://docs.python.org/3.10/library/dataclasses.html#dataclasses.KW_ONLY

[domdfcoding]

I think the _NotImplementedType issue might be sorted with Python 3.10.0rc1. I'm no longer seeing it in my own code where mypy used to flag it, and running the mypy testsuite I can't see any failures related to it.

[hauntsaninja]

Likely because python/typeshed#5044 made it onto mypy master

[cdce8p]

Another feature: PEP 612 (Parameter Specification Variables) #8645

[JukkaL]

Now that Python 3.10 is out, we should enable CI using the release version. A few tests seem to be failing, however, and may need to be fixed first. Anybody want to help with this (#11294)?

Once CI is up, we should start building 3.10 wheels in https://github.com/mypyc/mypy_mypyc-wheels.