improvement: Add AST subclass constructors

[bzoracler]

Fixes #8378

• ast.AST no longer owns attributes which may not exist on all subclasses (node locations, type_comment). These are all shifted to subclasses which actually own the attributes (d5e0838, 75a2e8e, f9a2cb9)
• ast.AST no longer has the constructor __init__(self, *args: Any, **kwargs: Any).
• All subclasses of ast.AST which have a non-empty _fields: ClassVar[tuple[str, ...]] or _attributes: ClassVar[tuple[str, ...]] now have their own __init__ constructor (d886a45). The constructor has:
  • Positional-or-keyword arguments in the order in which they appear in _fields, and
  • Non-mandatory keyword-only arguments from _attributes. These are expressed using a combination of typing.TypedDict and typing_extensions.Unpack.
    • These are non-mandatory because I expect most users won't pass explicit locations, but rather use something like ast.fix_missing_locations.

---

Updates:

• Arguments of type ast.expr_context are not required to be passed to the constructor, but they will not exist on node instances for Python < 3.13.
• Arguments of type list are not required to be passed to the constructor for Python >= 3.13.

[JelleZijlstra]

Thanks for this, I had been thinking of adding it but hadn't gotten around to it.

[JelleZijlstra]

These list fields should default to the empty list on 3.13+.

[bzoracler]

Addressed in c82a5fa

[JelleZijlstra]

I don't think type_comment needs to be keyword-only on this overload.

[bzoracler]

Addressed in 9655775

[bzoracler]

I didn't add these other changes from Python 3.13 runtime to this PR:

1. AST._field_types - not sure what the intention is for the values of these dictionaries; are they meant to be for runtime introspection, or can they be anything?
2. The default_value parameter for ast.TypeVar, ast.ParamSpec, ast.TypeVarTuple - the runtime suggests that this defaults to None if not passed, but the grammar does not mark this field as optional. (I misread the grammar, they're definitely optional)

[JelleZijlstra]

1. AST._field_types - not sure what the intention is for the values of these dictionaries; are they meant to be for runtime introspection, or can they be anything?

It's used by the constructor of AST nodes to figure out what defaults to use. I did document it, so I think we should add it to the ast.AST class as ClassVar[dict[str, Any]].

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

pyp (https://github.com/hauntsaninja/pyp) + pyp.py:585: error: "AST" has no attribute "lineno" [attr-defined] + pyp.py:586: error: "AST" has no attribute "end_lineno" [attr-defined] sphinx (https://github.com/sphinx-doc/sphinx) + sphinx/ext/autodoc/preserve_defaults.py: note: In function "get_default_value": + sphinx/ext/autodoc/preserve_defaults.py:114:12: error: "AST" has no attribute "lineno" [attr-defined] + sphinx/ext/autodoc/preserve_defaults.py:114:31: error: "AST" has no attribute "end_lineno" [attr-defined] + sphinx/ext/autodoc/preserve_defaults.py:115:26: error: "AST" has no attribute "lineno" [attr-defined] + sphinx/ext/autodoc/preserve_defaults.py:116:25: error: "AST" has no attribute "col_offset" [attr-defined] + sphinx/ext/autodoc/preserve_defaults.py:116:45: error: "AST" has no attribute "end_col_offset" [attr-defined] mypy (https://github.com/python/mypy) + mypy/fastparse.py:1998: error: "Index" has no attribute "col_offset" [attr-defined] + mypy/fastparse.py:2001: error: "Slice" has no attribute "col_offset" [attr-defined] + mypy/fastparse.py:2002: error: Argument 1 to "Tuple" has incompatible type "List[slice]"; expected "List[expr]" [arg-type] pytest (https://github.com/pytest-dev/pytest) + src/_pytest/assertion/rewrite.py:839: error: Argument 1 to "Dict" has incompatible type "list[Constant]"; expected "list[expr | None]" [arg-type] + src/_pytest/assertion/rewrite.py:839: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance + src/_pytest/assertion/rewrite.py:839: note: Consider using "Sequence" instead, which is covariant + src/_pytest/assertion/rewrite.py:932: error: Argument 3 to "If" has incompatible type "list[If]"; expected "list[stmt]" [arg-type] + src/_pytest/assertion/rewrite.py:932: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance + src/_pytest/assertion/rewrite.py:932: note: Consider using "Sequence" instead, which is covariant + src/_pytest/assertion/rewrite.py:938: error: Argument 1 to "Assign" has incompatible type "list[Name]"; expected "list[expr]" [arg-type] + src/_pytest/assertion/rewrite.py:938: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance + src/_pytest/assertion/rewrite.py:938: note: Consider using "Sequence" instead, which is covariant + src/_pytest/assertion/rewrite.py:963: error: Argument 1 to "Assign" has incompatible type "list[Name]"; expected "list[expr]" [arg-type] + src/_pytest/assertion/rewrite.py:963: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance + src/_pytest/assertion/rewrite.py:963: note: Consider using "Sequence" instead, which is covariant + src/_pytest/assertion/rewrite.py:1145: error: Argument 1 to "Tuple" has incompatible type "list[Constant]"; expected "list[expr]" [arg-type] + src/_pytest/assertion/rewrite.py:1145: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance + src/_pytest/assertion/rewrite.py:1145: note: Consider using "Sequence" instead, which is covariant + src/_pytest/assertion/rewrite.py:1146: error: Argument 1 to "Tuple" has incompatible type "list[Name]"; expected "list[expr]" [arg-type] + src/_pytest/assertion/rewrite.py:1146: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance + src/_pytest/assertion/rewrite.py:1146: note: Consider using "Sequence" instead, which is covariant + src/_pytest/assertion/rewrite.py:1147: error: Argument 1 to "Tuple" has incompatible type "list[Constant]"; expected "list[expr]" [arg-type] + src/_pytest/assertion/rewrite.py:1147: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance + src/_pytest/assertion/rewrite.py:1147: note: Consider using "Sequence" instead, which is covariant + src/_pytest/assertion/rewrite.py:1151: error: Argument 2 to "BoolOp" has incompatible type "list[Name]"; expected "list[expr]" [arg-type] + src/_pytest/assertion/rewrite.py:1151: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance + src/_pytest/assertion/rewrite.py:1151: note: Consider using "Sequence" instead, which is covariant + testing/test_assertrewrite.py:133: error: "AST" has no attribute "lineno" [attr-defined] + testing/test_assertrewrite.py:134: error: "AST" has no attribute "col_offset" [attr-defined] + testing/test_assertrewrite.py:135: error: "AST" has no attribute "end_lineno" [attr-defined] + testing/test_assertrewrite.py:136: error: "AST" has no attribute "end_col_offset" [attr-defined] werkzeug (https://github.com/pallets/werkzeug) + src/werkzeug/routing/rules.py:785: error: Argument 1 to "JoinedStr" has incompatible type "list[AST]"; expected "list[expr]" [arg-type] + src/werkzeug/routing/rules.py:788: error: List item 0 has incompatible type "AST"; expected "expr" [list-item] + src/werkzeug/routing/rules.py:788: error: List item 1 has incompatible type "AST"; expected "expr" [list-item] + src/werkzeug/routing/rules.py:818: error: "AST" has no attribute "lineno" [attr-defined] + src/werkzeug/routing/rules.py:820: error: "AST" has no attribute "end_lineno" [attr-defined] + src/werkzeug/routing/rules.py:820: error: "AST" has no attribute "lineno" [attr-defined] + src/werkzeug/routing/rules.py:822: error: "AST" has no attribute "col_offset" [attr-defined] + src/werkzeug/routing/rules.py:824: error: "AST" has no attribute "end_col_offset" [attr-defined] + src/werkzeug/routing/rules.py:824: error: "AST" has no attribute "col_offset" [attr-defined] ibis (https://github.com/ibis-project/ibis) + ibis/backends/bigquery/udf/core.py:66: error: Missing positional arguments "args", "keywords" in call to "Call" [call-arg] + ibis/backends/bigquery/udf/core.py:79: error: Missing positional arguments "args", "keywords" in call to "Call" [call-arg] + ibis/backends/bigquery/udf/core.py:85: error: Missing positional arguments "args", "keywords" in call to "Call" [call-arg] + ibis/backends/bigquery/udf/core.py:85: error: Missing positional argument "value" in call to "Attribute" [call-arg] + ibis/backends/bigquery/udf/core.py:95: error: Missing positional arguments "args", "keywords" in call to "Call" [call-arg] flake8-pyi (https://github.com/PyCQA/flake8-pyi) + pyi.py:2249: error: "AST" has no attribute "lineno" [attr-defined] + pyi.py:2249: error: "AST" has no attribute "col_offset" [attr-defined] streamlit (https://github.com/streamlit/streamlit) + lib/streamlit/runtime/scriptrunner/magic.py: note: In function "_build_st_write_call": + lib/streamlit/runtime/scriptrunner/magic.py:166:12: error: Unexpected keyword argument "kwargs" for "Call"; did you mean "args"? [call-arg] + note: "Call" defined here + lib/streamlit/runtime/scriptrunner/magic.py:166:12: error: Unexpected keyword argument "starargs" for "Call" [call-arg] + note: "Call" defined here 

[JelleZijlstra]

Thank you!