[ty] Infer type for implicit self parameters in method bodies (#20922)

## Summary Infer a type of `Self` for unannotated `self` parameters in methods of classes. part of astral-sh/ty#159 closes astral-sh/ty#1081 ## Conformance tests changes ```diff +enums_member_values.py:85:9: error[invalid-assignment] Object of type `int` is not assignable to attribute `_value_` of type `str` ``` A true positive :heavy_check_mark: ```diff -generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `Self@method2` -generics_self_basic.py:14:9: error[type-assertion-failure] Argument does not have asserted type `Self@set_scale ``` Two false positives going away :heavy_check_mark: ```diff +generics_syntax_infer_variance.py:82:9: error[invalid-assignment] Cannot assign to final attribute `x` on type `Self@__init__` ``` This looks like a true positive to me, even if it's not marked with `# E` :heavy_check_mark: ```diff +protocols_explicit.py:56:9: error[invalid-assignment] Object of type `tuple[int, int, str]` is not assignable to attribute `rgb` of type `tuple[int, int, int]` ``` True positive :heavy_check_mark: ``` +protocols_explicit.py:85:9: error[invalid-attribute-access] Cannot assign to ClassVar `cm1` from an instance of type `Self@__init__` ``` This looks like a true positive to me, even if it's not marked with `# E`. But this is consistent with our understanding of `ClassVar`, I think. :heavy_check_mark: ```py +qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__` +qualifiers_final_annotation.py:65:9: error[invalid-assignment] Cannot assign to final attribute `ID7` on type `Self@method1` ``` New true positives :heavy_check_mark: ```py +qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__` +qualifiers_final_annotation.py:57:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__` +qualifiers_final_annotation.py:59:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__` ``` This is a new false positive, but that's a pre-existing issue on main (if you annotate with `Self`): https://play.ty.dev/3ee1c56d-7e13-43bb-811a-7a81e236e6ab ❌ => reported as astral-sh/ty#1409 ## Ecosystem * There are 5931 new `unresolved-attribute` and 3292 new `possibly-missing-attribute` attribute errors, way too many to look at all of them. I randomly sampled 15 of these errors and found: * 13 instances where there was simply no such attribute that we could plausibly see. Sometimes [I didn't find it anywhere](https://github.com/internetarchive/openlibrary/blob/8644d886c6579a5f49faadc4cd1ba9992e603d7e/openlibrary/plugins/openlibrary/tests/test_listapi.py#L33). Sometimes it was set externally on the object. Sometimes there was some [`setattr` dynamicness going on](https://github.com/pypa/setuptools/blob/a49f6b927d83b97630b4fb030de8035ed32436fd/setuptools/wheel.py#L88-L94). I would consider all of them to be true positives. * 1 instance where [attribute was set on `obj` in `__new__`](https://github.com/sympy/sympy/blob/9e87b44fd43572b9c4cc95ec569a2f4b81d56499/sympy/tensor/array/array_comprehension.py#L45C1-L45C36), which we don't support yet * 1 instance [where the attribute was defined via `__slots__` ](https://github.com/spack/spack/blob/e250ec0fc81130b708a8abe1894f0cc926880210/lib/spack/spack/vendor/pyrsistent/_pdeque.py#L48C5-L48C14) * I see 44 instances [of the false positive above](astral-sh/ty#1409) with `Final` instance attributes being set in `__init__`. I don't think this should block this PR. ## Test Plan New Markdown tests. --------- Co-authored-by: Shaygan Hooshyari <sh.hooshyari@gmail.com>

Author: sharkdp

crates/ruff_benchmark/benches/ty.rs

@@ -667,7 +667,7 @@ fn attrs(criterion: &mut Criterion) {
  max_dep_date: "2025-06-17",
  python_version: PythonVersion::PY313,
  },
- 100,
+ 110,
  );
 
  bench_project(&benchmark, criterion);
@@ -684,7 +684,7 @@ fn anyio(criterion: &mut Criterion) {
  max_dep_date: "2025-06-17",
  python_version: PythonVersion::PY313,
  },
- 100,
+ 150,
  );
 
  bench_project(&benchmark, criterion);

crates/ruff_benchmark/benches/ty_walltime.rs

@@ -210,7 +210,7 @@ static TANJUN: Benchmark = Benchmark::new(
  max_dep_date: "2025-06-17",
  python_version: PythonVersion::PY312,
  },
- 100,
+ 320,
 );
 
 static STATIC_FRAME: Benchmark = Benchmark::new(
@@ -226,7 +226,7 @@ static STATIC_FRAME: Benchmark = Benchmark::new(
  max_dep_date: "2025-08-09",
  python_version: PythonVersion::PY311,
  },
- 630,
+ 750,
 );
 
 #[track_caller]

crates/ty_ide/src/completion.rs

@@ -1957,14 +1957,34 @@ class Quux:
 ",
  );
 
- // FIXME: This should list completions on `self`, which should
- // include, at least, `foo` and `bar`. At time of writing
- // (2025-06-04), the type of `self` is inferred as `Unknown` in
- // this context. This in turn prevents us from getting a list
- // of available attributes.
- //
- // See: https://github.com/astral-sh/ty/issues/159
- assert_snapshot!(test.completions_without_builtins(), @"<No completions found>");
+ assert_snapshot!(test.completions_without_builtins(), @r"
+ bar
+ baz
+ foo
+ __annotations__
+ __class__
+ __delattr__
+ __dict__
+ __dir__
+ __doc__
+ __eq__
+ __format__
+ __getattribute__
+ __getstate__
+ __hash__
+ __init__
+ __init_subclass__
+ __module__
+ __ne__
+ __new__
+ __reduce__
+ __reduce_ex__
+ __repr__
+ __setattr__
+ __sizeof__
+ __str__
+ __subclasshook__
+ ");
  }
 
  #[test]

crates/ty_ide/src/semantic_tokens.rs

@@ -1798,23 +1798,23 @@ class BoundedContainer[T: int, U = str]:
  "T" @ 554..555: TypeParameter
  "value2" @ 557..563: Parameter
  "U" @ 565..566: TypeParameter
- "self" @ 577..581: Variable
+ "self" @ 577..581: TypeParameter
  "value1" @ 582..588: Variable
  "T" @ 590..591: TypeParameter
  "value1" @ 594..600: Parameter
- "self" @ 609..613: Variable
+ "self" @ 609..613: TypeParameter
  "value2" @ 614..620: Variable
  "U" @ 622..623: TypeParameter
  "value2" @ 626..632: Parameter
  "get_first" @ 642..651: Method [definition]
  "self" @ 652..656: SelfParameter
  "T" @ 661..662: TypeParameter
- "self" @ 679..683: Variable
+ "self" @ 679..683: TypeParameter
  "value1" @ 684..690: Variable
  "get_second" @ 700..710: Method [definition]
  "self" @ 711..715: SelfParameter
  "U" @ 720..721: TypeParameter
- "self" @ 738..742: Variable
+ "self" @ 738..742: TypeParameter
  "value2" @ 743..749: Variable
  "BoundedContainer" @ 798..814: Class [definition]
  "T" @ 815..816: TypeParameter [definition]

crates/ty_python_semantic/resources/mdtest/annotations/self.md

@@ -56,24 +56,38 @@ In instance methods, the first parameter (regardless of its name) is assumed to
 
 ```toml
 [environment]
-python-version = "3.11"
+python-version = "3.12"
 ```
 
 ```py
 from typing import Self
 
 class A:
  def implicit_self(self) -> Self:
- # TODO: This should be Self@implicit_self
- reveal_type(self) # revealed: Unknown
+ reveal_type(self) # revealed: Self@implicit_self
 
  return self
 
- def a_method(self) -> int:
- def first_arg_is_not_self(a: int) -> int:
+ def implicit_self_generic[T](self, x: T) -> T:
+ reveal_type(self) # revealed: Self@implicit_self_generic
+
+ return x
+
+ def method_a(self) -> None:
+ def first_param_is_not_self(a: int):
  reveal_type(a) # revealed: int
- return a
- return first_arg_is_not_self(1)
+ reveal_type(self) # revealed: Self@method_a
+
+ def first_param_is_not_self_unannotated(a):
+ reveal_type(a) # revealed: Unknown
+ reveal_type(self) # revealed: Self@method_a
+
+ def first_param_is_also_not_self(self) -> None:
+ reveal_type(self) # revealed: Unknown
+
+ def first_param_is_explicit_self(this: Self) -> None:
+ reveal_type(this) # revealed: Self@method_a
+ reveal_type(self) # revealed: Self@method_a
 
  @classmethod
  def a_classmethod(cls) -> Self:
@@ -127,19 +141,16 @@ The name `self` is not special in any way.
 ```py
 class B:
  def name_does_not_matter(this) -> Self:
- # TODO: Should reveal Self@name_does_not_matter
- reveal_type(this) # revealed: Unknown
+ reveal_type(this) # revealed: Self@name_does_not_matter
 
  return this
 
  def positional_only(self, /, x: int) -> Self:
- # TODO: Should reveal Self@positional_only
- reveal_type(self) # revealed: Unknown
+ reveal_type(self) # revealed: Self@positional_only
  return self
 
  def keyword_only(self, *, x: int) -> Self:
- # TODO: Should reveal Self@keyword_only
- reveal_type(self) # revealed: Unknown
+ reveal_type(self) # revealed: Self@keyword_only
  return self
 
  @property
@@ -165,8 +176,7 @@ T = TypeVar("T")
 
 class G(Generic[T]):
  def id(self) -> Self:
- # TODO: Should reveal Self@id
- reveal_type(self) # revealed: Unknown
+ reveal_type(self) # revealed: Self@id
 
  return self
 
@@ -252,6 +262,20 @@ class LinkedList:
 reveal_type(LinkedList().next()) # revealed: LinkedList
 ```
 
+Attributes can also refer to a generic parameter:
+
+```py
+from typing import Generic, TypeVar
+
+T = TypeVar("T")
+
+class C(Generic[T]):
+ foo: T
+ def method(self) -> None:
+ reveal_type(self) # revealed: Self@method
+ reveal_type(self.foo) # revealed: T@C
+```
+
 ## Generic Classes
 
 ```py
@@ -342,31 +366,28 @@ b: Self
 
 # TODO: "Self" cannot be used in a function with a `self` or `cls` parameter that has a type annotation other than "Self"
 class Foo:
- # TODO: rejected Self because self has a different type
+ # TODO: This `self: T` annotation should be rejected because `T` is not `Self`
  def has_existing_self_annotation(self: T) -> Self:
  return self # error: [invalid-return-type]
 
  def return_concrete_type(self) -> Self:
- # TODO: tell user to use "Foo" instead of "Self"
+ # TODO: We could emit a hint that suggests annotating with `Foo` instead of `Self`
  # error: [invalid-return-type]
  return Foo()
 
  @staticmethod
- # TODO: reject because of staticmethod
+ # TODO: The usage of `Self` here should be rejected because this is a static method
  def make() -> Self:
  # error: [invalid-return-type]
  return Foo()
 
-class Bar(Generic[T]):
- foo: T
- def bar(self) -> T:
- return self.foo
+class Bar(Generic[T]): ...
 
 # error: [invalid-type-form]
 class Baz(Bar[Self]): ...
 
 class MyMetaclass(type):
- # TODO: rejected
+ # TODO: reject the Self usage. because self cannot be used within a metaclass.
  def __new__(cls) -> Self:
  return super().__new__(cls)
 ```

crates/ty_python_semantic/resources/mdtest/attributes.md

@@ -26,9 +26,7 @@ class C:
 c_instance = C(1)
 
 reveal_type(c_instance.inferred_from_value) # revealed: Unknown | Literal[1, "a"]
-
-# TODO: Same here. This should be `Unknown | Literal[1, "a"]`
-reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown
+reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown | Literal[1, "a"]
 
 # There is no special handling of attributes that are (directly) assigned to a declared parameter,
 # which means we union with `Unknown` here, since the attribute itself is not declared. This is
@@ -177,8 +175,7 @@ c_instance = C(1)
 
 reveal_type(c_instance.inferred_from_value) # revealed: Unknown | Literal[1, "a"]
 
-# TODO: Should be `Unknown | Literal[1, "a"]`
-reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown
+reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown | Literal[1, "a"]
 
 reveal_type(c_instance.inferred_from_param) # revealed: Unknown | int | None
 
@@ -399,9 +396,19 @@ class TupleIterable:
 
 class C:
  def __init__(self) -> None:
+ # TODO: Should not emit this diagnostic
+ # error: [unresolved-attribute]
  [... for self.a in IntIterable()]
+ # TODO: Should not emit this diagnostic
+ # error: [unresolved-attribute]
+ # error: [unresolved-attribute]
  [... for (self.b, self.c) in TupleIterable()]
+ # TODO: Should not emit this diagnostic
+ # error: [unresolved-attribute]
+ # error: [unresolved-attribute]
  [... for self.d in IntIterable() for self.e in IntIterable()]
+ # TODO: Should not emit this diagnostic
+ # error: [unresolved-attribute]
  [[... for self.f in IntIterable()] for _ in IntIterable()]
  [[... for self.g in IntIterable()] for self in [D()]]
 
@@ -598,6 +605,8 @@ class C:
  self.c = c
  if False:
  def set_e(self, e: str) -> None:
+ # TODO: Should not emit this diagnostic
+ # error: [unresolved-attribute]
  self.e = e
 
 # TODO: this would ideally be `Unknown | Literal[1]`
@@ -685,7 +694,7 @@ class C:
  pure_class_variable2: ClassVar = 1
 
  def method(self):
- # TODO: this should be an error
+ # error: [invalid-attribute-access] "Cannot assign to ClassVar `pure_class_variable1` from an instance of type `Self@method`"
  self.pure_class_variable1 = "value set through instance"
 
 reveal_type(C.pure_class_variable1) # revealed: str
@@ -885,11 +894,9 @@ class Intermediate(Base):
  # TODO: This should be an error (violates Liskov)
  self.redeclared_in_method_with_wider_type: object = object()
 
- # TODO: This should be an `invalid-assignment` error
- self.overwritten_in_subclass_method = None
+ self.overwritten_in_subclass_method = None # error: [invalid-assignment]
 
- # TODO: This should be an `invalid-assignment` error
- self.pure_overwritten_in_subclass_method = None
+ self.pure_overwritten_in_subclass_method = None # error: [invalid-assignment]
 
  self.pure_undeclared = "intermediate"
 
@@ -1839,6 +1846,7 @@ def external_getattribute(name) -> int:
 
 class ThisFails:
  def __init__(self):
+ # error: [invalid-assignment] "Implicit shadowing of function `__getattribute__`"
  self.__getattribute__ = external_getattribute
 
 # error: [unresolved-attribute]

crates/ty_python_semantic/resources/mdtest/call/dunder.md

@@ -205,7 +205,7 @@ class C:
  return str(key)
 
  def f(self):
- # TODO: This should emit an `invalid-assignment` diagnostic once we understand the type of `self`
+ # error: [invalid-assignment] "Implicit shadowing of function `__getitem__`"
  self.__getitem__ = None
 
 # This is still fine, and simply calls the `__getitem__` method on the class

crates/ty_python_semantic/resources/mdtest/class/super.md

@@ -163,14 +163,13 @@ class A:
 
 class B(A):
  def __init__(self, a: int):
- # TODO: Once `Self` is supported, this should be `<super: <class 'B'>, B>`
- reveal_type(super()) # revealed: <super: <class 'B'>, Unknown>
+ reveal_type(super()) # revealed: <super: <class 'B'>, B>
  reveal_type(super(object, super())) # revealed: <super: <class 'object'>, super>
  super().__init__(a)
 
  @classmethod
  def f(cls):
- # TODO: Once `Self` is supported, this should be `<super: <class 'B'>, <class 'B'>>`
+ # TODO: Once `cls` is supported, this should be `<super: <class 'B'>, <class 'B'>>`
  reveal_type(super()) # revealed: <super: <class 'B'>, Unknown>
  super().f()
 
@@ -358,15 +357,15 @@ from __future__ import annotations
 
 class A:
  def test(self):
- reveal_type(super()) # revealed: <super: <class 'A'>, Unknown>
+ reveal_type(super()) # revealed: <super: <class 'A'>, A>
 
  class B:
  def test(self):
- reveal_type(super()) # revealed: <super: <class 'B'>, Unknown>
+ reveal_type(super()) # revealed: <super: <class 'B'>, B>
 
  class C(A.B):
  def test(self):
- reveal_type(super()) # revealed: <super: <class 'C'>, Unknown>
+ reveal_type(super()) # revealed: <super: <class 'C'>, C>
 
  def inner(t: C):
  reveal_type(super()) # revealed: <super: <class 'B'>, C>
@@ -616,7 +615,7 @@ class A:
 class B(A):
  def __init__(self, a: int):
  super().__init__(a)
- # TODO: Once `Self` is supported, this should raise `unresolved-attribute` error
+ # error: [unresolved-attribute] "Object of type `<super: <class 'B'>, B>` has no attribute `a`"
  super().a
 
 # error: [unresolved-attribute] "Object of type `<super: <class 'B'>, B>` has no attribute `a`"

crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md

@@ -170,6 +170,7 @@ def f1(flag: bool):
  attr = DataDescriptor()
 
  def f(self):
+ # error: [invalid-assignment] "Invalid assignment to data descriptor attribute `attr` on type `Self@f` with custom `__set__` method"
  self.attr = "normal"
 
  reveal_type(C1().attr) # revealed: Unknown | Literal["data", "normal"]

crates/ty_python_semantic/resources/mdtest/named_tuple.md

@@ -208,8 +208,7 @@ class SuperUser(User):
  def now_called_robert(self):
  self.name = "Robert" # fine because overridden with a mutable attribute
 
- # TODO: this should cause us to emit an error as we're assigning to a read-only property
- # inherited from the `NamedTuple` superclass (requires https://github.com/astral-sh/ty/issues/159)
+ # error: 9 [invalid-assignment] "Cannot assign to read-only property `nickname` on object of type `Self@now_called_robert`"
  self.nickname = "Bob"
 
 james = SuperUser(0, "James", 42, "Jimmy")