Python linter behaviour for generic deepcopying - Stack Overflow

admin管理员组

文章数量:1122846

from typing import TypeVar

T = TypeVar("T")


def deepcopy(obj: T) -> T:
    t = type(obj)

    # ...
    # (handling of other types)
    # ...

    if isinstance(obj, list):
        ls = [deepcopy(item) for item in obj]

        # (1) linter cannot ensure correct type
        return ls

        # (2) i figured out, that i can cast/call the type
        # but the linter also doesn't like that
        return t(ls)

        # (3) finally no issues
        return t.__call__(ls)

    # ...
    # (handling of other types)
    # ...

    # object is of primitive type
    return obj

x = [[0, 1, ["a", "b", "c"]], [0, 22, 22222]]
y = deepcopy(x)

print(x)
print(y)

print(x == y)
print(x is y)

I'm working on a project and I want to do deep copy objects of any (basic) type. I came up with a solution (1), which does work successfully (as far as I tested) but the linter could not resolve the type of the resulting list and said that it is not the same as the object's type put it. Next I tried to cast/call the resulting list (2). This satisfied the linter for any other type like dict, but for lists this did not work. Coincidentally, I came across a Youtube short, were someone used the dunder method __call__ (for a singleton service) and I tried it out. Now the linter is happy (3) :).

I'm a bit confused because the types should be clear from the beginning (1). Maybe the generic TypeVar is causing issues?

Also what is the difference between calling with t() and calling with the dunder method t.__call__()?

Additional Info:
Linter: VSCode Pylance Extension (v2024.11.3)
python: 3.11.9

from typing import TypeVar

T = TypeVar("T")


def deepcopy(obj: T) -> T:
    t = type(obj)

    # ...
    # (handling of other types)
    # ...

    if isinstance(obj, list):
        ls = [deepcopy(item) for item in obj]

        # (1) linter cannot ensure correct type
        return ls

        # (2) i figured out, that i can cast/call the type
        # but the linter also doesn't like that
        return t(ls)

        # (3) finally no issues
        return t.__call__(ls)

    # ...
    # (handling of other types)
    # ...

    # object is of primitive type
    return obj

x = [[0, 1, ["a", "b", "c"]], [0, 22, 22222]]
y = deepcopy(x)

print(x)
print(y)

print(x == y)
print(x is y)

I'm working on a project and I want to do deep copy objects of any (basic) type. I came up with a solution (1), which does work successfully (as far as I tested) but the linter could not resolve the type of the resulting list and said that it is not the same as the object's type put it. Next I tried to cast/call the resulting list (2). This satisfied the linter for any other type like dict, but for lists this did not work. Coincidentally, I came across a Youtube short, were someone used the dunder method __call__ (for a singleton service) and I tried it out. Now the linter is happy (3) :).

I'm a bit confused because the types should be clear from the beginning (1). Maybe the generic TypeVar is causing issues?

Also what is the difference between calling with t() and calling with the dunder method t.__call__()?

Additional Info:
Linter: VSCode Pylance Extension (v2024.11.3)
python: 3.11.9

• python
• generics
• python-typing
• pyright

Share Improve this question Follow edited Nov 22, 2024 at 13:10 Mark Rotteveel 109k224224 gold badges155155 silver badges218218 bronze badges asked Nov 22, 2024 at 12:54 cornflaxcornflax 111 silver badge

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 1

There are multiple reasons for this.

I'm a bit confused because the types should be clear from the beginning [...]

The type of obj isn't clear. You did narrow it from T (basically object) to a narrower type (list) with isinstance(obj, list), but that does not tell what the types of the elements are.

In other words, obj is understood to be of type list[Unknown] (Unknown is an internal alias of Any that Pyright uses):

_(playground)

if isinstance(obj, list):
    reveal_type(obj)  # list[Unknown]

Also what is the difference between calling with t() and calling with the dunder method t.__call__()?

t.__call__() triggers no error only because its type is Callable[..., Any] (i.e., it takes in any and all arguments and might return anything). Any is compatible with everything, including T, so Pyright lets it pass.

_(playground)

t = type(obj)

reveal_type(t)           # type[T]
reveal_type(t.__call__)  # (...) -> Any

This is not wrong. t is a class, so t.__call__ refers to that class's unbound __call__ rather than the constructor:

class C:
    def __init__(self) -> None:
        print('__init__')
    def __call__(self) -> None:
        print('__call__')

>>> c = C()
__init__
>>> t = type(c)
>>> t.__call__(c)        # Not `C.__new__()` / `C.__init__()`
__call__
>>> type(t).__call__(C)  # This correctly calls the constructor of `C`
__init__
<__main__.C object at 0x0123456789ABCDEF>

All classes inherit from type, and according to the Liskov subsitution principle, a subtype's method must accept at least everything its counterpart in the supertype accepts. type(whatever).__call__ thus have the type of Callable[..., Any], the same as how type.__call__ is defined in typeshed:

class type:
    ...
    def __call__(self, *args: Any, **kwds: Any) -> Any: ...

Approach 1 (return ls) doesn't work because ls is an instance of list itself, but the type of obj might be a subtype thereof:

_(playground)

class CustomList[T](list[T]):
    def some_custom_method(self) -> None: ...

def deepcopy[T](obj: T) -> T:
    if isinstance(obj, list):
        return [item for item in obj]    # !!!
    ...

custom_list = CustomList([0, 1, 2])
custom_list_cloned = deepcopy(custom_list)

reveal_type(custom_list_cloned)          # CustomList
custom_list_cloned.some_custom_method()  # AttributeError at runtime

You might wonder why t isn't narrowed. The answer is that, once declared, the types of t and obj are separate and no longer depend on each other. Had you assigned to t within the if block, its type would have been affected by the narrowed type of obj.

_(playground)

outer_t = type(obj)
reveal_type(outer_t)         # type[T]

if isinstance(obj, list):
    inner_t = type(obj)

    reveal_type(obj)         # list[Unknown]
    reveal_type(outer_t)     # type[T] (not narrowed)
    reveal_type(inner_t)     # type[list[Unknown]] (deducted from `v`'s type)

    return inner_t(deepcopy(item) for item in obj)  # fine (in non-strict mode)

In conclusion:

_(playground)

def deepcopy[T](obj: T) -> T:
    if isinstance(obj, list):
        t = type(obj)
        ls = t([deepcopy(item) for item in obj])

        return ls
    
    return obj

When you want to manually tell the type checker that a value has a known type you can use typing.cast.

In your case, the type checker can deduce that the return value is a list but does not know what the list contains so is trying to return list[Unknown] (as shown when you use reveal_type) which does not match the expected return type T. You can manually override its assumption of the type of the return value using cast:

from typing import TypeVar, cast, reveal_type

T = TypeVar("T")


def deepcopy(obj: T) -> T:
    if isinstance(obj, list):
        ls = reveal_type([deepcopy(item) for item in obj])
        return cast(T, ls)

    # object is of primitive type
    return obj

本文标签： Python linter behaviour for generic deepcopyingStack Overflow