Python 3 type hints and static analysis

Python 3.5 introduces the new type module, which provides standard library support for using function annotations to provide optional type hints. This opens the door to static type checking (like mypy) and possibly automated type-based optimization in the future. Type hints are specified in PEP-483 and PEP-484.

In this tutorial, I'll explore the possibilities of type hint rendering and show you how to use mypy to statically analyze your Python programs and significantly improve code quality.

Type hint

Type hints are built on top of function annotations. In short, function annotations allow you to annotate the parameters and return values ​​of a function or method with arbitrary metadata. Type hints are a special case of function annotations that specifically annotate function parameters and return values ​​with standard type information. General function annotations and special type hints are completely optional. Let's look at a simple example:

def reverse_slice(text: str, start: int, end: int) -> str:
    return text[start:end][::-1]
    
reverse_slice('abcdef', 3, 5)
'ed'

Parameters are annotated with their type and return value. But it's important to realize that Python completely ignores this. It provides type information through the annotation properties of the function object, and nothing more.

reverse_slice.__annotations
{'end': int, 'return': str, 'start': int, 'text': str}

To verify that Python really ignores type hints, let's mess with type hints completely:

def reverse_slice(text: float, start: str, end: bool) -> dict:
    return text[start:end][::-1]
    
reverse_slice('abcdef', 3, 5)
'ed'

As you can see, the code behaves the same regardless of type hints.

Motivation for type hints

OK. Type hints are optional. Python completely ignores type hints. So what's their point? Well, there are a few good reasons:

• Static Analysis
• IDE support
• Standard Document

I will use Mypy for static analysis later. IDE support has started with PyCharm 5 support for type hints. Standard documentation is useful for developers who can easily find out the types of parameters and return values ​​just by looking at the function signature, as well as an automatic documentation generator that can extract type information from hints.

typing Module

The input module contains types designed to support type hints. Why not just use existing Python types like int, str, list, and dict? You can definitely use these types, but due to Python's dynamic typing, you won't get much information beyond the basic types. For example, if you want to specify that a parameter can be a map between strings and integers, you can't do this using standard Python types. Using the input module, it's as simple as this:

Mapping[str, int]

Let's look at a more complete example: a function with two parameters. One of them is a list of dictionaries, where each dictionary contains string keys and integer values. The other parameter is a string or integer. Type modules allow such complex parameters to be specified precisely.

from typing import List, Dict, Union

def foo(a: List[Dict[str, int]],
        b: Union[str, int]) -> int:
    """Print a list of dictionaries and return the number of dictionaries
    """
    if isinstance(b, str):
        b = int(b)
    for i in range(b):
        print(a)


x = [dict(a=1, b=2), dict(c=3, d=4)]
foo(x, '3')

[{'b': 2, 'a': 1}, {'d': 4, 'c': 3}]
[{'b': 2, 'a': 1}, {'d': 4, 'c': 3}]
[{'b': 2, 'a': 1}, {'d': 4, 'c': 3}]

Useful types

Let's look at some of the more interesting types in the typing module.

The Callable type allows you to specify functions that can be passed as arguments or returned as results, because Python treats functions as first-class citizens. The syntax for a callable object is to provide an array of parameter types (also from the typing module), followed by a return value. If this is confusing, here's an example:

def do_something_fancy(data: Set[float], on_error: Callable[[Exception, int], None]):
    ...
    

The on_error callback function is specified as a function that accepts an Exception and an integer as arguments and returns nothing.

Any type means that the static type checker should allow any operation and assignment to any other type. Every type is a subtype of Any.

The Union type you saw earlier is useful when arguments can be of multiple types, which is common in Python. In the following example, the verify_config() function accepts a configuration parameter, which can be a Config object or a file name. If it is a filename, call another function to parse the file into a Config object and return it.

def verify_config(config: Union[str, Config]):
    if isinstance(config, str):
        config = parse_config_file(config)
    ...
    
def parse_config_file(filename: str) -> Config:
    ...
    

Optional type means that the parameter can also be None. Optional[T] Equivalent to Union[T, None]

There are many more types representing various functions, such as Iterable, Iterator, Reversible, SupportsInt, SupportsFloat, Sequence, MutableSequence and IO. Check out the typing module documentation for a complete list.

Best of all, you can specify the types of arguments in a very fine-grained way, supporting the Python type system with high fidelity and allowing for generics and abstract base classes.

Forward Quote

Sometimes you want to reference a class in a type hint for one of its methods. For example, suppose class A can perform some merge operation that takes another instance of A, merges it with itself, and returns the result. Here's a naive attempt to specify it using type hints:

class A:
    def merge(other: A) -> A:
        ...

      1 class A:
----> 2         def merge(other: A = None) -> A:
      3                 ...
      4

NameError: name 'A' is not defined

what happened? When Python checks the type hints for its merge() method, class A is not yet defined, so class A cannot be used (directly) at this time. The solution is very simple and I've seen it used with SQLAlchemy before. You just specify the type hint as a string. Python will understand it's a forward reference and do the right thing:

class A:
    def merge(other: 'A' = None) -> 'A':
        ...

输入别名

对长类型规范使用类型提示的一个缺点是，即使它提供了大量类型信息，它也会使代码变得混乱并降低可读性。您可以像任何其他对象一样为类型添加别名。很简单：

Data = Dict[int, Sequence[Dict[str, Optional[List[float]]]]

def foo(data: Data) -> bool:
    ...

get_type_hints() 辅助函数

类型模块提供 get_type_hints() 函数，该函数提供有关参数类型和返回值的信息。虽然 annotations 属性返回类型提示，因为它们只是注释，但我仍然建议您使用 get_type_hints() 函数，因为它可以解析前向引用。另外，如果您为其中一个参数指定默认值 None，则 get_type_hints() 函数将自动将其类型返回为 Union[T, NoneType]（如果您刚刚指定了 T）。让我们看看使用 A.merge() 方法的区别之前定义：

print(A.merge.__annotations__)

{'other': 'A', 'return': 'A'}

annotations 属性仅按原样返回注释值。在本例中，它只是字符串“A”，而不是 A 类对象，“A”只是对其的前向引用。

print(get_type_hints(A.merge))

{'return': , 'other': typing.Union[__main__.A, NoneType]}

由于 None 默认参数，get_type_hints() 函数将 other 参数的类型转换为 A（类）和 NoneType 的并集。返回类型也转换为 A 类。

装饰器

类型提示是函数注释的特殊化，它们也可以与其他函数注释一起工作。

为了做到这一点，类型模块提供了两个装饰器：@no_type_check和@no_type_check_decorator。 @no_type_check 装饰器可以应用于类或函数。它将 no_type_check 属性添加到函数（或类的每个方法）。这样，类型检查器就会知道忽略注释，它们不是类型提示。

这有点麻烦，因为如果你编写一个将被广泛使用的库，你必须假设将使用类型检查器，并且如果你想用非类型提示来注释你的函数，你还必须装饰它们与@no_type_check。

使用常规函数注释时的一个常见场景也是有一个对其进行操作的装饰器。在这种情况下，您还想关闭类型检查。一种选择是除了装饰器之外还使用 @no_type_check 装饰器，但这会过时。相反，@no_Type_check_decorator可用于装饰您的装饰器，使其行为类似于@no_type_check（添加no_type_check属性）。 p>

让我来说明所有这些概念。如果您尝试在使用常规字符串注释的函数上使用 get_type_hint() （任何类型检查器都会这样做），则 get_type_hints() 会将其解释为前向引用：

def f(a: 'some annotation'):
    pass

print(get_type_hints(f))

SyntaxError: ForwardRef must be an expression -- got 'some annotation'

要避免这种情况，请添加 @no_type_check 装饰器，get_type_hints 仅返回一个空字典，而 __annotations__ 属性返回注释：

@no_type_check
def f(a: 'some annotation'):
    pass
    
print(get_type_hints(f))
{}

print(f.__annotations__)
{'a': 'some annotation'}

现在，假设我们有一个打印注释字典的装饰器。您可以使用 @no_Type_check_decorator 装饰它，然后装饰该函数，而不用担心某些类型检查器调用 get_type_hints() 并感到困惑。对于每个使用注释操作的装饰器来说，这可能是最佳实践。不要忘记@functools.wraps，否则注释将不会被复制到装饰函数中，一切都会崩溃。 Python 3 函数注释对此进行了详细介绍。

@no_type_check_decorator
def print_annotations(f):
    @functools.wraps(f)
    def decorated(*args, **kwargs):
        print(f.__annotations__)
        return f(*args, **kwargs)
    return decorated

现在，您可以仅使用 @print_annotations 来装饰该函数，并且每当调用它时，它都会打印其注释。

@print_annotations
def f(a: 'some annotation'):
    pass
    
f(4)
{'a': 'some annotation'}

调用 get_type_hints() 也是安全的，并返回一个空字典。

print(get_type_hints(f))
{}

使用 Mypy 进行静态分析

Mypy 是一个静态类型检查器，它是类型提示和类型模块的灵感来源。 Guido van Rossum 本人是 PEP-483 的作者，也是 PEP-484 的合著者。

安装 Mypy

Mypy 正处于非常活跃的开发阶段，截至撰写本文时，PyPI 上的软件包已经过时，并且无法与 Python 3.5 一起使用。要将 Mypy 与 Python 3.5 结合使用，请从 GitHub 上的 Mypy 存储库获取最新版本。很简单：

pip3 install git+git://github.com/JukkaL/mypy.git

使用 Mypy

一旦安装了 Mypy，您就可以在您的程序上运行 Mypy。以下程序定义了一个需要字符串列表的函数。然后它使用整数列表调用该函数。

from typing import List

def case_insensitive_dedupe(data: List[str]):
    """Converts all values to lowercase and removes duplicates"""
    return list(set(x.lower() for x in data))


print(case_insensitive_dedupe([1, 2]))

运行程序时，显然在运行时失败并出现以下错误：

python3 dedupe.py
Traceback (most recent call last):
  File "dedupe.py", line 8, in 
    print(case_insensitive_dedupe([1, 2, 3]))
  File "dedupe.py", line 5, in case_insensitive_dedupe
    return list(set(x.lower() for x in data))
  File "dedupe.py", line 5, in 
    return list(set(x.lower() for x in data))
AttributeError: 'int' object has no attribute 'lower'

这有什么问题吗？问题在于，即使在这个非常简单的案例中，也无法立即弄清楚根本原因是什么。是输入类型的问题吗？或者代码本身可能是错误的，不应该尝试调用“int”对象的 lower() 方法。另一个问题是，如果您没有 100% 的测试覆盖率（老实说，我们都没有），那么此类问题可能潜伏在一些未经测试、很少使用的代码路径中，并在生产中最糟糕的时间被检测到。

静态类型在类型提示的帮助下，通过确保您始终使用正确的类型调用函数（用类型提示注释），为您提供了额外的安全网。这是 Mypy 的输出：

(N) > mypy dedupe.py
dedupe.py:8: error: List item 0 has incompatible type "int"
dedupe.py:8: error: List item 1 has incompatible type "int"
dedupe.py:8: error: List item 2 has incompatible type "int"

这很简单，直接指出问题，并且不需要运行大量测试。静态类型检查的另一个好处是，如果您提交它，则可以跳过动态类型检查，除非解析外部输入（读取文件、传入的网络请求或用户输入）。就重构而言，它还建立了很大的信心。

结论

类型提示和类型模块对于 Python 的表达能力来说是完全可选的补充。虽然它们可能不适合每个人的口味，但对于大型项目和大型团队来说它们是不可或缺的。证据是大型团队已经使用静态类型检查。现在类型信息已经标准化，共享使用它的代码、实用程序和工具将变得更加容易。像 PyCharm 这样的 IDE 已经利用它来提供更好的开发人员体验。

The above is the detailed content of Python 3 type hints and static analysis. For more information, please follow other related articles on the PHP Chinese website!