How to draw common activation function curves using Python?

Preparation work: Download numpy, matplotlib, sympy

pip install numpy matplotlib sympy

Find the documentation for the corresponding library:

numpy documentation matplotlib documentation sympy documentation

When writing code, I found that vscode was not working Will format my python? After checking, it turns out that flake8 and yapf need to be installed. One is a code specification tool and the other is a formatting tool. Then configure the setting.json

"python.linting.flake8Enabled": true, // 规范检查工具
"python.formatting.provider": "yapf", // 格式化工具
"python.linting.flake8Args": ["--max-line-length=248"], // 设置单行最长字符限制
"python.linting.pylintEnabled": false, // 关闭pylint工具

The preparations are completed. Next, let’s see how to write the code

The first step is to create a new py file

First write out the function expression of the activation function. There are two ways. If you just get the calculation result, the other It is enough to use numpy, but if you have to find the derivation yourself, you need to use sympy to write the functional expression.

The way sympy expresses functions is like this:

from sympy import symbols, evalf, diff
# 我们先要定义自变量是什么，这边按需求来，这是文档的例子有两个变量
x, y = symbols('x y')
# 然后我们写出函数表达式
expr = x + 2*y
# 输出看一下是什么东西
expr # x + 2*y
# 接着就要用我们定义的函数了
expr.evalf(subs={x: 10, y: 20}) # 50.000000
# 再对我们的函数求导
diff(expr, x, 1) # 对x进行求导得出结果 1，这也是表达式

diff is the derivation function of sympy

sympy.core.function.diff(f, *symbols , **kwargs)

Then we define the expression of the activation function

def sigmoid():
    """
    定义sigmoid函数
    """
    x = symbols('x')
    return 1. / (1 + exp(-x))

def tanh():
    """
    定义tanh函数
    """
    x = symbols('x')
    return (exp(x) - exp(-x)) / (exp(x) + exp(-x))

def relu():
    """
    定义ReLU函数
    """
    x = symbols('x')
    return Piecewise((0, x < 0), (x, x >= 0))

def leakyRelu():
    """
    定义Leaky ReLu函数
    """
    x = symbols('x')
    return Piecewise((0.1 * x, x < 0), (x, x >= 0))

def softMax(x: np.ndarray):
    """
    定义SoftMax函数\n
    """
    exp_x = np.exp(x)
    print(exp_x, np.sum(exp_x))
    return exp_x / np.sum(exp_x)

def softmax_derivative(x):
    """
    定义SoftMax导数函数\n
    x - 输入x向量
    """
    s = softMax(x)
    return s * (1 - s)

Then we define a derivation function

def derivate(formula, len, variate):
    """
    定义函数求导
      formula：函数公式
      len：求导次数
      variate：自变量
    """
    return diff(formula, variate, len)

There is a question here, why All other functions are one, but the softMax function has two, one is the definition of the softMax function, and the other is the definition of its derivative function?

Let’s take a look at what the softMax function looks like

#The denominator of the softMax function needs to write the accumulation process. Using numpy.sum cannot be used to derive the derivation through sympy (someone can , I don’t know why, it may be due to different usage methods. If you know, you can communicate with me) and using sympy.Sum or sympy.summation can only accumulate from i to n in units of 1 each time

For example: Assume There is an expression for m**x (m raised to the power of x) sympy.Sum(m**x, (x, 0, 100)), then the result is m**100 m**99 m**98 … m**1, and the ndarray I defined is np.arange(-10, 10, 0.05), which cannot meet the requirements and the derivation cannot be performed.

So I wrote two functions, one is the original function definition, and the other is the derivative function definition. As mentioned before, if it is evaluation, it can actually be completed only with numpy.

At this point, all functions and derivative functions have been defined by us

The second step is to use matplotlib to draw the curve

First, we need to know matplotlib What is there?

matplotlib mainly includes Figure, Axes, Axis, and Artist. I understand that figure is the canvas, and we must prepare the canvas before drawing the figure; axes and axis are both translated as axes, but axes should be the coordinate axis, and axis is one of the coordinate axes; artist is other that can be added Element

If you want to draw a simple graph, you can do this

x = np.linspace(0, 2, 100)  # Sample data.

# Note that even in the OO-style, we use `.pyplot.figure` to create the Figure.
fig, ax = plt.subplots(figsize=(5, 2.7), layout='constrained')
ax.plot(x, x, label='linear')  # Plot some data on the axes.
ax.plot(x, x**2, label='quadratic')  # Plot more data on the axes...
ax.plot(x, x**3, label='cubic')  # ... and some more.
ax.set_xlabel('x label')  # Add an x-label to the axes.
ax.set_ylabel('y label')  # Add a y-label to the axes.
ax.set_title("Simple Plot")  # Add a title to the axes.
ax.legend()  # Add a legend.

Then we are ready to draw our function curve

plt.xlabel('x label') // 两种方式加label，一种为ax.set_xlabel（面向对象），一种就是这种（面向函数）
plt.ylabel('y label')

After adding laben, I considered two things There are two ways to draw, one is to draw all the curves in one figure, but divide them into different axes

Use the subplot function to divide the figure into 2 rows and 2 columns of axes

plt.subplot(2, 2, 1, adjustable='box') # 1行1列
plt.subplot(2, 2, 2, adjustable='box') # 1行2列

The second one is to draw the specified function by inputting the function name

do = input( 'input function expression what you want draw(sigmoid, tanh, relu, leakyRelu, softMax)\n' )

After getting the input

 try:
        plt.xlabel('x label')
        plt.ylabel('y label')
        plt.title(do)
        if (do == 'softMax'):
            plt.plot(num, softMax(num), label='Softmax')
            plt.plot(num, softmax_derivative(num), label='Softmax Derivative')
        else:
            plt.plot(
                num,
                [eval(f'{do}()').evalf(subs={symbols("x"): i}) for i in num])
            plt.plot(num, [
                derivate(eval(f'{do}()'), 1, 'x').evalf(subs={symbols('x'): i})
                for i in num
            ])

        plt.tight_layout()
        plt.show()
    except TypeError:
        print(
            'input function expression is wrong or the funciton is not configured'
        )

This is done, attached is a seller's show

The above is the detailed content of How to draw common activation function curves using Python?. For more information, please follow other related articles on the PHP Chinese website!