Introduction to Neural Network Algorithm Basics

The hottest technology right now is definitely artificial intelligence.

The underlying model of artificial intelligence is "neural network". Many complex applications (such as pattern recognition, automatic control) and advanced models (such as deep learning) are based on it. Learning artificial intelligence must start with it.

1. Perceptron

Historically, scientists have always hoped to simulate the human brain and create something that can think. machine. Why can people think? Scientists have discovered that the reason lies in the body's neural network.

#1. External stimulation is converted into electrical signals through nerve endings and transduced to nerve cells (also called neurons).

2. Countless neurons constitute the nerve center.

3. The nerve center synthesizes various signals and makes judgments.

4. The human body responds to external stimuli according to instructions from the nerve center.

2. Weights and Thresholds

Seeing this, you will definitely ask: If some factors are established and other factors are not established, what is the output? For example, the weather is nice on the weekend and the tickets are not expensive, but Xiao Ming cannot find a companion. Should he still go to visit?

In reality, various factors rarely have equal importance: some factors are decisive, others are secondary. Therefore, these factors can be assigned weights, representing their different importance.

Weather: Weight is 8

Companion: Weight is 4

Price: Weight is 4

The above weight indicates that weather is the decisive factor, companion and price are secondary factors.

If the three factors are all 1, the sum of them multiplied by the weight is 8 4 4 = 16. If the weather and price factors are 1 and the companion factor is 0, the total becomes 8 0 4 = 12. At this time, you also need to specify a threshold (threshold). If the sum is greater than the threshold, the perceptron outputs 1, otherwise it outputs 0. Assume that the threshold is 8, then 12 > 8, Xiao Ming decides to visit. The level of the threshold represents the strength of the will. The lower the threshold, the more you want to go. The higher the threshold, the less you want to go.

The above decision-making process is expressed mathematically as follows.

3. Decision-making model

A single perceptron constitutes a simple decision-making model, which is ready for use. In the real world, the actual decision-making model is much more complex and is a multi-layer network composed of multiple perceptrons.

In the above figure, the bottom layer perceptron receives external input, makes a judgment, and then sends a signal as the input of the upper layer perceptron until the final result is obtained. (Note: There is still only one output of the perceptron, but it can be sent to multiple targets.)

In this picture, the signals are one-way, that is, the output of the lower layer perceptron is always the output of the upper layer perceptron. enter. In reality, cyclic transmission may occur, that is, A is passed to B, B is passed to C, and C is passed to A. This is called a "recurrent neural network"

4. Vectorization

In order to facilitate the subsequent discussion, some mathematical processing needs to be performed on the above model.

External factors x1, x2, x3 are written as vectors , abbreviated as x

Weights w1, w2, w3 are also written as vectors (w1, w2, w3), abbreviated Define the operation w⋅x = ∑ wx for w

, that is, the point operation of w and x, which is equal to the sum of the products of factors and weights

Define b equal to the negative threshold b = -threshold

The perceptron model becomes as follows.

5. The operation process of neural network

To build a neural network, three conditions need to be met.

1. Input and output

2. Weight (w) and threshold (b)

3. The structure of multi-layer perceptron

That is Say, you need to draw the picture that appears above in advance

The most difficult part is to determine the weight (w) and threshold (b). So far, these two values ​​have been given subjectively, but in reality it is difficult to estimate their values, and there must be a way to find out the answer.

This method is trial and error. Keeping other parameters unchanged, small changes in w (or b) are recorded as Δw (or Δb), and then observe the changes in the output. Repeat this process until we get the set of w and b that corresponds to the most accurate output, which is the value we want. This process is called model training.

#So, the neural network operates as follows.

1. Determine the input and output

2. Find one or more algorithms that can get the output from the input

3. Find a set of known answers The data set is used to train the model and estimate w and b

4. Once new data is generated and input into the model, the results can be obtained, and w and b are corrected at the same time

6. Output continuity

The above model has an unresolved problem. According to the assumption, the output has only two results: 0 and 1. However, the model requires small changes in w or b, which will cause changes in the output. If you only output 0 and 1, it would be too insensitive and the accuracy of the training cannot be guaranteed. Therefore, the "output" must be transformed into a continuous function. This requires a little simple mathematical modification.

First, record the calculation result wx b of the perceptron as z.

z = wx b

Then, calculate the following formula and record the result as σ(z).

σ(z) = 1 / (1 e^(-z))

This is because if z tends to positive infinity z → ∞ (indicating strong perceptron matching), Then σ(z) → 1; if z tends to negative infinity z → -∞ (indicating a strong perceptron mismatch), then σ(z) → 0. In other words, as long as σ(z) is used as the output result, the output will become a continuity function.

The original output curve is as follows.

Now this is it:

In fact, it can also be proved that Δσ satisfies the following formula.

That is, there is a linear relationship between Δσ, Δw and Δb, and the rate of change is a partial derivative. This is helpful to accurately calculate the values ​​of w and b.

The above is the detailed content of Introduction to Neural Network Algorithm Basics. For more information, please follow other related articles on the PHP Chinese website!