The face recognition algorithm refers to the recognition algorithm that after detecting the face and locating the key facial feature points, the main face area can be cropped out, and after preprocessing, fed into the back-end recognition algorithm. The recognition algorithm must complete the extraction of facial features and compare them with known faces in the inventory to complete the final classification. The principle of the face recognition algorithm: the system inputs one or a series of face images with undetermined identities, as well as several face images with known identities in the face database or corresponding codes, and its output is a series of The similarity score indicates the identity of the face to be recognized.
The operating environment of this tutorial: Windows 7 system, Dell G3 computer.
Facial Recognition is to obtain the user’s facial image through a video collection device, and then use the core algorithm to calculate and analyze the position, shape and angle of the facial features, and then compare it with its own database Compare existing templates to determine the user's true identity.
The face recognition algorithm refers to the recognition algorithm that after detecting the face and locating the key facial feature points, the main face area can be cropped out, and after preprocessing, fed into the back-end recognition algorithm. The recognition algorithm must complete the extraction of facial features and compare them with known faces in the inventory to complete the final classification.
There are four types of face recognition algorithms: recognition algorithms based on facial feature points, recognition algorithms based on the entire face image, template-based recognition algorithms, and recognition algorithms using neural networks.
Principle of face recognition algorithm:
The system input is generally one or a series of face images containing unidentified faces, and a face database There are several face images with known identities or corresponding codes in it, and its output is a series of similarity scores, indicating the identity of the face to be recognized.
Three classic algorithms for face recognition
1. Eigenfaces algorithm
## Eigenfaces is the name of a set of feature vectors used in computer vision problems of face recognition. Eigenfaces is based on PCA (principal component analysis), so learning Eigenfaces requires us to understand the principles of PCA.
Basic idea
Principal component analysis (PCA) is a matrix compression algorithm that reduces the matrix dimension as much as possible. Retaining the information of the original matrix simply means converting the n×m matrix into an n×k matrix, retaining only the main features present in the matrix, which can greatly save space and data volume. The implementation of PCA requires dimensionality reduction, that is, transforming the matrix from a higher dimension to a lower dimension. However, the dimensionality reduction of PCA is inseparable from the covariance matrix. Variance describes the deviation of a one-dimensional data sample from the mean. It is a statistic used to measure the relationship between two random variables. From a perspective, the smaller the angle, the larger the value, and the more similar the direction. That is, the more positive the correlation is. In addition to the relationship between two random variables, the covariance matrix measure is also the relationship between dimensions, not the relationship between samples.
To learn a new thing, especially knowledge, we need to understand the ideas inknowledge. When I understand and learn the Eigenface algorithm, its idea is that image recognition must first choose a suitable subspace, concentrate all images into this subspace, and then measure the similarity or perform classification learning in this subspace, and then talk about it. The subspace is transformed into another space. The effect of this is that images of the same category are closer, and images of different categories are farther apart. In this way, images separated by linear classification can be easily transformed into the new space. separate. At the same time, eigenface technology will look for the basic elements of face image distribution, that is, the eigenvectors of the covariance matrix of the face image sample set, to characterize the face image. There are many basic elements of human face images, such as eyes, cheeks, lips and other basic elements. The subspace correspondingly generated by these feature vectors in the image space of eigenface is called subface space. After generating the subspace, it is necessary to construct the space. So how to construct the space? First, we need to look for the commonalities in human faces, secondly, we need to look for the differences between individuals and commonalities, and third, we need to understand that commonalities are actually spaces, and individuals are vectors. Use the covariance matrix to decompose the eigenvalues of all face images in the target set to obtain the corresponding eigenvectors. These eigenvectors are "
eigenface". Find the characteristics of the eigenvectors and linearly combine them. In the space based on each characteristic sub-face, each face is a point, and the coordinates of this point are the projection coordinates of each face under each characteristic base.
Eigenfaces algorithm process
Add M faces in the corresponding dimensions, and then average to get the "average face";
Subtract the average face from each image Vector;
Calculate covariance matrix;
Use Eigenfaces memory face recognition;
Algorithm practice process;
Training image
Find the average face
Obtain the characteristic sub-face
Perform image reconstruction
Find face images with high similarity.
2. FisherFace algorithm
FisherFace is the application of Fisher linear discrimination in face recognition. Linear discriminant analysis (LDA) algorithm idea was first proposed by Ronald Fisher (Ronald), a British statistician and geneticist and one of the founders of modern statistical science. The LDA algorithm uses statistical methods to try to find a linear combination of features between objects and consider category information while reducing dimensionality. The linear combination obtained by this algorithm can be used as a linear classifier or to achieve dimensionality reduction.
Basic idea
## The basic idea of linear discriminant analysis is: project high-dimensional pattern samples to low-dimensional The best vector space can achieve the effect of extracting important classification information and compressing the dimension of feature space. After projection, it is guaranteed that the pattern sample has the largest inter-class distance and the smallest intra-class distance in the new subspace, that is, the pattern has the smallest distance in this space. Excellent separability. The theory is similar to the Eigenfaces used in eigenfaces. They are both methods of overall dimensionality reduction and mapping of the original data to a low-dimensional space. Both fisherfaces and Eigenfaces start from the overall data and are different from LBP in extracting local texture features.
Use the Fisher linear discriminant method on the dimensionally reduced samples to determine an optimal projection direction, construct a one-dimensional sign space, project the multi-dimensional face image into the fisherfaces feature space, and use intra-class samples The data forms a set of feature vectors, and this set of feature vectors represents the characteristics of the human face. We know that this algorithm minimizes the intra-class distance and maximizes the inter-class distance after the sample data is mapped to another feature space. The LDA algorithm can be used for dimensionality reduction. The principle of this algorithm is very similar to the PCA algorithm, so the LDA algorithm can also be used in the field of face recognition. The algorithm for face recognition using the PCA algorithm is called the Eigenface method, while the algorithm for face recognition using the LDA algorithm is called the Fisher face method.LDACompared with PCA:
Fisherfaces algorithmCompared with Eigenfaces algorithm:
Fisherface algorithm process
3. LBPH (Local Binary Patter Histogram) algorithm
Local Binary Patterns Histograms are the statistical histograms of LBP features Figure,LBPH combines LBP (local binary coding) features with the spatial information of the image. If the LBP encoded image is used directly for face recognition. In fact, there is not much difference from not extracting LBP features. Therefore, in actual LBP applications, the statistical histogram of the LBP encoded image is generally used as the feature vector for classification and recognition.
The position of this pixel is marked as 1, otherwise it is 0. In this way, 8 points in the 33 neighborhood can be compared to produce an 8-bit binary number (usually converted into a decimal number, that is, LBP code, 256 types in total), that is, the LBP value of the pixel in the center of the window is obtained, and this value is used to reflect the area texture characteristics. The dimension of LBPH: There are 8 sampling points. If the original LBP or Extended LBP feature is used, the LBP feature value mode is 256, then the LBP feature vector dimension of an image is: 64256 =16384 dimensions, and if the UniformPatternLBP feature is used, the LBP value pattern is 59, and the feature vector dimension is: 6459=3776 dimensions. It can be seen that using the equivalent pattern feature, the feature vector dimension is greatly reduced. This This means that the time for learning using machine learning methods will be greatly reduced without greatly affecting performance.
Basic idea
Face recognition method based on LBPHBasic idea
is as follows: First, take each pixel as the center, judge and The relationship between the gray value of the surrounding pixels is binary encoded to obtain the LBP encoded image of the entire image; then the LBP image is divided into regions, the LBP encoded histogram of each region is obtained, and then the LBP of the entire image is obtained. Encoding histograms achieve the purpose of face recognition by comparing LBP encoding histograms of different face images. The advantage is that it will not be affected by illumination, scaling, rotation and translation. The LBPH algorithm "people" is as its name suggests. The identification method used is the local feature extraction method. This is the biggest difference from the first two methods.LBPH algorithm process
4. Replication code of the algorithm
1), EigenFaces algorithm#encoding=utf-8
import numpy as np
import cv2
import os
class EigenFace(object):
def __init__(self,threshold,dimNum,dsize):
self.threshold = threshold # 阈值暂未使用
self.dimNum = dimNum
self.dsize = dsize
def loadImg(self,fileName,dsize):
'''
载入图像，灰度化处理，统一尺寸，直方图均衡化
:param fileName: 图像文件名
:param dsize: 统一尺寸大小。元组形式
:return: 图像矩阵
'''
img = cv2.imread(fileName)
retImg = cv2.resize(img,dsize)
retImg = cv2.cvtColor(retImg,cv2.COLOR_RGB2GRAY)
retImg = cv2.equalizeHist(retImg)
# cv2.imshow('img',retImg)
# cv2.waitKey()
return retImg
def createImgMat(self,dirName):
'''
生成图像样本矩阵，组织形式为行为属性，列为样本
:param dirName: 包含训练数据集的图像文件夹路径
:return: 样本矩阵，标签矩阵
'''
dataMat = np.zeros((10,1))
label = []
for parent,dirnames,filenames in os.walk(dirName):
# print parent
# print dirnames
# print filenames
index = 0
for dirname in dirnames:
for subParent,subDirName,subFilenames in os.walk(parent+'/'+dirname):
for filename in subFilenames:
img = self.loadImg(subParent+'/'+filename,self.dsize)
tempImg = np.reshape(img,(-1,1))
if index == 0 :
dataMat = tempImg
else:
dataMat = np.column_stack((dataMat,tempImg))
label.append(subParent+'/'+filename)
index += 1
return dataMat,label
def PCA(self,dataMat,dimNum):
'''
PCA函数，用于数据降维
:param dataMat: 样本矩阵
:param dimNum: 降维后的目标维度
:return: 降维后的样本矩阵和变换矩阵
'''
# 均值化矩阵
meanMat = np.mat(np.mean(dataMat,1)).T
print '平均值矩阵维度',meanMat.shape
diffMat = dataMat-meanMat
# 求协方差矩阵，由于样本维度远远大于样本数目，所以不直接求协方差矩阵，采用下面的方法
covMat = (diffMat.T*diffMat)/float(diffMat.shape[1]) # 归一化
#covMat2 = np.cov(dataMat,bias=True)
#print '基本方法计算协方差矩阵为',covMat2
print '协方差矩阵维度',covMat.shape
eigVals, eigVects = np.linalg.eig(np.mat(covMat))
print '特征向量维度',eigVects.shape
print '特征值',eigVals
eigVects = diffMat*eigVects
eigValInd = np.argsort(eigVals)
eigValInd = eigValInd[::-1]
eigValInd = eigValInd[:dimNum] # 取出指定个数的前n大的特征值
print '选取的特征值',eigValInd
eigVects = eigVects/np.linalg.norm(eigVects,axis=0) #归一化特征向量
redEigVects = eigVects[:,eigValInd]
print '选取的特征向量',redEigVects.shape
print '均值矩阵维度',diffMat.shape
lowMat = redEigVects.T*diffMat
print '低维矩阵维度',lowMat.shape
return lowMat,redEigVects
def compare(self,dataMat,testImg,label):
'''
比较函数，这里只是用了最简单的欧氏距离比较，还可以使用KNN等方法，如需修改修改此处即可
:param dataMat: 样本矩阵
:param testImg: 测试图像矩阵，最原始形式
:param label: 标签矩阵
:return: 与测试图片最相近的图像文件名
'''
testImg = cv2.resize(testImg,self.dsize)
testImg = cv2.cvtColor(testImg,cv2.COLOR_RGB2GRAY)
testImg = np.reshape(testImg,(-1,1))
lowMat,redVects = self.PCA(dataMat,self.dimNum)
testImg = redVects.T*testImg
print '检测样本变换后的维度',testImg.shape
disList = []
testVec = np.reshape(testImg,(1,-1))
for sample in lowMat.T:
disList.append(np.linalg.norm(testVec-sample))
print disList
sortIndex = np.argsort(disList)
return label[sortIndex[0]]
def predict(self,dirName,testFileName):
'''
预测函数
:param dirName: 包含训练数据集的文件夹路径
:param testFileName: 测试图像文件名
:return: 预测结果
'''
testImg = cv2.imread(testFileName)
dataMat,label = self.createImgMat(dirName)
print '加载图片标签',label
ans = self.compare(dataMat,testImg,label)
return ans
if __name__ == '__main__':
eigenface = EigenFace(20,50,(50,50))
print eigenface.predict('d:/face','D:/face_test/1.bmp')2 ), FisherFaces algorithm
#encoding=utf-8
import numpy as np
import cv2
import os
class FisherFace(object):
def __init__(self,threshold,k,dsize):
self.threshold = threshold # 阈值，暂未使用
self.k = k # 指定投影w的个数
self.dsize = dsize # 统一尺寸大小
def loadImg(self,fileName,dsize):
'''
载入图像，灰度化处理，统一尺寸，直方图均衡化
:param fileName: 图像文件名
:param dsize: 统一尺寸大小。元组形式
:return: 图像矩阵
'''
img = cv2.imread(fileName)
retImg = cv2.resize(img,dsize)
retImg = cv2.cvtColor(retImg,cv2.COLOR_RGB2GRAY)
retImg = cv2.equalizeHist(retImg)
# cv2.imshow('img',retImg)
# cv2.waitKey()
return retImg
def createImgMat(self,dirName):
'''
生成图像样本矩阵，组织形式为行为属性，列为样本
:param dirName: 包含训练数据集的图像文件夹路径
:return: 包含样本矩阵的列表，标签列表
'''
dataMat = np.zeros((10,1))
label = []
dataList = []
for parent,dirnames,filenames in os.walk(dirName):
# print parent
# print dirnames
# print filenames
#index = 0
for dirname in dirnames:
for subParent,subDirName,subFilenames in os.walk(parent+'/'+dirname):
for index,filename in enumerate(subFilenames):
img = self.loadImg(subParent+'/'+filename,self.dsize)
tempImg = np.reshape(img,(-1,1))
if index == 0 :
dataMat = tempImg
else:
dataMat = np.column_stack((dataMat,tempImg))
dataList.append(dataMat)
label.append(subParent)
return dataList,label
def LDA(self,dataList,k):
'''
多分类问题的线性判别分析算法
:param dataList: 样本矩阵列表
:param k: 投影向量k的个数
:return: 变换后的矩阵列表和变换矩阵
'''
n = dataList[0].shape[0]
W = np.zeros((n,self.k))
Sw = np.zeros((n,n))
Sb = np.zeros((n,n))
u = np.zeros((n,1))
N = 0
meanList = []
sampleNum = []
for dataMat in dataList:
meanMat = np.mat(np.mean(dataMat,1)).T
meanList.append(meanMat)
sampleNum.append(dataMat.shape[1])
dataMat = dataMat-meanMat
sw = dataMat*dataMat.T
Sw += sw
print 'Sw的维度',Sw.shape
for index,meanMat in enumerate(meanList):
m = sampleNum[index]
u += m*meanMat
N += m
u = u/N
print 'u的维度',u.shape
for index,meanMat in enumerate(meanList):
m = sampleNum[index]
sb = m*(meanMat-u)*(meanMat-u).T
Sb += sb
print 'Sb的维度',Sb.shape
eigVals, eigVects = np.linalg.eig(np.mat(np.linalg.inv(Sw)*Sb))
eigValInd = np.argsort(eigVals)
eigValInd = eigValInd[::-1]
eigValInd = eigValInd[:k] # 取出指定个数的前k大的特征值
print '选取的特征值',eigValInd.shape
eigVects = eigVects/np.linalg.norm(eigVects,axis=0) #归一化特征向量
redEigVects = eigVects[:,eigValInd]
print '变换矩阵维度',redEigVects.shape
transMatList = []
for dataMat in dataList:
transMatList.append(redEigVects.T*dataMat)
return transMatList,redEigVects
def compare(self,dataList,testImg,label):
'''
比较函数，这里只是用了最简单的欧氏距离比较，还可以使用KNN等方法，如需修改修改此处即可
:param dataList: 样本矩阵列表
:param testImg: 测试图像矩阵，最原始形式
:param label: 标签矩阵
:return: 与测试图片最相近的图像文件夹，也就是类别
'''
testImg = cv2.resize(testImg,self.dsize)
testImg = cv2.cvtColor(testImg,cv2.COLOR_RGB2GRAY)
testImg = np.reshape(testImg,(-1,1))
transMatList,redVects = fisherface.LDA(dataList,self.k)
testImg = redVects.T*testImg
print '检测样本变换后的维度',testImg.shape
disList = []
testVec = np.reshape(testImg,(1,-1))
sumVec = np.mat(np.zeros((self.dsize[0]*self.dsize[1],1)))
for transMat in transMatList:
for sample in transMat.T:
disList.append( np.linalg.norm(testVec-sample))
print disList
sortIndex = np.argsort(disList)
return label[sortIndex[0]/9]
def predict(self,dirName,testFileName):
'''
预测函数
:param dirName: 包含训练数据集的文件夹路径
:param testFileName: 测试图像文件名
:return: 预测结果
'''
testImg = cv2.imread(testFileName)
dataMat,label = self.createImgMat(dirName)
print '加载图片标签',label
ans = self.compare(dataMat,testImg,label)
return ans
if __name__=="__main__":
fisherface = FisherFace(10,20,(20,20))
ans = fisherface.predict('d:/face','d:/face_test/8.bmp')
print ans3), LBPH algorithm
#encoding=utf-8
import numpy as np
import os
import cv2
class LBP(object):
def __init__(self,threshold,dsize,blockNum):
self.dsize = dsize # 统一尺寸大小
self.blockNum = blockNum # 分割块数目
self.threshold = threshold # 阈值，暂未使用
def loadImg(self,fileName,dsize):
'''
载入图像，灰度化处理，统一尺寸，直方图均衡化
:param fileName: 图像文件名
:param dsize: 统一尺寸大小。元组形式
:return: 图像矩阵
'''
img = cv2.imread(fileName)
retImg = cv2.resize(img,dsize)
retImg = cv2.cvtColor(retImg,cv2.COLOR_RGB2GRAY)
retImg = cv2.equalizeHist(retImg)
# cv2.imshow('img',retImg)
# cv2.waitKey()
return retImg
def loadImagesList(self,dirName):
'''
加载图像矩阵列表
:param dirName:文件夹路径
:return: 包含最原始的图像矩阵的列表和标签矩阵
'''
imgList = []
label = []
for parent,dirnames,filenames in os.walk(dirName):
# print parent
# print dirnames
# print filenames
for dirname in dirnames:
for subParent,subDirName,subFilenames in os.walk(parent+'/'+dirname):
for filename in subFilenames:
img = self.loadImg(subParent+'/'+filename,self.dsize)
imgList.append(img) # 原始图像矩阵不做任何处理，直接加入列表
label.append(subParent+'/'+filename)
return imgList,label
def getHopCounter(self,num):
'''
计算二进制序列是否只变化两次
:param num: 数字
:return: 01变化次数
'''
binNum = bin(num)
binStr = str(binNum)[2:]
n = len(binStr)
if n < 8:
binStr = "0"*(8-n)+binStr
n = len(binStr)
counter = 0
for i in range(n):
if i != n-1:
if binStr[i+1] != binStr[i]:
counter += 1
else:
if binStr[0] != binStr[i]:
counter += 1
return counter
def createTable(self):
'''
生成均匀对应字典
:return: 均匀LBP特征对应字典
'''
self.table = {}
temp = 1
print type(temp)
for i in range(256):
if self.getHopCounter(i) <= 2:
self.table[i] = temp
temp += 1
else:
self.table[i] = 0
return self.table
def getLBPfeature(self,img):
'''
计算LBP特征
:param img:图像矩阵
:return: LBP特征图
'''
m = img.shape[0];n = img.shape[1]
neighbor = [0]*8
featureMap = np.mat(np.zeros((m,n)))
for y in xrange(1,m-1):
for x in xrange(1,n-1):
neighbor[0] = img[y-1,x-1]
neighbor[1] = img[y-1,x]
neighbor[2] = img[y-1,x+1]
neighbor[3] = img[y,x+1]
neighbor[4] = img[y+1,x+1]
neighbor[5] = img[y+1,x]
neighbor[6] = img[y+1,x-1]
neighbor[7] = img[y,x-1]
center = img[y,x]
temp = 0
for k in range(8):
temp += (neighbor[k] >= center)*(1<<k)
featureMap[y,x] = self.table[temp]
featureMap = featureMap.astype('uint8') # 数据类型转换为无符号8位型，如不转换则默认为float64位，影响最终效果
return featureMap
def calcHist(self,roi):
'''
计算直方图
:param roi:图像区域
:return: 直方图矩阵
'''
hist = cv2.calcHist([roi],[0],None,[59],[0,256]) # 第四个参数是直方图的横坐标数目，经过均匀化降维后这里一共有59种像素
return hist
def compare(self,sampleImg,testImg):
'''
比较函数，这里使用的是欧氏距离排序，也可以使用KNN，在此处更改
:param sampleImg: 样本图像矩阵
:param testImg: 测试图像矩阵
:return: k2值
'''
testImg = cv2.resize(testImg,self.dsize)
testImg = cv2.cvtColor(testImg,cv2.COLOR_RGB2GRAY)
testFeatureMap = self.getLBPfeature(testImg)
sampleFeatureMap = self.getLBPfeature(sampleImg)
# 计算步长，分割整个图像为小块
ystep = self.dsize[0]/self.blockNum
xstep = self.dsize[1]/self.blockNum
k2 = 0
for y in xrange(0,self.dsize[0],ystep):
for x in xrange(0,self.dsize[1],xstep):
testroi = testFeatureMap[y:y+ystep,x:x+xstep]
sampleroi =sampleFeatureMap[y:y+ystep,x:x+xstep]
testHist = self.calcHist(testroi)
sampleHist = self.calcHist(sampleroi)
k2 += np.sum((sampleHist-testHist)**2)/np.sum((sampleHist+testHist))
print 'k2的值为',k2
return k2
def predict(self,dirName,testImgName):
'''
预测函数
:param dirName:样本图像文件夹路径
:param testImgName: 测试图像文件名
:return: 最相近图像名称
'''
table = self.createTable()
testImg = cv2.imread(testImgName)
imgList,label = self.loadImagesList(dirName)
k2List = []
for img in imgList:
k2 = self.compare(img,testImg)
k2List.append(k2)
order = np.argsort(k2List)
return label[order[0]]
if __name__ == "__main__":
lbp = LBP(20,(50,50),5)
ans = lbp.predict('d:/face','d:/face_test/9.bmp')
print ans
Expand knowledge: Difficulties in face recognition algorithm researchFace recognition algorithms have been studied for a long time. Most algorithms can handle it well when the background is simple. However, face recognition has a wide range of applications, and simple image testing is far from meeting practical needs. Therefore, there are still many difficulties in the face recognition algorithm. Lighting
The lighting problem is an old problem in machine vision, and it is particularly obvious in face recognition. The algorithm has not reached the level of use.Posture
Similar to the lighting problem, the posture problem is also a technical difficulty that needs to be solved in face recognition research. There are relatively few studies on posture. Most face recognition algorithms are mainly aimed at frontal or near-frontal face images. When the pitch or left or right side is severe, the recognition rate of the face recognition algorithm will also be lower. A sharp decline.Occlusion
For face image collection in non-cooperative situations, the occlusion problem is a very serious problem, especially in a surveillance environment, where the monitored objects are often They all wear glasses, hats and other accessories, which may make the collected face images incomplete, which will affect subsequent feature extraction and recognition, and even cause the failure of the face recognition algorithm.Age changes
As age changes, facial appearance also changes, especially for teenagers, this change is more obvious. For different age groups, the recognition rates of face recognition algorithms are also different.Image quality
The sources of face images may be diverse. Due to different collection equipment, the quality of the obtained face images is also different, especially for those with low How to perform effective face recognition on face images with high resolution, high noise, and poor quality is an issue that requires attention. Similarly, for high-resolution images, the impact on face recognition algorithms also requires further study.Lack of samples
Face recognition algorithm based on statistical learning is the mainstream algorithm in the field of face recognition, but statistical learning methods require a lot of training. Since the distribution of face images in high-dimensional space is an irregular popular distribution, the samples that can be obtained only sample a very small part of the face image space. How to solve the statistical learning problem under small samples needs further research. Research.Large amount of data
Traditional face recognition algorithms such as PCA, LDA, etc. can be easily trained and learned in small-scale data. However, for large amounts of data, the training process of these methods is difficult and may even collapse.Large-scale face recognition
As the size of the face database increases, the performance of the face algorithm will decline. ###For more related knowledge, please visit the FAQ column!
The above is the detailed content of What does face recognition algorithm mean?. For more information, please follow other related articles on the PHP Chinese website!