Python realizes PD text recognition, extraction and writing into CSV file script sharing

1. Foreword

2. Requirements description

3. Start using your brain

3.1 Install relevant third-party packages

3.2 Import the required third-party libraries

3.3 Read the pdf file and identify the content

3.4 Process the identified data and write it to the csv file

Summary

1. Foreword

Scanned documents have always been popular among the public. Any paper documents can be archived after being scanned, and can be opened on your mobile phone when you want to use them, saving worry and effort. However, the advantages of the scanned document also lead to a disadvantage. Because it is scanned through an electronic device, what comes out is an image. If you want to process the content on the file, direct operation is not possible.

What if you want to quote the content? Don't worry, Python will help you solve the problem.

2. Requirement Description

There is a pdf scan. We want to extract the text and write it into a csv document in three columns. The content and effect are as follows:

pdfexample

csvexample

3. Start using your hands and brain

The pdf scan is a document scan It is converted into a computer picture format, and extracting the text is equivalent to identifying the text in the picture. Therefore, our job is to convert the PDF into a picture, and then use the OCR tool to extract the text in the picture.

3.1 Install relevant third-party packages

pip3 install pdf2image pytesseract

3.2 Import the required third-party libraries

import os #处理文件
from pdf2image import convert_from_path# pdf转图片
import pytesseract# 识别图片文字
import csv# 处理csv文件

3.3 Read pdf files and identify the content

tess_ocr(pdf_path, lang, first_page, last_page)

Split the pdf file into pictures, and extract the text and write it into the text file

• pdf_path: The storage path of the pdf file
• image: List of PIL images representing each page of the PDF document
• first_page: allows setting the first page processed by pdftoppm;
• last_page: allows setting the last page processed by pdftoppm
• fmt: Allows specifying the output format. Currently supported formats are jpg, png and ppm;
• output_folder: image saving path

def tess_ocr(pdf_path, lang,first_page,last_page):
# 创建一个和pdf同名的文件夹
 images = convert_from_path(pdf_path, fmt='png',first_page=first_page,last_page=last_page,output_folder=imagefolder,userpw='site')# 转成图片
text = ''
for img in images:
 text += pytesseract.image_to_string(img, lang=lang) # 识别图片文字
 with open(r'exampledata.txt' 'a', encoding='utf-8') as f: #写入txt文件
 f.write(text)

Running results

Generate a folder with the same name to store the split Picture, then extract the picture text and write it into data.txt

image-20211215212147760

Running problem

" " Problem throws 1:

pdf2image.exceptions.PDFInfoNotInstalledError: Unable to get page count. Is poppler installed and in PATH? ”

Solution: Download poppler.

>1 Method 1: Set the environment variable poppler/bin;

>2 Method 2:

The parameter specifies the absolute path:

images = convert_from_path( pdf_path=pdf_file_path, poppler_path=r'the address of the bin file in poppler')

" " Problem throws 2:

pytesseract.pytesseract.TesseractNotFoundError: tesseract is not installed or it's not in your PATH . See README file for more information. ”

Solution: additionally download and install tesseract-ocr and configure environment variables.

3.4 Process the identified data and write it into a csv file

modification(infile, outfile)

Clean the generated text document

• infile: The address of the file that needs to be processed
• outfile: The address of the new file generated after processing

def modification(infile, outfile):
infp = open(infile, "r",encoding='utf-8')
outfp = open(outfile, "w",encoding='utf-8')
lines = infp.readlines() #返回列表，包含所有的行。
#依次读取每行
for li in lines:
if li.split(): #str.split(str="", num=string.count(str)),过滤文件中的空行
# 根据识别情况对数据进行清洗
li = li.replace('[', ' ').replace(']', '')
outfp.writelines(li)
infp.close()
outfp.close()

Running result

Generate a New txt file, the new file deletes the blank lines in data.txt, and replaces the incorrectly identified content in the original file with the correct one.

writercsv(intxt,outcsv)

Write the text file into the csv table separated by spaces

• intxt: text file address
• outcsv: Newly generated csv file

def writercsv(intxt,outcsv):
# 使用newlines=''可保证存储的数据不空行。
csvFile = open(outcsv, 'a',newline='', encoding='utf-8') 
writer = csv.writer(csvFile)
csvRow = []
f = open(intxt,'r',encoding='utf-8')
for line in f:
csvRow = line.split() #以空格为分隔符
if len(csvRow)>1 and len(csvRow)<=3:#约束条件，视情况而定
 writer.writerow(csvRow)
f.close()
csvFile.close()

Running result

Generates a three-column csv file, the first column is the English name, and the second column is the Chinese name , the third column is the country

image-20211215204846623

image-20211215204941725

Summary

Through this study, it is possible to extract text from scanned documents and write the content into different formats as required. Documentation requirements.

At first I thought that the library for extracting pdf would also be suitable for scanned documents, so I tried the Pdfplumber library and the PyPDF2 library.

Practice found that Pdfplumber can only recognize watermarks in scanned PDFs, and is not applicable to scanned PDFs. The PyPDF2 library runs and reports an error: NotImplementedError: only algorithm code 1 and 2 are supported.

The reason is that this encrypted PDF may come from a higher version of acrobot, so the corresponding encryption algorithm code is '4'. However, the existing pypdf2 module only supports the encryption algorithm code '4'. 1' or '2' pdf encrypted file.

The above is the detailed content of Python realizes PD text recognition, extraction and writing into CSV file script sharing. For more information, please follow other related articles on the PHP Chinese website!