感兴趣内容1
感兴趣内容2
……
感兴趣内容n
内容1
内容2
……
内容n
例如 具有如下结构的html文件
代码如下:
……
……
我们尝试获得'感兴趣内容'
对于文本内容,我们保存到IDList中。
可是如何标记我们遇到的文本是感兴趣的内容呢，也就是，处于
代码如下:
这里的内容
还有这里
……
以及这里的内容
思路如下
- 遇到设置标记flag = True后 设置标记flag = False
- 遇到
- 当flag 为True时遇到
设置标记getdata = True
- 遇到 且getdata = True,设置getdata = False
python为我们提供了SGMLParser类，SGMLParser 将 HTML 分析成 8 类数据[1]，然后对每一类调用单独的方法:使用时只需继承SGMLParser 类，并编写页面信息的处理函数。
可用的处理函数如下：
- 开始标记 (Start tag)
是一个开始一个块的 HTML 标记，象 ，， 或等，或是一个独一的标记，象 <br> 或 <img alt="使用python提取html文件中的特定数据的实现代码" > 等。当它找到一个开始标记 tagname，SGMLParser 将查找名为 <em><strong>start_tagname</strong></em> 或 <strong><em>do_tagname</em></strong> 的方法。例如，当它找到一个 <pre class="brush:php;toolbar:false"> 标记，它将查找一个 start_pre 或 do_pre 的方法。如果找到了，SGMLParser 会使用这个标记的属性列表来调用这个方法；否则，它用这个标记的名字和属性列表来调用 <strong><em>unknown_starttag</em></strong> 方法。
- 结束标记 (End tag)
是结束一个块的 HTML 标记，象 ，， 或 等。当找到一个结束标记时，SGMLParser 将查找名为 end_tagname 的方法。如果找到，SGMLParser 调用这个方法，否则它使用标记的名字来调用 unknown_endtag 。
- 字符引用 (Character reference)
用字符的十进制或等同的十六进制来表示的转义字符，象 。当找到，SGMLParser 使用十进制或等同的十六进制字符文本来调用 handle_charref 。
- 实体引用 (Entity reference)
HTML 实体，象 ©。当找到，SGMLParser 使用 HTML 实体的名字来调用 handle_entityref 。
- 注释 (Comment)
HTML 注释, 包括在 之间。当找到，SGMLParser 用注释内容来调用 handle_comment。
- 处理指令 (Processing instruction)
HTML 处理指令，包括在 ... > 之间。当找到，SGMLParser 用处理指令内容来调用 handle_pi。
- 声明 (Declaration)
HTML 声明，如 DOCTYPE，包括在 之间。当找到，SGMLParser 用声明内容来调用 handle_decl。
- 文本数据 (Text data)
文本块。不满足其它 7 种类别的任何东西。当找到，SGMLParser 用文本来调用 handle_data。
代码如下:
from sgmllib import SGMLParser
class GetIdList(SGMLParser):
def reset(self):
self.IDlist = []
self.flag = False
self.getdata = False
SGMLParser.reset(self)
def start_div(self, attrs):
for k,v in attrs:#遍历div的所有属性以及其值
if k == 'class' and v == 'entry-content':#确定进入了
self.flag = True
return
def end_div(self):#遇到
self.flag = False
def start_p(self, attrs):
if self.flag == False:
return
self.getdata = True
代码如下:
def end_p(self):#遇到
if self.getdata:
self.getdata = False
def handle_data(self, text):#处理文本
if self.getdata:
self.IDlist.append(text)
代码如下:
def printID(self):
for i in self.IDlist:
print i
上面的思路存在一个bug
遇到后 设置标记flag = False
如果遇到div嵌套怎么办？
代码如下:
我是来捣乱的
感兴趣
在遇到第一个之后标记flag = False，导致无法的到‘感兴趣内容'。
怎么办呢？如何判断遇到的是和
匹配的哪个呢？
很简单，
和很简单，
是对应的，我们可以记录他所处的层数。进入子层div verbatim加1,退出子层div verbatim减1.这样就可以判断是否是同一层了。
self.flag = True
return
if self.verbatim == 0:
self.flag = False
if self.flag == True:#退出子层div了，层数减1
self.verbatim -=1
if self.getdata:
self.getdata = False
self.flag = True
return
if self.verbatim == 0:
self.flag = False
if self.flag == True:#退出子层div了，层数减1
self.verbatim -=1
if self.getdata:
self.getdata = False
test
……
……
'''
lister = GetIdList()
lister.feed(the_page)
lister.printID()
参考文献
[1] 深入 Python:Dive Into Python 中文版
修改后 如下
代码如下:
from sgmllib import SGMLParser
class GetIdList(SGMLParser):
def reset(self):
self.IDlist = []
self.flag = False
self.getdata = False
self.verbatim = 0
SGMLParser.reset(self)
def start_div(self, attrs):
if self.flag == True:
self.verbatim +=1 #进入子层div了，层数加1
return
for k,v in attrs:#遍历div的所有属性以及其值
if k == 'class' and v == 'entry-content':#确定进入了
self.flag = True
return
def end_div(self):#遇到
if self.verbatim == 0:
self.flag = False
if self.flag == True:#退出子层div了，层数减1
self.verbatim -=1
def start_p(self, attrs):
if self.flag == False:
return
self.getdata = True
def end_p(self):#遇到
if self.getdata:
self.getdata = False
def handle_data(self, text):#处理文本
if self.getdata:
self.IDlist.append(text)
def printID(self):
for i in self.IDlist:
print i
最后 建立了我们自己的类GetIdList后如何使用呢？
简单建立实例 t = GetIdList()
the_page为字符串，内容为html
t.feed(the_page)#对html解析
t.printID()打印出结果
全部测试代码为
代码如下:
from sgmllib import SGMLParser
class GetIdList(SGMLParser):
def reset(self):
self.IDlist = []
self.flag = False
self.getdata = False
self.verbatim = 0
SGMLParser.reset(self)
def start_div(self, attrs):
if self.flag == True:
self.verbatim +=1 #进入子层div了，层数加1
return
for k,v in attrs:#遍历div的所有属性以及其值
if k == 'class' and v == 'entry-content':#确定进入了
self.flag = True
return
def end_div(self):#遇到
if self.verbatim == 0:
self.flag = False
if self.flag == True:#退出子层div了，层数减1
self.verbatim -=1
def start_p(self, attrs):
if self.flag == False:
return
self.getdata = True
def end_p(self):#遇到
if self.getdata:
self.getdata = False
def handle_data(self, text):#处理文本
if self.getdata:
self.IDlist.append(text)
def printID(self):
for i in self.IDlist:
print i
##import urllib2
##import datetime
##vrg = (datetime.date(2012,2,19) - datetime.date.today()).days
##strUrl = 'http://www.nod32id.org/nod32id/%d.html'%(200+vrg)
##req = urllib2.Request(strUrl)#通过网络获取网页
##response = urllib2.urlopen(req)
##the_page = response.read()
the_page ='''
title
我是来捣乱的
感兴趣内容1
感兴趣内容2
……
感兴趣内容n
我是来捣乱的2
我是来捣乱的3
内容1
内容2
……
内容n
'''
lister = GetIdList()
lister.feed(the_page)
lister.printID()
执行后 输出为
代码如下:
感兴趣内容1
感兴趣内容2
感兴趣内容n
参考文献
[1] 深入 Python:Dive Into Python 中文版
Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Python: Automation, Scripting, and Task ManagementApr 16, 2025 am 12:14 AMPython excels in automation, scripting, and task management. 1) Automation: File backup is realized through standard libraries such as os and shutil. 2) Script writing: Use the psutil library to monitor system resources. 3) Task management: Use the schedule library to schedule tasks. Python's ease of use and rich library support makes it the preferred tool in these areas.
Python and Time: Making the Most of Your Study TimeApr 14, 2025 am 12:02 AMTo maximize the efficiency of learning Python in a limited time, you can use Python's datetime, time, and schedule modules. 1. The datetime module is used to record and plan learning time. 2. The time module helps to set study and rest time. 3. The schedule module automatically arranges weekly learning tasks.
Python: Games, GUIs, and MoreApr 13, 2025 am 12:14 AMPython excels in gaming and GUI development. 1) Game development uses Pygame, providing drawing, audio and other functions, which are suitable for creating 2D games. 2) GUI development can choose Tkinter or PyQt. Tkinter is simple and easy to use, PyQt has rich functions and is suitable for professional development.
Python vs. C : Applications and Use Cases ComparedApr 12, 2025 am 12:01 AMPython is suitable for data science, web development and automation tasks, while C is suitable for system programming, game development and embedded systems. Python is known for its simplicity and powerful ecosystem, while C is known for its high performance and underlying control capabilities.
The 2-Hour Python Plan: A Realistic ApproachApr 11, 2025 am 12:04 AMYou can learn basic programming concepts and skills of Python within 2 hours. 1. Learn variables and data types, 2. Master control flow (conditional statements and loops), 3. Understand the definition and use of functions, 4. Quickly get started with Python programming through simple examples and code snippets.
Python: Exploring Its Primary ApplicationsApr 10, 2025 am 09:41 AMPython is widely used in the fields of web development, data science, machine learning, automation and scripting. 1) In web development, Django and Flask frameworks simplify the development process. 2) In the fields of data science and machine learning, NumPy, Pandas, Scikit-learn and TensorFlow libraries provide strong support. 3) In terms of automation and scripting, Python is suitable for tasks such as automated testing and system management.
How Much Python Can You Learn in 2 Hours?Apr 09, 2025 pm 04:33 PMYou can learn the basics of Python within two hours. 1. Learn variables and data types, 2. Master control structures such as if statements and loops, 3. Understand the definition and use of functions. These will help you start writing simple Python programs.
How to teach computer novice programming basics in project and problem-driven methods within 10 hours?Apr 02, 2025 am 07:18 AMHow to teach computer novice programming basics within 10 hours? If you only have 10 hours to teach computer novice some programming knowledge, what would you choose to teach...
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
Assassin's Creed Shadows: Seashell Riddle Solution
2 weeks agoByDDD
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Chat Commands and How to Use Them
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
Hot Tools
DVWA
Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software
SublimeText3 Chinese version
Chinese version, very easy to use
MantisBT
Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.
SublimeText3 English version
Recommended: Win version, supports code prompts!
mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),
