Each HTML page is a DOM tree. When crawling, record the position of each sensitive word in the node, and then compare it in the database to complete the job.
The idea of recording the IDs that have been crawled is right. Even if it is a reply, you can do it this way. Just record the ID, time and other identifying information of the reply
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
Each HTML page is a DOM tree. When crawling, record the position of each sensitive word in the node, and then compare it in the database to complete the job.
Please use bloom filter
The idea of recording the IDs that have been crawled is right. Even if it is a reply, you can do it this way. Just record the ID, time and other identifying information of the reply