When it comes to making a crawler, the first thing that everyone may think of is Python. In fact, PHP can also be used to write crawler programs. PHP has always been simple and easy to use. I personally tested that I can write a simple crawler using the PHPspider framework.
##The matching method uses XPach syntax. (Recommended learning: PHP video tutorial)
<?php
require '/vendor/autoload.php';
use phpspider\core\phpspider;
/* Do NOT delete this comment */
/* 不要删除这段注释 */
$configs = array(
'name' => '简书',
'log_show' =>false,
'tasknum' => 1,
//数据库配置
'db_config' => array(
'host' => '127.0.0.1',
'port' => 3306,
'user' => 'root',
'pass' => '',
'name' => 'demo',
),
'export' => array(
'type' => 'db',
'table' => 'jianshu', // 如果数据表没有数据新增请检查表结构和字段名是否匹配
),
//爬取的域名列表
'domains' => array(
'jianshu',
'www.jianshu.com'
),
//抓取的起点
'scan_urls' => array(
'https://www.jianshu.com/c/V2CqjW?utm_medium=index-collections&utm_source=desktop'
),
//列表页实例
'list_url_regexes' => array(
"https://www.jianshu.com/c/\d+"
),
//内容页实例
// \d+ 指的是变量
'content_url_regexes' => array(
"https://www.jianshu.com/p/\d+",
),
'max_try' => 5,
'fields' => array(
array(
'name' => "title",
'selector' => "//h1[@class='title']",
'required' => true,
),
array(
'name' => "content",
'selector' => "//div[@class='show-content-free']",
'required' => true,
),
),
);
$spider = new phpspider($configs);
$spider->start();Then cmd, enter
php -f d:\jianshu\spider.php
and run as follows
The above is the detailed content of How to run php crawler. For more information, please follow other related articles on the PHP Chinese website!
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
