Preface
Everyone should have some experience. Many times when doing web crawling, it is particularly necessary to process the hyperlinks searched by the crawler, and uniformly change them to Absolute path, so this article wrote a regular expression to process the searched links. Not much to say below, let’s take a look at the detailed introduction.
Usually we may search for the following links:
<!-- 空超链接 --> <a href=""></a> <!-- 空白符 --> <a href=" " rel="external nofollow" > </a> <!-- a标签含有其它属性 --> <a href="index.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" alt="超链接"> index.html </a> <a href="/" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" target="_blank"> / target="_blank" </a> <a target="_blank" href="/" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" alt="超链接" > target="_blank" / alt="超链接" </a> <a target="_blank" title="超链接" href="/" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" alt="超链接" > target="_blank" title="超链接" / alt="超链接" </a> <!-- 根目录 --> <a href="/" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" > / </a> <a href="a" rel="external nofollow" > a </a> <!-- 含参数 --> <a href="/index.html?id=1" rel="external nofollow" > /index.html?id=1 </a> <a href="?id=2" rel="external nofollow" > ?id=2 </a> <!-- // --> <a href="//index.html" rel="external nofollow" > //index.html </a> <a href="//www.mafutian.net" rel="external nofollow" > //www.mafutian.net </a> <!-- 站内链接 --> <a href="http://www.hole_1.com/index.html" rel="external nofollow" > //m.sbmmt.com/ </a> <!-- 站外链接 --> <a href="http://www.mafutian.net" rel="external nofollow" > //m.sbmmt.com/ </a> <a href="http://www.numberer.net" rel="external nofollow" > //m.sbmmt.com/ </a> <!-- 图片,文本文件格式的链接 --> <a href="1.jpg" rel="external nofollow" > 1.jpg </a> <a href="1.jpeg" rel="external nofollow" > 1.jpeg </a> <a href="1.gif" rel="external nofollow" > 1.gif </a> <a href="1.png" rel="external nofollow" > 1.png </a> <a href="1.txt" rel="external nofollow" > 1.txt </a> <!-- 普通链接 --> <a href="index.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" > index.html </a> <a href="index.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" > index.html </a> <a href="./index.html" rel="external nofollow" > ./index.html </a> <a href="../index.html" rel="external nofollow" > ../index.html </a> <a href=".../" rel="external nofollow" > .../ </a> <a href="..." rel="external nofollow" > ... </a> <!-- 非链接,含有链接冒号 --> <a href="javascript:void(0)" rel="external nofollow" > javascript:void(0) </a> <a href="a:b" rel="external nofollow" > a:b </a> <a href="/a#a:b" rel="external nofollow" > /a#a:b </a> <a href="mailto:'mafutian@126.com'" rel="external nofollow" > mailto:'mafutian@126.com' </a> <a href="/tencent://message/?uin=335134463" rel="external nofollow" > /tencent://message/?uin=335134463 </a> <!-- 相对路径 --> <a href="." rel="external nofollow" > . </a> <a href=".." rel="external nofollow" > .. </a> <a href="../" rel="external nofollow" > ../ </a> <a href="/a/b/.." rel="external nofollow" > /a/b/.. </a> <a href="/a" rel="external nofollow" > /a </a> <a href="./b" rel="external nofollow" > ./b </a> <a href="./././././././././b" rel="external nofollow" > ./././././././././b </a> <!-- 其实就是 ./b --> <a href="../c" rel="external nofollow" > ../c </a> <a href="../../d" rel="external nofollow" > ../../d </a> <a href="../a/../b/c/../d" rel="external nofollow" > ../a/../b/c/../d </a> <a href="./../e" rel="external nofollow" > ./../e </a> <a href="http://www.hole_1.org/./../e" rel="external nofollow" > //m.sbmmt.com/ </a> <a href="./.././f" rel="external nofollow" > ./.././f </a> <a href="http://www.hole_1.org/../a/.../../b/c/../d/.." rel="external nofollow" > //m.sbmmt.com/ </a> <!-- 带有端口号 --> <a href=":8081/index.html" rel="external nofollow" > :8081/index.html </a> <a href="http://www.mafutian.net:80/index.html" rel="external nofollow" > :80/index.html </a> <a href="http://www.mafutian.net:8081/index.html" rel="external nofollow" > //m.sbmmt.com/:8081/index.html </a> <a href="http://www.mafutian.net:8082/index.html" rel="external nofollow" > //m.sbmmt.com/:8082/index.html </a>
## The first step in processing is to set it to an absolute path:
http:// ... / ../ ../
function url_to_absolute($relative) { $absolute = ''; // 去除所有的 './' $absolute = preg_replace('/(?<!\.)\.\//','',$relative); $count = preg_match_all('/(?<!\/)\/([^\/]{1,}?)\/\.\.\//',$absolute,$res); // 迭代去除所有的 '/abc/../' do { $absolute = preg_replace('/(?<!\/)\/([^\/]{1,}?)\/\.\.\//','/',$absolute); $count = preg_match_all('/(?<!\/)\/([^\/]{1,}?)\/\.\.\//',$absolute,$res); }while($count >= 1); // 除去最后的 '/..' $absolute = preg_replace('/(?<!\/)\/([^\/]{1,}?)\/\.\.$/','/',$absolute); $absolute = preg_replace('/\/\.\.$/','',$absolute); // 除去存在的 '../' $absolute = preg_replace('/(?<!\.)\.\.\//','',$absolute); return $absolute; } $relative = 'http://www.mytest.org/../a/.../../b/c/../d/..'; var_dump(url_to_absolute($relative)); // 输出:string 'http://www.mytest.org/a/b/' (length=26)
More PHP uses regular expressions For related articles on how to convert a relative path into an absolute path, please pay attention to the PHP Chinese website!