Python爬虫解析库之pyquery详解

版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接: https://blog.csdn.net/qq_44205272/article/details/102065813

PyQuery解析库·详解

安装

pip3 install pyquery

初始化的三种方法

字符串初始化

from pyquery import PyQuery  as pq
html = '''
<div>
    <url>
        <li class='item-0'>first item</li>
        <li class='item-1'><a href='link3.html'><span class='bold'>third item</span></a></li>
    </url>
</div>
'''
doc = pq(html)
print(doc('li'))# 这里的选择与css选择器一样,选class加点,选id加#,选标签什么都不加
输出结果为:
<li class="item-0">first item</li>
        <li class="item-1"><a href="link3.html"><span class="bold">third item</span></a></li>

url初始化

from pyquery import PyQuery  as pq
url='http://www.baidu.com'
doc = pq(url)
print(doc('head'))

文件初始化
在这里就不举出例子了,与前面两种方法类似

基本CSS选择器

from pyquery import PyQuery as pq
 
html = '''
<div id='container'>
    <ul class='list'>
        <li class='item-0'>first item</li>
        <li class='item-1'><a href='link2.html'>second item</a></li>
        <li class='item-0 active'><a href='link3.html'><span class='bold'>third item</span></a></li>
        <li class='item-1 active'><a href='link4.html'>fourth item</a></li>
        <li class='item-0'><a href='link5.html'>fifth item</a></li>
    </url>
</div>
'''
doc = pq(html)
print(doc('#container .list li'))
 
输出结果为:
<li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>

查找元素

查找子元素,find方法,查找元素里面包含的元素

from pyquery import PyQuery as pq
 
html = '''
<div id='container'>
    <ul class='list'>
        <li class='item-0'>first item</li>
        <li class='item-1'><a href='link2.html'>second item</a></li>
        <li class='item-0 active'><a href='link3.html'><span class='bold'>third item</span></a></li>
        <li class='item-1 active'><a href='link4.html'>fourth item</a></li>
        <li class='item-0'><a href='link5.html'>fifth item</a></li>
    </url>
</div>
'''
doc = pq(html)
items = doc('.list')
print(type(items))
print(items)
lis = items.find('li')
print(type(lis))
print(lis)
 
输出结果为:
<class 'pyquery.pyquery.PyQuery'>
<ul class="list">
        <li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>
     
</ul>
<class 'pyquery.pyquery.PyQuery'>
<li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>

查找父元素

from pyquery import PyQuery as pq
 
html = '''
<div id='container'>
    <ul class='list'>
        <li class='item-0'>first item</li>
        <li class='item-1'><a href='link2.html'>second item</a></li>
        <li class='item-0 active'><a href='link3.html'><span class='bold'>third item</span></a></li>
        <li class='item-1 active'><a href='link4.html'>fourth item</a></li>
        <li class='item-0'><a href='link5.html'>fifth item</a></li>
    </url>
</div>
'''
doc = pq(html)
items = doc('.list')
print(items.parent())
输出结果为:
<div id="container">
    <ul class="list">
        <li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>
     
</ul></div>

还有parents方法,查找祖先节点,不只是父节点,父节点的父节点也会查找到

查找兄弟元素

from pyquery import PyQuery as pq
 
html = '''
<div id='container'>
    <ul class='list'>
        <li class='item-0'>first item</li>
        <li class='item-1'><a href='link2.html'>second item</a></li>
        <li class='item-0 active'><a href='link3.html'><span class='bold'>third item</span></a></li>
        <li class='item-1 active'><a href='link4.html'>fourth item</a></li>
        <li class='item-0'><a href='link5.html'>fifth item</a></li>
    </url>
</div>
'''
doc = pq(html)
items = doc('.list .item-0.active')
print(items.siblings())
print(items.siblings('.active'))<br>#在查找的时候,可以进行进一步满足条件的筛选
输出结果为:
<li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0">first item</li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>
     
<li class="item-1 active"><a href="link4.html">fourth item</a></li>

遍历,items()方法:实际上就是产生了一个产生器,再用for循环进行遍历

from pyquery import PyQuery as pq
 
html = '''
<div id='container'>
    <ul class='list'>
        <li class='item-0'>first item</li>
        <li class='item-1'><a href='link2.html'>second item</a></li>
        <li class='item-0 active'><a href='link3.html'><span class='bold'>third item</span></a></li>
        <li class='item-1 active'><a href='link4.html'>fourth item</a></li>
        <li class='item-0'><a href='link5.html'>fifth item</a></li>
    </url>
</div>
'''
doc = pq(html)
lis = doc('li').items()
for li in lis:
    print(li)
 
输出结果为:
<li class="item-0">first item</li>
         
<li class="item-1"><a href="link2.html">second item</a></li>
         
<li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         
<li class="item-1 active"><a href="link4.html">fourth item</a></li>
         
<li class="item-0"><a href="link5.html">fifth item</a></li>

获取信息

获取属性,比如要获取item元素的属性:item.attr(‘属性名称’),或者:item.attr.属性名称

from pyquery import PyQuery as pq
 
html = '''
<div id='container'>
    <ul class='list'>
        <li class='item-0'>first item</li>
        <li class='item-1'><a href='link2.html'>second item</a></li>
        <li class='item-0 active'><a href='link3.html'><span class='bold'>third item</span></a></li>
        <li class='item-1 active'><a href='link4.html'>fourth item</a></li>
        <li class='item-0'><a href='link5.html'>fifth item</a></li>
    </url>
</div>
'''
doc = pq(html)
li = doc('.item-0.active a')
print(li.attr.href)
print(li.attr('href'))
 
输出结果为:
link3.html
link3.html  

获取文本,text()方法

还有DOM操作,因为使用的比较少,这里就不详细介绍了

下面是pyquery的官方文档
https://pyquery.readthedocs.io/en/latest/api.html

欢迎各位朋友的关注与评论

猜你喜欢

转载自blog.csdn.net/qq_44205272/article/details/102065813