urllib3库的学习

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/weixin_39997829/article/details/80179636

参考链接:https://urllib3.readthedocs.io/en/latest/user-guide.html#using-timeouts

0x01:

Making Requests

首先我们肯定要安装urllib3这个库了(python3支持)

pip install urllib3

然后我们在编写任何脚本之前需要导入这个模块

import urllib3

现在我们需要一个PoolManager去建立一个request,这个的connection pooling的作用是为了thread safety

语句为:http=urllib3.PoolManager()

现在我们尝试一下get请求

import urllib3

http=urllib3.PoolManager()

r=http.request('GET','http://httpbin.org/robots.txt')

print(r.data)

>>b'User-agent: *\nDisallow: /deny\n'

request()返回一个HTTPResponse对象,我们可以使用request()对任何HTTP verb做请求

如下所示:post请求

import urllib3

http=urllib3.PoolManager()

r=http.request('POST','http://httpbin.org/post',fields={'hello':'world'})

0X02:

Response content

这个HTTPResponse对象提供了status,data,headers等属性

import urllib3

http=urllib3.PoolManager()

r=http.request('GET','http://httpbin.org/ip')

print(r.status)

print(r.data)

print(r.headers)

200

b'{\n  "origin": "120.203.18.108"\n}\n'

HTTPHeaderDict({'Connection': 'keep-alive', 'Server': 'gunicorn/19.7.1', 'Date': 'Thu, 03 May 2018 06:50:51 GMT', 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Credentials': 'true', 'X-Powered-By': 'Flask', 'X-Processed-Time': '0', 'Content-Length': '33', 'Via': '1.1 vegur'})

0x03:Json content

json文本在加载之前需要进行一次utf-8解密,将这个data进行一次反序列化,当然记得导入json模块

import json

r=http.request('GET','http://httpbin.org/ip')
print(json.loads(r.data.decode('utf-8')))

{'origin': '120.203.18.108'}

0x04:Binary content

data响应的属性始终设置为表示响应内容的字节字符串

r=http.request('GET','http://httpbin.org/bytes/8')

print(r.data)

b';P\x8aq\x97uO.'

注意:对于很大的response,我们有的时候需要去stream这个response

Streaming and IO

我们设置preload_content to False是为了urllib3能够stream这个请求文本,streamI()能够让我们迭代响应内容文本

注意:我们在使用preload_content=False的时候,我们应该调用release_conn()将http连接释放

import urllib3

http=urllib3.PoolManager()

r=http.request('GET','http://httpbin.org/bytes/1024',preload_content=False)
for chunk in r.stream(32):

    print(chunk)

r.release_conn()

我们也可以将HTTPResponse实例视为文件类对象,这样的话可以让我们做下缓冲

import urllib3

http=urllib3.PoolManager()

r=http.request('GET','http://httpbin.org/bytes/1024',preload_content=False)

r.read()

Calls toread将阻塞,直到有更多的相应的数据可用

r=http.request('GET','http://httpbin.org/bytes/1024',preload_content=False)
reader=io.BufferedReader(r,8)
print(reader.read(4))

r.release_conn()

b'\x959\xc3\x10'

我们也可以使用类文件对象来执行诸如codecs以下内容解码内容的操作

import codecs

reader=codecs.getreader('utf-8')
r=http.request('GET','http://httpbin.org/ip',preload_content=False)

print(json.load(reader(r)))

>>{'origin': '120.203.18.108'}

0x05:Request data

Headers

我们可以在headers参数中指定标题作为字典request()

import urllib3

http=urllib3.PoolManager()

r=http.request('GET','http://httpbin.org/headers',headers={'X-Something':'value'})

print(json.loads(r.data.decode('utf-8'))['headers'])

>>{'Accept-Encoding': 'identity', 'Connection': 'close', 'Host': 'httpbin.org', 'X-Something': 'value'}

Query parameters

对于get,head和delete请求,我们可以简单的传递参数作为一个字典fields参数request()

r=http.request('GET','http://httpbin.org/get',fields={'arg':'value'})

print(json.loads(r.data.decode('utf-8'))['args'])

>>{'arg': 'value'}

对于post和put请求,我们需要在url中手动编码查询参数

import urllib3

from urllib.parse import urlencode

encoded_args=urlencode({'arg':'value'})
url='http://httpbin.org/post?'+encoded_args
r=http.request('POST','url')

print(json.loads(r.data.decode('utf-8'))['args'])

Form data

对于put和post请求,urllib3将会自动的对表单提供给参数的字典进行编码

r=http.request('POST','http://httpbin.org/post',fields={'field':'value'})

print(json.loads(r.data.decode('utf-8'))['form'])

>>{'field': 'value'}

Json

我们可以通过将编码指定为body参数并在Content-Type调用的时候设置标题,来向json来发送请求

data={'attribute':'value'}
encoded_data=json.dumps(data).encode('utf-8')
r=http.request('POST','http://httpbin.org/post',body=encoded_data,headers={'Content-Type':'application/json'})
print(json.loads(r.data.decode('utf-8'))['json'])

>>{'attribute': 'value'}

Files&binary data

要使用multipart/form-data编码上传文件,可以使用与表单数据相同的方法,并将文件字段指定为一下元组(file_name,file_data)

with open('test.txt')as fp:
    file_data=fp.read()

r=http.request('POST','http://httpbin.org/post',fields={'filefields':('test.txt',file_data)})

print(json.loads(r.data.decode('utf-8'))['files'])

尽管指定文件名不是严格要求的,但是为了匹配浏览器的行为,建议使用它。当然我们还可以传递元组的第三项来显式的指定文件的MIME类型

with open('test.txt')as fp:
    file_data=fp.read()

r=http.request('POST','http://httpbin.org/post',fields={'filefields':('test.txt',file_data,'text/plain')})

print(json.loads(r.data.decode('utf-8'))['files'])

要发送原始的二进制数据,只需要指定body参数,还建议设置Content-Type标题

with open('test.jpg','rb')as fp:
    binary_data=fp.read()
r=http.request('POST','http://httpbin.org/post',body=binary_data,headers={'Content-Type':'image/jpeg'})

print(json.loads(r.data.decode('utf-8'))['data'])

>>data:application/octet-stream;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAIBAQIBAQICAgICAgICAwUDAwMDAwYEBAMFBwYHBwcGBwcICQsJCAgKCAcHCg0KCgsMDAwMBwkODw0MDgsMDAz/2wBDAQICAgMDAwYDAwYMCAcIDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCABIAGADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD3r49Wty3irT7vUNP1/VfCV9ZXWgeJYdEAk1SGznuLaUXVvHlWkeHyJNojbzYpZILmNZZLVYZd/wCGmt3XjG4uNaTSb7Ro9Sj+0zpdNPvmvbi4ub+9aM3Cpcm2F3e3Cwm5jjmESIrRgKrN0Gga+/iqOaWazFsRPIuFD7GCyMAw3qrYKgHkDrWjcX8mk6NfzxQxPcQWkz2scgYpLOEYxo23nDPtBI7E9OterjMyxMML/ZdS1oNre/Xydnr17LQwwKo1lHGU72kk1dNOzXVOzWnR7dVc6r4eeAY/HkUol8UeHvD08IDINW1H7KZeSPk4OcY5+ory/wDtjUdc8SazF/bV7BdQSx2809xryR/b1RQIzG5lzJGgIA5wMkADmui8NeJdH17w39p8SobbVIw7fYtOsJpVkAQFUE0uArF9y5KkAYPrUepp4NvbNZIIfEEbNIpaCaxiLRxEncNyvhpFGOMBSQ2GxgnzMLelJVn2vpLlflrvdb2Mszbrc+DimnzOF3T546K7fZwauua9ruxz8Omp4c16WyM2+S1WXe8dybhHZgHLCTJDZ3E5z/KpZZBd2j7ZC2B2Y1n+J7nT9Il8T3OkteLo1lpdzcWzX6pDOiLDuYuFJVcEN36AGvnL45/s23P7Xn/CKeJfDf7Qvir4X2tnoUdte23hrXFt7W6n3GVnndZ0UTKJApDAthV6V5+ITqVZSk923e973ffr6nqYVqnRhBRtotLcttO3T06H0n4TlaGxlHmOP3p7n0Fdj4S/sgala/8ACRavcaNpEl0kNzexK0jwKY5WG1Qr5JaNV+62Mk47181fDPVdM/Z78FaV4RvPippnje40VnWTXNd8V6XHqOqK8jSq0olut+VVxGM5+WNeTXtnwq/ah0Twv4s0PVha22rQafqUOoL/AGf4j0iV5FSOZcJ/pWCSZF79AaUYT7G0qke59M6L8M/gP45sms/7f8S+I7ZgJXtrrSJJ422kOrMhs8cEBgSOCARyK7PSNN+CVr4dtYrey1u+tBGPJaDSNTbKY+XHlRAAYxjAAxjFeffEb/goT4bv/hdqUuoWPibw1bT6ZNZ/btVNr5XzROm8vFOwxl1zhg3pzipf2a/+Cvf7PHiPQfDfg+0+IllDqFnawack11G8dtPLGqoVWYBkyWGAGIz06kCuxR1svyOST01OE/affwvc+J7dPCVnqmmWEdmqTx3ttd2kxmLMSdtyFkxsKYbGD2714/b6akF6HT/WFgGfPzNyK+iv2sL5Zfj/AK1cp5s0ccUUXy453WsfAyenzfqa8XksJ7zUJJZUIUuNg4zj8P8A69ZTjY1jI9M1T4Y39jqbQ2iC4U5OBpcgIOSCNzThOgB69/bFeX+Nr/W77436dpXh7xZ4Jh8NixCapa3NsH1K1vhbzBl81XlhVPtBgIDrwiSDczMpHsPjD9vvwh4cuITbeE3gN9OsMDeINbt9KSZ3YKoVoxcZySMY5PGK8q/aG8RXPiD4i3N/d20NjLfW0EzW0Ny1zFARGsZVXZVLD5M5KqeeQOletSxEZxjanFOL7Xv63ep4McsqqvUq1K9S01bluko+cbJNPzuehaH8HvE2l6PZzSweGfEEd7cpGjGeZJZRJsRQksCxx7c7m/1RZcHDOCFXP8A/D/Vda+AcHi+z0+bU11iKW6trW9inikbDuqQrJcSpsBMeFdo48q2SBmvP7jw3+0X8btB8OaH4T8XXvwt8LeG5ZZoPEFxbWbzqzWE8UDWduqmSdQ9wd4u3iUqpMTB9ki+z/s/fs62fwD+FeieG5dc13xnc6PG6tqerznzbh3d5HcIuEj+d2PyqWPBd5G3O2dTHVeaWkUpNN2jFbdtLpehpTyWnywjOU5ciaTlOb33v73vPzd7Hyh+0R8TdI8AXmuWmpat4dvLnVLObTLrS7TUoleyaSMxywTNE22F0XcWIfKkjLZIz8IeMP2l/C3w18RWHg6xk8NXPg/ToFnsLXw/aM1np10XcEeSgCTSDIJlYbiCTvBLKf1m+KX/BMv8AZ/8AjBqEl9rvwi8EPdy5Ek2n2jaVPJli5LPaNE5JZmbOeWZj1JJ8m8Sf8ELf2db+ffpuieM/DLjp/Zni6/kH/fN3JOv6V5tSMZz55q7vc9X6vNQ9nCXurofljB8NLH45+JvEXiK4s4o7Czm+z2moSXiQLKqJhnbKncBwxdsDcSOcEV4RZeINevPiYNBstShl1S7vZLWGezuALWdnYnduUhWXk4KjAHQcV+w/jv8A4IBfCDXtIFqfHfxhsbZX3og1bTnRTnP3WscNznrnrnrzXD63/wAEGfgto/iCz1CbxB8TtRe0jSJbZLzTdOgmAyN8hhsllLdy6srkqMMGwadrsyWEkr3Z8q+JPhLrmkfsn6fbaAYDq/iDU7jxLqFrHG00uoadZSzWVpGFCHehmj1W5XIGUuIyOleR/DG11z4gfFi80jwpoVxrR1HTIJru2htVaGxMiYZJC4CQEIsfL7Dyf7xz+h/if4XeOPgd8b9c8U/DT4m2/gvSrzw5pXhW30Sy8N299/ZWn6fERDFHPdPIf9Y8p5QE5XeZCAawrj4keJ7S6kbxL4w1vxhqNxIJ31DUEtLV2AGPL8u2hiTYAMDgN8x3FuAOyFPlQOi29T1z9hHx9408L/DTw94J+J19p+q+J182K1mtJ5LlorWPAhjllZRuZE2r1OAAAzBc19BS26Ic9Dx/PFfnLYftJeK9K/bH+G3h7RlgXStT+2Ta3dFNzwA2N49ssfOEINq7MSOd0QHVs/RGt/GDxdHPE9n4olQIy/uZLC2ljkOeNxMYfrz8rDpWEuWOjOmMW9uh5B+xd8MdK8caaLqP4b+I/CZ0m0tEtdb1+zthdawwUgyKQzSFl2KxLhQS+QuOno+v/G/x38Lr3SfFPxM1PR/EnhXwxJay6vH4f8PC21HVbb7UkYhInujArvJKqEjy1w/VMbq6/QvG1v4H+G2l/wBqXdrp1vbRmBZrqdYUO04CgsQCQMDA5o+G3x7+BnxU1TxF8P8A4i69py+D/GOj/YLy4v7yXSLCWRbu2mii+1l4ijmWNSvIRyuzLbwjb4elbRys29hSnJpyUbpI+ofhJ/wUm+B3x+0mW90rx1FYSWyM1xBrNnNp4tNpHmfv3T7LKsfWR7eaWOMZLuoBNe3W+gtqukQ6habLywuV3w3VuwkgmX1V1yrD3BNfnB4y/Yn8K+Bz8RPCvhzx3b6X4hWKGxuT40tDB5VqrSiOOPUIYPIS3dRIkZj035GQMxdtpPztN/wTi+Pnwq8aXmsaH8Nb+x0u6DSx6h8OPE0MJvH3Db5t9E02uSHg5Z4Ycng7RyPSrZfOFudWv3Vjz6eJT0TP2YvNPeJwux3djhEVCzN7ADrXl/xo/ak+GfwKEqeN/iJ4D8Ezxj/j11bXLe3vX9hC77z+Ck8ivzE+Jn/BPv8Aam+PGk3yeN9N+LOpaRbXC6Zo2iD4rQXDzW20PLf38mo6jKrIZPkitwrTANKJCqrEGib/AIJI6t8StW8V6tqnwP8AA3wh1S8dLzRv7N+I0U0Fi8cEMaWj21rp726RySrLPJMiPJnYqKPnds5YNK1jRYh9T7O0r/gp78H/AImfGDSvAvhnW9avfEOvQXM9h9p8O6jp8d2II3lkCS3cMW7CJIQyZU7CN2SFPBftx/tx+Cv2S/BUOpeONdj0camzR2dpGrS3N4wwWCIgLNjcCTgKNy7iNy58s/ZW/YC1L9nz45XvxH8aeLNJ8SeJ5dJXSba3061nNtp6LHHF532idvNnl8iPy9zRqcSy9Qyqn5mfH7Xb/wD4Kc/ti+NPEc11fTeHNJu49L0JIj/o9np0cj+bdFuPlEMN1dFRy2xhnCqK56lJQWm5tGrJn6HWv7QWmfFjwpp2t+Hb9dX0rW7cXNlcRqyiRCxXG0gMGVgykEAhlYHpXnvizxLO+p6kLW1uNavtGtJ7zUEhgkuYdLjiQu7TLGC0jgAnyY/m4+YoCDXzB8Iv2itH/ZM/ZM8K6TrWryJqJGoQW+laZIBquoIupXfz+cCy2Vsc5MuDJJwItwEhHGeKf+Co/iS8+DniLwNoVhpuh6Z4isptOmhsIvLAhlUoyc5ZgFYgbm7kkliScXNLc25nbQ7L9nL9tB/iD8T/ABZ4n0+KW3lutRiX7VeKn2hY1tEtrRFx+6j8zF47ogxuMa7m2IT77F+1hrZjjDzxPhhw8Sc/kor8+P2YtLGka1JK0So6IcuANy5bGAe3Q1783i3yQoExJBGMjNedWqe8d+GpPluz1LVvFl3qN891fXd1qF6/DzzyvNI/plmJY/56VUtPiLoEGn61pfi/w7H4r8K+ItPOmarY/b5NPujH50M6S210isbeeOaCJlcpIhCsrxyI7Kc28uWuZZto2I3U4x+Y/wA/SuK8eGS00q5lyWVEJGBzx069K5aeIakmtz1a1FOLi1oz6m+H/wC3D4jnt18PeE/jB4G+M2nTQyW3/CCfGz7N4X8RosqmBobfWmYade7opHj3SXNvM6sw8gBiD7L40/4ONvHP7MvxEfw98YP2ZfEXgOyuIoF0xL3VJNNvXSOCNJpI2uLYWl3G0is6GOaJQjqoMhG5vxC8bB9WvZnm5LscgcnBrtPgV+3B8aP2V/D8+i+B/iDrem+GLksbjw5eeXqegXRY5Jl027WW0kJOTloic89a9+lmVRRUXLRfd9x83icK5PXX8/vP27sP+C7vwi8ePqcSySaXFp6xva3F1eQyWuqo5ySk0HmJG6BhvjnMcg7K2DXiN9/wcM+ANdWO30/wd4jutTvJY4YLGO7ge43u+xU8sAFnIywWMvyVQlSTt/O3Tv8AgrJ4n0i3Avvgp+yxrF7nLXlz8ItIgmc+pFvHEg/BBWzb/wDBdL46aBp89t4TsPhd4CjmTy2/4RvwXY2J2nt908V2f2hdL3rfI4VhrO1j9P8A9qX9obxB4E/YQ8WeMPEFja+HPFY8MSyy2UFwZU028mjKxwCQ43MjOqFhwzKSvBAr8t/2RdR0/wCC/wADvB3xG1KC61RtD8UJqMmkTX72+n3uj2U0MtwJSoZwZ7pltwEBBUNmN+CPB/j1+1r8Vf2pZIx478a654jhhfzI7WaUR2sb44ZYIwsYbGeQucZqz+zp440X4eW2qy+IdLvtekiVZdGsGuJFsUuSyh5mRWAEoRQFc5A5IUsqFeLE4tSlzR6HTSoy2OJ1Ga/1jxJfXmpRyrqN/O9zdB0KMZJCXJKnpndn6Gum8GeFWuJzIwCDuzd/YVPL9s8Za1LqV7sRpeAEBEcKj7saAkthRxkkn1JOTXbeF9Aaa1yiKyrwACRknsMcf5NeTVrdT2MNg7tXL/g2IaAsnlsg807jweldHp+pXuv6tbWFlHLcXl9KltBGgy0sjsFVQPUkgf8A1hXMX5ksZjvQggYPzd/T6f8A16+mf2Mfg8PBHhN/iVrUANxMrW/h63cYLbso1x/wL5lU9kEjchlNc8W27nbVSpqxvfG34P8Air4FeLbjRfF2jXejX6MfLd13xXag48yGQfI6dOVJxnBAIIHlfiW+MlnPGqttZSCWGckg/wCeKKK55RUaritkddCTqUlOW7PC/FPggROxNuYRIT+8HBP4D+n5iuT1bwxLCQ5QOpOBwf8AI/ziiiutyatYwdKLi2zNu/DLRuf3LMP723O3n8v5/nxVY+HHSXAtmXnJZlI/AUUVUaje5zyoxJYPDc6xBvICxg/xHBPr+HvW1oPhFJgHcA85I/hPt/j9PwooqJTdjejh4X1OntfC21lPmKE652kAD0z261s6OyaRFvG9SwIUY6j/ABJ/kKKK5pNvRnWoqK5kdl+zn8HX/aG+Mun6HKsq6ZETeapJGTmG0QjcAR0ZyViUjkM4PQHH158TPFEOv66kFkkUWk6YBbWcUK7YsKApZQOAoACKOm1cjhyKKK66KSVzzMU252P/2Q==

0x06:Certificate verfication

强烈建议始终使用ssl验证。默认情况下,urllib3不验证https请求

为了启用验证,我们需要一组根证书。最简单而且最可靠的方法就是使用火狐提供的根证书包的certifi包

pip install certifi

获得证书后,我们可以创建一个PoolManager在发出请求的时候验证证书的方法

http=urllib3.PoolManager(cer_reqs='CERT_REQUIRED',ca_certs=certifi.where())

这个PoolManager将会自动的处理证书的验证,并在SSLError再验证失败的时候提出

r=http.request('GET','https://google.com')

r=http.request('GET', 'https://expired.badssl.com')

注意:如果需要,你可以使用操作系统提供的证书。只需要指定证书包的完整路径作为ca_carts参数而不是certifi.where()。例如,大多数linux操作系统存储证书/etc/ssl/certs/ca-certificates.crt。其他的操作系统可能很难。

0x07:Using timeouts

超时允许你控制请求在被中止前运行多长时间。再简单情况下,你可以将超时指定为float to request()

http=urllib3.PoolManager()

r=http.request('GET','http://httpbin.org/delay/3',timeout=4.0)

print(r.status)

>>200

http=urllib3.PoolManager()
r=http.request('GET','http://httpbin.org/delay/3',timeout=2.5)
print(r.status)

>>ReadTimeoutError

对于更加精细的控制,你可以使用一个Timeout实例,让你指定单独的连接和读取超时

http=urllib3.PoolManager()

r=http.request('GET','http://httpbin.org/delay/3',timeout=urllib3.Timeout(connect=1.0))

http=urllib3.PoolManager()
r=http.request('GET','http://httpbin.org/delay/3',timeout=urllib3.Timeout(connect=1.0,read=2.0))
print(r.status)


如果你希望所有的请求都受到相同的超时时间限制的话,我们可以在PoolManager级别上指定超时值

http=urllib3.PoolManage(timeout=3.0)

http=urllib3.PoolManage(timeout=3.0,read=2.0)

当然我们任然可以指定timeout来覆盖这个PoolManager超时的request()

0x08:Retrying requests

urllib3可以自动的重试idempotent等请求。同样的机制可以处理重定向。你可以使用retries参数to来控制重试requests(),默认情况下,urllib3将重试请求3次冰球跟随最多3次重定向。要更改重试次数,只需要指定一个整数即可

http=urllib3.PoolManager()
r=http.request('GET','http://httpbin.org/ip',retries=10)
print(r.status)

为了能够取消所有的重试和重定向,可以使用retries=False

http=urllib3.PoolManager()
r=http.request('GET','http://httpbin.org/redirect/1',retries=False)

print(r.status)

>>302

要禁用重定向但是需要保留重试,请指定redirect=False

http=urllib3.PoolManager()

r=http.request('GET','http://httpbin.org/redirect/1',redirect=False)

print(r.status)

>>302

为了更细化的控制,你可以使用一个retry实例。这个类允许你更好的控制请求重试的方式

例如:总共执行3次重试,但仅限于2次重定向。

http=urllib3.PoolManager()
r=http.request('GET','http://httpbin.org/redirect/1',retries=urllib3.Retry(3,redirect=2))

print(r.status)

当然你也可以禁用太多的重定向的异常,并且返回302响应

http=urllib3.PoolManager()
r=http.request('GET','http://httpbin.org/redirect/1',retries=urllib3.Retry(redirect=2,raise_on_redirect=False))

print(r.status)

如果你希望所有的请求都遵循相同的重试策略,那门可以在PoolManager级别上指定重试。

http=urllib3.PoolManager(retries=False)

http=urllib3.PoolManager(retries=urllib3.retry(5,redirect=2))

当然我们同样是可以使用retries来覆盖这个PoolManager的重试策略request()

0x09:Errors&Exceptions

urllib3包装较低级别的异常,例如:

http=urllib3.PoolManager()
try:
    r=http.request('GET', 'nx.example.com', retries=False)
    print(r.status)
except urllib3.exceptions.NewConnectionError:

    print("urllib3 connection error!")

0x010:Logging

如果你使用的是标准库logging模块,urllib3将会发出多个日志。在某些情况下,这不可能是不可取的。你可以使用标准记录器界面来更改urllib3记录器的日志级别

logging.getLogger('urllib3').setLevel(logging.WARNING)






























猜你喜欢

转载自blog.csdn.net/weixin_39997829/article/details/80179636