学习爬虫以及数据分析中, 整理一下常用的request学习方法。
传输url请求
payload = {'key1': 'value1', 'key2': 'value2'}
r = requests.get("http://httpbin.org/get", params=payload)
输出的结果为
http://httpbin.org/get?key2=value2&key1=value1
可以用于搜索关键词的网址,比如淘宝百度一类。同时还可以作为列表的值加入。
>>> payload = {'key1': 'value1', 'key2': ['value2', 'value3']} >>> r = requests.get('http://httpbin.org/get', params=payload) >>> print(r.url) http://httpbin.org/get?key1=value1&key2=value2&key2=value3
用于相应内容
>>> import requests
>>> r = requests.get('https://api.github.com/events')
>>> r.text
u'[{"repository":{"open_issues":0,"url":"https://github.com/...
同时也可以查看编码的方式,并且改变编码的方式
>>> r.encoding
'utf-8'
>>> r.encoding = 'ISO-8859-1'
定制请求表头
>>> url = 'https://api.github.com/some/endpoint'
>>> headers = {'user-agent': 'my-app/0.0.1'}
>>> r = requests.get(url, headers=headers)
post请求
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.post("http://httpbin.org/post", data=payload)
>>> print(r.text)
{
...
"form": {
"key2": "value2",
"key1": "value1"
},
...
}
或者也可以给data参数设置添加一个元组
>>> payload = (('key1', 'value1'), ('key1', 'value2'))
>>> r = requests.post('http://httpbin.org/post', data=payload)
>>> print(r.text)
{
...
"form": {
"key1": [
"value1",
"value2"
]
},
...
}