python-common modules

Commonly used built-in modules

datetime

From datetime import datetime
Datetime.now() returns the current date and time type is datetime
Datetime(y,m,d,H,i,s) parameter year month day hour minute second returns datetime type
Datetime () .timestamp () converts datetime to timestamp
Datetime.fromtimestamp(t) converts timestamp to datetime type
Datetime.utcfromtimestamp(t) converts timestamp to UTC standard time
Datetime.strptime('2015-6-1 18:19:59' , '%Y-%m-%d %H:%M:%S' ) convert string to datetime type
Datetime.now().strftime(%a, %b %d %H:%M) convert datetime type to string type
datetime加减 from datetime import datetime,timedelta
datetime.now()+timedelta(days=1) tomorrow's date and time
datetime.now()-timedelta(hours=10) time ten hours ago
datetime.now()+timedelta(days=1 , hours=10) the date ten hours after a day
Timezone(timedelta(hours=8)) creates a time zone
datetime.now().replace(tzinfo=Timezone(timedelta(hours=8))) to force timezone
Datetime.utcnow().replace(tzinfo=timezone.utc) to get the utc time and force the time zone
Time zone object . astimezone(timezone(timedelta(hours=8))) converts time zone to Beijing time zone

Collections collection module

The namedtuple function, used to create a custom tuple object , specifies the number of tuple elements, and can use attributes instead of indexes to refer to an element of the tuple
- from collections import namedtuple
- Point = namedtuple(‘Point’,[‘x’,y])
- p=Point(1,2)
- P.x
The one-way list list can be deleted by del l[index] and l.pop(value)
Deque two-way list, which can efficiently implement insertion and deletion operations
- from collections import deque
- q=also(['a','b','c'])
- q.append(‘x’)
- Q.appendleft(‘y’)
- append(),pop(),appendleft(),popleft()等操作

When the key referenced by defaultdict does not exist when using dict , it will throw a keyerror using defaultdict and return a default value when the key does not exist
- from collections import defaultdict
- dd=defaultdict(lambda: ’N/A’)
- dd[‘key1’]=‘abc’
- dd['key1']#key1 has a return value
- Dd['key2']#key2 does not exist and returns the default value N/A

Ordereddict can guarantee the order of keys
In dict , keys are unordered, and the order of keys cannot be determined
- from collections import OrderedDict
- by = OrderedDict ()
- from ['z'] = 1
- from ['y'] = 2
- from ['x'] = 3
- list(od.keys()) returns in the order of insertion

Counter _

from collections import deque

>>> q = deque(['a', 'b', 'c'])

>>> q.append('x')

>>> q.appendleft('y')

>>> q

Base64

A way to represent arbitrary binary data in 64 characters
- import base64
- Base64.b64encode() encoding
- Base64.b64decode() decoding
- Base64.urlsafe_b64encide()
- base64.urlsafe_b64decode()

Struct

Convert bytes to other binary data
- import struct
- Struct.pack('>I',10240099) Convert any data type to bytes
  - I represents a 4 -byte unsigned integer H represents a 2 -byte unsigned integer
- struct.unpack('>IH',b'\xf0\xf0\xf0\xf0\x80\x80') turns bytes into the corresponding data type

Hashlib

Provides MD5 , SHA1 algorithm , converts data of any length into a data string of fixed length
- import hashlib
- md5=hashlib.md5()
- Md5.update(‘string’.encode(‘utf-8’))
- md5.hexdigest()
The usage of sha1 is consistent with that of md5

Hmac

A standard algorithm, the enhancement of md5
- import hmac
- message=b’hello,world!’;
- key=b’secret’
- h=hmac.new(key,message,digestmod=‘MD5’)
- h.hexdigest()# View the generated data string

Itertools

for manipulating iterable objects
The count() function creates an infinite iterator
- import itertools
- natuals=itertools.count(1)
- For n in natuals:
  - print(n)
cycle() will repeat the incoming sequence indefinitely
- import itertools
- cs=itertools.cycle('ABC')# String is also a kind of sequence
- for c in cs:
  - print(c)

The -repeat() function repeats an element infinitely, specifying the second parameter is the number of repetitions

- itertools.repeat(‘A’,3)

chain() concatenates a set of iterator objects to form a larger iterator
- for c in itertools.chain(‘ABC’,’XYZ’):
  - print(c)

groupby() picks out adjacent duplicate elements in the iterator and puts them together
- for key, group in itertools.groupby('AAABBBCCAAA'):
- print(key,list(group))

Contextlib

Urllib

The request module of urllib can easily grab the URL content, that is, send a GET request to the specified page, and then return the HTTP response:
Crawl the url
- from urllib import request
- With request.urlopen(‘url’) as f:
  - data=f.read();
  - print(‘status:’,f.status,f.reason)
  - For k,v in f.getheaders():
    - print ('% s:% s'% (k, v))
  - print(‘data’,data.decode(‘utf-8’))
The get request uses the request object (note that there is no s ), and header information needs to be added
- from urllib import request
- req = request.Request('http://www.douban.com/')
- req.add_header('User-Agent', 'Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25')
- with request.urlopen(req) as f:
  - data=f.read();
  - print(‘status:’,f.status,f.reason)
  - For k,v in f.getheaders():
    - print ('% s:% s'% (k, v))
  - print(‘data’,data.decode(‘utf-8’))

post request, need to pass parameters

from urllib import request, parse

print('Login to weibo.cn...')

email = input('Email: ')

passwd = input('Password: ')

login_data = parse.urlencode([

('username', email),

('password', passwd),

('entry', 'mweibo'),

('client_id', ''),

( 'savestate' , '1' ),

('ec', ''),

('pagerefer', 'https://passport.weibo.cn/signin/welcome?entry=mweibo&r=http%3A%2F%2Fm.weibo.cn%2F')

])

req = request.Request('https://passport.weibo.cn/sso/login')

req.add_header('Origin', 'https://passport.weibo.cn')

req.add_header('User-Agent', 'Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25')

req.add_header('Referer', 'https://passport.weibo.cn/signin/login?entry=mweibo&res=wel&wm=3349&r=http%3A%2F%2Fm.weibo.cn%2F')

python-common modules

Guess you like