Python爬虫【第3篇】【多线程】

一、多线程

Python标准库提供2个模块，thread是低级模块，threading是高级模块

1.threading模块创建多线程

方式1：把1个函数传入并创建Thread实例，然后调用start方法开始执行

import random
import time,threading
# 新线程执行的代码
def thread_run(urls):
    print 'Current %s is running...' % threading.current_thread().name
    for url in urls:
        print '%s ---->>> %s' % (threading.current_thread().name,url)
        time.sleep(random.random())
    print '%s ended' % threading.current_thread().name
    
print '%s is running...' % threading.current_thread().name
t1 = threading.Thread(target=thread_run,name='Thread_1',args=(['url_1','url_2','url_3'],))
t2 = threading.Thread(target=thread_run,name='Thread_2',args=(['url_4','url_5','url_6'],))
t1.start()
t2.start()
t1.join()
t2.join()
print '%s ended' % threading.current_thread().name

执行结果：
MainThread is running...
Current Thread_1 is running...
Thread_1 ---->>> url_1
Current Thread_2 is running...
Thread_2 ---->>> url_4
Thread_1 ---->>> url_2
Thread_2 ---->>> url_5
Thread_2 ---->>> url_6
Thread_1 ---->>> url_3
Thread_1 ended
Thread_2 ended
MainThread ended

方式2：从threading.Thread继承并创建线程类，然后重写__init__方法和run方法