Based on the thread pool queue queue implementation

This article by article syncing pushed to blog garden, display layout may be wrong, please forgive me!

Write the previous text: in Python available to multi-process the process pool for the thread, Python2 does not directly provide the thread pool class (provided thread pool function Python3 in), and the thread pool in parallel in the wider application, thus achieving a process pool function is necessary. Based Queue (queue) function to achieve the thread pool function.

In Python3 standard library provides a thread pool , pool process function, it is recommended to use the standard library.

from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import ProcessPoolExecutor

Implementation code:

# / usr / bin / the env to python3! 
# - * - Coding: UTF-. 8 - * - 
__auth__ = "SongWei" 
Import Threading, queue, Time 
 
class Threadpool: 
    '' 'based on the thread pool queue queue implemented' '' 
 
    DEF the __init__ (Self, max_thread = 1): 
        '' 'the process of creating the queue' '' 
        self.queue = Queue.Queue (MAXSIZE = max_thread) 
 
    DEF the Apply (Self, target = None, args = (), callback = None, calljoin = True , ** kwargs): 
        '' ': param callback callback function when running after the end of the sub-thread function returns the value passed in the callback function 
            : param calljoin Boolean value callback function is blocked process pool by default True only if the target and callback functions are executed after the end of the thread is seen as the end of the 
            other parameters are the same threading.Thread class 
            Note: only when the target and callback functions are executed after the end of the message queue will retrieve the value (that is, the callback function will block the thread pool) 
        '' ' 
        iF not callback:
            callback = self._callback
        of the threading.Thread = T (= self._decorate target (target, the callback, calljoin), args = args, ** kwargs) 
        self.queue.put (T) 
        t.start () 
 
    DEF the Join (Self): 
        '' ' 
            when there blocked thread pool when the sub-thread end of the main thread is not performed 
            Note: calljoin = False when the callback is executed because it is only after the join to retrieve the message queue will not wait for a callback function 
        '' ' 
        the while self.queue.qsize ( ): 
            the time.sleep (0.05) 
 
    DEF _decorate (Self, target, the callback, calljoin): 
        '' ': param receiving a target objective function 
            : param the callback function to accept a callback 
            : param backjoin Boolean true if the callback function when otherwise, when released after the end of the queue after the queue is released objective function performed 
            on this function is essentially a decorator, i.e., after the operation the objective function is performed to retrieve queue (self.queque.get ()), and the return value as parameters callback function. 
        '' '
        warpper DEF (* args, ** kwargs): 
            RES = target (* args, ** kwargs) 
            IF calljoin: 
                the callback (RES) 
                self.queue.get () 
            the else: 
                self.queue.get () 
                the callback (RES) 
            return RES 
        return wrapper 
 
    DEF _callback (Self, * args, ** kwargs): 
        '' 'when you did not pass a callback function to do nothing' '' 
        Pass 
 
invocation example: 
result_list = [] 
DEF FUNC (Arg): 
    Print ( 'being awaiting execution S% '% Arg) 
    the time.sleep (10) 
    return Arg 
 
DEF Back (RES): 
    Print (' I have retrieved the data:% S '% RES) 
    result_list.append (RES)
 
pool = Threadpool(max_thread=20)
for i in range(40):
    pool.apply(target=func,args=(i,),callback=back)
pool.join()
print(result_list)