用ZMQ实现任务调度

首先做个测试：

1，一个性能低下的服务（或者页面）

<?php

$time = rand(1,5);

//current task costs $time seconds
sleep($time);

//result
echo "Hello_$time";

?>

用ab测试一下结果如下

ciaos:~/penjin # ab2 -c 500 -n 10000 http://127.0.0.1/index.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Finished 10000 requests


Server Software:        nginx
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /index.php
Document Length:        166 bytes

Concurrency Level:      500
Time taken for tests:   6.71600 seconds
Complete requests:      10000
Failed requests:        418
   (Connect: 0, Length: 418, Exceptions: 0)
Write errors:           0
Non-2xx responses:      9582
Total transferred:      3040612 bytes
HTML transferred:       1593538 bytes
Requests per second:    1647.01 [#/sec] (mean)
Time per request:       303.580 [ms] (mean)
Time per request:       0.607 [ms] (mean, across all concurrent requests)
Transfer rate:          489.00 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   15 209.5      0    3003
Processing:     1  138 711.0      3    6054
Waiting:        0  138 711.0      3    6054
Total:          1  153 767.3      3    6064

Percentage of the requests served within a certain time (ms)
  50%      3
  66%      4
  75%      4
  80%      4
  90%      4
  95%      6
  98%   4007
  99%   5024
 100%   6064 (longest request)

失败的请求达到418个，rps = 1647，最长请求6秒结束，性能不是很好。下面我们可以用zmq实现个简单的任务调度

2，使用zmq的router/dealer模型（模型图如下）

首先编写broker.php实现将前台请求路由给后台的worker

<?php

// Prepare our context and sockets
$context = new ZMQContext();
$frontend = new ZMQSocket($context, ZMQ::SOCKET_ROUTER);
$backend = new ZMQSocket($context, ZMQ::SOCKET_DEALER);
$frontend->bind("tcp://*:5559");
$backend->bind("tcp://*:5560");

// Initialize poll set
$poll = new ZMQPoll();
$poll->add($frontend, ZMQ::POLL_IN);
$poll->add($backend, ZMQ::POLL_IN);
$readable = $writeable = array();

// Switch messages between sockets
while (true) {
    $events = $poll->poll($readable, $writeable);

    foreach ($readable as $socket) {
        if ($socket === $frontend) {
            // Process all parts of the message
            while (true) {
                $message = $socket->recv();
                // Multipart detection
                $more = $socket->getSockOpt(ZMQ::SOCKOPT_RCVMORE);
                $backend->send($message, $more ? ZMQ::MODE_SNDMORE : null);
                if (!$more) {
                    break; // Last message part
                }
            }
        } elseif ($socket === $backend) {
            $message = $socket->recv();
            // Multipart detection
            $more = $socket->getSockOpt(ZMQ::SOCKOPT_RCVMORE);
            $frontend->send($message, $more ? ZMQ::MODE_SNDMORE : null);
            if (!$more) {
                break; // Last message part
            }
        }
    }
}
?>

编写worker.php，开启多个worker提供服务，我这里开启5个作为测试

<?php

function worker_thread()
{

        $context = new ZMQContext();

        // Socket to talk to clients
        $responder = new ZMQSocket($context, ZMQ::SOCKET_REP);
        $responder->connect("tcp://localhost:5560");

        while (true) {
                // Wait for next request from client
                $string = $responder->recv();

                // Do some 'work'
                sleep($string);

                // Send reply back to client
                $responder->send("Hello_".$string);
        }

}

for ($worker_nbr = 0; $worker_nbr < 5; $worker_nbr++) {
        $pid = pcntl_fork();
        if ($pid == 0) {
                worker_thread();

                return;
        }
}
?>

修改index.php如下：（index.php不再处理业务，而是发给后台的任务调度器分发处理）

<?php

$time = rand(1,5);

$context = new ZMQContext();

$requester = new ZMQSocket($context, ZMQ::SOCKET_REQ);
$requester->connect("tcp://localhost:5559");

$requester->send($time);
$string = $requester->recv();
printf ("Received reply [%s]%s", $string, PHP_EOL);

?>

再用ab测试一下

ciaos:~/penjin # ab2 -c 500 -n 10000 http://127.0.0.1/index.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Finished 10000 requests


Server Software:        nginx
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /index.php
Document Length:        0 bytes

Concurrency Level:      500
Time taken for tests:   3.627223 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Non-2xx responses:      10000
Total transferred:      1340000 bytes
HTML transferred:       0 bytes
Requests per second:    2756.93 [#/sec] (mean)
Time per request:       181.361 [ms] (mean)
Time per request:       0.363 [ms] (mean, across all concurrent requests)
Transfer rate:          360.61 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0  109 561.5      0    3000
Processing:    17   53  49.9     38    1467
Waiting:       17   53  49.8     38    1466
Total:         31  162 596.6     38    3619

Percentage of the requests served within a certain time (ms)
  50%     38
  66%     38
  75%     39
  80%     72
  90%     75
  95%    162
  98%   3230
  99%   3241
 100%   3619 (longest request)

可以看到请求全部成功，最长请求耗费3.6秒，rps达到2700多个，处理能力大大加强。比起用gearman来要简单不少。

用ZMQ实现任务调度

猜你喜欢