【翻译】Why is reading lines from stdin much slower in C++ than Python?

Why is reading lines from stdin much slower in C++ than Python? - Stack Overflow

I wanted to compare reading lines of string input from stdin using Python and C++ and was shocked to see my C++ code run an order of magnitude slower than the equivalent Python code. Since my C++ is rusty and I’m not yet an expert Pythonista, please tell me if I’m doing something wrong or if I’m misunderstanding something.

有道：我想比较使用Python和c++从stdin中读取字符串输入的行数，却惊讶地发现我的c++代码的运行速度比等效的Python代码慢了一个数量级。由于我的c++很生疏，而且我还不是python的专家，如果我做错了什么，或者我误解了什么，请告诉我。

我想要比较用python和C++从stdin中读取的输入的字符串行数，但是震惊的发现我C++的代码比起相同的python代码慢了一个数量级，因为我的C++是生疏的，并且我也不是一个python专家，如果我做错了或者理解错了什么东西，请告诉我。

(TLDR answer: include the statement: cin.sync_with_stdio(false) or just use fgets instead.

TLDR results: scroll all the way down to the bottom of my question and look at the table.)

(TLDR的答案:包含语句:cin.sync_with_stdio(false)或者直接使用fgets。

TLDR结果:一直滚动到我的问题的底部，看看表格。)

~~这里不清楚TLDR是什么东西~~

TLDR 答案：包含声明：cin.sync_with_stdio(false)或者仅使用fgets代替

TLDR结果：一直滚动带我的问题的底部并且查看表格

C++ code:

#include <iostream>
#include <time.h>

using namespace std;

int main() {
    string input_line;
    long line_count = 0;
    time_t start = time(NULL);
    int sec;
    int lps;

    while (cin) {
        getline(cin, input_line);
        if (!cin.eof())
            line_count++;
    };

    sec = (int) time(NULL) - start;
    cerr << "Read " << line_count << " lines in " << sec << " seconds.";
    if (sec > 0) {
        lps = line_count / sec;
        cerr << " LPS: " << lps << endl;
    } else
        cerr << endl;
    return 0;
}

// Compiled with:
// g++ -O3 -o readline_test_cpp foo.cpp

Python Equivalent:

#!/usr/bin/env python
import time
import sys

count = 0
start = time.time()

for line in  sys.stdin:
    count += 1

delta_sec = int(time.time() - start_time)
if delta_sec >= 0:
    lines_per_sec = int(round(count/delta_sec))
    print("Read {0} lines in {1} seconds. LPS: {2}".format(count, delta_sec,
       lines_per_sec))

Here are my results:

$ cat test_lines | ./readline_test_cpp
Read 5570000 lines in 9 seconds. LPS: 618889

$ cat test_lines | ./readline_test.py
Read 5570000 lines in 1 seconds. LPS: 5570000

解释一下，把文件当做输入源，测试读取整个文件的行数需要多久，C++采用getline读取，python还不会，大概意思也是这样。

测试结果得出C++读5570000行需要9s,1s读61889行，而Python需要1s，每秒5570000行

I should note that I tried this both under Mac OS X v10.6.8 (Snow Leopard) and Linux 2.6.32 (Red Hat Linux 6.2). The former is a MacBook Pro, and the latter is a very beefy server,not that this is too pertinent.

我应该注意，我在Mac OS X v10.6.8(雪豹)和Linux 2.6.32 (Red Hat Linux 6.2)下都尝试了这个。前者是一台MacBook Pro，后者是一台非常强大的服务器，但这不是说得太切题了。

我应该注意到，我在Mac OS X v10.6.8（Snow Leopard）和Linux 2.6.32(Red Hat Linux 6.2)。前者是MacBook Pro，后者是一台非常强大的服务器，但是这不是特别切题。

$ for i in {1..5}; do echo "Test run $i at `date`"; echo -n "CPP:"; cat test_lines | ./readline_test_cpp ; echo -n "Python:"; cat test_lines | ./readline_test.py ; done

Test run 1 at Mon Feb 20 21:29:28 EST 2012
CPP:   Read 5570001 lines in 9 seconds. LPS: 618889
Python:Read 5570000 lines in 1 seconds. LPS: 5570000
Test run 2 at Mon Feb 20 21:29:39 EST 2012
CPP:   Read 5570001 lines in 9 seconds. LPS: 618889
Python:Read 5570000 lines in 1 seconds. LPS: 5570000
Test run 3 at Mon Feb 20 21:29:50 EST 2012
CPP:   Read 5570001 lines in 9 seconds. LPS: 618889
Python:Read 5570000 lines in 1 seconds. LPS: 5570000
Test run 4 at Mon Feb 20 21:30:01 EST 2012
CPP:   Read 5570001 lines in 9 seconds. LPS: 618889
Python:Read 5570000 lines in 1 seconds. LPS: 5570000
Test run 5 at Mon Feb 20 21:30:11 EST 2012
CPP:   Read 5570001 lines in 10 seconds. LPS: 557000
Python:Read 5570000 lines in  1 seconds. LPS: 5570000

上面作者写了脚本对数据进行了多次测试

Tiny benchmark addendum and recap

微小的基准补充和概述

简单的基准附录和摘要

For completeness, I thought I’d update the read speed for the same file on the same box with the original (synced) C++ code. Again, this is for a 100M line file on a fast disk. Here’s the comparison, with several solutions/approaches:

出于完整性考虑，我认为应该用原始的(同步的)c++代码更新同一机器上相同文件的读取速度。同样，这是针对高速磁盘上的100M行文件。以下是一些解决方案/方法的比较:

出于完整性考虑，我觉得我应该更新在相同环境（机器）下用原始的（同步的）C++代码读取相同文件的速度。同样，这是一个在高速磁盘上的100M的行文件，以下是一些解决方案/方法的比较。

Implementation	Lines per second
python (default)	3,571,428
cin (default/naive)	819,672
cin (no sync)	12,500,000
fgets	14,285,714
wc (not fair comparison)	54,644,808

上面表格作者比较了默认下的Pythn,C++，关闭同步的C++和gets的速度，第五个wc是一次读取多行。

这也对上了原来的一个观点，cin关闭同步后性能和scanf差不多。

精选评论：

Did you run your tests multiple times? Perhaps there is a disk cache issue. –
Vaughn Cato

你做过多次测试吗?可能是磁盘缓存问题。- - - - - -沃恩卡托

你运行过你的测试用例多次吗？可能是磁盘缓存的问题

@VaughnCato Yes, and on two different machines as well. – JJC

@VaughnCato是的，而且是在两台不同的机器上。- - - - - -JJC

对的，并且是在两台不同的机器上

wc -l is fast because it reads the stream more than one line at a time (it might be fread(stdin)/memchr('\n') combination). Python results are in the same order of magnitude e.g., wc-l.py – jfs

Wc -l速度很快，因为它一次读取流不止一行(它可能是fread(stdin)/memchr(‘\n’)组合)。Python的结果具有相同的数量级，例如:wc-l.py -jfs

wc-l更快是因为它一次读取的流超过一行（它可能是fread(stdin)/memchr(‘\n’)的结合体）。Python的结果是相同的数量级。

Since nobody seems to have mentioned why you get an extra line with C++: Do not test against cin.eof()!! Put the getline call into the 'if` statement

因为似乎没有人提到为什么在c++中要多写一行:不要对cin.eof()进行测试!!将getline调用放到’if '语句中

似乎没有人提到为什么你用C++多写了一行,不要对cin.eof()进行测试，把getline放入if中

The problem is synchronization with stdio – see my answer. – [Vaughn Cato]

问题是与stdio同步——参见我的答案。——(沃恩卡托)

问题是与stdio的同步——参见我的回答

【翻译】Why is reading lines from stdin much slower in C++ than Python?

猜你喜欢