[spirit] benchmarks/qi benchmarks valid?
Hey list, I was merely wondering if what I'm doing is right or if I fail at basic benchmarking: impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out /////////////////////////////////////////////////////////////////////////// Numbers to test: 1 digit number:-5 2 digit number:-54 3 digit number:541 4 digit number:-9604 5 digit number:-50733 6 digit number:-162974 7 digit number:-3133813 8 digit number:39950230 9 digit number:298344332 /////////////////////////////////////////////////////////////////////////// atoi_test: 5.0041852930 [s] {checksum: 13f6bf40} strtol_test: 5.4193627260 [s] {checksum: 13f6bf40} spirit_int_test: 2.2137115780 [s] {checksum: 13f6bf40} impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out /////////////////////////////////////////////////////////////////////////// Numbers to test: 1 digit number:5 2 digit number:-55 3 digit number:-978 4 digit number:2835 5 digit number:-78343 6 digit number:354806 7 digit number:9413631 8 digit number:-10824620 9 digit number:-369204759 /////////////////////////////////////////////////////////////////////////// atoi_test: 0.5902040060 [s] {checksum: e9ed193a} strtol_test: 0.6001010490 [s] {checksum: e9ed193a} spirit_int_test: 3.6562933090 [s] {checksum: e9ed193a} According to this, the optimized binary is 10 times slower wrt to the atoi/strtol implementation. I find that somehow quite hard to believe, is my toolchain broken, the testsuite or is it really true? The level of optimization doesn't matter btw, I get similar results with -O[,1,2] Any ideas are welcome. -- Mierswa, Daniel If you still don't like it, that's ok: that's why I'm boss. I simply know better than you do. --- Linus Torvalds, comp.os.linux.advocacy, 1996/07/22
On 14 Sep 2009, at 05:25, Daniel Mierswa wrote:
Hey list, I was merely wondering if what I'm doing is right or if I fail at basic benchmarking:
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/ qi $ g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out /////////////////////////////////////////////////////////////////////////// Numbers to test: 1 digit number:-5 2 digit number:-54 3 digit number:541 4 digit number:-9604 5 digit number:-50733 6 digit number:-162974 7 digit number:-3133813 8 digit number:39950230 9 digit number:298344332 /////////////////////////////////////////////////////////////////////////// atoi_test: 5.0041852930 [s] {checksum: 13f6bf40} strtol_test: 5.4193627260 [s] {checksum: 13f6bf40} spirit_int_test: 2.2137115780 [s] {checksum: 13f6bf40}
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/ qi $ g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out /////////////////////////////////////////////////////////////////////////// Numbers to test: 1 digit number:5 2 digit number:-55 3 digit number:-978 4 digit number:2835 5 digit number:-78343 6 digit number:354806 7 digit number:9413631 8 digit number:-10824620 9 digit number:-369204759 /////////////////////////////////////////////////////////////////////////// atoi_test: 0.5902040060 [s] {checksum: e9ed193a} strtol_test: 0.6001010490 [s] {checksum: e9ed193a} spirit_int_test: 3.6562933090 [s] {checksum: e9ed193a}
According to this, the optimized binary is 10 times slower wrt to the atoi/strtol implementation. I find that somehow quite hard to believe, is my toolchain broken, the testsuite or is it really true? The level of optimization doesn't matter btw, I get similar results with -O[,1,2]
Any ideas are welcome.
First obvious thought: Why are the numbers changing. Parsing different numbers can take different amounts of time (although I'm surprised at that large a change). Chris
Christopher Jefferson wrote:
First obvious thought: Why are the numbers changing. Parsing different numbers can take different amounts of time (although I'm surprised at that large a change).
Random numbers with 1-9 digits for each run. One set of numbers for all parsers per run. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net http://doiop.com/jdg_facebook
AMDG Daniel Mierswa wrote:
I was merely wondering if what I'm doing is right or if I fail at basic benchmarking:
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out <snip> atoi_test: 5.0041852930 [s] {checksum: 13f6bf40} strtol_test: 5.4193627260 [s] {checksum: 13f6bf40} spirit_int_test: 2.2137115780 [s] {checksum: 13f6bf40}
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out <snip> atoi_test: 0.5902040060 [s] {checksum: e9ed193a} strtol_test: 0.6001010490 [s] {checksum: e9ed193a} spirit_int_test: 3.6562933090 [s] {checksum: e9ed193a}
-O0 means no optimization.
According to this, the optimized binary is 10 times slower wrt to the atoi/strtol implementation. I find that somehow quite hard to believe, is my toolchain broken, the testsuite or is it really true? The level of optimization doesn't matter btw, I get similar results with -O[,1,2]
In Christ, Steven Watanabe
Steven Watanabe wrote:
AMDG
Daniel Mierswa wrote:
I was merely wondering if what I'm doing is right or if I fail at basic benchmarking:
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out <snip> atoi_test: 5.0041852930 [s] {checksum: 13f6bf40} strtol_test: 5.4193627260 [s] {checksum: 13f6bf40} spirit_int_test: 2.2137115780 [s] {checksum: 13f6bf40}
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out <snip> atoi_test: 0.5902040060 [s] {checksum: e9ed193a} strtol_test: 0.6001010490 [s] {checksum: e9ed193a} spirit_int_test: 3.6562933090 [s] {checksum: e9ed193a}
-O0 means no optimization.
According to this, the optimized binary is 10 times slower wrt to the atoi/strtol implementation. I find that somehow quite hard to believe, is my toolchain broken, the testsuite or is it really true? The level of optimization doesn't matter btw, I get similar results with -O[,1,2]
This has been discussed in the Spirit list. Seems that the test is skewed bigtime with no optimization. The benchmarks were designed with full optimization, of course. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net http://doiop.com/jdg_facebook
On 14 Sep 2009, at 16:21, Joel de Guzman wrote:
Steven Watanabe wrote:
AMDG Daniel Mierswa wrote:
I was merely wondering if what I'm doing is right or if I fail at basic benchmarking:
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/ qi $ g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out <snip> atoi_test: 5.0041852930 [s] {checksum: 13f6bf40} strtol_test: 5.4193627260 [s] {checksum: 13f6bf40} spirit_int_test: 2.2137115780 [s] {checksum: 13f6bf40}
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/ qi $ g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out <snip> atoi_test: 0.5902040060 [s] {checksum: e9ed193a} strtol_test: 0.6001010490 [s] {checksum: e9ed193a} spirit_int_test: 3.6562933090 [s] {checksum: e9ed193a} -O0 means no optimization. According to this, the optimized binary is 10 times slower wrt to the atoi/strtol implementation. I find that somehow quite hard to believe, is my toolchain broken, the testsuite or is it really true? The level of optimization doesn't matter btw, I get similar results with -O[,1,2]
This has been discussed in the Spirit list. Seems that the test is skewed bigtime with no optimization. The benchmarks were designed with full optimization, of course.
I think the problem is why do atoi and strtol get 10x faster with no optimisation? Chris
Christopher Jefferson wrote:
On 14 Sep 2009, at 16:21, Joel de Guzman wrote:
Steven Watanabe wrote:
AMDG Daniel Mierswa wrote:
I was merely wondering if what I'm doing is right or if I fail at basic benchmarking:
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out <snip> atoi_test: 5.0041852930 [s] {checksum: 13f6bf40} strtol_test: 5.4193627260 [s] {checksum: 13f6bf40} spirit_int_test: 2.2137115780 [s] {checksum: 13f6bf40}
impulze@istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out <snip> atoi_test: 0.5902040060 [s] {checksum: e9ed193a} strtol_test: 0.6001010490 [s] {checksum: e9ed193a} spirit_int_test: 3.6562933090 [s] {checksum: e9ed193a} -O0 means no optimization. According to this, the optimized binary is 10 times slower wrt to the atoi/strtol implementation. I find that somehow quite hard to believe, is my toolchain broken, the testsuite or is it really true? The level of optimization doesn't matter btw, I get similar results with -O[,1,2]
This has been discussed in the Spirit list. Seems that the test is skewed bigtime with no optimization. The benchmarks were designed with full optimization, of course.
I think the problem is why do atoi and strtol get 10x faster with no optimisation?
Not in my machine. I can't reproduce his results. So, I don't know. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
On 14.09.2009 18:20, Christopher Jefferson wrote:
I think the problem is why do atoi and strtol get 10x faster with no optimisation? Precisely, I'm thinking my toolchain is hoaxed or something therein, I wish I would be more knowledgable to debug this further but all I can do is post here (and on spirit as was already mentioned).
-- Mierswa, Daniel If you still don't like it, that's ok: that's why I'm boss. I simply know better than you do. --- Linus Torvalds, comp.os.linux.advocacy, 1996/07/22
participants (4)
-
Christopher Jefferson -
Daniel Mierswa -
Joel de Guzman -
Steven Watanabe