Author Topic: Quick&dirty C++ language conformance tests (Read 19887 times)

Michael · « **on:** December 16, 2005, 02:45:09 pm »

Hello,

I have found this website which provides quick&dirty C++ language conformance tests for the Win32 platform. As C::B supports them, this website could be useful for deciding which compiler is best suitable for a specific project.

Michael

takeshimiya · « **Reply #1 on:** December 16, 2005, 06:19:08 pm »

It appears then, that the most c++ standard compliant compiler is:
-GCC >=3.2 at the top
-DMars >=8.45 very good also
-MSVC >=7.1 rather good nonetheless

-OpenWatcom <=1.3 pretty bad
-OpenWatcom >=1.4 Dev catching up, but still worse than others

Michael · « **Reply #2 on:** December 16, 2005, 06:28:27 pm »

Despite all, GNU GCC seems to be a good choice (I think there was a thread about comparing speed, executable size, etc of several compiler. May be an Intel Compiler C++ thread).

MSVC is also improving. MSVC 2005 Express seems also to compile library as Loki (see http://msdn.microsoft.com/vstudio/express/visualc/features/language/default.aspx), which is not so evident.

But none is absolutly perfect...there is still some work to do

.

Michael

thomas · « **Reply #3 on:** December 16, 2005, 09:22:20 pm »

I think this comparison (although still in favour to gcc) is quite unfair. They compare a 3 year old version of gcc with the current release of DM and the current release of Microsoft Visual C++. However, it makes me smile that gcc still beats them.

On a different subject, I recently found out something that I'd like to share with the many people who keep bashing gcc for its poor executable size. Read the manual. Yes, it sounds like a stupid recommendation, but seriously... do it.

You can not only tell the compiler to optimize the code, but you can also pass optimization flags to the linker. This is something few people seem to know (I did not know myself until two weeks ago). By passing -O2 to the linker, I managed to reduce the executable size of a program of mine by 18%.

takeshimiya · « **Reply #4 on:** December 16, 2005, 09:38:26 pm »

GNU/GCC is not that bad regarding executable size (still worse compared to others).

When it looses and badly, is in compilation/linkage time. I mean, compiling wx+cb can take more than 30 minutes in my new pc with mingw 3.4.
However, I have hopes that GCC post-4.1 will be much faster thanks to the new optimizations and features it introduces.

Ceniza · « **Reply #5 on:** December 17, 2005, 04:45:59 pm »

-O2 to the linker? 18%? I just tried that and the filesize is still the same.

-Wl,-O2

Am I missing something?

thomas · « **Reply #6 on:** December 17, 2005, 05:18:05 pm »

Quote from: Ceniza on December 17, 2005, 04:45:59 pm

-O2 to the linker? 18%? I just tried that and the filesize is still the same.

-Wl,-O2

Am I missing something?

I don't know really, lol. When I read that in the manpages, I immediately tested it, and it turned a 802kB app into a 658kB app, which made me go "wowww!". I then tried recompiling wxWidgets in that manner, but it did not seem to make a difference (although I am not sure whether or not I did it correctly. You know compiling wxWidgets is not precisely intuitive...). And since building wx takes forever, I did not bother any further. So far my experience with it.
Personally, I have currently no need to squeeze every bit out of my applications, so I prefer the linker running faster instead

But still, there is more in it than the eye meets.

Although I don't know how the linker optimizes at all, I guess it is mostly useful on large executables because you have the minimum overhead of 4k due to sections (at least using PE, don't know the limits of ELF) which is less noticeable the bigger the executable is, and because most likely "optimize" means dead-stripping unused code and playing with function alignments where it does not break anything.

The most notable thing, however, is that nobody (including myself) ever reads the documentation (yes, reading documentation sucks almost as much as writing), but everybody thinks he knows the tools he is using.
And then one day after many years, you discover that you don't know anything at all... I have half an hour of reading documentations on my daily schedule now

Urxae · « **Reply #7 on:** December 17, 2005, 05:55:08 pm »

Quote from: thomas on December 17, 2005, 05:18:05 pm

Quote from: Ceniza on December 17, 2005, 04:45:59 pm
-O2 to the linker? 18%? I just tried that and the filesize is still the same.

-Wl,-O2

Am I missing something?
I don't know really, lol. When I read that in the manpages, I immediately tested it, and it turned a 802kB app into a 658kB app, which made me go "wowww!". I then tried recompiling wxWidgets in that manner, but it did not seem to make a difference (although I am not sure whether or not I did it correctly. You know compiling wxWidgets is not precisely intuitive...). And since building wx takes forever, I did not bother any further. So far my experience with it.
Personally, I have currently no need to squeeze every bit out of my applications, so I prefer the linker running faster instead
But still, there is more in it than the eye meets.

I have noticed that in code containing a lot of template instantiations (in my case, anything non-trivial using Boost.Spirit) using optimizations can really reduce linker times. My guess is that inlining removes a lot of symbols that the linker would otherwise have to check for duplicates etc. It may be helped by the fact that the linker uses less memory and doesn't need to swap as much.

Ceniza · « **Reply #8 on:** December 17, 2005, 07:15:24 pm »

Quote from: thomas

And since building wx takes forever...

real 6m26.734s
user 2m59.290s
sys 1m25.821s

I find it quite acceptable, even more when real becomes closer to user

me22 · « **Reply #9 on:** December 17, 2005, 08:59:04 pm »

Quote from: Michael on December 16, 2005, 02:45:09 pm

I have found this website which provides quick&dirty C++ language conformance tests for the Win32 platform. As C::B supports them, this website could be useful for deciding which compiler is best suitable for a specific project.

Gotta love that of the C++ tests, MinGW ( not even native g++ ) only fails 2, one of which is "throwing destructor" which you shouldn't ever be using anyways.

And what a surprise, the old mingw fails C++0x extension tests *rolls eyes*

Michael · « **Reply #10 on:** December 17, 2005, 09:49:37 pm »

Quote from: thomas on December 16, 2005, 09:22:20 pm

I think this comparison (although still in favour to gcc) is quite unfair. They compare a 3 year old version of gcc with the current release of DM and the current release of Microsoft Visual C++. However, it makes me smile that gcc still beats them.

Yes, I have remarked too that the GCC version was relatively old, but the C++ conformance is a surprise

. I have though that Visual Studio 2005 had a better conformance, especially after all the bla bla of M$. But in the real not :?. I just wonder how could be the conformance of a latest GCC version (and also of the unix GCC original version).

Quote from: me22 on December 17, 2005, 08:59:04 pm

Gotta love that of the C++ tests, MinGW ( not even native g++ ) only fails 2, one of which is "throwing destructor" which you shouldn't ever be using anyways.

And what a surprise, the old mingw fails C++0x extension tests *rolls eyes*

I also like such tests

. When I have some free time, I search in Internet some comparison between C++ and other languages (recently I have found an interesting comparision/fight between C++ and Fortran. Fortran is much better than C++, C++ is not so OOP, C++ is quite a dead language,...

.

Anyway, I suppose that C++0x are too new stuff for the old GCC used in the tests. Visual Studio 2005 also do not that better. It shoulds, at least theoretically...Na ja, M$ is full of surprises

.

Michael

Urxae · « **Reply #11 on:** December 17, 2005, 10:23:50 pm »

Quote from: me22 on December 17, 2005, 08:59:04 pm

Quote from: Michael on December 16, 2005, 02:45:09 pm
I have found this website which provides quick&dirty C++ language conformance tests for the Win32 platform. As C::B supports them, this website could be useful for deciding which compiler is best suitable for a specific project.

Gotta love that of the C++ tests, MinGW ( not even native g++ ) only fails 2, one of which is "throwing destructor" which you shouldn't ever be using anyways.

Note that at the top he mentions he's evaluating compilers for Win32. Native g++ doesn't qualify, mingw32-g++ does.
That said:

Code

D:\Temp>(g++ throwing_destructor.cpp -o throwing_destructor.exe && throwing_destructor.exe && echo Success ) || echo Failure
Success

D:\Temp>(g++ friend_name_injection.cpp -o friend_name_injection.exe && friend_name_injection.exe && echo Success ) || echo Failure
Failure

D:\Temp>g++ --version
g++ (GCC) 3.4.2 (mingw-special)
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(Note: this isn't the latest version, but it's all I have at the moment)
So even though you normally shouldn't be throwing from destructors, by now it's handled correctly if it happens anyway.
Indeed weird that they use MSVC 2005, but such an old MinGW when 2004+ versions are available.

Michael · « **Reply #12 on:** December 18, 2005, 01:36:48 am »

Just for curiosity, I have tried to build the tests on C++ Language Features that failed with MinGW and I could build:

throwing destructor
friend name injection
right angle brackets

The other tests still fail

. Anyway, there are improvments

.

I used mingw32-g++.exe that come with C::B RC2 (gcc version 3.4.4 (mingw special)).

Michael

Urxae · « **Reply #13 on:** December 18, 2005, 02:05:21 am »

Quote from: Michael on December 18, 2005, 01:36:48 am

Just for curiosity, I have tried to build the tests on C++ Language Features that failed with MinGW and I could build:

throwing destructor
friend name injection
right angle brackets

The other tests still fail . Anyway, there are improvments .

You realize they shouldn't just build but also return a non-zero exit code when you run them, right?

Michael · « **Reply #14 on:** December 18, 2005, 02:52:04 am »

Quote from: Urxae on December 18, 2005, 02:05:21 am

You realize they shouldn't just build but also return a non-zero exit code when you run them, right?

Yes, at least two of them do it. Let me check the missing one.

Michael

Michael · « **Reply #15 on:** December 18, 2005, 03:20:30 am »

Hmmm, the last two of the list return 1, but the first (throwing destructor) returns 0. Visual Studio C++ .NET 2003 and gcc version 3.4.4 (mingw special) gives the same results and behaves (for what I could see) in the same way.

Michael

grv575 · « **Reply #16 on:** December 18, 2005, 04:32:42 am »

Quote from: Ceniza on December 17, 2005, 04:45:59 pm

-O2 to the linker? 18%? I just tried that and the filesize is still the same.

-Wl,-O2

Am I missing something?

-Os optimizes size
and it doesn't need to be passed directly to the linker.
Still optimizing for speed is more important (-O2, -O3)

thomas · « **Reply #17 on:** December 18, 2005, 03:27:39 pm »

Quote from: grv575 on December 18, 2005, 04:32:42 am

-Os optimizes size
and it doesn't need to be passed directly to the linker.

I can only repeat: read the manual.

Quote

-O level
If level is a numeric values greater than zero ld optimizes the output. This might take significantly longer and therefore
probably should only be enabled for the final binary.

grv575 · « **Reply #18 on:** December 18, 2005, 11:42:25 pm »

Code

man gcc says:

-O1 Optimize.
-O2 Optimize even more...performs nearly all supported optimizations that do not involve a space-speed tradeoff.
-O3 Optimize yet more.
-O0 Do not optimize.  This is the default.
-Os Optimize for size.  -Os enables all -O2 optimizations that do not typically increase code size.   It also performs further optimizations designed to reduce code size.

thomas · « **Reply #19 on:** December 19, 2005, 12:01:05 am »

Try man ld

280Z28 · « **Reply #20 on:** January 22, 2006, 04:28:07 pm »

If someone tried any of the ones MSVC 2005 fails, I would want to slap them for making me go :? as I tried to read their code.

Michael · « **Reply #21 on:** January 22, 2006, 05:33:30 pm »

Quote from: 280Z28 on January 22, 2006, 04:28:07 pm

If someone tried any of the ones MSVC 2005 fails, I would want to slap them for making me go :? as I tried to read their code.

Which one for example?

Michael

280Z28 · « **Reply #22 on:** January 22, 2006, 05:35:27 pm »

Quote from: Michael on January 22, 2006, 05:33:30 pm

Quote from: 280Z28 on January 22, 2006, 04:28:07 pm
If someone tried any of the ones MSVC 2005 fails, I would want to slap them for making me go :? as I tried to read their code.

Which one for example?

Michael

Code

int main(int argc, char *argv[])
{
  int i = compl((2 bitor 4) xor 4) bitand 3;
  bool b = not ((true or false) and true);

  b and_eq true;
  b or_eq false;
  b xor_eq true;
  b not_eq not b;

  return 0;
}

Michael · « **Reply #23 on:** January 22, 2006, 05:49:33 pm »

Quote from: 280Z28 on January 22, 2006, 05:35:27 pm

Code

int main(int argc, char *argv[])
{
  int i = compl((2 bitor 4) xor 4) bitand 3;
  bool b = not ((true or false) and true);

  b and_eq true;
  b or_eq false;
  b xor_eq true;
  b not_eq not b;

  return 0;
}

Yes, I have to admit it has made me confusing too

. Anyway, I like C++, because it can always surprise me with something I did not even know possible and therefore stimulate my desire to continue studying and using it.

Anyway, e.g., compl is an alternative to the ~ operator. The others are explained here.

Michael

280Z28 · « **Reply #24 on:** January 22, 2006, 06:16:36 pm »

Quote from: Michael on January 22, 2006, 05:49:33 pm

Yes, I have to admit it has made me confusing too . Anyway, I like C++, because it can always surprise me with something I did not even know possible and therefore stimulate my desire to continue studying and using it.

Anyway, e.g., compl is an alternative to the ~ operator. The others are explained here.

Michael

No, no. compl is a way to get coworkers to :x and then kill you. :lol:

Michael · « **Reply #25 on:** January 22, 2006, 06:22:05 pm »

Quote from: 280Z28 on January 22, 2006, 06:16:36 pm

No, no. compl is a way to get coworkers to :x and then kill you. :lol:

Yes, it is a possible consequence

.

Michael

News:

Author Topic: Quick&dirty C++ language conformance tests (Read 19887 times)

takeshimiya

takeshimiya

grv575

grv575