Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => Topic started by: MHaggag on October 14, 2005, 08:12:35 pm

Title: Binary size reduction - UPX
Post by: MHaggag on October 14, 2005, 08:12:35 pm
Using upx (http://upx.sourceforge.net/#download), I've been able to reduce the binary size by half, from 8.42 Mb to 4.21 Mb:
Code
        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
    292864 ->    103936   35.49%    win32/pe     codeblocks.exe
     21504 ->     10752   50.00%    win32/pe     console_runner.exe
   2043904 ->    666112   32.59%    win32/pe     codeblocks.dll
    418304 ->    200704   47.98%    win32/pe     exchndl.dll
     15960 ->     12888   80.75%    win32/pe     mingwm10.dll
   6037465 ->   3426777   56.76%    win32/pe     wxmsw242.dll
   --------------------   ------   -----------   -----------
   8830001 ->   4421169   50.07%                 [ 6 files ]

Might be a good idea to use upx in the build/packaging chain to reduce the package size. 56K users would be very thankful (Not me! :))

EDIT: Also, by linking to wxWidgets statically instead of dynamically, you guarantee that you only pull stuff you use. Are there any good reasons for using wxWidgets dynamically, aside from the INSANELY slow linking by gcc? (Last time I tried, it took half an hour to link wxWidgets statically into an application).
Title: Re: Binary size reduction - UPX
Post by: MHaggag on October 14, 2005, 08:32:14 pm
Quote from: Muhammad Haggag on October 14, 2005, 08:12:35 pm
EDIT: Also, by linking to wxWidgets statically instead of dynamically, you guarantee that you only pull stuff you use. Are there any good reasons for using wxWidgets dynamically, aside from the INSANELY slow linking by gcc? (Last time I tried, it took half an hour to link wxWidgets statically into an application).
Don't bother replying to this one. Most, if not all, plugins use wxWidgets, and thus the DLLs version is shared among them.
Title: Re: Binary size reduction - UPX
Post by: thomas on October 14, 2005, 08:44:49 pm
UPX is a mighty fine tool, but do note a few things:

- The installer compresses the installation files. Compressing an already compressed file usually gains very little, rarely more than 3-5%. There is even the possibility that double-compression is worse than only compressing once (the same is true for many things, think for example of checksums or MACs).

- Decompression takes time and has other side effects, for example the executable memory image is no longer simply a memory mapped file that can be paged in and out "for free" (paging is never "free", but you know what I mean). An IDE is an application that is loaded, minimized, and maximized many times per day. Granted, UPX is fast, but I do nevertheless perceive it as noticeable for the IDE (only just, but still).

-exchn.dll and mingwm10.dll do not like being UPXed. I am surprised it works for you. UPX reliably breaks the versions of these libraries that come with RC2 and is not able to restore them (I tried that several times for this (http://forums.codeblocks.org/index.php/topic,994.msg7304.html#msg7304) thread).
Title: Re: Binary size reduction - UPX
Post by: MHaggag on October 14, 2005, 10:35:42 pm
Quote from: thomas on October 14, 2005, 08:44:49 pm
-exchn.dll and mingwm10.dll do not like being UPXed. I am surprised it works for you. UPX reliably breaks the versions of these libraries that come with RC2 and is not able to restore them (I tried that several times for this (http://forums.codeblocks.org/index.php/topic,994.msg7304.html#msg7304) thread).
Ah, silly me. I've not actually tested it after compressing these 2 files.

Thanks for the reply.
Title: Re: Binary size reduction - UPX
Post by: takeshimiya on October 14, 2005, 11:00:02 pm
You can actually choose less options when compiling wxWidgets dinamically, as the wxWidgets dll will be only used by the Code::Blocks vendor.

There are some things that are know to take a lot of space, namely TIFF support, ODBC support, exceptions, etc.

Also, compiling with other compiler that produces significant less size (Digital Mars, Borland, MSVC6).

See here: http://wiki.wxwidgets.org/wiki.pl?Reducing_Executable_Size
Title: Re: Binary size reduction - UPX
Post by: thomas on October 14, 2005, 11:32:25 pm
Your remark about TIFF and ODBC is truly a good objection. Those are huge beasts, and I guess you can safely assume nobody would miss them in an IDE. What about joysticks and sounds? :lol:
Title: Re: Binary size reduction - UPX
Post by: takeshimiya on October 15, 2005, 05:56:37 am
I'd suggest, for now, setting the following to zero:

#define wxUSE_JOYSTICK 0
#define wxUSE_MEDIACTRL 0
#define wxUSE_QUICKTIME 0
#define wxUSE_DIRECTSHOW 0
#define wxUSE_GSTREAMER 0
#define wxUSE_GLCANVAS 0
#define wxUSE_ODBC 0
#define wxUSE_LIBTIFF 0
#define wxUSE_GIF 0
#define wxUSE_PNM 0
#define wxUSE_PCX 0
#define wxUSE_IFF 0

And, try to make C::B compilable with DMars, MSVC, and other compilers (there is anyone working on this?).
Title: Re: Binary size reduction - UPX
Post by: thomas on October 15, 2005, 02:01:29 pm
Quote from: takeshimiya on October 15, 2005, 05:56:37 am
And, try to make C::B compilable with DMars, MSVC, and other compilers (there is anyone working on this?).

Hmm... I would not recommed that.

The major reason to use DM is its compilation speed, but gcc is on the order of 4 times faster with precompiled headers, too (thanks to tiwag, I finally managed to get these to work).
Compiling Code::Blocks out of the box (clean rebuild) takes just under 7 minutes for me, so assumed precompiled headers were used, that would be less than 2 minutes -- quite acceptable actually. In particular, during development, one typically changes only one or two files at a time (with maybe 4-5 dependencies), so we are talking about 5-6 seconds, which is nothing, really.

Supporting many compilers is a lot of extra work, very painful (not only to implement and to maintain compatibility, but also to follow the actual code), and brings a lot of disadvantages (guess why wxWindows is such a piece of crap - it is because they support so many compilers that don't know C++).
Of course, supporting only one compiler may be somewhat limiting (but name any half-way modern platform that gcc does not support). On the positive side, however, you have a much happier life and can take advantage of all the strengths the compiler offers while gcc will compile it on almost every platform the same way.

Realistically, most people will just use the plain vanilla binary for Windows anyway - let's face the facts.
Linux users in lack of a binary RPM may need to compile from sources, but they will likely use gcc (or the intel compiler, which is said to be compatible now), so that is fine. The same is true for any other platform. There is no MSVC or DM on Solaris, MacOS, or any other OS, so whether you support these or not, people have to use gcc anyway.
And for those few who really want to compile Code::Blocks themselves on Windows, getting a copy of gcc is really not asked too much, I think.

Execution speed may be another issue, as DM is said to produce faster code, but hey, we're talking about an IDE, not about a realtime MRI reconstruction application ;).

My two cents :)
Title: Re: Binary size reduction - UPX
Post by: takeshimiya on October 15, 2005, 02:44:29 pm
Mm. Little off-topic, but I don't think it's good to only support GCC...

You'd blame wxWidgets for being a crap, but the problem is that wx supports compilers that doesn't know what a standard means. But it is not so with newer compilers.
Just support the compilers that are more or less compliant with the standards (I'm not talking about MSVC4 or GCC1.0).


Aside from that, due to the user doesn't having the GCC compiler or other reasons (libraries onlycompileable with borland).
For example, if C::B can't be compiled with OpenWatcom, you'll loose some specific platforms where it can work: Linux hardware (similar to framebuffer), Win32, X11 and OS/2, QNX, SMX and even DOS, through MGL.

As for precompiler headers, there are a lot of people that doesn't know how to use them, or don't want to (for the side effects).
I can estimate that you can compile with DMars, for example, Code::Blocks entirely in 20 seconds or less without precompiler headers.

Anyways, C::B is written in a very cross-compiler way (ie. trying to not use STL), so I think there would be little things here and there to check, for supporting other compilers.


As a bottom note, GCC isn't the universe, it's slow, bloated, makes big executables, the win32 port of the linker have problems with removing dead code, and more important, GCC is not 100% compliant with the standard (althrough there isn't almost any compiler 100% compliant). But, I still like GCC. :)
Title: Re: Binary size reduction - UPX
Post by: thomas on October 15, 2005, 03:56:48 pm
The point I am having about gcc is that it works on every serious platform (except the C64 and the ZX80 maybe ;)).
Most likely, it even works without changing one line of code. This is no longer true if you support other compilers. As a matter of fact, DM does not work for me at all on anything non-trivial. This may be my fault of course, but as a matter of fact, gcc runs everything I feed to it out of the box.

I think development time and energy is best spent where you get the largest benefit for the largest number of people. Trying to get everything right for everybody can break your neck.
Probably 75% of all people will use the standard Windows executable without ever thinking about compiling their own. Most people use Windows, this is a fact, whether we like it or not, and most people do not want to compile the software they use, they just expect it to run off an installer.
Out of the remaining quarter, a few dozen people will actually be developing Code::Blocks, the vast majority will be Linux/x86 users (and a few FreeBSDers). All other users together might make up around 1-2% of the total population (and most of them will actually have gcc available).
Title: Re: Binary size reduction - UPX
Post by: takeshimiya on October 15, 2005, 04:37:20 pm
Well, in reality, the point of making available in compiling with other compilers, is that it's never a good thing to stick to only one compiler, because any compiler have bugs, in any version, or maybe, for example, it can compile without errors something in your code that is not compliant (this especially happens when going from GCC2 to GCC3, etc), and then you'll notice that some of your code wasn't right.

Well, it seems that you only care about windows users, and I must say, that I've saw various posts about 'how I can compile C::B with MSVC...'
You are ignoring the fact that there are people that doesn't like at all GCC (SLOW, bloated, ...), or are more comfortable with other compiler, but wants to help developing plugins, etc.

You would be right in thinking that is not worth the effort, but it's not the case with Code::Blocks, because it's written in cross-compiler way, there are probably very few things to 'fix' to make it compile with let's say MSVC7, DMars, and OpenWatcom.

I would like to see C::B running in, for example, QNX (one of the most widely used real time OS for embedded systems), or in Linux 'framebuffer'. And that is only possible when compiled in OpenWatcom.

About the compilation time, I can assure you, that with DMars you'll be compiling the entire C::B in 20 seconds (on your pc) instead of 7 mins with GCC.

Back to the topic, we got here :D because GCC is one of the worst when it comes to executable size (especially true in win32).

wxWidgets minimal when compiled stripped with GCC tipically weights 3.4 MB, and compiled with, for example MSVC6/2003 weights 0.8 MB. That alone, justifies the little effort in supporting that compiler.
Title: Re: Binary size reduction - UPX
Post by: thomas on October 15, 2005, 07:37:00 pm
Well, I just gave DM (version 8.4.5) another chance, I still think that I'll stick with gcc ;)

Code
Project   : Console application
Compiler  : Digital Mars Compiler (called directly)
Directory : D:\Desktop\Neuer Ordner\
--------------------------------------------------------------------------------
Switching to target: default
Compiling: main.cpp
nbytes = 65520, ph_maxsize = 65520
Internal error: ph 1848
--- errorlevel 1
Process terminated with status 1 (0 minutes, 19 seconds)
0 errors, 0 warnings
Title: Re: Binary size reduction - UPX
Post by: Ceniza on October 16, 2005, 02:50:38 am
Trying to avoid the most off-topic of it...

UPX would only help if you want to reduce the size of your DLLs and EXEs in the installed application. Compressing with UPX to make a installer (which will also compress it) later wouldn't help that much (even though that's what I do with the CVS binary snapshots, but without installer).

I think Code::Blocks' size installed (with all its plugins and resources) not being UPXed isn't that big, even more having in mind the high capacity of HDDs nowdays, so it wouldn't really have an use there.

What I would really like to see is how much could the wxWidgets DLL be reduced disabling those things Code::Blocks won't need for sure, just like takeshimiya pointed.

About making Code::Blocks compile with most compilers (the ones supported by wxWidgets): it would be good and shouldn't be that hard (for recent compilers, of course).

Digital Mars in your case is giving you internal errors. Nothing to do there but wait for Digital Mars developers to solve it.

In fact, I had problems trying to compile wxWidgets as DLL, without debugging information, monolithic, and maybe a few little things more, with Digital Mars. I had to create the custom Makefile (with bakefile), fix a lot of things in there, change the name of output object files ('cause it was being unable to link so many files with those long names or in other words: limitations of command-line length in cmd) and even modify a bit of a wxWidgets header file. Final problem: internal error of the linker.
Title: Re: Binary size reduction - UPX
Post by: MHaggag on October 16, 2005, 07:51:54 am
Quote from: takeshimiya on October 15, 2005, 05:56:37 am
And, try to make C::B compilable with DMars, MSVC, and other compilers (there is anyone working on this?).
Yes, I'm working on compiling it with VC++.NET (7.1). I've got the main application compiled and working so far, and am currently doing the plugins.

Without going into much discussion on the single vs multiple compiler issue, I believe at least Visual C++ should be supported - it's by far the best IDE/Compiler for windows, compiles very fast, produces small executables, and optimizes pretty well. I can understand not wanting to complicate matters or wasting precious development time, but then, this is an open source application. You don't have to work "officially" on keeping it in sync with VS.NET - I will, officially or not :)

In fact, my current VS.NET project setup does not reside in the CodeBlocks folder, but rather in a sibling folder, until it's done and it's decided whether or not it'll make it into the CVS.
Title: Re: Binary size reduction - UPX
Post by: takeshimiya on October 16, 2005, 11:18:46 am
Great! :D
Title: Re: Binary size reduction - UPX
Post by: Ceniza on October 16, 2005, 03:31:08 pm
Sorry, but I cannot agree with Muhammad: The best C/C++ compiler for Windows is Intel C++.

It's a shame it needs the libraries, headers and some programs of Visual C++ to work, but it's The One :)
Title: Re: Binary size reduction - UPX
Post by: takeshimiya on October 16, 2005, 04:42:30 pm
I disagree with both. :lol:

There isn't anything like THE compiler (at least for now). Every compiler have his own pros and cons.
A rough guide of the current most used compilers would be:

MSVC 7
-Binary size: Smallest
-Compilation time: Very good
-Speed of execution: Very good
-Platform: Windows

GCC 3
-Binary size: Biggest
-Compilation time: The worst
-Speed of execution: Very good
-Platform: Everywhere

DMars 8
-Binary size: Good
-Compilation time: The best
-Speed of execution: Average
-Platform: Windows

Intel 8
-Binary size: Big
-Compilation time: Slow
-Speed of execution: The best
-Platform: Windows, Linux

OpenWatcom 1.1
-Binary size: Good
-Compilation time: Average
-Speed of execution: Slowest
-Platform: Windows, OS/2

Borland 5.5
-Binary size: Average
-Compilation time: Very good
-Speed of execution: Very good
-Platform: Windows

Hence, a choice could be: DMars while developing, MSVC for win32 release, GCC for everything else.
Title: Re: Binary size reduction - UPX
Post by: rickg22 on October 17, 2005, 01:16:11 am
Don't forget that Intel compiled executables are slower when run in AMD processors.
http://www.theinquirer.net/?article=24574
Title: Re: Binary size reduction - UPX
Post by: Ceniza on October 17, 2005, 01:48:24 am
Hmmm, that article is related to a problem already fixed in the Intel Fortran compiler.

A few years ago I was helping test some code with the Intel C++ compiler vs GNU G++ and the tests ran in an AMD machine.

In fact the generated code was pretty good and he hadn't any problems running it.

It's true Intel compilers are optimized for Intel processors and technologies (I think the reason is obvious) but compiling with the right flags the executables will run just fine in both Intel and AMD processors.

BTW, there's an article around about GCC vs Intel C++. It measures performance, compilation time and executable size. It shows that both Intel C++ and GCC are pretty good competitors, but it also shows that in some cases Intel C++ generates much better code.

Oh, and we're going off-topic again, don't we?
Title: Re: Binary size reduction - UPX
Post by: takeshimiya on October 17, 2005, 04:04:58 pm
Yep, just to note that there are some hacks to disable the "if it's an AMD CPU then I'll unoptimize the code" check.
You had to #define something, and that's all to disable the lame check.

Anyways, AMD bring out Intel to the court regarding this, so it's possible that new versions of the compiler will not include the lame check.

The funny part, it's that even with those unoptimizations, on an AMD CPU, the compiler is in most benchmarks better than GCC or MSVC (regarding speed of execution).

But I think that the Intel Compiler isn't very helpful for compiling C::B, as it would provide only an advantage regarding speed.

I still think that a good approach is:
DMars while developing for it's amazing compilation time,
MSVC for win32 release for it's amazing binary size, and
GCC for everything else for it's amazing availability everywhere. :)
Title: Re: Binary size reduction - UPX
Post by: Ptomaine on November 14, 2005, 03:57:08 pm
BTW, Apple has chosen the Intel C++ compiler as the default compiler for their MacOSX for the Intel platforms. The release date is late March or so.
So, you have to correct the item like:

Intel 9.x
-Binary size: Big
-Compilation time: Slow
-Speed of execution: The best
-Platform: Windows, Linux, Mac OSX

P.S. The Intel C++ compiler is the only compiler that can produce optimized code for two types of processors and switch to optimized code run-time!