Single PCH file for building the CodeBlocks_wx31_64.cbp

[ollydbg]

Several years ago, I posted some patches to build CodeBlocks_wx30.cbp which only need to use single PCH, also, we don't need to export unnecessary symbols from the codeblocks.dll. Now those patches are manually applied to the CodeBlocks_wx31_64.cbp and related files.

You can find it here: https://github.com/asmwarrior/codeblocks_sf/tree/small_codeblocks_dll

[LETARTARE]

Hello,
what are these fixes?

[ollydbg]

One issue I try to fix is that I don't think it is necessary to export all the symbols  from the codeblocks.dll, only a few of them need to be exported, so this can reduce the size of the codeblocks.dll.

Another issue is: I always see two PCH file generated when building. In-fact, we can use only one PCH file for both building sdk target and other src or plugin targets.

EDIT:
http://forums.codeblocks.org/index.php/topic,20607.msg140408.html#msg140408

This is the old discussion about this issue.

[oBFusCATed]

1. In my opinion this PCH stuff causes more problems than it solves. For me even slows the compilation down, because I edit some of the headers included in it very often. But most of these headers are not used everywhere, but I get a full rebuild anyway. I'll be happy to just remove it. Also full rebuilds are serialized on modern machines, because they are waiting on the 2 pch compilations.
2. Your branch contains other changes.
3. You reuse the DLLIMPORT macro for different libs, don't do it. It will cause problems some day.
4. I'm 25% into the process of removing SqPlus.
5. I'm pretty sure you've broken the --disable-pch autotools build, but I have not tested.

[ollydbg]

1. In my opinion this PCH stuff causes more problems than it solves. For me even slows the compilation down, because I edit some of the headers included in it very often. But most of these headers are not used everywhere, but I get a full rebuild anyway. I'll be happy to just remove it. Also full rebuilds are serialized on modern machines, because they are waiting on the 2 pch compilations.
2. Your branch contains other changes.

My branch has manily two things:
1, PCH related changes, from your point, non-pch build is better, so, we can drop the PCH related changes.
2, some export related changes, I think we don't need to export every symbols from the codeblocks.dll, which makes the codeblocks.dll smaller.

3. You reuse the DLLIMPORT macro for different libs, don't do it. It will cause problems some day.

which commit?

4. I'm 25% into the process of removing SqPlus.

I don't know SqPlus' relatationship with my branch.

5. I'm pretty sure you've broken the --disable-pch autotools build, but I have not tested.

I have only Windows build system, so, I can't test it.

[ollydbg]

About the visibility issue, for example:

- export cbSmartIndentPlugin class from the codeblocks.dll · asmwarrior/codeblocks_sf@8259e0c — https://github.com/asmwarrior/codeblocks_sf/commit/8259e0c0a4271aecda50b3a3f22c2b5ae96a8f9a

I think this change is needed either we enable PCH or disable PCH.

EDIT:

Since we build the codeblocks.dll as a shared library, and this dll include all the tinyxml source code, I add export decorations in tinyxml header files. (Which is suggested in this post: TinyXML / Discussion / Developer: TinyXML shared lib)

[oBFusCATed]

1. Are PCH changes related to the visibility changes? Or in other words: can we fix the PCH without forcing visibility and vice versa?
2. Your branch: has the patch to re-enable the symbol browser and another patch to fix some hidpi issue.
3. The one which adds tinyxml DLLIMPORT to tinyxml. Use prefix marcos for different libs.
4. The major blocker for fixing the visibility was sqplus and its use in plugins. You're changing SqPlus to export stuff, so it is related.
5. You'll either have to setup a vm and test it there or ask someone to do this for you.

[ollydbg]

1. Are PCH changes related to the visibility changes? Or in other words: can we fix the PCH without forcing visibility and vice versa?

NO, PCH issue and visibility issue are independent.

Basically, I think visibility is about to reduce the codeblocks.dll's size, currently when building this dll, we use a "-Wl,--export-all-symbols" option. I think we don't need to use this option, we can only export the symbols we needed, so we need add some "__declspec(dllexport)" decoration to the symbols.


PCH issue is mainly about the building speed, this has lower priority as I think.

[ollydbg]

3. The one which adds tinyxml DLLIMPORT to tinyxml. Use prefix marcos for different libs.

you mean the name "DLLIMPORT" is not good, I may change a another name like "TINYXMLIMPORT"?

4. The major blocker for fixing the visibility was sqplus and its use in plugins. You're changing SqPlus to export stuff, so it is related.

By default, the SqPlus export all the symbols, this is done by the linker option  "-Wl,--export-all-symbols" when building the codeblocks.dll.

Code

 src/include/scripting/include/squirrel.h | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/src/include/scripting/include/squirrel.h b/src/include/scripting/include/squirrel.h
index 9b1b6e9c9..7375b3860 100644
--- a/src/include/scripting/include/squirrel.h
+++ b/src/include/scripting/include/squirrel.h
@@ -36,8 +36,14 @@ extern "C" {
 #endif
 
 #ifndef SQUIRREL_API
-#define SQUIRREL_API extern
-#endif
+#ifdef _WIN32
+    // since we build the squirrel static library, and we need to export all the symbols
+    // the codeblocks.dll, we need to add the dllexport decoration
+    #define SQUIRREL_API extern __declspec(dllexport)
+#else
+    #define SQUIRREL_API extern
+#endif // _WIN32
+#endif // SQUIRREL_API
 
 #if (defined(_WIN64) || defined(_LP64))
 #ifndef _SQ64

What I change here is that those SqPlus symbols are explicitly exported(since I remove the "-Wl,--export-all-symbols"), so the final exported symbols of SqPlus in codeblocks.dll are not changed after my changes.

[stahta01]

WARNING: You must include the PCH file under windows directly!
You can not just include a header that includes the PCH file and expect it to work.

Tim S.

[ollydbg]

WARNING: You must include the PCH file under windows directly!
You can not just include a header that includes the PCH file and expect it to work.

Tim S.

Hi, tim, I know this, and my solution is to explicitly include this pch file in the command line, for example, when build the "src" target, it has such command line

g++.exe -Wall -g -pipe -mthreads -m64 -fmessage-length=0 -fexceptions -Winvalid-pch -std=gnu++11 -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DcbDEBUG -DwxUSE_UNICODE -D_WIN64 -DCB_PRECOMP -include "sdk_precomp.h" -DEXPORT_LIB -DEXPORT_EVENTS -DWXMAKINGDLL_SCI -iquote.objs31_64\include -I.objs31_64\include -I. -IE:\code\wxWidgets-3.1.4\include -IE:\code\wxWidgets-3.1.4\lib\gcc_dll\mswu -Isdk\wxscintilla\include -Iinclude\tinyxml -Iinclude -Iinclude\scripting\include -Iinclude\scripting\sqplus -Iexchndl\win64\include -c D:\code\cb\codeblocks_sf_smalldll\src\src\main.cpp -o .objs31_64\src\main.o

You see, the "-include "sdk_precomp.h" command line option, this header file(and in-fact the .gch PCH file) will always be included in the first place.

The trick here is: you should have the SAME c preprocessor definition set when we build the sdk(the sdk_precomp.h PCH file) and the src/plugins target.

[oBFusCATed]

Trick question: Do you get a build time decrease if you use a pch? How much is it? Is it worth it (for me it isn't)?

[stahta01]

Trick question: Do you get a build time decrease if you use a pch? How much is it? Is it worth it (for me it isn't)?

I just tried adding PCH to an open source small project and I got a speed up doing all the things to speed up building except on using an PCH it actually slowed down the building. (Removing unneeded includes from headers and adding forward declares instead)

NOTE: I do have changes to Code::Blocks that speed up the building using an smaller set of includes inside the two PCH files used by CB.

But, I think that removing the use of PCH by all the contrib plugins is likely much easier than getting and maintaining the CB contrib project to build correctly using PCH.

Tim S.

[ollydbg]

Trick question: Do you get a build time decrease if you use a pch? How much is it? Is it worth it (for me it isn't)?

I just did a test of rebuilding CodeBlocks_wx31_64.cbp. (windows 7 64bit, mingw-w64's gcc 8.1)

I have two working tree, folder A is the master(https://github.com/asmwarrior/codeblocks_sf/tree/master),  folder B is the the single pch branch (https://github.com/asmwarrior/codeblocks_sf/tree/small_codeblocks_dll).

The building time is: (I first rebuild A, then rebuild B)

A: 8min 19sec
B: 4min 45sec

After those steps, I try to do rebuild A again, and the time is 8min 15 sec.

So, my conclusion is: we can save about 50% time. 

[oBFusCATed]

No this is not how you benchmark. Apply all changes in your branch, then try to measure PCH-on versus PCH-off.
Currently you're measuring noPCH+massive amount of symbols and PCH+smaller amount of symbols. The linking step in C::B is serial, so it has big effects on the overall time. Smaller number of symbols improves linking performance.
Also what happens if you just put only wx related includes in the PCH or you use the wxprec.h or whatever is called?

p.s. My build time on Linux, GCC 9.3.0, Amd Ryzen7 1700 (relatively slow CPU in 2020, it is slower than a mobile cpu produced by a fashion/fruit company) is:
1. 93 sec with PCH.
2. 123 sec without PCH.

So, you need to buy better CPU and/or switch to better compiler. GCC on windows is massively slow. So working to switch to something like clang would be a big improvement in both speed and compiler quality (GCC on windows is pretty bad compared to how it behaves on Linux). I've not use clang on windows, but I doubt it would be as bad as GCC is. On linux clang and GCC are comparable.

And I pay 60 seconds or more every time I edit a header which is in the PCH, super annoying.

p.p.s. The compilation is really slow if the compiler emits too many errors/warnings. If you're not error/warning free you'll get horrendous build times. This is something which needs to be investigated quite a bit.
p.p.p.s. The compilation is non-optimal (and it will get worse on newer non-fruit cpus), because it cannot schedule work from multiple projects. Linking is blocking progress . This is clearly visible when building any plugins. Someday I'll write a ninja-build generator, but there are more important projects/problems first.
p.p.p.p.s. The switch from wx28 to wx30 made compilation visibly slower. I've not debugged why, but my guess is the use of basic_string as the base for wxString.