Developer forums (C::B DEVELOPMENT STRICTLY!) > Development

Parallel builds

<< < (3/4) > >>

thomas:
Have you ever looked at what happens during linking? It's 99% seeking and waiting for IO, and the CPU hardly ever gets to do something. Running several links in parallel is pouring oil into the fire.

oBFusCATed:

--- Quote from: thomas on November 13, 2011, 08:14:00 pm ---A "true parallel" build will likely not be any faster, and possibly be slower due to caching effects, since unrelated files are compiled concurrently and will less likely share the same headers and their object files will be less likely found in caches at link time. It will also be less robust.

--- End quote ---
It will, because linking c++ projects is slow and single core and during linking time, some object files could be complied on the other cores.
And c++ compilation is not IO bound. Someone should play with it and see if there is any benefit.


--- Quote from: Alpha on November 13, 2011, 08:25:31 pm ---There is a situation in which I would find this method parallel compiling to be faster.  When I have tested various modifications to the core of a program (for example, Code::Blocks), many of the extra plugins must be re-linked to the sdk, but not necessarily re-compiled (if they do not include a modified header).  When running on multiple cores, this results in only a single core being used for the majority of the time.

--- End quote ---
I doubt parallel linking will help here. What we should do is remove the external link dependencies in the C::B project, because they are not needed.

Alpha:

--- Quote from: oBFusCATed on November 13, 2011, 08:46:31 pm ---I doubt parallel linking will help here. What we should do is remove the external link dependencies in the C::B project, because they are not needed.

--- End quote ---
You are correct here, however, I had been giving Code::Blocks as an example.  If I am linking with a static library and have only changed the implementation of a function (in the static library), a re-link is required.

IvanBatrakov:
Migrating from Windows to Linux platform needed replacement of Visual Stuido to CodeBlocks.

Visual Stuido builds the same workspace at least 10-20 times faster.

Already made changes to CodeBlocks editor behaivour similar to Visual Studio and even more features to work comfortable and faster.

But without parallel builds migration to CodeBlocks is impossible due to slow down development.

thomas:
Well, feel free to submit a patch. I'm just telling you that I don't believe making it more parallel will make anything faster, except maybe in some very contrieved example cases. But again, feel free to experiment. If you really manage to make my builds faster, I will not complain. :)

Visual Studio compiles a lot faster because MSVC is a much faster (and lower quality) compiler than gcc. This is more true under Windows than under Linux too, because the GNU-style absurd number of process creations is not very efficient. It is not uncommon that for compiling a single source, a chain of 5 or 6 executables is launched. This somewhat less of a problem under Linux, since it is not that painfully devastative, but it is much more disturbing under Windows. Visual Studio only ever loads the compiler once, for the entire workspace.

Also, in particular GNU ld is the most terrible thing on earth, which is why link times often dominate the build process by far.

Though you can use an alternative linker, which is even officially supported via plugins in the mean time, if that is a problem. There exist some which are an order of magnitude faster, too.

On my development systems, compile times under Windows are bound by both process creation and IO. Under Linux, they are bound by IO alone. No matter how many parallel builds I start (the sweet spot is 5 for me), CPU usage never comes even close to 100%.
Linking using ld literally causes millions of seeks on moderately large projects. Solid state and ram drives help, but do not make the problem go away. As such, IO is a dominating factor, and my guess is that adding more seeks does not improve the situation.

Navigation

[0] Message Index

[#] Next page

[*] Previous page