Code::Blocks Forums

User forums => Using Code::Blocks => Topic started by: alexchen on July 08, 2020, 02:48:40 am

Title: Cannot increase the number of parallel build processes beyond 16
Post by: alexchen on July 08, 2020, 02:48:40 am

I am using the latest Code::Blocks 20.  I try to increase the number of parallel builds to 32 but it can only go up to 16. 
It will no go up even if I change the setting <PARALLEL_PROCESSES int="32" /> in $HOME/.config/codeblocks/default.conf and restart Code::Blocks. 
What is the factor that caps this value?

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: BlueHazzard on July 08, 2020, 08:15:06 am

I think it is locked to the maximum count of (virtual) cpus your pc has, because it makes no sense to increase the number past this number

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: alexchen on July 09, 2020, 08:04:45 pm

I don't think that's the case.  My CentOS VM starts with 4 CPU on a 12 CPU MacPro (the cylindrical model) and I can have set up to 16 parallel builds.  Even if I bump the VM CPU to 8 and eventually to 12 CPU with 32 GB memory and 128 GB HD.
But  CodeBlocks still max out at 16 parallel builds.

I assume Parallel builds is at thread level to launch multiple compiler processes. 
Number of CPU can affect the performance but should not be the limiting factor.

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: sodev on July 09, 2020, 09:05:05 pm

I think that value is hardcoded and its maximum got increased in the past, sadly i can't find the commit. Im my build (r12183) i can increase the number to 65K.

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: alexchen on July 09, 2020, 09:08:01 pm

That's what I suspect.  Maybe CodeBlocks cannot handle more than 16 concurrent builds due to some architectural limitation or unresolved issues?

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: Miguel Gimenez on July 09, 2020, 09:17:19 pm

Revision 10822:

https://sourceforge.net/p/codeblocks/code/10822/

EDIT: if you select 0 then std::max(1, wxThread::GetCPUCount()) is used.

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: alexchen on July 09, 2020, 09:52:17 pm

Currently I can set up to 16 parallel builds on a VM with 12 CPU.  If I set to 0 according the link mentioned, wouldn't it mean that I can have only 12 parallel builds? That is even worse.

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: oBFusCATed on July 10, 2020, 12:16:40 am

But why do you need it more than 12 in the first place? What do you think it will happen if you set it to 32 or 64 or some other high number? (I know the limitation is not really good idea)

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: alexchen on July 10, 2020, 12:20:19 am

I do not need to bump it up that much really.  I just want to know whether it is a hard limit regardless how powerful your machine is or some other issues.  When I do a big project build wit Code::Blocks command line tool in a batch build, it can sometime take several hours to finish.

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: oBFusCATed on July 10, 2020, 12:29:14 am

Actually the limit has been removed in r10822.
Are you sure you have a clean install of 20.03?
On my machine I can set it to 32 no problem.

Most of the time this limit is not what is limiting scalability/parallelism in cb projects/workspaces.
The main problem is that targets/projects cannot be run in parallel.
So if you have many small targets they are run in serial almost mode with very little use of CPU resources.

Currently the only solution is to switch to cmake+ninja or something similar.

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: alexchen on July 10, 2020, 12:34:38 am

This is my version.  I downloaded it from CB's download site directly

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: oBFusCATed on July 10, 2020, 01:55:27 am

Obviously you have to install the latest and greatest to get this feature.

Title: Re: Cannot increase the number of parallel build processes beyond 16
Post by: sodev on July 10, 2020, 07:02:39 am

So you said you are using CodeBlocks 20 which i assumed is 20.03, but this version was build 2016 and is the release from 2013... scary things happening here :o ::)