Author Topic: custom makefile support mutliple cpus  (Read 10098 times)

Offline mattn2

  • Multiple posting newcomer
  • *
  • Posts: 18
custom makefile support mutliple cpus
« on: October 14, 2012, 08:57:06 pm »
Hi,

we are looking for a way to get the amount of cpus in a codeblocks variable that can be used in the custom makefile parameters

something like:

$make -r -R -j $cpus -f $makefile $target

is something like this available?

thanks a lot
martin

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: custom makefile support mutliple cpus
« Reply #1 on: October 14, 2012, 09:36:49 pm »
I think per project variables is what you are looking for.

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: custom makefile support mutliple cpus
« Reply #2 on: October 14, 2012, 10:52:40 pm »
Hi,

we are looking for a way to get the amount of cpus in a codeblocks variable that can be used in the custom makefile parameters

something like:

$make -r -R -j $cpus -f $makefile $target

is something like this available?

thanks a lot
martin

On linux you should be able to use something like (note the backticks):
Quote
$make -r -R -j `getconf _NPROCESSORS_ONLN` -f $makefile $target

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: custom makefile support mutliple cpus
« Reply #3 on: October 15, 2012, 12:25:58 am »
Isn't it a bit odd that a compiler that supports multiple CPUs doesn't have a switch to detect the number of CPUs?

Offline mattn2

  • Multiple posting newcomer
  • *
  • Posts: 18
Re: custom makefile support mutliple cpus
« Reply #4 on: October 15, 2012, 12:54:57 pm »
it's not the compiler, but make

on windows i could also use the env var %NUMBER_OF_PROCESSORS% - but i'm looking for a portable way to handle this.

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: custom makefile support mutliple cpus
« Reply #5 on: October 15, 2012, 01:39:48 pm »
You could try something like:
Quote
$make -r -R -j $if(PLATFORM == PLATFORM_MSW){%NUMBER_OF_PROCESSORS%}{`getconf _NPROCESSORS_ONLN`} -f $makefile $target
the linux part should work, but I'm not sure about the windows part, you might need someting similar to linux' "echo".

Offline mattn2

  • Multiple posting newcomer
  • *
  • Posts: 18
Re: custom makefile support mutliple cpus
« Reply #6 on: October 25, 2012, 10:09:38 pm »
thanks - but there is also mac - which does this again a little bit different. and c::b must already know the amount of cpus, no?

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: custom makefile support mutliple cpus
« Reply #7 on: October 25, 2012, 10:29:42 pm »
wxWidgets has a static function to determine the umber of cpu's, but I don't know how good it works.
But it could surely be used to add a cpu_count macro, that can be used wherever macro-/variable-expansion is done.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: custom makefile support mutliple cpus
« Reply #8 on: October 25, 2012, 10:34:10 pm »
wxWidgets has a static function to determine the umber of cpu's, but I don't know how good it works.
But it could surely be used to add a cpu_count macro, that can be used wherever macro-/variable-expansion is done.
We have also a setting for the number of cpus used for building, this could be provided here.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: custom makefile support mutliple cpus
« Reply #9 on: October 25, 2012, 11:06:20 pm »
wxWidgets has a static function to determine the umber of cpu's, but I don't know how good it works.
But it could surely be used to add a cpu_count macro, that can be used wherever macro-/variable-expansion is done.
We have also a setting for the number of cpus used for building, this could be provided here.
Both could be provided:
the number of CPU's determined by wxw and the number of parallel processes for building set by the user in the compiler options dialog.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: custom makefile support mutliple cpus
« Reply #10 on: October 26, 2012, 06:17:54 am »
wxWidgets has a static function to determine the umber of cpu's, but I don't know how good it works.
It works pretty well - I am using it in another of my projects (You meant the static wxThread based method, right?)
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: custom makefile support mutliple cpus
« Reply #11 on: October 26, 2012, 09:36:22 am »
wxWidgets has a static function to determine the umber of cpu's, but I don't know how good it works.
It works pretty well - I am using it in another of my projects (You meant the static wxThread based method, right?)
Yes

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: custom makefile support mutliple cpus
« Reply #12 on: October 27, 2012, 06:16:39 pm »
Yes
Reminds me: Recently had the chance to work on a 16 core "enterprise PC". What a mind-blowing fast compilation of everything you do there with C::B, having setup 16 parallel build processes. Also, the wxThread methods worked fine there, too. Now all I need is somebody giving me enough money because I want to have such a beast, too. :-)
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: custom makefile support mutliple cpus
« Reply #13 on: October 27, 2012, 06:47:00 pm »
Yes
Reminds me: Recently had the chance to work on a 16 core "enterprise PC". What a mind-blowing fast compilation of everything you do there with C::B, having setup 16 parallel build processes. Also, the wxThread methods worked fine there, too. Now all I need is somebody giving me enough money because I want to have such a beast, too. :-)
That's more or least what I need too, with at least 32 GB of Ram.

An alternative would be a quadcore laptop with 32 GB installable Ram.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: custom makefile support mutliple cpus
« Reply #14 on: October 29, 2012, 07:53:22 pm »
Moar cores, moar...

Jen: why would you need so much ram? ;D
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: custom makefile support mutliple cpus
« Reply #15 on: October 29, 2012, 08:30:40 pm »
Jen: why would you need so much ram? ;D
Did you even compile on a RAM drive? If not - now you know, why... ;D

BTW: The computer I was on had 128GB of RAM.

(Maybe we should either split the topic or just shut up... ;-))
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: custom makefile support mutliple cpus
« Reply #16 on: October 29, 2012, 08:56:08 pm »
Jens: why would you need so much ram? ;D

I have several virtual machines to test and compile and they benefit more from ram than from cpu's.
More cores are of course much better, but I need a laptop to be more flexible/mobile.
And laptops with up to 32 GB and a quadcore are common and not too expensive.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: custom makefile support mutliple cpus
« Reply #17 on: October 29, 2012, 09:34:48 pm »
Did you even compile on a RAM drive? If not - now you know, why... ;D
No, what is the difference in comparison to a desktop (7200rpm produced in the last 2-3 years) hard disk?
I would have upgraded to a ssd, but looking at the reports on the internet, there is no big difference, that's why I've not upgraded.
As far as I can tell disk performance will be important only to the linking phase.

(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]