Author Topic: ccache to speed up builds  (Read 22755 times)

Offline Smirftsch

  • Multiple posting newcomer
  • *
  • Posts: 22
ccache to speed up builds
« on: March 28, 2010, 12:25:33 am »
I digged a while and tried some settings, but I can't seem to be able to add ccache in CodeBlocks to speed up my builds. Where do I have to add it? Probably I'm just blind...:)

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13439
    • Travis build status
Re: ccache to speed up builds
« Reply #1 on: March 28, 2010, 12:42:50 am »
Probably the easiest way is to make a bash script and replace your toolchain executable with the script ( settings -> compiler & debugger -> toolchain executable).

Create a file ccache_gcc.sh and fill it with:
Code
#!/bin/bash
ccache gcc "$@"

The in the field for C compiler type ccache_gcc.sh, do the same for C++...
Also you should change your "Compiler's installation dir and do some symbolic linking for the non ccached tools.

p.s. I've not tested the script, so it is possible that it doesn't work (I'm not a ccache user)
p.p.s. you could try to use "ccache gcc" directly, but I doubt it will work
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline Smirftsch

  • Multiple posting newcomer
  • *
  • Posts: 22
Re: ccache to speed up builds
« Reply #2 on: March 28, 2010, 09:17:40 am »
No, ccache gcc doesn't work. Tried that already ;) - but I'm surprised that no one here seems to use it,even a search in the forums showed nothing really related. While it is some really important thing in my eyes, especially when comparing against vc++ it really takes way longer to recompile my project in C::B without it.

Maybe most people are just not aware about it?
http://ccache.samba.org/performance.html
just for information.

I really suggest to add such an option to use it in C::B maybe as checkbox. If testing minor changes in big projects it can safe hours of work.

PS: Thx for the idea with the script, seems to work fine ;)

Update: while it executes ccache now it doesn't seem to do its work. Maybe some path issue, so its not working yet...
« Last Edit: March 28, 2010, 12:55:25 pm by Smirftsch »

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #3 on: April 03, 2010, 01:42:30 am »
#!/bin/sh
ccache gcc "$@"

works fine here :) need to tell the program youre compiling that it needs to use the wrapper.

i.e
export CC="ccache_gcc"


Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #4 on: April 03, 2010, 05:14:34 am »
forgot to mention dont call it ccache_gcc.sh just ccache_gcc .sh files call bash directly while scripts with #!/bin/bash
are actually handled like real executables

Offline frithjofh

  • Regular
  • ***
  • Posts: 376
Re: ccache to speed up builds
« Reply #5 on: April 03, 2010, 07:55:31 pm »
hi,

I just applied the changes to my settings and the compilation time of my rather small project went from 42 sec the first time down to 10 sec the second and every other time. great. I hope I am not mistaken, as there is no output whatsoever if or if not ccache really is taking part in the process, but from the times measured, it should be.

as I am on linux, I added a custom variable under "project build options"->"custom variables"  : CC=ccache_gcc . the script I put in /usr/bin

thanks

greetings from asturias

frithjofh
architect with some spare time  -  c::b compiled from last svn  -   openSuSE leap x86_64  -  AMD FX-4100

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #6 on: April 05, 2010, 10:07:51 am »
happy it worked out :)

aye ccache can really speed things up  :D

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5403
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ccache to speed up builds
« Reply #7 on: May 11, 2010, 02:00:57 am »
I'm interest in using it, but I found there is no simple instruction for Win32. No binaries can be found for Windows system. :(
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #8 on: May 11, 2010, 03:30:48 am »
on the contrary :) ftp://90.184.233.166:21/ccache.exe

mingw win32 build.

the wrapper scripts above cannot however be used on windows unless you have the msys shell in path (baaad idea) and codeblocks bitches loudly if i use bat files as wrappers  :? so have to define CC="ccache gcc" etc directly in the makefiles.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5403
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ccache to speed up builds
« Reply #9 on: May 11, 2010, 03:39:39 am »
on the contrary :) ftp://90.184.233.166:21/ccache.exe

mingw win32 build.


Thanks, I also get a response to the author of ccache, who gives me another link in : http://ramiro.arrozcru.org/ ccache-win32-1.exe

Quote
the wrapper scripts above cannot however be used on windows unless you have the msys shell in path (baaad idea) and codeblocks bitches loudly if i use bat files as wrappers  :? so have to define CC="ccache gcc" etc directly in the makefiles.
But I'm not fully understand these sentence, seems we can only use ccache in MSYS shell. So, I can't directly used it in CodeBlocks IDE?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #10 on: May 11, 2010, 03:59:26 am »
unfortunatly not atleast untill some adds the option to codeblocks :/ using a batch file as a wrapper causes codeblocks to spit out invalid compiler [YOUR ANSWER IS ALREADY THERE. SEARCH THE FORUMS!] errors but i suspect you could do it from inside the project properties by defining CC=ccache gcc to the preprocessor variables. same goes for g++ ;)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5403
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ccache to speed up builds
« Reply #11 on: May 11, 2010, 04:53:18 am »
Sorry, I'm still confused.
As you said:
Quote
i suspect you could do it from inside the project properties by defining CC=ccache gcc to the preprocessor variables. same goes for g++

So, what's the exact steps I should do?

1, I just copy ccache.exe to the mingw/bin
2, I create a console project by Codeblocks' project wizard.
3, then, what should I do? Do I need to do some hack on the project build options?? or other modifications on project properties??? I have just add a Custom variables : CC = "ccache gcc", but it does Noting...
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #12 on: May 11, 2010, 03:36:12 pm »
pretty much ;) its not visible but try a larger project and notice the compile times if it works it should compile quite a lot faster :)

a way to check if its working is to show hidden files in explorer then navigate to user\your name\AppData\Roaming And look for a folder named .ccache if its there it works :).

a god project to try it out on could be codeblocks itself because of its size you should notice some better compile times. codeblocks is c++ though so the variable would be CXX="ccache g++".

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5403
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ccache to speed up builds
« Reply #13 on: May 11, 2010, 04:54:01 pm »
I just change the Menu-settings-compiler and debugger-Tool chain executes -
I change the
C++ compiler from
Quote
mingw32-g++.exe
to
Quote
ccache.exe mingw32-g++.exe

But still can't get things to work.

With the ccache.exe from your ftp site ftp://90.184.233.166:21/ccache.exe
It works!!

Here is the build log:
Quote
-------------- Build: Debug in test_cc_cache ---------------

[ 50.0%] ccache mingw32-g++.exe -Wall -fexceptions  -g     -c C:\codeblocks_test_code\test_cc_cache\main.cpp -o obj\Debug\main.o
[100.0%] mingw32-g++.exe  -o bin\Debug\test_cc_cache.exe obj\Debug\main.o   
Output size is 1.01 MB
Process terminated with status 0 (0 minutes, 4 seconds)
0 errors, 0 warnings



with the ccache.exe from another link: http://ramiro.arrozcru.org/ ccache-win32-1.exe

I get the error log below:
Quote
-------------- Build: Debug in test_cc_cache ---------------

[ 50.0%] ccache.exe mingw32-g++.exe -Wall -fexceptions  -g     -c C:\codeblocks_test_code\test_cc_cache\main.cpp -o obj\Debug\main.o
ccache: FATAL: Unable to determine cache directory
Process terminated with status 1 (0 minutes, 0 seconds)
1 errors, 0 warnings
 

 :(



 
« Last Edit: May 11, 2010, 05:13:03 pm by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #14 on: May 11, 2010, 08:54:46 pm »
yep thats the error i got if i use it in the compiler properties.

just made a test and it works ok if done like in the image.



Uploaded with ImageShack.us

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #15 on: May 11, 2010, 08:58:45 pm »
oups didnt notice that it indeed worked  :lol:

aye my ccache version had a few patches to fix some broken behaviour :)

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #16 on: May 23, 2010, 12:30:40 am »
just made a test and it works ok if done like in the image.

Could you please clarify? The *only* change you made (besides copying ccache.exe to the mingw bin directory) was to set that preprocessor variable and nothing else? You left the toolchain properties untouched, is that it?

Ken

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #17 on: May 23, 2010, 12:22:45 pm »
i believe so as that is what i do when i use it :)

but i may be wrong.

problem on my side is i cant set it in the toolchain or codeblocks will throw an invalid compiler [YOUR ANSWER IS ALREADY THERE. SEARCH THE FORUMS!] selected at me :)

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #18 on: May 23, 2010, 06:36:53 pm »
i believe so as that is what i do when i use it :)

but i may be wrong.

problem on my side is i cant set it in the toolchain or codeblocks will throw an invalid compiler [YOUR ANSWER IS ALREADY THERE. SEARCH THE FORUMS!] selected at me :)

I had the same problem when I tried to set the toolchain compilers, but I can't tell if just setting the var is making any difference. Under Linux I had both to use a script *and* set an environment variable (but not where you set it, It's on the "Custom variables" tab which is at the same level as the "toolchain executables" tab, just to the right of it). Under Linux it clearly works.

Windows. Meh.

Ken

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #19 on: May 23, 2010, 06:43:21 pm »
I had the same problem when I tried to set the toolchain compilers, but I can't tell if just setting the var is making any difference.

I set the "show full command line" option on, and I can confirm that ccache is *not* being called.

Any suggestions, anyone?

Ken

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7254
Re: ccache to speed up builds
« Reply #20 on: May 23, 2010, 07:07:32 pm »
Using a script on linux works, if you either put it in the same place the real compiler resides (usually /usr/bin, needs root-rights) or add the directory with the script to the Additional paths in the Toolchain executables-tab.

for gcc I use ccache-gcc with the following content:
Code
#!/bin/sh
ccache gcc $@

and for g++ I called my script ccache-g++:
Code
#!/bin/sh
ccache g++ $@


No environment-variables needed.
don't forget to make the scripts executable (
Code
chmod +x <scriptname>
[/b])

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #21 on: May 23, 2010, 10:42:44 pm »
one way to check if its working with the CC=ccache gcc variable on windows is looking in c:\username\appdata\roaming if theres a .ccache folder there with a lot of odly named files in it then it works ;)

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #22 on: May 23, 2010, 11:20:42 pm »
Using a script on linux works, if you either put it in the same place the real compiler resides (usually /usr/bin, needs root-rights) or add the directory with the script to the Additional paths in the Toolchain executables-tab.

Hi Jens,

I got it working under Linux, the problem is with windows...

Thanks anyway,
Ken

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #23 on: May 23, 2010, 11:40:22 pm »
one way to check if its working with the CC=ccache gcc variable on windows is looking in c:\username\appdata\roaming if theres a .ccache folder there with a lot of odly named files in it then it works ;)

There is only one file in that folder, called "stats".

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7254
Re: ccache to speed up builds
« Reply #24 on: May 24, 2010, 12:05:07 am »
ccache-gcc.bat:
Code
@echo off
ccache "gcc %*"

ccache-g++.bat:
Code
@echo off
ccache "g++ %*"

Put them into the system-path or MinGW's bin-folder should work.
I used the "Additional paths" in "Toolchain executables"-tab.
Tested on my virtual(kvm/qemu)-vista-system.

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #25 on: May 24, 2010, 12:21:10 am »
ccache-gcc.bat:
Code
@echo off
ccache "gcc %*"

ccache-g++.bat:
Code
@echo off
ccache "g++ %*"

Put them into the system-path or MinGW's bin-folder should work.
I used the "Additional paths" in "Toolchain executables"-tab.
Tested on my virtual(kvm/qemu)-vista-system.


It works here - I used mingw32-gcc and mingw32-g++, I don't know if this makes any difference.

Thanks once again!

Ken

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #26 on: May 24, 2010, 12:26:27 am »
Now, is there anything that can speed up linking as well?

Ken

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #27 on: May 24, 2010, 01:46:42 am »
nothing usable from within codeblocks i believe :(

theres a tool from thor lilquist he uses for gtk building but it relies heavily on bash and is more of a replacement for libtool.

ld isnt that slow to be frank since its a real program i dont find linking times to be any scary with gcc :) not even for huge libraries like wxwidgets monolithic.

and libtool can make a grown man cry on windows  :lol:

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13439
    • Travis build status
Re: ccache to speed up builds
« Reply #28 on: May 24, 2010, 02:04:54 am »
ld isnt that slow to be frank since its a real program i dont find linking times to be any scary with gcc :) not even for huge libraries like wxwidgets monolithic.
Try to code using boost::bind and boost::function.... my app was 10k LOC and the linking was slow.

For linux there is the gold linker that is said to be faster than ld, but it is linux only. http://en.wikipedia.org/wiki/Gold_%28linker%29
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #29 on: May 24, 2010, 02:21:33 am »

ld isnt that slow to be frank since its a real program i dont find linking times to be any scary with gcc :) not even for huge libraries like wxwidgets monolithic.

and libtool can make a grown man cry on windows  :lol:

Hehe. :)

You do have a point about the linking times not being that large, but anything that you can shave off during a heavy debug-rewrite-recompile-debug session is welcome...

Ken

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #30 on: May 24, 2010, 02:27:30 am »

For linux there is the gold linker that is said to be faster than ld, but it is linux only. http://en.wikipedia.org/wiki/Gold_%28linker%29


The upp guys made some modifications to ld that seem to speed it up, but I don't know if it can be used disassociated from their IDE. That article on gold mentions an interesting direction, incremental linking. That would be a welcome addition to the GNU toolchain... (ld has a -i option, but I am not sure whether it works the same way that incremental linking works in Borland or Microsoft compilers under windows).

Ken

Offline reckless

  • Regular
  • ***
  • Posts: 338
Re: ccache to speed up builds
« Reply #31 on: May 24, 2010, 02:28:27 am »
true ;)

and sounds interresting gonna have a look :)

Offline kencamargo

  • Multiple posting newcomer
  • *
  • Posts: 110
Re: ccache to speed up builds
« Reply #32 on: May 24, 2010, 03:18:27 am »
true ;)

and sounds interresting gonna have a look :)

Keep us posted!