I ran into the same problem and it confused the hell outta me too. The problem is two-fold. 1) The IDE sets up certain flags automatically when a new project is created using the project wizards. Namely, certain debugging, warning, and optimization settings. 2) When you attempt to change the toolset used to build a specific target, C::B attempts to "carry over" those flags (and any others currently specified), even though that usually makes no sense.
You can tell this has happened by going to:
(for C::B 10.05)
Project > Build options... > (select a build target) >
(select a compiler that is NOT the default nor the one used when project was created) >
Compiler settings tab > Other options tab
There you'll find your extraneous flags. This is much more obvious if the default compiler settings have a lot of warning flags (-Wall, -pedantic, etc). You can test it by selecting a bunch of flags and then changing compilers. All the new flags will be listed in the Other options tab of the newly selected compiler.
There is a very simple solution. Separate build targets for each compiler/toolset you want to use. I personally have 6 build targets in my test bed project. A debug and release target using each of mingw, openwatcom, and bcc55. You can add new targets under Project > Properties... > Build targets tab. Unfortunately, you'll have to set up all desired flags manually so you need to know the command-line options of that compiler pretty well, but I prefer that method anyway. To build for a specific target, just select it in the drop-down box of the Build toolbar. Easy. One final note, you can either have all your build targets build to the same (default) directories, or modify them in the Project > Properties... > Build targets tab to point to specific directories.
tl;dr: Simply changing compilers in a build target is a bad way to use multiple compilers in a project. Set up different build targets for each compiler you wish to use.