Author Topic: Problem creating same output as VS 2005  (Read 7545 times)

Offline yamokosk

  • Single posting newcomer
  • *
  • Posts: 7
Problem creating same output as VS 2005
« on: June 21, 2007, 06:21:37 pm »
I am sure you devs already hear this enough but.. wow, CB is amazing. I finally gave up on VS 2005 after being annoyed to no end by countless hang ups and a very, very sluggish intellijoke system. Great work guys! Keep it up.

However, I have run into a little bit of a snag. I have been using VS 2005 to compile Matlab MEX files (really just a DLL with a special exported function - mexFunction). But I can't seem to get CB to properly build these same MEX projects. To help debug this problem, I created a simple "Hello World" MEX project in Visual Studio. When I compile the library and then dumpbin the contents I get:

Quote
Microsoft (R) COFF/PE Dumper Version 8.00.50727.762
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file SimpleMexVS.mexw32

File Type: DLL

  Section contains the following exports for SimpleMex.dll

    00000000 characteristics
    467A8F0C time date stamp Thu Jun 21 10:45:32 2007
        0.00 version
           1 ordinal base
           1 number of functions
           1 number of names

    ordinal hint RVA      name

          1    0 0001101E mexFunction = @ILT+25(_mexFunction)

  Summary

        1000 .data
        1000 .idata
        2000 .rdata
        1000 .reloc
        1000 .rsrc
        4000 .text
       10000 .textbss

Then in CB (nightly build, Jun 18, 2007) I have created an identical project (DLL target, using VS 2005 compiler, same source code, and meticulously copied every option from VS to CB). But when I compile in CB then dumpbin the contents of the library I get something different:

Quote
Microsoft (R) COFF/PE Dumper Version 8.00.50727.762
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file SimpleMexCB.mexw32

File Type: DLL

  Section contains the following exports for SimpleMex.dll

    00000000 characteristics
    467AA204 time date stamp Thu Jun 21 12:06:28 2007
        0.00 version
           1 ordinal base
           1 number of functions
           1 number of names

    ordinal hint RVA      name

          1    0 00001005 mexFunction = @ILT+0(_mexFunction)

  Summary

        1000 .data
        1000 .idata
        2000 .rdata
        1000 .reloc
        1000 .rsrc
        4000 .text

This difference is important because when I try to load each respective function in MATLAB I get an error from the CB compiled version. Anyone have any ideas where to proceed. As I said, I have double checked that all compile/linker options, preprocessor symbols, etc are the same between the VS and CB projects. Are there any other debugging ideas anyone can think of?

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4291
    • Code::Blocks IDE
Re: Problem creating same output as VS 2005
« Reply #1 on: June 21, 2007, 09:48:55 pm »
Morten us building matlab dlls with C::B but he's not using the VC compiler. He's using LCC or GCC.
Maybe he has something to share...
Be patient!
This bug will be fixed soon...

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9493
Re: Problem creating same output as VS 2005
« Reply #2 on: June 21, 2007, 09:55:51 pm »
Maybe he has something to share...
I have. ;-)
Well - I can't really help you with VC2005 as I don't have it. But for this case I think you are still missing some compiler flags. Have a look at the mex batch file that is used (setup) for your compiler... (I assume you know the interna of the mex build process). This was my starting point, too when it came to what compiler/linker flags to use. In addition it could be that you are linking against the wron MSVC lib - remember that Matlab ships with several one, depending on the VC compiler you are using. Hopefully you didn-'t forget to link against the required Matlab libs, too... but this should throw a linker error otherwise.

I am using the Matlab built-in LCC (I have a special version of C::B for that - including support for that compiler) or the MinGW/GCC. The latter is not easy to configure, but it is possible. I can guide you through it if you like. If you need the VC compiler (I can imagine several reason why this could be the case) try to compile with another version in addition - VC6, if this is available to you. Search the C::B WiKi for information how to setup this compiler in C::B. I only have that compiler so I could help you better then... maybe... ;-)

With regards, Morten.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline yamokosk

  • Single posting newcomer
  • *
  • Posts: 7
Re: Problem creating same output as VS 2005
« Reply #3 on: June 21, 2007, 10:39:54 pm »
Morten-

Good point about checking the Matlab options file for my compiler. There was one or two options listed in it that neither VS nor C::B contained. But adding them to C::B did not remedy the situation.

Here is what I don't understand. C::B is merely an IDE. Under the covers, it is doing everything VS is doing.. calling cl.exe + my options, calling link.exe + my options, etc. And if I can verify that "my options" are the same between both programs (which I am guessing they are not), then they should both produce identical output.

Is there any way to capture the command terminal steps of both programs? So I can see what is actually being passed to cl.exe by VS and cl.exe by C::B?

(btw, if anyone was wondering I did track down one of the differences in the dumpbin's between the two compiled libraries. From MS documentation, "Section .textbss is only required by Microsoft Visual C++ 7.0 and higher if incremental linking is used." I verified that /INCREMENTAL is a linker option in both C::B and VS... But the C::B dumpbin does not contain a .textbss section. Is the /INCREMENTAL option not getting passed to link.exe in C::B?)
« Last Edit: June 21, 2007, 10:47:55 pm by yamokosk »

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4291
    • Code::Blocks IDE
Re: Problem creating same output as VS 2005
« Reply #4 on: June 21, 2007, 10:45:53 pm »
Quote
Is there any way to capture the command terminal steps of both programs? So I can see what is actually being passed to cl.exe by VS and cl.exe by C::B?

I can't help you with VS (as I don't have it), but for C::B do this:
Go to "Settings->Compiler and debugger", select your compiler from the drop-down list at the top and then click on the "Other" page. There, select the logging option and set it to "Full logging". This will print the full build commands, as executed.
Be patient!
This bug will be fixed soon...

Offline TDragon

  • Lives here!
  • ****
  • Posts: 936
    • Twilight Dragon Media
Re: Problem creating same output as VS 2005
« Reply #5 on: June 21, 2007, 10:49:40 pm »
Is there any way to capture the command terminal steps of both programs? So I can see what is actually being passed to cl.exe by VS and cl.exe by C::B?
VS2005 generally creates a build log in the intermediate files directory called BuildLog.htm. In Code::Blocks, you can see the full command line in the Build Log window by changing the Compiler logging option in Settings->Compiler and debugger->Global compiler settings->Other settings to "Full command line".

Edit: Beatz0r'ed. :P
http://tdm-gcc.tdragon.net/ - TDM-GCC compiler suite for Windows (GCC 5.1.0 2015-06-28, 32/64-bit, no extra DLLs)

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9493
Re: Problem creating same output as VS 2005
« Reply #6 on: June 21, 2007, 10:50:03 pm »
Is there any way to capture the command terminal steps of both programs? So I can see what is actually being passed to cl.exe by VS and cl.exe by C::B?
In Matlab there is a verbose switch (or similar) for the compiler... I don't recall exactly - but I can have a look when I'm at work if you like. For C::B have a look at my sig. It will tell you what to do... ;-)

And yes: If the command line produced by C::B is the same and the one from Matlab AND the path settings are correct (e.g. consider environment variables, too maybe) then it *will* produce identical output (besides date stamps maybe). For sure.

With regards, Morten.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9493
Re: Problem creating same output as VS 2005
« Reply #7 on: June 21, 2007, 10:51:07 pm »
VS2005 generally creates a build log in the intermediate files directory called BuildLog.htm.
I believe he isn't compiling in VC2005 but in Matlab - but even there *could* be created a build log... if it is setup (which is the default IMHO).
With regards, Morten.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline yamokosk

  • Single posting newcomer
  • *
  • Posts: 7
Re: Problem creating same output as VS 2005
« Reply #8 on: June 21, 2007, 10:59:42 pm »
I am moving from VS to C::B.. well to be more specific it was more like Matlab > VS > C::B. Just to review, I can compile and run mex files from VS with no problems. Its only when I try and do the same thing in C::B that I am running into trouble.

At any rate, I found that build log produced by VS (I wish it was more of an integrated option like it is for C::B..). But at any rate, I will pour over the two logs to see whats missing or not getting included.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9493
Re: Problem creating same output as VS 2005
« Reply #9 on: June 21, 2007, 11:17:44 pm »
Just to review, I can compile and run mex files from VS with no problems.
Oh! Ok... this was my misunderstanding. I though you only use the compiler of VS2005 but compile inside Matlab. (I always did that.) Ok then... probably in this case the batch file from Matlab is incorrect. Make sure you do a "mex -setup" before (on the Matlab prompt) and configure Matlab appropriately for the VC compiler. But you can also parse the log... this should contain the most important information, too.
With regards, Morten.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline yamokosk

  • Single posting newcomer
  • *
  • Posts: 7
Re: Problem creating same output as VS 2005
« Reply #10 on: June 22, 2007, 01:11:23 am »
Ok, I did a lot of testing and discovered two things: one possible bug in C::B and the other is a major difference in the tool paths between C::B and VS.

First the possible bug. After thoroughly scouring the build logs of both VS and C::B, I found that the link option "/DEBUG" was present in VS but not in C::B. To try and fix this, I went to C::B->Build Options->Link Settings->Other linker options and added /DEBUG. But whenever I hit "Ok" and recompiled, the "/DEBUG" option did not show up. In fact when I went back to the build options page, what I had entered disappeared?! Maybe this has already been reported? Anways, as a quick fix, I went to Settings->Compiler and debugger...->Other settings->Advanced options... and essentially hardcoded "/DEBUG" into the "Link object files to dynamic library" command. After the fix, what do you know.. The dumpbin for both the C::B compiled library and the C::B compiled mex file were identical!

But that still did not completely fix my problems. I am not a C++/Windows/VS guru but what is a manifest? This seems to be where the C::B tool path differs significantly from VS. More specifically, when I instruct VS to not "Embed the Manifest", I get the same bad mex file produced by C::B (aka Matlab gives the same error message for both mex files). But when I embed the manifest in VS, I get a working mex file.

Also one other observation. VS allows options to be specified for the resource compiler tool. Is this functionality in the works for a future release of C::B? What about this manifest thing?

Offline Biplab

  • Developer
  • Lives here!
  • *****
  • Posts: 1874
    • Biplab's Blog
Re: Problem creating same output as VS 2005
« Reply #11 on: June 22, 2007, 01:56:27 pm »
The problem is wizard doesn't setup a Post-build Job to embed manifests. I believe this is the root cause.

MSVC 8 (MSVC 2005) requires a special manifest file (An XML file) to load debug CRT. As you're compiling DLL in debug mode, it needs that manifest file either in embedded form or in the same directory where your DLL resides.

I'm updating the wizard so that a post-build job gets executed to embed the manifest file.

First the possible bug. After thoroughly scouring the build logs of both VS and C::B, I found that the link option "/DEBUG" was present in VS but not in C::B. To try and fix this, I went to C::B->Build Options->Link Settings->Other linker options and added /DEBUG. But whenever I hit "Ok" and recompiled, the "/DEBUG" option did not show up. In fact when I went back to the build options page, what I had entered disappeared?! Maybe this has already been reported? Anways, as a quick fix, I went to Settings->Compiler and debugger...->Other settings->Advanced options... and essentially hardcoded "/DEBUG" into the "Link object files to dynamic library" command. After the fix, what do you know.. The dumpbin for both the C::B compiled library and the C::B compiled mex file were identical!

This is the full commandline which C::B generates and it has /DEBUG directive.
Code: [Select]
link.exe /dll /nologo /LIBPATH:"C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\lib" /LIBPATH:"C:\Program Files\Microsoft Visual Studio 8\VC\lib"  /out:bin\Debug\TestDLL.dll user32.lib  obj\Debug\main.obj   /DEBUG
« Last Edit: June 22, 2007, 02:07:27 pm by Biplab »
Be a part of the solution, not a part of the problem.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9493
Re: Problem creating same output as VS 2005
« Reply #12 on: June 22, 2007, 02:31:01 pm »
I went to C::B->Build Options->Link Settings->Other linker options and added /DEBUG. But whenever I hit "Ok" and recompiled, the "/DEBUG" option did not show up.
Sure not. If you enter such option by hand that is part of the "official" list of compiler/linker settings it will be "transferred" into the appropriate checkbox being activated. It works like that:
- parse the list of compiler/linker options
- if it's found in the "official" list -> activate the appropriate checkbox
- if it cannot be found add it to the text field "just as it is"
So in your case if you added "/DEBUG" by hand int he text field next time you enter the dialog the appropriate checkbox is activated -> the settings do not get lost.
With regards, Morten.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline Biplab

  • Developer
  • Lives here!
  • *****
  • Posts: 1874
    • Biplab's Blog
Re: Problem creating same output as VS 2005
« Reply #13 on: June 22, 2007, 02:35:54 pm »
The updated wizard is in Rev 4153. So wait for todays (or tonight's ;) ) nightly. It should solve your problem.

Please note that in the new wizard, incremental linking is set to NO. This is necessary to embed the manifest in a simple way.
« Last Edit: June 22, 2007, 02:37:43 pm by Biplab »
Be a part of the solution, not a part of the problem.

Offline yamokosk

  • Single posting newcomer
  • *
  • Posts: 7
Re: Problem creating same output as VS 2005
« Reply #14 on: June 22, 2007, 04:57:28 pm »
Quote
The updated wizard is in Rev 4153.

Great! I will give the new build a test run when it comes out.

Quote
So in your case if you added "/DEBUG" by hand int he text field next time you enter the dialog the appropriate checkbox is activated -> the settings do not get lost.

Quote
This is the full commandline which C::B generates and it has /DEBUG directive.

Maybe I overlooked something but I am almost positive that the full command line that C::B generated did not have the /DEBUG option in it. That was after adding it to the extra options list about 3 or 4 times... but if you say its there, then maybe it was a user error. Morten - Thanks for the explanation of the disappearing options. I had a feeling something like that was going on behind the scenes because I witnessed similar behavior for other options.

Quote
Please note that in the new wizard, incremental linking is set to NO. This is necessary to embed the manifest in a simple way.

This is no problem for my project. I toggled this option in VS and Matlab didn't seem to care either way.. which makes sense I guess.