Author Topic: Linking fails with A LOT of object files  (Read 5261 times)

webmonarch

  • Guest
Linking fails with A LOT of object files
« on: November 03, 2008, 07:19:46 pm »
Hey all,

This could very well be a mingw / windows issue, but I wanted to put it out there:

I was trying to compile a project called ogre4j http://ogre4j.sourceforge.net/ but was having trouble due to the number of object files it needs to link at once. Specifically, there are 749 files, and the command line that code::blocks generates is something like 45k characters long.  Ultimately, things don't work. :)  Unfortunately, there isn't an error message or anything helpful afterwards.  It just says the command failed, and the next line says "Nothing to do".  Suspecting the number of files was the problem, i removed as many as i could from the codebase without losing features that I need, and I was eventually able to link things. 

My questions are:

(1) is this a known problem?  Is it with MinGW? Windows?
(2) is there a work around I can use to compile the whole program?  Incremental linking?  Linking to intermediate objects?

Appreciate the help.  I am a Java guy, and needing to put together some C++ code for my current project.

cheers,
eric

Offline odubtaig

  • Single posting newcomer
  • *
  • Posts: 9
Re: Linking fails with A LOT of object files
« Reply #1 on: November 03, 2008, 11:39:14 pm »
I expect this is a project that comes with a makefile for mingw that compiles in stages and builds groups of objects into larger objects which are then linked at a later stage.

Have a look at virtual build targets: virtual targets are specifically to contain multiple targets and the order in which those targets are built is controlled by you. Some of those targets can be compile only, some can be link only.

I do worry about a project which would need to link that many files at once, I doubt it's actually necessary so I would look into that. 45k characters is easily enough to write a final year project thesis for a degree so it's hard to criticise any program that would choke on that as a single command line.

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Linking fails with A LOT of object files
« Reply #2 on: November 04, 2008, 11:06:23 am »
Specifically, there are 749 files, and the command line that code::blocks generates is something like 45k characters long.  Ultimately, things don't work.
This is a known problem[1], but one that cannot be fixed, since it bases on a Microsoft Windows limitation. You will have to work around it, for example by building a few static libraries from a subset of the object files first, and linking those instead.

Quote from: MSDN, CreateProcess Function
lpCommandLine [in, out, optional]
    The command line to be executed. The maximum length of this string is 32,768 characters, including the Unicode terminating null character. If lpApplicationName is NULL, the module name portion of lpCommandLine is limited to MAX_PATH characters.


[1] Technically, it's not a "known problem", since you're the first one to report it, but Yiannis will remember I once said "what a piece of shit, this will bite us some day". It seems today is the day.
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: Linking fails with A LOT of object files
« Reply #3 on: November 04, 2008, 11:35:32 am »
[1] Technically, it's not a "known problem", since you're the first one to report it, but Yiannis will remember I once said "what a piece of shit, this will bite us some day". It seems today is the day.

Indeed, I do remember :).
Be patient!
This bug will be fixed soon...

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Linking fails with A LOT of object files
« Reply #4 on: November 04, 2008, 01:19:12 pm »
http://ogre4j.sourceforge.net/
I wonder how this is compiled on Windows than anyways?! I mean: Wouldn't even a Makefile based setup end up with the same problem??? Seems really you have to work-around with static libraries as Thomas said. However - I don't know this project but the number 749 makes me feel that I don't want to know it... ;-)
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline stahta01

  • Lives here!
  • ****
  • Posts: 7588
    • My Best Post
Re: Linking fails with A LOT of object files
« Reply #5 on: November 04, 2008, 02:12:03 pm »
It looks to be an Java Wrapper around Ogre; unless you wish to maintain ogre4j I would just use the standard MSVC build project file or Ant build script. The number of files combined with the file name lengths are just a lot.

Edited: The other options, as other have said, is to add adding static libs that are combined into the final DLL. I would also if I was maintaining it reorganize the the Folders and see if short names are possible. But, if just using the lib would leave file naming and structure alone.

Tim S
« Last Edit: November 04, 2008, 02:22:26 pm by stahta01 »
C Programmer working to learn more about C++ and Git.
On Windows 7 64 bit and Windows 10 64 bit.
--
When in doubt, read the CB WiKi FAQ. http://wiki.codeblocks.org