Author Topic: Automaticlly building all files in folder without a custom makefile?  (Read 5489 times)

Offline Eriond

  • Single posting newcomer
  • *
  • Posts: 8
Hi,

As far as I understand it, C::B geneates its own makefile at compile time, based on which files are in your project. I have a script that generates some files at compile time, and I'd like to find a way to add these files to the makefile that C::B generates, without actually going through the trouble of adding in the files to the project everytime a new one's generated. It seems that using a custom makefile might be somewhat of a hassle (I'm not so experienced with the things; all I know is configure, make, and make install), but if it's easier than I'm thinking, please tell me, and I guess I'll use that instead.

Basically, my question is, does anyone know if it's possible to add all *.cpp files in a particular directory (or any glob for that matter) to the build list at compile time?

Thanks
« Last Edit: June 20, 2011, 11:17:44 pm by Eriond »

Offline ouch

  • Almost regular
  • **
  • Posts: 221
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #1 on: June 20, 2011, 11:08:50 pm »
codeblocks does not create it's own makefile it passes the commands to the toolchain on the fly as you have set the project up.

right click on your project and select add files recursively. That's probably what you want from the sounds of things.

Offline Eriond

  • Single posting newcomer
  • *
  • Posts: 8
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #2 on: June 20, 2011, 11:12:51 pm »
Ah, okay.

As for recursively adding files, yeah, that's what I've been doing currently, it's just a hassle to keep doing it every time the generated file composition changes (plus, when certain generated files get deleted, I have to go in a remove them manually from the project, or they just sit as dead links in the project list). Is there any way to automate this, or, at the very least, send a message to the C::B process to refresh its project list when the actual project file is modified (I could just append the files to the .cbp manually via a pre-build script, if this is possible, I guess)
« Last Edit: June 20, 2011, 11:16:50 pm by Eriond »

Offline ouch

  • Almost regular
  • **
  • Posts: 221
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #3 on: June 20, 2011, 11:27:33 pm »
unfortunately the only way I know of would be to dive to scripting, or possibly making a custom makefile. Might even go for a strait command line shell/batch script as well depending on the situation.

But what are you building that is generating a bunch of cpp files anyway?

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 101
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #4 on: June 20, 2011, 11:31:53 pm »
You can ask your script to generate a couple of files with #include directives for all other generated files and then add only that couple to CB project.
Another solution is to use autogenerated CB project or CMake makefile that can be converted to both CB project and GNU makefile.

Offline Eriond

  • Single posting newcomer
  • *
  • Posts: 8
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #5 on: June 20, 2011, 11:58:21 pm »
Hmm. Well, I guess I'll look into using makefiles then; I've been trying to put it off for as long as possible, but I guess there's no more delaying of the inevitable.

The thing about generating the include directives, is that if I have a large amount of these files, this should increase compile time, as far as I can figure; by forcing the system recompile literally all the generated files every time one is changed; I'd like to avoid that if possible.

As for what's being generated, it's a few things. Among others, it's additional class information that a perl script scrapes from the header files; I put in a bunch of defines that mimic Unreal Engine style .uc files, so that classes that inherit from a particular one define a super, automatically serialize certain variables that aren't marked 'transient', automatically serialize pointers to anything else that inherits from the base class properly so that I don't have to do any of the dirty work every time I create a class, am able to have some richer reflexive information than would otherwise be possible, etc..
« Last Edit: June 21, 2011, 12:03:14 am by Eriond »

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 101
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #6 on: June 21, 2011, 12:42:20 am »
Interesting. Is your project an open source project?

Offline Eriond

  • Single posting newcomer
  • *
  • Posts: 8
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #7 on: June 21, 2011, 12:55:44 am »
Unfortunately no; I do have plans at some point to actually sell something made off it (not incompatible with open-source, I know!), but I haven't actually gotten around choosing a license or anything yet, so I am somewhat hesitant to make the whole thing public. It's likely at a future point in time, the engine itself will be made open-source, while the games which link with it will be closed-source.

That being said, if you're interested, I'd be glad to share some of the codebase, though I'll be honest with you, some of the perl and C++ is not very clean :p

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 101
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #8 on: June 21, 2011, 01:16:58 am »
I just thought that looking at the code might help to produce a better idea of how to solve your problem, but that's not really necessary. An example of data conversion scheme (which files become which other files for which purpose) and a couple of use cases would be enough. Besides, I don't use Perl, so I can't dig into it anyway :) By saying "interesting" I meant that it is interesting what ideas might have lead to the very existence of the problem you're facing now.

Offline Eriond

  • Single posting newcomer
  • *
  • Posts: 8
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #9 on: June 21, 2011, 01:44:14 am »
Ah, I see, lol.

Well, essentially the script rescursively goes through every *.h file in the source directory, and essentially generates one file per *.h that has at least 1 line which contains the macro "REGISTER_...". From there, it parses information about the rest of the class, and generates a .cpp in a subfolder off the root source directory, which contain static functions to initialize the class information, as well as one static member function inside the program's main class that calls the entire set of initialization functions.

For example, this, in "Source/CaeBackdrop.h"
Code: [Select]
class Backdrop : public Base {
        REGISTER_CLASS(Backdrop, Base);

        Sky* Parent;

        float3 Position;
        bool Visible;
        float Scale;
        Texture* Image;
        float4 Color;

        Backdrop(Sky* Parent);
        virtual ~Backdrop();

        virtual void Draw(const Camera* Eye) const;
        virtual void Step(float Delta);
};

Must generate this, in "Source/Generated/CaeBackdropGenerated.cpp"
Code: [Select]
void Backdrop::InitializeClassAttributes()
{
if (Properties)
return;
ClassProperties* ParentProperties = Super::GetClassProperties();
Properties = &Base::RuntimeDatabase->Classes["Backdrop"];
if (ParentProperties)
Properties->Inherit(ParentProperties);
Properties->Variables["Parent"] = BaseProperty("Parent", TYPE_BASE, __builtin_offsetof(Backdrop, Parent), BaseProperty::FLAG_NONE);
Properties->Variables["Position"] = BaseProperty("Position", TYPE_FLOAT3, __builtin_offsetof(Backdrop, Position), BaseProperty::FLAG_NONE);
Properties->Variables["Visible"] = BaseProperty("Visible", TYPE_BOOL, __builtin_offsetof(Backdrop, Visible), BaseProperty::FLAG_NONE);
Properties->Variables["Scale"] = BaseProperty("Scale", TYPE_FLOAT, __builtin_offsetof(Backdrop, Scale), BaseProperty::FLAG_NONE);
Properties->Variables["Image"] = BaseProperty("Image", TYPE_ASSET, __builtin_offsetof(Backdrop, Image), BaseProperty::FLAG_NONE);
Properties->Variables["Color"] = BaseProperty("Color", TYPE_FLOAT4, __builtin_offsetof(Backdrop, Color), BaseProperty::FLAG_NONE);
InitializeClassMacro();
}

IMPLEMENT_CLASS(Backdrop);

My issue is that if I create a new class in a new header with a REGISTER_... macro, or delete an old header which has a macro in it, the generated folder will be updated to reflect this, but the actual C::B project will not be; this is where the problem really stems from. I can't quite think of another way to really add this generated code in any other way into the build list, without inducing somewhat more severe and unparallelizable recompile times (as would be the case in some sort of #include scheme), beyond actually using a custom makefile and putting in something to the effect of 'compile every cpp in this folder.'

I hope that makes things somewhat more clear; sorry.
« Last Edit: June 21, 2011, 01:48:24 am by Eriond »

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 101
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #10 on: June 21, 2011, 02:10:17 am »
So you're trying to attach some kind of RTTI to unmanaged code to access object fields by name (string parameter). Am I right?

Hmm... Why don't you #include generated .cpp into manually created .cpp corresponding to .h file?
Changing one file at a time won't turn into a complete rebuild of the project... however, forgetting a file is still possible :(
« Last Edit: June 21, 2011, 02:26:50 am by mirai »

Offline Eriond

  • Single posting newcomer
  • *
  • Posts: 8
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #11 on: June 21, 2011, 02:45:22 am »
Yeah, creating some RTTI as well as some context about that particular class is the goal.

I'd like to avoid changing the actual .h files (or for that matter, any files that the user will generally be looking at) as much as possible; I'd like to make the system completely transparent to the programmer, though including things in the individual header files is one thing I hadn't considered. I suppose that would indeed be a very acceptable way of including the file with minimum fuss. I'll try it out, and see how I like the feel of it. Thanks!

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 101
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #12 on: June 21, 2011, 03:15:00 am »
OK :D

By the way, I would suggest to try another really different approach as an alternative to scripting. Here's the idea: every manged field should be an object of a class which constructor registers it in a "database", performs initialization, connects all necessary pointers, tunes callbacks, etc.; managed field classes may be just envelope classes for basic types without any real overhead. A database then should be a static member of the base class or even a regular procedural interface providing named access to registered entities. This way you probably won't have to generate anything by scripts or macros at all. But if you already have a lot of working code... taking this approach may become a problem.



Offline Eriond

  • Single posting newcomer
  • *
  • Posts: 8
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #13 on: June 23, 2011, 04:47:04 am »
It's an interesting idea, but as you say, I've already got a ton of code that uses the system and supports it :P. Also, how would the managed fields get their stringified names, without the use of macros? I didn't think there was any way to actually get the name of any member without using macros or actually typing in the name of each and every single member variable twice. Plus, there's a bunch of other things that automatically get done in the class generation script that you can't really do (I think!) with the perl, like local enum transformations (forward and back) from strings to ints, etc..
 
Though, I will admit, for some reason, I never really considered using templates and classes like that to make the managed fields. It's definitely something to think about!  :D

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 101
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #14 on: June 24, 2011, 02:49:57 pm »
Hmm... macros might be inevitable :? However, some things can be resolved with initializers, I suppose. About typing names twice... those names may be different - internal and public name - internal name may have a prefix like "m_" (member) or be an abbreviation ("pos" for position, "dir" for direction) or contain some characters to look different to some other unmanaged field or comply with coding style, while public names may be longer and more self-descriptive and comply with configuration style. Registering a managed field in a database establishes certain mapping between internal (private) names with their style and external (public) names with their probably different style. Macros won't be very helpful in this case.

Plus, there's a bunch of other things that automatically get done in the class generation script that you can't really do (I think!) with the perl, like local enum transformations (forward and back) from strings to ints, etc..
Can you tell more about this part?

Offline Eriond

  • Single posting newcomer
  • *
  • Posts: 8
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #15 on: June 24, 2011, 07:13:46 pm »
Well, for example, if an enum is defined in a managed class header, the script will pick up on it, and register the enum, and stick the stringified enum values into a map, as well as the actual values of those enums; allowing me to to translate to ints from strings with those enum values (useful in config files, and in-game script), as well as back from ints (useful in writing the config back to the file, etc..).

Granted, I could easily register these manually, but it seems to me that maintaining a lot of interdependent duplicate code is probably more trouble than maintaining the script, even if not using the script would allow me to keep everything in one language, and reduce complexity slightly (the script is only ~300 lines).

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 101
Re: Automaticlly building all files in folder without a custom makefile?
« Reply #16 on: June 24, 2011, 11:53:06 pm »
Yeah, it's hard to overcome the power of meta-programming with regular language features :D

The C++ as we know it was born as preprocessed extension of the C language, and now it seems that we need a new language built upon meta-programming paradigm with plain C at the bottom of a language extension stack.

As for your initial question, I think auto-generated projects or makefiles (probably, template-based) is the way to go.