Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => Topic started by: zieQ on July 13, 2005, 02:33:27 pm

Title: Dependencies handling ?
Post by: zieQ on July 13, 2005, 02:33:27 pm

Well, I'm trying to solve the visual c++ workspace bad imports, and i have problem converting project dependencies to codeblocks.

In Visual C++, I think there's 2 sorts of dependencies:
- inter-project dependencies: say there's 2 project A and B, if A depends on B, A should be recompiled if B is modified and A should link B.
- intra-project dependencies: some external files (i.e. not .cpp or .h) should issue a compilation when modified, often due to a custom rule. For example, lexer files, resources files produce source or object files to compile/link with.

I had a deep look into Code::Blocks but i'm lost with the dependencies features !
- GetExternalDeps : i think it's the intra-project dependencies
- SetDepsOutput: I don't understand what it is for !

My questions are:
- is inter-project dependencies exist in Code::Blocks ?
- if yes, how could i say that a project A depend on a project B
- if not, could this be implemented, or at least the methods in the project class so i could finish the visual workspace loader ?
- should I use a workaround, that is linking explicitly my library B to A ?

zieQ

Title: Dependencies handling ?
Post by: mandrav on July 13, 2005, 03:26:10 pm

Quote

My questions are:
- is inter-project dependencies exist in Code::Blocks ?
- if yes, how could i say that a project A depend on a project B
- if not, could this be implemented, or at least the methods in the project class so i could finish the visual workspace loader ?
- should I use a workaround, that is linking explicitly my library B to A ?

ProjectBuildTarget::SetExternalDeps() is what you want. Any file you put there, its timestamp is checked and if more recent than the current target's output, it forces re-linking.

Yiannis.

Title: Dependencies handling ?
Post by: zieQ on July 13, 2005, 03:57:24 pm

Where are the intra-project dependencies then ? That is, how can I set a custom rule that compile a lexer file and produce a source file, and the project to recompile the generated file when the lexer file is modified ! Must I add the generated source file somewhere to be compiled ?

I also made a search for GetExternalDeps calls, they seem not to be used when compiling ?!! I'll make some quick test to check that !

And what file must I add to External Deps ?? lib file may not have been generated !

Title: Dependencies handling ?
Post by: zieQ on July 13, 2005, 04:20:23 pm

I've just tested the external dependencies "feature" with 2 projects, one building a static library and one exe to link with the library.

When i click to set external deps, I'm only able to set a (existing) file, not a project name ! I set the .cdb file and the library file as dependencies of the exe project. Now the exe project is set as active. When I change the .cpp file of the library, the library is not recompiled ! When I change the .h of the library, only the exe project recompile (since the .h file is included in the main.cpp). So the external deps are not used (or nothing indicates that they are used).

So I conclude: there's no inter-project dependencies in codeblocks !! No ! Where are my answers ?  :P

Title: Dependencies handling ?
Post by: zieQ on July 18, 2005, 05:48:47 pm

UP ? How could I set-up inter-project dependencies ?
Do you plan to add the related functions in the sdk soon ?

Title: Dependencies handling ?
Post by: mandrav on July 18, 2005, 06:00:26 pm

Quote from: zieQ

UP ? How could I set-up inter-project dependencies ?
Do you plan to add the related functions in the sdk soon ?

Projects are sets of build targets. This means a project may have more than one target generating output (an executable, a library, etc). There is no way to "link" two projects together in the form of one being depended on the other.
You can, however, make a target's output dependent on another target's output (even if it belongs to a different project). That's why you select as external dependencies only executables or libraries (shared or static).

If you don't understand what I 'm saying, say so and I 'll give you an example.

Yiannis.

Title: Dependencies handling ?
Post by: zieQ on July 18, 2005, 06:36:02 pm

Quote from: mandrav

There is no way to "link" two projects together in the form of one being depended on the other.

We agree on that !

Quote from: mandrav

You can, however, make a target's output dependent on another target's output (even if it belongs to a different project). That's why you select as external dependencies only executables or libraries (shared or static).

I really understand but this is not equivalent! I expect both targets to be recompiled if I modify something in the dependent target. Actually, if A is the currently selected target, A depend on B (a library) and I modify a .cpp file of B, none are recompiled except if I manually recompile B. The right way to do it is to depend on all executables/libraries/source files/user dependencies from the dependent target and that's a pain to set.

So, maybe you could add the "project dependencies" or "target dependency" feature. The main thing to do is to check that all the dependent project are up-to-date before proceeding to the compilation of the current project, and if not, perform the compilation of the dependent out-to-date project and relink.
In the SDK, 2 additionnal methods need to be implemented in cbproject.h /.cpp:

Code

addProjectDependency(wxString projectName);
wxArrayString getProjectDependencies();

What do you think about that ? Could this be implemented in the SDK even if methods are empty, so that i will be able to solve a part of the visual import bug ?!

I advocate I really need this feature. IMHO workspaces are useless if there's no way to make dependencies between projects/targets. What's the point of having many projects displayed simultaneously if they are no dependencies between them, or if they do not recompile properly when we edit some files?! Code::Blocks is great but without this feature it's unusable to me. I'll switch off using it if if this is not implemented but that would be a shame !

I keep trying to solve the visual workspace import bug but this is the main problem to the conversion for now.

Title: Dependencies handling ?
Post by: kagerato on July 18, 2005, 07:45:20 pm

If you need the functionality that badly and believe it is simple to implement (judging from your post, it seems to safe to establish both premises), it seems quite reasonable to code it yourself.

I don't agree that workspaces were developed for the sole purpose of managing interdependent projects, and you truly have not offered any evidence for that assertion.

Title: Dependencies handling ?
Post by: mandrav on July 18, 2005, 08:07:13 pm

Quote

I really understand but this is not equivalent! I expect both targets to be recompiled if I modify something in the dependent target. Actually, if A is the currently selected target, A depend on B (a library) and I modify a .cpp file of B, none are recompiled except if I manually recompile B. The right way to do it is to depend on all executables/libraries/source files/user dependencies from the dependent target and that's a pain to set.

I don't believe you understand what I 'm saying.

Example:

Project A is a static library.
Project B is an executable that is linked with the static library of project A.
We want B to depend on A, so if anything is changed on A, B is updated.
Do we agree so far?

C::B allows you to set A's product, the static library, as a dependency for B's product, the executable. This is done by going to project B's properties and adding A's library as an external dependency to the B's executable target.
Clear so far?

When you change something in A, say a .cpp file, this makes A dirty so it will be compiled and linked.
When A's static library is linked again, B's executable has dirty external dependency (the relinked A's static library).
Here is the point now: if nothing's changed in B and you try to compile it, it will get re-linked because it depends on a static library which is set as an external dependency.

This is the best I can explain it to you. If you still can't understand it, maybe someone who does and is more fluent with English than me can step in and explain it better.

On another note, it just occured to me you might be talking about projects build order inside a workspace. If that's the case, then the only useful hint I can give you is that projects are built top-to-bottom, as seen in the workspace tree. This means if a project A must be compiled before project B, A must be above B in the project tree. Then a "Compile->Compile all projects" will do the trick (i.e. build all in the right order).

You have to excuse me because I have never used any Microsoft IDE, so their terminology eludes me...

HTH,
Yiannis.

Title: Dependencies handling ?
Post by: Profic on July 18, 2005, 10:58:54 pm

I seemed understand what zieQ wants (using A and B from your post):
he wants automatically (not manually) compile and link A if something changed in it when he tells to compile and link B. And after this compile and link B.
Is it clear? Or I misunderstand something, too...

Title: Dependencies handling ?
Post by: David Perfors on July 18, 2005, 11:01:25 pm

isn't it easier then to make 1 project from it with diffrent build targets?

Title: Dependencies handling ?
Post by: rickg22 on July 18, 2005, 11:34:30 pm

If you read the first post, you'd see that he's "trying to solve the visual c++ workspace bad imports", and his main problem so far has been "converting project dependencies to codeblocks. "

Title: Dependencies handling ?
Post by: David Perfors on July 18, 2005, 11:36:11 pm

I didn't say anything :oops: :P

Title: Dependencies handling ?
Post by: zieQ on July 19, 2005, 09:37:33 am

Quote from: kagerato

If you need the functionality that badly and believe it is simple to implement (judging from your post, it seems to safe to establish both premises), it seems quite reasonable to code it yourself.

Hey, I am new to C::B! First I don't have big insights about C::B project compilation so I am not able implement it for now. Second, I want the main developers approval/opinion before anything! But IMHO it should not be difficult to do ! And please remember I'm already trying to solve visual workspace imports, I don't have the time to do everything in C::B!  :shock:

Quote from: kagerato

I don't agree that workspaces were developed for the sole purpose of managing interdependent projects, and you truly have not offered any evidence for that assertion.

I can't see any use of workspaces for independent projects, please give me some examples that could not be handled by separate projects files !? If workspaces are just used to collect projects in the same place, there is no added functionnality in that. So why not just keeping independent projects in independent places and use the recent list to edit them ?

Mandrav, I'll suggest you try it, that's what I made. Profic has understood my point. When I edit a .cpp file of a library B, then ask C::B to compile A (which depend on B via library dependency as you suggest, and A is set as active project), nothing get recompiled since the library is not active (so not recompiled) ! I need to manually compile B before compiling A. That's my problem, I would like it to be automatic, via inter-project dependencies. Worse, If I edit a .h file of B, A is also set as active and use the .h file, only A is recompiled. I need to manually compile B and then A is relinked automatically. Making a dependency with the target output (library in this case) is NOT sufficient to achieve the proper behaviour I expect! That's why I keep asking for inter-project dependencies!

I think using the order of the projects to get things compiled in the proper order is not the best idea. Worse, there's no easy way to move projects up and down in the workspace project list! For sure, using project dependencies would be better for that.

Hope I am clearer!

Title: Dependencies handling ?
Post by: zieQ on July 19, 2005, 09:38:31 am

Quote from: mispunt

isn't it easier then to make 1 project from it with diffrent build targets?

But are inter-target dependencies implemented ?  :wink:

Title: Dependencies handling ?
Post by: Urxae on July 19, 2005, 10:30:18 am

Inter-target dependencies are implemented in the same way as inter-project dependencies: add the produced file to the external dependency list.

Title: Dependencies handling ?
Post by: mandrav on July 19, 2005, 11:32:48 am

Quote from: zieQ

I think using the order of the projects to get things compiled in the proper order is not the best idea.

May be so, but in the end that's what it comes down to: a list of projects to build, one after another. It just so happens that this list is visible all the time: It's the workspace view :)

Quote from: zieQ

Worse, there's no easy way to move projects up and down in the workspace project list! For sure, using project dependencies would be better for that.

Try Ctrl-Shift-UpArrow and Ctrl-Shift-DownArrow ;)
Or programmatically:

Code

Manager::Get()->GetProjectManager()->MoveProjectUp(cbProject*);
// or
Manager::Get()->GetProjectManager()->MoveProjectDown(cbProject*);

These are not easy ways?

Quote from: mandrav

On another note, it just occured to me you might be talking about projects build order inside a workspace. If that's the case, then the only useful hint I can give you is that projects are built top-to-bottom, as seen in the workspace tree. This means if a project A must be compiled before project B, A must be above B in the project tree. Then a "Compile->Compile all projects" will do the trick (i.e. build all in the right order).

And, as you can see, I finally understood what you 're saying :)

Quote from: mandrav

You have to excuse me because I have never used any Microsoft IDE, so their terminology eludes me...

I thought I 'd quote this to emphasize it. In simple words: if you can't contribute code that does what you 're saying, the least you could do is provide detailed info on this thing and possible implementation details.

Yiannis.

Title: Dependencies handling ?
Post by: zieQ on July 19, 2005, 12:05:39 pm

Quote from: mandrav

Try Ctrl-Shift-UpArrow and Ctrl-Shift-DownArrow  
[...]
These are not easy ways?

Sorry I tried dragging the project and looking at the contextual menu and I didn't see anything about that! Ok  :roll:

Not all my points are related to the build order: whatever the order I choose, the projects are not compiled as I expect it to be!

Quote from: mandrav

I thought I 'd quote this to emphasize it. In simple words: if you can't contribute code that does what you 're saying, the least you could do is provide detailed info on this thing and possible implementation details.

I keep trying...  :roll:  I will explain how it is perform in Visual Studio: say we have a project A (exe) which depend on B (lib), then we have to:
- ensure B is up-to-date before any compilation/link for A
- if not, recompile B
- A will have the include paths of B added in its include paths (do not appear in the include options of the project), so as to compile properly with B.
- A is linked to B (do not appear in the link options of the project)

The extension to multiple dependencies:
- before compiling the currently selected project/target, compile all dependent projects (if necessary, but visual studio compile all and issue "up-to-date" strings)
- then compile/link the current project

Title: Dependencies handling ?
Post by: RShadow on July 19, 2005, 01:02:45 pm

Quote from: zieQ

I keep trying...  :roll:  I will explain how it is perform in Visual Studio: say we have a project A (exe) which depend on B (lib), then we have to:
- ensure B is up-to-date before any compilation/link for A
- if not, recompile B
- A will have the include paths of B added in its include paths (do not appear in the include options of the project), so as to compile properly with B.
- A is linked to B (do not appear in the link options of the project)

The extension to multiple dependencies:
- before compiling the currently selected project/target, compile all dependent projects (if necessary, but visual studio compile all and issue "up-to-date" strings)
- then compile/link the current project

forgive me if I put my foot in my mouth on this one.. I'm jumping into the conversation kind of late, but let me see if I get this right:

Lets see.. we have one workspace with two projects.

first let me get rid of the project a b c x y z stuff because it hurts my head :)

Project MyLib (library .. doesn't realy matter if its shared or static)
Project MyApp (executable that links to MyLib)

What you are wanting is when you manualy compile MyApp for it to automaticly detect any changes to MyLib and recompile MyLib if changes are present?  If that is the case I believe that is what mandrav explained like 3 posts up.  I'm not sure if C::B automaticly detects this, but it seems a little backwards to me.  If I make changes in MyLib then I would compile MyLib ..which would make MyApp's depends list dirty and force MyApp to recompile as well.  You want MyApp to autocheck MyLib for changes, but if MyLib changes then I already know about it.. well because I changed it.. I guess it just makes sense to me to build bottom up.. If I make a change to a library then I'm going to recompile that library before I recompile the app that links to it.  anyways having C::B setup a dependency list / build order wouldn't be too difficult, however I think most of the dev's are working on other stuff right now(like unicode support, and documentation to name a few).. perhaps you could submit a patch set with limitied funcationality of what you would like and then somebody could take it from there.

You also want MyApp's include path to have the same include path as MyLib (I think this is already possible, just add the common paths to the global compiler include paths)..

One thing I would keep in mind is that MSVC has been in development since the 80's and C::B is still a beta software..so you can't expect C::B to mimick the exact functionality of MSVC.  Personaly I think an IDE should only provide basic linking capabilities internaly.  If a more complex build solution is needed there are a lot of tools for that. (Perhaps writing plugins to interface with those tools is a good idea.. some of which are already being worked on). There are lots of tools to solve complex build senerios that have been around for a long time, and they still don't get it completly right.

Title: Dependencies handling ?
Post by: zieQ on July 19, 2005, 01:34:30 pm

Quote from: RShadow

I'm not sure if C::B automaticly detects this, but it seems a little backwards to me.

The answer is not! I do not figure out why it is backwards to you ?!

Quote from: RShadow

If I make changes in MyLib then I would compile MyLib ..which would make MyApp's depends list dirty and force MyApp to recompile as well.  You want MyApp to autocheck MyLib for changes, but if MyLib changes then I already know about it.. well because I changed it.. I guess it just makes sense to me to build bottom up..

Yes, that's really simple in your case: one library, one executable. To illustrate my problem, I'm currently working on a big project. There's about 30 projects in my workspace, with complex dependencies (more than 1 level, an executable which depend on a library with depend on...), with many files opened at the same time from several projects. In my case, it is not easy, evident, straightforward to remember all the dependencies, or worse which projects the files belongs to.

That's why I need the dependency feature, I'm could not live without. Projects dependencies allow faster coding since we do not have to remember all the dependencies, and to compile every time I modify a file. That's my habits :roll: , and it seems really useful to every people I know using the Visual environment.

Quote from: RShadow

One thing I would keep in mind is that MSVC has been in development since the 80's and C::B is still a beta software..so you can't expect C::B to mimick the exact functionality of MSVC.

That's not what I want. I just want to discuss it, to ear the opinion of the main developer and issue a feature request! As I said, I could not live without, so if the main developers agree with this new functionality, ok, I will help implementing it in the core SDK or give some feedback, otherwise I'll switch off using CodeBlocks. So I want some feedback from them. Moreover, this could not be implemented as a plugin since it imply modifying core sdk files (cbproject files) !

PS: could I commit modifications on the CVS, how ? Should I be registered as a regular developer ? (for visual workspaces imports)

Title: Dependencies handling ?
Post by: mandrav on July 19, 2005, 02:35:20 pm

Quote

That's why I need the dependency feature, I'm could not live without. Projects dependencies allow faster coding since we do not have to remember all the dependencies, and to compile every time I modify a file. That's my habits Rolling Eyes , and it seems really useful to every people I know using the Visual environment.

Point taken already.

Quote

That's not what I want. I just want to discuss it, to ear the opinion of the main developer and issue a feature request! As I said, I could not live without, so if the main developers agree with this new functionality, ok, I will help implementing it in the core SDK or give some feedback, otherwise I'll switch off using CodeBlocks. So I want some feedback from them. Moreover, this could not be implemented as a plugin since it imply modifying core sdk files (cbproject files) !

I have no problem adding this functionality, but I don't have the spare time now. I 'm doing bug-fixes, then I 'll release RC1 and then I 'll take my vacation :)
Anyway, since this kind of changes would be pretty heavy for the build system, they will have to be postponed for post-1.0 (no matter who writes the code).

Quote

PS: could I commit modifications on the CVS, how ? Should I be registered as a regular developer ? (for visual workspaces imports)

Please submit (http://sourceforge.net/tracker/?func=add&group_id=126998&atid=707418) a patch.

Yiannis.

Title: Dependencies handling ?
Post by: rickg22 on July 19, 2005, 06:35:55 pm

OK what I see could be done is this:

* Add the option to specify project files as dependencies for a project (cbproject.cpp etc).
* Keep a stack of "inter-project dependencies", as which project is on the bottom of the stack. When a project depends on other projects, push it to the stack and get a list of inter-project dependencies. (This would need modification on the compiler plugin)
* If any of the inter-proj dependencies is already present on the stack, ask to abort (circular dependency error), or ignore (don't try to compile project already in the stack).

Doesn't seem as hard to implement. ZieQ, we'd need your help on this. ANY VOLUNTEERS TO HELP???

Also, please tell us about the *OTHER* problems regarding visual studio imports. (Please explain them further in your sourceforge bug report).

I wanted to tackle these before 1.0 but I'm afraid this might not be possible before RC1 at least.

Title: Re: Dependencies handling ?
Post by: zieQ on July 29, 2005, 02:42:51 pm

I posted a patch for visual imports  :P

About dependencies, I finally think it should be "target dependencies" since we may have one projects with many targets that depends on each other! As you said, we just have to keep a list of dependent targets (pointers) in each target. The idea of the stack is good if we want to avoid testing a target (for up-to-date) more than once (consider the exemple below). But it is more complicated to get it work than just testing/compile the dependent projects one by one.

Here is a sketch of dependencies handling for compilation. The contents of the stack is in brackets.

                      -->  { A }
     A               -->  { A, B, C } (for the current element of the stack, A here, we add its dependencies at the end, i.e B, C)
   /   \
  B     \            --> { A, B, C, E } (dependencies of B added)
  |      C           --> { A, B, C, E, D } (dependencies of C added)
  |      |
  |      D          ---> { A, B, C, D, E }  (E was already in the stack, it is moved to the end since it must be compiled before)
   \   /
     E


And finally we compile projects from the end of the stack towards the beginning, that is E, D, C, B, A (issue up-to-date if there's nothing to do) and it should work fine.

As a side note, I said that the include of dependent projects should be added in the current project, but that is not true in Visual. We must set includes by hand. The dependency feature just add the dependent library (virtually, do not appear in the project settings) to the link libs of the current target.

Hope this helps. I may help implement this feature in the week if everything is ok.

Title: Re: Dependencies handling ?
Post by: zieQ on July 31, 2005, 09:36:34 pm

Note that I'm working on a patch to implement inter-target dependencies in C::B  :)

Title: Re: Dependencies handling ?
Post by: troels on August 01, 2005, 10:12:22 am

Quote from: zieQ on July 31, 2005, 09:36:34 pm

Note that I'm working on a patch to implement inter-target dependencies in C::B  :)

Can't wait to see it. Killer-feature!
/Troels