New UI: Project Globs

[BlueHazzard]

Hi,
after a long time i finished my rework of globs to a point where i can show something. It is still work in progress, but it should work fully and can be tested with this git hub repo:
https://github.com/bluehazzard/codeblocks_sf/tree/wip/globs

What are globs?
Globs are folders on the file system from what files are loaded automatically. This feature was hidden in codeblocks since many years, but had no UI.
negative points about old implementation:
- No UI, only modifying the project file in xml
- The loading of new files was only done during project loading.
- Glob files where not added to the project, so you could not set any individual file/compiler options and save them

I have reworked it and now it reloads the files in "real time". I also added an UI for managing globs

Features:
* New UI for project globs:
+ Adding, deleting globs
+ Context menu and main menu entry to manage globs per project or per active project
* Automatically load new and remove non existent files from the file system
+ Reloading is handled in 1 sec interval for reduced UI clogging when adding a LOT files
* File options are saved for glob files
* Added UI to project file options to display that this file is from a glob

Things i am not 100% sure i like how i implemented it:
~ A hash is calculated for each glob and this hash is used as id to save in the project file
~ Removing of glob files is still possible. This should give a warning or should not be possible at all
~ Symlinks are not handled ( I do not plan to add support for them at the moment)

Feel free to test and comment

greetings

PS. I have not a lot time at the moment... Food giving work is really demanding at the moment...

For refenrence:
Ticket: https://sourceforge.net/p/codeblocks/tickets/729/
Old forum thread: http://forums.codeblocks.org/index.php/topic,22783.msg154862/topicseen.html (i have started a new thread, because the new logic is quite different from the old thread)

[omlk]

This part C::B IDE, can it be done as a plugin?

[BlueHazzard]

It is part of the core Application,
So you can write a plugin that uses this functionality,
or even use scripting (not fully supported at the moment)

[omlk]

I understood correctly?

[oBFusCATed]

or even use scripting (not fully supported at the moment)

I hope you haven't changed any scripting related code.
It will make the days of someone pretty funny if you did (the first pushing his stuff wins).

[BlueHazzard]

Code

I hope you haven't changed any scripting related code.
It will make the days of someone pretty funny if you did (the first pushing his stuff wins).

Added one function... No big deal...

[oBFusCATed]

Yes, no big deal if you're not the one fixing the conflicts.
All binding code is 100% rewritten in my version.

[oBFusCATed]

BTW:
What is the sales pitch for these globs?
Why would you want to introduce such a complex system?

At work we've used them in cmake and they are a total disaster, so we're slowly replacing them with hard-coded list of files.

[BlueHazzard]

The initial idea was (and still is if i find the time) that you can use generated projects, like from STCube (Microcontroller)
They generate a "generic" makefile you can import in eclipse and i have written a codeblocks import too, but in this file the source files are not listed, only a top level folder is listed. You have to add the whole folder, and every time you regenerate the code you would have to reimport the whole folder and remove the non existent files. Eclipse does this by default.
With this function this would happen automatically also in codeblocks.

PS. This was 3 years ago, i do not know if they have updated their project export file to the standard (https://www.keil.com/pack/doc/CMSIS/Build/html/element_cprj.html ) where each file is listed...

[oBFusCATed]

The initial idea was (and still is if i find the time) that you can use generated projects, like from STCube (Microcontroller)

This is the job for the generator.

They generate a "generic" makefile you can import in eclipse and i have written a codeblocks import too, but in this file the source files are not listed, only a top level folder is listed. You have to add the whole folder, and every time you regenerate the code you would have to reimport the whole folder and remove the non existent files. Eclipse does this by default.
With this function this would happen automatically also in codeblocks.

The generators in CMake already do this.
Does this folder parsing works with multi-folder projects? Would you be able to use these globs with a projects as big as Codeblocks.cbp (which is relatively small to medium size project in 2020)?

PS. This was 3 years ago, i do not know if they have updated their project export file to the standard (https://www.keil.com/pack/doc/CMSIS/Build/html/element_cprj.html ) where each file is listed...

Even if they have not they would have to do it in order to generate the <glob> stuff.

If this is the only reason for this feature to exists, I doubt that it is a good idea to add it.

If I were doing it I would monitor the original project definition for changes and just generate plain cbp files. It would be a lot simpler and easier to do. These FS watchers seem scary and inefficient to me and the OS differences are quite massive under the hood...

[BlueHazzard]

I think you have not understood me:
MXCube (a project generator for ST Microcontroller) generates source files and a generic project file (i am not quite sure, but i think it has this format: https://www.keil.com/pack/doc/CMSIS/Build/html/element_cprj.html). This is not a makefile. This is also not a codeblocks project.

This is a commercial product and the code is not public or i have any possibility to intervene...
This code generates a lot sub folders with code, asambler and other files.
This code generator generates a generic project file where he describes some sub folders that have to be added to the build system, but it does not describe each file.
The structure of this folders changes a lot and is quite dynamic.

If i can not use globs with file system watcher i have to reimport the whole project each time. This is not acceptable. And other IDEs do this automatically. Why should codeblocks not do it?

This is the job for the generator.

Well i can not modify the generator....

Does this folder parsing works with multi-folder projects? Would you be able to use these globs with a projects as big as Codeblocks.cbp

yes.

If this is the only reason for this feature to exists, I doubt that it is a good idea to add it.

This is a example of the usage.... Look at eclipse they use this concept for their projects....
here is a other user: http://forums.codeblocks.org/index.php/topic,22783.msg157016.html#msg157016

If I were doing it I would monitor the original project definition for changes and just generate plain cbp files. It would be a lot simpler and easier to do. These FS watchers seem scary and inefficient to me and the OS differences are quite massive under the hood...

Well i would still need the fs watcher... If you do not use project globs you have absolute no impact (beside the 8 Bytes per project file for the ID)....

[oBFusCATed]

I think you have not understood me:
MXCube (a project generator for ST Microcontroller) generates source files and a generic project file (i am not quite sure, but i think it has this format: https://www.keil.com/pack/doc/CMSIS/Build/html/element_cprj.html). This is not a makefile. This is also not a codeblocks project.

I know what it is. I don't know why people want to use it

If i can not use globs with file system watcher i have to reimport the whole project each time. This is not acceptable. And other IDEs do this automatically. Why should codeblocks not do it?

This is what cmake does - it generates a new project file every time.

Anyway. I have an opinion, but the decision is up to you.
If you find it useful then fine. Just keep in mind that you'll be the owner of this part of the code and you would have the responsibility to maintain it.

[BlueHazzard]

I understood correctly?

No, the UI is in Project->Project Globs
There you can add Folders that are added automatically to the project.

[omlk]

The initial idea was (and still is if i find the time) that you can use generated projects, like from STCube (Microcontroller)
They generate a "generic" makefile you can import in eclipse and i have written a codeblocks import too, but in this file the source files are not listed, only a top level folder is listed. You have to add the whole folder, and every time you regenerate the code you would have to reimport the whole folder and remove the non existent files. Eclipse does this by default.
With this function this would happen automatically also in codeblocks.

PS. This was 3 years ago, i do not know if they have updated their project export file to the standard (https://www.keil.com/pack/doc/CMSIS/Build/html/element_cprj.html ) where each file is listed...

+ Now I understand what you offered.

[killerbot]

BTW:
What is the sales pitch for these globs?
Why would you want to introduce such a complex system?

At work we've used them in cmake and they are a total disaster, so we're slowly replacing them with hard-coded list of files.

We are doing the  same thing, moving away from globs in cmake. But we used to glob in cmake, and I also have cbp project files for every library, executable we have in our codebase, and to keep it aligned (with those globbed cmakelists files) and avoid that it had to be adjusted each time files came and go, I created this glob feature (but kept it simple and only in the xml file, no gui).

Though we are, as said, no longer using glob in cmake I still think our CB project variants are having added value from the fact they glob. They are a second tool, not the official build tool (cmake/make is).
So personally I think the improvements are nice, the fact it can happen at runtime, and not only at project load. A gui in front of it, is also nice. The saving of some settings per "globbed file", might be tricky, and I something I wouldn't do (my feeling : to much added complexity).


I once already experimented with the generated cbp files from cmake, but CB never was able to launch the entire workspace, and after 30 minutes I gave up. Still something I am gonna further investigate.

[oBFusCATed]

I once already experimented with the generated cbp files from cmake, but CB never was able to launch the entire workspace, and after 30 minutes I gave up. Still something I am gonna further investigate.

Why are you using the workspace? It should have tons of duplicated stuff. I think cmake generates full projects for every cmake target. Full means all dependent targets are present as C::B targets.
Generally I'm using the main cbp file generated from cmake (the one in the root of your build directory) for a fairly big project (2m+ lines, 100+ executables/dlls) and it works fine at least for compilation, editing, CC. Running and debugging is another story. Loading time for such a project is something like 2-3 seconds.

[BlueHazzard]

The saving of some settings per "globbed file", might be tricky, and I something I wouldn't do (my feeling : to much added complexity).

The behavior has changed from your implementation. In your implementation the glob file was special, you could change the compiler options but they were not saved. This was confusing for me on the first use...
In my implementation the glob files are like regular files. They are added to the project, like if you add a file with Project->Add file, just automatically, no special handling, no additional complexity.
This comes with the specialty, that your project file gets changed every time the globs change...