Author Topic: UI for project globs aka automatic source directories  (Read 12663 times)

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
UI for project globs aka automatic source directories
« on: February 04, 2023, 12:14:53 am »
Hi,
today i pushed (r13191) a UI for project globs aka automatic source directories.
This adds
+ an UI for the user to easy enter directories and specify if they should be added recursively and with file masks (found under the menu Project->Automatic source paths)
+ a logic based on wxFileSystemWatcher that watches the above added directories for file changes and adds and removes the file as they are added and removed from the file system
+ a rudimentary script binding

BEWARE: this changes behaviour of old file globs, that was only accessible when editing the project file directly.
The old implementation (loading the directories only on project load, and not adding them in the project file directly) made problems with code completion and compiler settings ecc...
I think this implementation is a lot better, but it changes the project file every time a file is added in the directory (as a makefile would).

i worked for this a while ago, obfuscated made some reviews and was ok with it, but because the wxFileSystemWatcher was not available on all platforms in wx2.8 this work was lost. Now that we are on wx3.0 a was thinking to add this to trunk.
Any comments are welcome.
thank you!

Offline Miguel Gimenez

  • Developer
  • Lives here!
  • *****
  • Posts: 1553
Re: UI for project globs aka automatic source directories
« Reply #1 on: February 04, 2023, 10:51:44 am »
I get this warnings/errors when trying to compile in MSW-32 bits:
Code
projectfile.h||In constructor 'ProjectFile::ProjectFile(cbProject*)':|
projectfile.h|237|warning: 'ProjectFile::project' will be initialized after [-Wreorder]|
projectfile.h|195|warning:   'GlobId ProjectFile::globId' [-Wreorder]|
projectfile.cpp|26|warning:   when initialized here [-Wreorder]|

projectmanagerui.o||In function `ZN16ProjectManagerUI13OnManageGlobsER14wxCommandEvent':|
C:\Codeblocks\src\src\projectmanagerui.cpp|1657|undefined reference to `ManageGlobsDlg::ManageGlobsDlg(cbProject*, wxWindow*, int, wxPoint const&, wxSize const&)'|
C:\Codeblocks\src\src\projectmanagerui.cpp|1657|undefined reference to `ManageGlobsDlg::~ManageGlobsDlg()'|
C:\Codeblocks\src\src\projectmanagerui.cpp|1657|undefined reference to `ManageGlobsDlg::~ManageGlobsDlg()'|
The undefined references appear because you added the new files to all projects but CodeBlocks_wx32.cbp. I have attached a patch for this part.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #2 on: February 04, 2023, 04:44:11 pm »
Thank you! Fixed in trunk

Offline gd_on

  • Lives here!
  • ****
  • Posts: 796
Re: UI for project globs aka automatic source directories
« Reply #3 on: February 11, 2023, 07:50:24 pm »
I tried to understand how this "globs" work, how I could eventually use them, and how I can translate the few items.
Until now, no real success.
Could you give us some examples or better add them and a detailed description in the CB wiki (Then, I will be able to add this in the documentation).
Windows 11 64 bits (23H2), svn C::B (last version or almost!), wxWidgets 3.2.4 (tests with 3.3), Msys2 Compilers 13.2.0, 64 bits (seh, posix : gcc, g++ and gfortran in C:\msys64\mingw64) or 32 bits (dwarf2, posix  in C:\msys64\mingw32).

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #4 on: February 11, 2023, 09:31:37 pm »
Quick and dirty wiki article...
i do not know how to describe it better:
https://wiki.codeblocks.org/index.php/Automatic_source_paths

Offline gd_on

  • Lives here!
  • ****
  • Posts: 796
Re: UI for project globs aka automatic source directories
« Reply #5 on: February 12, 2023, 10:10:19 am »
Thanks. It's more clear.
I still have a problem to find an equivallent word for "glob" in french. I don't think that "global' is good. If somebody has an idea ... You can set a translation directly in lauchpad.
« Last Edit: February 12, 2023, 10:12:29 am by gd_on »
Windows 11 64 bits (23H2), svn C::B (last version or almost!), wxWidgets 3.2.4 (tests with 3.3), Msys2 Compilers 13.2.0, 64 bits (seh, posix : gcc, g++ and gfortran in C:\msys64\mingw64) or 32 bits (dwarf2, posix  in C:\msys64\mingw32).

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #6 on: February 12, 2023, 05:47:52 pm »
I think we should remove the word "glob" from the user perspective... I think "automatic source folder" is a better (but no perfect) naming for user. Glob is only used internally, because of historic reasons....

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #7 on: February 13, 2023, 08:11:33 am »
My point of view is, that the project file is like a make file. You have to modify a makefile if you add source files to be able to build it.
If you do not add them to the project we have some problems:
1) you can not modify properties of them: Only build with certain build targets ecc.
2) I had other problems with the old implementation, i can not remember any more exactly (it is a 3 years), i think code completion did not work correctly...

We have discussed this with the problems of the original implementation here: http://forums.codeblocks.org/index.php/topic,22783.msg154862/topicseen.html

My opinion on this: This feature was never/seldom used by user before, because it had no ui/documentation. So the risk of breaking some project files is minimal. I also pointed this out in the announcement post (https://forums.codeblocks.org/index.php/topic,25276.0.html). Now it has UI and documentation and we can expect it to be used... I for example need it for an upcoming plugin i currently am working on...
If you are absolutely against this implementation i can try to look into it and add some flag for old and new behavior?

I will look into the problems with the ids. Thank you for testing and pointing it out.


When I created this feature, long time ago, the intention is to mimic the glob feature of cmake. We have been relying on this feature extensively, and indeed it is not documented, since one needs to know how to use it, it has limitations, eg when new file is added, one needs to reload the project  to pick up the new files. Code completion works, but not always, but with the old code completion plug-in it also worked like this, sometimes yes, sometimes no, but I can indeed think with this glob it is a bit worse.


Now that said, the cmake world is also no longer in favor of their glob feature (it also has some problems), and want sources to be explicitly added. Which we could translate to CB as :
  • we also don't want this feature
  • we remove both new and old behavior
  • we keep the old behavior as is for backwards compatibility

I do think we need to keep the old behavior for backwards compatibility, since it was there to mimic cmake glob (and then you also did not have to touch the CMakeLists.txt anymore), this needs to be re-instated, because overhere everything is now collapsing.

So please provide the old way, and when the new way is wanted, have CB add per directory an extra flag to say new behavior (old behavior should need no changes to be made by the user), absence of this flag means old behavior.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #8 on: February 15, 2023, 08:28:06 am »
any fixes coming up ?

I also note other strange behaviors, not sure due to this topic, but when I save files that were adjusted, they save and then they auto close.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #9 on: February 15, 2023, 09:43:18 am »
I work on it. The earliest i can do is this weekend...


Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #10 on: February 15, 2023, 11:21:25 am »
Is it also not desired, that the globs get automatically reloaded when new files are added in the directory? So no automatic reloading?

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #11 on: February 15, 2023, 05:37:25 pm »
Is it also not desired, that the globs get automatically reloaded when new files are added in the directory? So no automatic reloading?

I think this would be a very welcome addition.

Offline Miguel Gimenez

  • Developer
  • Lives here!
  • *****
  • Posts: 1553
Re: UI for project globs aka automatic source directories
« Reply #12 on: February 15, 2023, 06:18:05 pm »
After commiting this feature files can't be removed from projects (I am not using globs). See this forum thread.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #13 on: February 16, 2023, 10:40:20 pm »
Quote
After commiting this feature files can't be removed from projects (I am not using globs). See this forum thread.
fixed in trunk

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #14 on: February 17, 2023, 12:57:07 am »
I have a working test here: https://github.com/bluehazzard/codeblocks_sf/tree/fix/globs/fix_not_add_to_prj/1

@killerbot: can you test it?

Be aware:
*) I found out, that i have to add some option to add globs only to certain targets, so first work for this is in this commit also, but not functional at the moment.
*) I have not looked at the Ids yet.
*) I have added a checkbox  "Add to project" in the globs settings. This naming is not right and i will rename it in "Add files to project file". If you create a new glob this will be set by default. If you load a glob from a project file with this option not present it will be deactivated.

More work on the weekend...


[Edit:] I also saw that the comments in project file are not right...

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #15 on: February 18, 2023, 10:07:03 pm »
I tried a build I made from your repo, and I can confirm my project files remain unaltered  :)

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #16 on: February 21, 2023, 09:35:19 am »
please activate this on trunk if you can, since your repo/clone does not contain clangd client, which means many functionality is gone.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #17 on: February 21, 2023, 11:47:05 pm »
Quote
since your repo/clone does not contain clangd client
i do not understand? This is in the git repo, i use it all the time...

Quote
please activate this on trunk if you can
is in trunk, with many other fixes...

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #18 on: February 23, 2023, 07:43:04 pm »
I just switched back to trunk, so no longer from your repo, and the behavior of modified cbp files is back.
 And clangd client completion is back.

The diff is due to the following options being saved in the cbp file:
* addToProject=0
* id = some magic value
* wildcard now after the recursive


If this is stable, then I could indeed do a one time commit of all my changed project files ...


UPDATE :
when opening the project file once more, and closing it, CB again says there are changes, I select yes to save them once more, diffing the outcome ==> no change

So when opening my workspace and closing it I have to click 200 times :-(
Please fix this part , since this is rather troublesome
« Last Edit: February 23, 2023, 07:49:30 pm by killerbot »

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #19 on: February 24, 2023, 12:23:16 am »
Quote
The diff is due to the following options being saved in the cbp file:
* addToProject=0
* id = some magic value
* wildcard now after the recursive
I do not see any "easy" possibility to not add this to the project glob project file entry, and keep the features. I would have to store if this were in the origin project file and save only if there. But this would make the code a lot more complicated and not really dev friendly...

Quote
when opening the project file once more, and closing it, CB again says there are changes,
Can you try my last changes? If they do not fix it, can you give me a minimal example project, i can not reproduce it with my test cases...

[edit:] And thank you a lot for testing!

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #20 on: February 24, 2023, 11:27:42 am »
still the same, will provide a small project later today

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #21 on: February 25, 2023, 08:59:00 am »
attached a little project
open it up in CB, allow it to save the adjusted project file (make a cop of it)
open it up again, once it still thinks there are changes, allow it to save, compare to the copy ==> identical
==> this repeats forever

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #22 on: February 26, 2023, 12:59:24 am »
Thank you a lot!
I think my last commit fixes the error.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #23 on: February 26, 2023, 09:14:47 am »
Confirmed, and it doesn't even try to add the id field, and switch the order.
So it nicely keeps old style projects untouched :-)

many thanks.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #24 on: March 23, 2023, 10:46:20 pm »
I spotted another regression, when a workspace is opened which contains several projects (say all glob ones, in my case), only the active project has been "scanned", all the other projects have no sources (in the project tree view they also don't have the expand indicator), once you activate the project manually, they get populated.
As such CB is completely useless for such workspace, because trying to build the workspace or a project that has one of those unpopulated ones as dependencies leads to nothing.

Could this be fixed urgently ?

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #25 on: March 23, 2023, 11:16:56 pm »
I can reproduce it, at the moment i have no idea why and i will look into it.
But it can take until weekend....

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #26 on: March 24, 2023, 01:26:23 am »
Ok, i think i fixed it in trunk. This should emulate the old behaviour. On loading the workspace all globs get updated. After loading workspace Globs get only updated if you
a) activate the project
b) reload the project

The question is if it would be better to activate file system watches for all projects (also inactive, but this would probably impact performance) or to update globs before building, what also would impact the build performance... Any suggestion?

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #27 on: March 24, 2023, 07:40:10 am »
I can confirm it is fixed. Many thanks.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #28 on: May 07, 2023, 05:02:03 pm »
could you please fix the problem which I think is a result of the adjusted glob, when I have such a project open, and I edit a file, and I safe that file (or by compiling it autosaves), it always closes.
I mentioned this problem before  I think, but I had to upgrade now also CB on my main system, and the annoyance is now so big I can no longer use CB in production (before it was on my secondary system which I don't use that much, so I tried to live with it there).


Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #29 on: May 08, 2023, 10:58:47 pm »
I never encountered this problem.
I reckon your code generation deletes the files and regenerates them instead of overwriting?
I will try to reproduce and find a fix....

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #30 on: May 09, 2023, 07:39:06 am »
there is no code generation.
The files are there, the project is open, a file is open in the editor pane, edit in the file, save --> editor gets closed.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #31 on: May 09, 2023, 11:38:49 pm »
Quote
The files are there, the project is open, a file is open in the editor pane, edit in the file, save --> editor gets closed.
I can not reproduce this...
I have also tried to switch save strategy with no difference
What operating system?


Only as i thought when the file gets deleted and regenerated.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #32 on: May 09, 2023, 11:58:25 pm »
Can you share a example project?

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #33 on: May 15, 2023, 11:56:58 pm »
Can you test this patch?
I still need some more testing, but it is a start...

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #34 on: May 16, 2023, 08:12:43 pm »
on my first test example, which had 100% reproducibility this works ok now :-)

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #35 on: May 16, 2023, 09:43:43 pm »
It would still be interesting to know why your system does this...

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #36 on: May 18, 2023, 06:40:09 pm »
indeed I also haven't found a real explanation for this. And sometimes it does not happen ..

Can you commit the patch ?

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #37 on: May 19, 2023, 12:54:51 am »
There is still a issue with the fortran plugin i have to fix first...

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #38 on: May 19, 2023, 11:37:12 pm »
Can you test this second patch? Specially code completion and so on...
Also it would be interesting, if you could test if you specifically remove the link project file tick in file->properties->build and if this still is persisted in your case
I think this would not work, and i do not think there is a solution for this...
Are you using windows? Are you using some sync application on the source files?

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: UI for project globs aka automatic source directories
« Reply #39 on: May 29, 2023, 06:21:35 pm »
this patch replace the previous one, or is top of the previous one.

One thing I have noticed, that after a compile, or other actions, CB crashed a a lot since I applied the first patch.
Will inspect next time the crash xml.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #40 on: May 29, 2023, 08:35:17 pm »
Instead of the first one...

Quote
One thing I have noticed, that after a compile, or other actions, CB crashed a a lot since I applied the first patch.
yea, this are my worries.... The problem is i can not test this as elaborate as you, because i can not reproduce... If you could attach a debugger or a trace it would be nice. I suspect the code completion plugin? Can you try to disable it and see if the crashes repeat?

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: UI for project globs aka automatic source directories
« Reply #41 on: June 06, 2023, 12:13:08 am »
Any updates?