[Solved] Distracting SymbolBrowser Behaviour

[tigerbeard]

A couple of days ago I had updated to trunk rev13403 and noted a different behaviour of the Symbol Browser. The last version I used one was about a year older (can't recall the svn). The test was in Ububtu 22.04.

its about how the symbol tree updates. I use the setting "own parser per project". The tree is rebuild after slight code changes. I can reproduce it by adding a space to a header file and saving it. The tree disappears for about half a second and then is reset to fully closed. The problem I have is that my last tree position & open state is lost.

My old version did do that too, but a lot less often. On the other hand I had situations where a wrong click to the tree would crash code blocks. That happened about once or twice a day, but I never lost code (CB is very good keeping disk files up to date  ) and it was rather predictable so I would re-scan manually.

Now subjectively every few minutes I have to reclick through the tree (I use hierarchical namespaces a lot) and would like to do something about it. For me the crashing version was more productive.

Also I noted that with the setting "use one parser for the whole workspace" the tree closes when I double click on a class in a file not yet open in the editor. This was with only 1 project open. I think this makes the symbol tree extremly hard to work with.

Can someone confirm this behaviour?



If that is intended behaviour, I would like to try to implement something to get the tree into the same open position as before. As I do not know much about CodeCompletion I would be glad if someone can tell me in which function this tree rebuild is triggered. Maybe there is a way to find the first open tree leaf, save it and after the tree update search it and re-open it if found there.

[Pecan]

Which code completion plugin are you using?
The legacy CodeCompletion or the Clangd_client?

Could you post a copy of the Code::Blocks log. It shows a list of plugins activated just after startup. Right click on the tab titled "Code::Blocks" and copy it to the clipboard then paste it here within code tags (the # icon above  a reply box).

[tigerbeard]

Which code completion plugin are you using?
The legacy CodeCompletion or the Clangd_client?

No I am not using clangd nor have I configured it (ManagePlugins shows it as disabled). The CodeCompletion is Version 1.0 and enabled.

Here is it in the log.

Code

...
BrowseTracker
ToolsPlus
SpellChecker
CodeCompletion
cbDragScroll
EditorConfig
SmartIndentHDL
...

[Pecan]

Which code completion plugin are you using?
The legacy CodeCompletion or the Clangd_client?

No I am not using clangd nor have I configured it (ManagePlugins shows it as disabled). The CodeCompletion is Version 1.0 and enabled.

Here is it in the log.

Code

...
BrowseTracker
ToolsPlus
SpellChecker
CodeCompletion
cbDragScroll
EditorConfig
SmartIndentHDL
...

Thanks, I'll see if I can recreate the situation and figure it out.
If you have any hints about how to recreate the problem, I'd appreciate it.

[tigerbeard]

Thats very kind, thanks.

I have a test using the workspace  trunk/src/CodeBlocks_wx30-unix.workspace

First Test

• Set Settings/Editor/CC/CCParser/UseOneParserForWholeWorkspace
  
• Set Symbol Browser to "Current Projects Symbols"
  
• Make sure no files are open in the editor
  
• Activate Project wxSmith/ContribItems
  
• Switch to symbol Browser
  
• Open a class in the tree, e.g. wxsAxis and double clikc on the constructor
  
• Obvservation: The file is opened, and the tree is refeshed and closed
  Expected: The file is opened, and the tree remains open and wxAxis ctor is selected.
  

[tigerbeard]

Second Test

• Set CC option to OneParserPerProject. It will ask to reparse. Set OK
  
• Repeat Test1. it should show the expected behaviour
  
• Enter Project Tab and select project Clangd client
  
• Switch to Symbol Browser Tab.
  
• Set Symbol Browser to "Current Projects Symbols".
  
• Observe Popup : "This feature is not supported in combination with the option "one parser per whole workspace".
  If you got the popup there are several ways to get to the correct display. Beside restarting CB switching to FileSymbold and selecting a file in the new project and switching back worked.
  
• Open naemspace Doxygen
  
• double click on DoxygenParser class -> header opens. tree is stable
  
• double click on DoxygenParser ctor -> cpp opens. tree is stable
  
• Add a space in the open doxygen_parser.h file and press CTRL-S to save
  
• Obvservation: The tree is refeshed and closed
  Expected: The tree is refeshed and remains open
  

[Pecan]

Thats very kind, thanks.

I have a test using the workspace  trunk/src/CodeBlocks_wx30-unix.workspace

First Test

• Set Settings/Editor/CC/CCParser/UseOneParserForWholeWorkspace
  
• Set Symbol Browser to "Current Projects Symbols"
  
• Make sure no files are open in the editor
  
• Activate Project wxSmith/ContribItems
  
• Switch to symbol Browser
  
• Open a class in the tree, e.g. wxsAxis and double clikc on the constructor
  
• Obvservation: The file is opened, and the tree is refeshed and closed
  Expected: The file is opened, and the tree remains open and wxAxis ctor is selected.
  

I think this is actually working as expected.
For the tree to remain stable after the user selects an item, "Display bottom tree" option has to be selected from the tree root context menu.

I'm guessing this is how the original developer meant it to work. For this reason:
The code checks for "Display bottom tree" option enabled to determine whether to update the tree to the next project of the workspace, or leave the tree as is.

This behavior could be changed by someone with the cohones to modify legacy CC, but that would not include me since a single click solves the problem. If a user does not want to see the bottom tree, just pull the split window slider down.

[tigerbeard]

I think this is actually working as expected.
For the tree to remain stable after the user selects an item, "Display bottom tree" option has to be selected from the tree root context menu.

Thanks that you took time to test it. A great tip to just move down the split window (I found that mode hard to use). It seems that on my systems the effect is the same with the bottom tree mode enabled. So maybe there is an additional factor I need to find. Or did you test under Windows?

The code checks for "Display bottom tree" option enabled to determine whether to update the tree to the next project of the workspace, or leave the tree as is.
This behavior could be changed by someone with the cohones to modify legacy CC, but that would not include me

I would like to have a go, but looking through the CC code bought me nowhere yet, so I need to debug myself into the issue. The problem is that bebugging CB on my computers is really slow (5sec+ per step). So it would help me to know which CC function did the check you described above. Maybe from there I could reverse engineer the system and understand the the problem.

Would you mind to check the other test as well?
The second issue is more annoying part because I do not have a workaround for it yet.

legacy CC

I do not want to hijack my own thread but you mentioned that before and I'd be interested why you say that.
I am aware that you are an clang expert. I have to admit that I never used it but I checked the CB wiki and the wikipedia page. It tells me basically its alternative C++ token parser that must run as server process on my pc and it has some Apple history and comes from another compiler suite. I did not see any advantages it would have compared to using the "legacy" system. From what I read I only know the disadvantages like another tool to track, configure and maintain, separate installiation on every system, maybe even starting the server before running CB. Unfortunately I do not know any benefit compared to my current "legacy CC" when I am not planning to change the gcc compilers?

[Pecan]

Thanks that you took time to test it. A great tip to just move down the split window (I found that mode hard to use). It seems that on my systems the effect is the same with the bottom tree mode enabled. So maybe there is an additional factor I need to find. Or did you test under Windows?

I now think I'm wrong. When debugging with your second test, I see some problems.
When I went back to the first test, I could cause the liinux problem on windows also. So I'm still working on this.

I would like to have a go, but looking through the CC code bought me nowhere yet, so I need to debug myself into the issue. The problem is that bebugging CB on my computers is really slow (5sec+ per step). So it would help me to know which CC function did the check you described above. Maybe from there I could reverse engineer the system and understand the the problem.

Here are the routines I'm circling in on:
void ClassBrowser::UpdateClassBrowserView
void NativeParser::OnEditorActivated(EditorBase* editor)
void NativeParser::OnParserEnd(wxCommandEvent& event)

Would you mind to check the other test as well?
The second issue is more annoying part because I do not have a workaround for it yet.

This is the test that convinced me that there was a problem.
I got thrown by the fact that legacy CC (LCC) requires that a file of a newly activated project has to be opened before LCC will fill the Symbols tree. The bottom tree of the split window may have nothing to do with it.

From what I read I only know the disadvantages like another tool to track, configure and maintain, separate installiation on every system, maybe even starting the server before running CB. Unfortunately I do not know any benefit compared to my current "legacy CC" when I am not planning to change the gcc compilers?

The Clangd_client plugin starts the server in another process for you, communication to CB with pipes. the advantage of the plugins is the support that Clangd provides for parsing code using C++11 onward.
LCC will not be updated to support modern compilers.

To run Clangd_client , you need only install clangd and tell the plugin where it lives.
clangd , itself, uses the parsing stage of your compiler to mark errors w/o having to compile the code. It's much faster to create clean code that way. But it also eats up a process running clangd iin the background. LCC may be more useful on slow hardware.

However, since I stole a lot of code from LCC, I'm continuing to work on this problem you've presented here because I want to make sure that Clangd_client plugin is not having LCC like problems with the Symbols Browser.

Thanks for reporting, we'll get it fixed yet. Or know the reason why we cannot fix it.

[tigerbeard]

Here are the routines I'm circling in on:
void ClassBrowser::UpdateClassBrowserView
void NativeParser::OnEditorActivated(EditorBase* editor)
void NativeParser::OnParserEnd(wxCommandEvent& event)

Great that is really helpful. I'll have a look there.

LCC will not be updated to support modern compilers.

You mean like LLVM or does that include newer gcc versions?

clangd , itself, uses the parsing stage of your compiler to mark errors w/o having to compile the code. It's much faster to create clean code that way.

interesting. That might be worth checking out...

Clangd_client plugin is not having LCC like problems with the Symbols Browser.

I wondered whether or not you would have and own symbol broswer for the plugin.

[Pecan]

@ tigerbeard
I've managed to re-create the "first test" problem on linux, but not on windows. That would point a finger at a possible problem in wxWidgets (or the way we use it).

I'm using Mint 21.x and wxWidget 3.0 .
On windows 11 I'm using wxWidget 3.2.4

Tell us what wxWidgets version you're using on linux.
And what linux version you're using.
Thanks

Please also see: https://sourceforge.net/p/codeblocks/tickets #1444

LCC will not be updated to support modern compilers.

You mean like LLVM or does that include newer gcc versions?

I mean that LCC will not be updated to support any compiler.
At least when it comes to parsing new features for any compilers using  C++11, C++17, C++20, additions etc.
There is not enough CB resources (esp. team resources) to add new C++?? features to LCC.

clangd takes care of that for us. There are also many clangd extensions that we haven't even explored yet. such as suggested fixes while the user is editing (codeActions) and code snippet suggestions while editing.

[tigerbeard]

Tell us what wxWidgets version you're using on linux.
Please also see: https://sourceforge.net/p/codeblocks/tickets #1444

Have added the info to the ticket.

clangd takes care of that for us. There are also many clangd extensions that we haven't even explored yet. such as suggested fixes while the user is editing (codeActions) and code snippet suggestions while editing.

Cool, thanks that good to know.

[tigerbeard]

Solved by a code fix for next nightly by @pecan, thanks a lot!

And there is an information for the issue from Test1: (Symbol Tree when switching between projects):

• When changing projects in the "Project" tab you must open any code file from the new project before switching to the "Symbols" tab.  I that is done the symbold browser tree will be re-parsed with the correct project. It needs at least one open file from the project.

I will mark the thread closed

[Pecan]

Solved by a code fix for next nightly by @pecan, thanks a lot!

And there is an information for the issue from Test1: (Symbol Tree when switching between projects):

• When changing projects in the "Project" tab you must open any code file from the new project before switching to the "Symbols" tab.  I that is done the symbold browser tree will be re-parsed with the correct project. It needs at least one open file from the project.

I will mark the thread closed

I'd like to add one important distinction:
We should have used the word activate rather than open.

We do not need to open a file every time we switch to a project in codecompletion plugin, but we do need at least one file of the project open in an editor and activated. Code completion follows the currently activated editor to update the Symbols browser.

To activate an editor, just move the mouse inside that editor (if focus follows mouse) or click inside the editor or it's tab.

Thanks to tigerbeard for his patience, fix and testing.

[Pecan]

Request for comment:

Here is the code that creates the root of the ClassBrowser tree. From a separate thread it sends a thread function's local pointer to the main GUI thread which create the root node for the ClassBrowser symbols tree

Code

void ClassBrowserBuilderThread::FillGUITree(bool top)
{
    CCTree* localTree = top ? m_CCTreeTop : m_CCTreeBottom;

    ... ... ... ... ... code snipped out ... ... ... ...
    
    CCTreeItem* sourceRoot = localTree->GetRootItem();
    if (sourceRoot)
    {
        m_Parent->CallAfter(&ClassBrowser::TreeOperation, ClassBrowser::OpAddRoot, sourceRoot);
        AddItemChildrenToGuiTree(localTree, sourceRoot, true);
        m_Parent->CallAfter(&ClassBrowser::TreeOperation, top ? ClassBrowser::OpExpandRoot : ClassBrowser::OpExpandAll, nullptr);
    }

    ... ... ... ... ... code snipped out  ... .. .. ... .. .. .. ..
}

1) The variable "sourceRoot" is a local pointer which will be deleted at the end of the function.
2) Even if only the value of "sourceRoot" mattered in the CallAfter() statement "localTree" is also a variable soon to be deleted. (I'm not sure that really matters);
3) The data referenced by the "souceRoot" value (pointer) is subject to change and deletion without the Main GUI threads knowledge.
4) I've verified that the data seen by the GUI can be garbage even though it was perfectly valid when sent from the threads CallAfter() statement.
5) Placing a "wxMilliSleep(128)" after the CallAfter() statement resolved all crashes which, to me indicates a race condition.
6) Note that a CallAfter() simply places the request at the end of the GUI's event queue. No telling when it'll get executed.

What looks to me like "it should work", is, on occasion, causing garbage at the GUI end and crashing CB.

Your thoughts please. I'm not an expert on CallAfter().
Thanks in advance.

 

[Miguel Gimenez]

I wrote that code three years ago, but I have forgotten the details.

CallAfter() is executed in call order (it uses the event queue), but it is possible for a second call to execute while the first is still running. Probably a mutex in ClassBrowser::TreeOperation() fixes this, or waiting for a "finished" flag after each call to CallAfter().

[sodev]

I don't know the code you are referring to but your description of how CallAfter() works is wrong. CallAfter() does always post to the main thread, so even if the code executed by CallAfter() does use CallAfter() itself, that code won't execute immediately, your described interleaving of calls can't happen.

[Pecan]

I wrote that code three years ago, but I have forgotten the details.

CallAfter() is executed in call order (it uses the event queue), but it is possible for a second call to execute while the first is still running. Probably a mutex in ClassBrowser::TreeOperation() fixes this, or waiting for a "finished" flag after each call to CallAfter().

Thank you You're suggestion worked.
By placing a semafore.Wait() after the CallAfter()s and  a semaphore.Post() in the CallAfter()'s target function, all the crashes that I could previously create disappeared.

Whew !!, Thanks again.

[Miguel Gimenez]

Happy it worked, good job.

@sodev, what I am telling is: if you execute two consecutive CallAfter(foo) in the worker thread, and foo() in the main thread calls p.e. Yield(), directly or indirectly, then the second foo() will start executing before the first has ended. Control will return to the first foo() when the second ends.

[sodev]

That makes sense, i didn't take the dangerous Yield() into account.

[Pecan]

Head ver 13432 contains many fixes to the ClassBrowser after treating it badly with stress testing and beating out every crash I could find.

Give it a try,  beat up on it,and let us know how it goes.
Backup your current codecompletion.dll and copy the the devel version to the output folder version so that any crash shows us the line numbers in the backtrace file.

Thanks

[tigerbeard]

Give it a try,  beat up on it,and let us know how it goes.

Thanks for your work. I am currently testing that as my main development version. Its very stable so far but I need more testing.
I am testing svn13342 under a Linux Ubuntu 22.04 environment.

Some early feedback
* when opening a new project the symbol tree is not shown unless I click into the empty tab. I am not sure if its a 100% rule, but I did not have another case yet

* during work sometimes the three would not rebuild and an empty tab is shown with no working context menu. It was notable, because I found no other way to display the symbol tree again other than rebuilding the project. That happening out of nowhere would be a bit annoying for a new user. It happened during a full work day about 5 times, so I only can say there seems to be something, but i need to find a way to reproduce it.

* Subjectively the symbol tree seems to re-build a bit less often than before, which is a good thing. I had not crashes at all.

I keep on testing.

[Miguel Gimenez]

Using svn13436, if I open C::B workspace, wait for CC to complete and close the program then codeblocks.exe still shows in the task manager (tested on MSW).

I have detected this after compiling svn13437: executing update32.bat shows a lot of "file in use" errors.

EDIT: nightly 13430 works OK, nightly 13434 fails. This leaves 13432 as the suspect, probably the thread is waiting for a semaphore.

[Pecan]

Using svn13436, if I open C::B workspace, wait for CC to complete and close the program then codeblocks.exe still shows in the task manager (tested on MSW).

I have detected this after compiling svn13437: executing update32.bat shows a lot of "file in use" errors.

EDIT: nightly 13430 works OK, nightly 13434 fails. This leaves 13432 as the suspect, probably the thread is waiting for a semaphore.

Thanks, I'll look into it.

[Pecan]

Using svn13436, if I open C::B workspace, wait for CC to complete and close the program then codeblocks.exe still shows in the task manager (tested on MSW).

I have detected this after compiling svn13437: executing update32.bat shows a lot of "file in use" errors.

EDIT: nightly 13430 works OK, nightly 13434 fails. This leaves 13432 as the suspect, probably the thread is waiting for a semaphore.

I could not recreate this problem, so I changed all CallAfter semaphore Wait()s to WaitTimeout(500).

Please test to see if this solved the problem.
Thanks

[Miguel Gimenez]

It works now, thank you.