Symbols browser issue of CC has been fixed for my Linux

[Miguel Gimenez]

The second (dump)


Edit: both are executed in the main thread, and the first number was mistakenly including the second. In my next message the numbers are corrected and compared with the new version

[Miguel Gimenez]

I have changed the std::list to a kind of intrusive list (each element has pointers to parent, previous and next siblings and first child).

I still have to adapt the quicksort algorithm, but the timings with CB project look better. Below are the old times (tree creation times revised) wth the new ones at the right side (remember, no sort).

- Current projects symbols
    - tree creation 36 ms, now 8 ms
    - top tree dump 227 ms, now 15 ms
    - bottom tree (creation+dump)
        - Global functions 40+132, now 21+105
        - Global typedefs 16+29, now 13+26
        - Global variables 32+97, now 17+77
        - Macro definitions 390+1093, now 167+707

- Everything
    - tree creation 427 ms, now 241 ms
    - top tree dump 452 ms, now 159 ms
    - bottom tree (creation+dump)
        - Global functions 1720+7702, now 838+2264
        - Global typedefs 634+2003, now 329+1496
        - Global variables 183+526, now 114+435
        - Macro definitions 12321+26790, now 5803+14174

The top tree creation is made in the worker thread, while the dump and the bottom tree work are made in the main thread.

[Miguel Gimenez]

Quicksort reactivated; the timings for tree creation have increased, but not much

21 -> 30
13 -> 14
17 -> 23
167 -> 215
838 -> 978
329 -> 384
114 -> 143
5803 -> 6211

[oBFusCATed]

The posts you're making aren't very clear.
As I've said earlier we're interested only/mostly in the timings in the main thread.
How long does it take to execute the workers is not that important (it is, but it is secondary).
The question is what is taking the time? You'll have to use a profiler. Try either vtune or perf.
VTune is easier to setup for stack-gathering and it recently switched to be free instead of being mightily expensive.

As I've said previously the main thread must not pause for more than 10ms during this operation.
After you know what is slow you could decide what is best course of action. I won't be surprised that the tree-ctrl api is slow.

[ollydbg]

One idea: why not creating the whole tree, is there any optimized way we can build some part of the tree which are shown to the user. That's only a very small part of the whole tree. If user scroll up and down the tree, we have to build the tree gradually.

[oBFusCATed]

For lists this concept/mode is called virtual list. We use this to make opening the alt-g dialog to be really fast no matter the size of the project (the dialog opens in <100ms even on the linux kernel project (40+k files)).

I'm not sure this is available for trees though Probably we could do create-children-on-expand thing, but I'm not sure if this would benefit us.

As I've said we first need to know what is slow then we could find a solution. And more importantly why it is slow. Then we'll be able to find a solution. Guessing about performance is never a good idea.

[Miguel Gimenez]

I was using wxStopWatch.

The code works the same as before, only the dump parts have been added, and they use the main thread because it is the only one allowed to access the CCTreeCtrl.

- The top tree creation is made in the worker thread as before.
- The worker thread only activates (now and before) when the top tree changes due to a code completion token tree change.
- The top tree dump is made by the main thread when the tree is finished (this part is new).
- The bottom tree creation and dump are made by the main thread in the OnSelect event (just like before). Of course, the dump adds delay.

The tree leaves are created dynamically when expanding their parent, and deleted when the parent is collapsed, just like in the original code. This work is (and was) made by the main thread.

I find it quite usable if you don't use the Everything option.

I have attached my final changes, just in case anybody wants to test/profile/modify or reuse parts.

[earlgrey]

Seems to work, thank you.

[ollydbg]

I was using wxStopWatch.

The code works the same as before, only the dump parts have been added, and they use the main thread because it is the only one allowed to access the CCTreeCtrl.

- The top tree creation is made in the worker thread as before.
- The worker thread only activates (now and before) when the top tree changes due to a code completion token tree change.
- The top tree dump is made by the main thread when the tree is finished (this part is new).
- The bottom tree creation and dump are made by the main thread in the OnSelect event (just like before). Of course, the dump adds delay.

The tree leaves are created dynamically when expanding their parent, and deleted when the parent is collapsed, just like in the original code. This work is (and was) made by the main thread.

I find it quite usable if you don't use the Everything option.

I have attached my final changes, just in case anybody wants to test/profile/modify or reuse parts.

I try to apply the patch file "symbol_browser.patch" in my local git repo.

But it looks like I have no way to apply the patch.

Code

$ patch -p0 ../symbol_browser.patch

This will make the patch command hangs without  returning, I have to press the "CTRL+C" to halt the patch command. (I'm using the shell from git-for-windows)


EDIT:
There is another issue, I see this patch contains some changes like:

Code

-            const TokenIdxSet* tokens = tree->GetTokensBelongToFile(*itf);
+            const TokenIdxSet *tokens = tree->GetTokensBelongToFile(*itf);

I would still prefer the old styling. 

[Miguel Gimenez]

Here you have a revised copy with the coding standard restored. I prefer the other way, but obviously the existing standard must be respected.

The patch is generated with svn (svn diff > symbol_browser.patch), I can generate it with svn diff --git if you need it.
in the next message you have a version generated with --git option.

I have been testing timings in Symbol Browser with 17.12, and the delays in main thread are similar (as I said above, most of the work in the bottom tree is, and was, made by the main thread).

Edit 22-10-2020: published new patch correcting an uninitialized pointer, see http://forums.codeblocks.org/index.php/topic,23580.msg164959.html#msg164959

[Miguel Gimenez]

I have attached a patch file generated with svn diff --git

[kipade]

I have attached a patch file generated with svn diff --git

after several busy days, I tried this patch and it works find for me.
I use git, use this command to do:

Code

git apply ~/Downloads/symbol_browser_git.patch  -p2

Thanks for this perfect work and patch.

[oBFusCATed]

Thanks for this perfect work and patch.

I guess your project is either small or you could tolerate long UI pauses.

[Miguel Gimenez]

Most of the pauses are already in the original code, the greatest one is in the OnSelect() event due to full bottom tree creation in main thread.

[kipade]

Most of the pauses are already in the original code, the greatest one is in the OnSelect() event due to full bottom tree creation in main thread.

Yes, you are right. there are some short pauses when click the top tree items.
Since it have to be done in the main UI thread, I think no way to avoid that if did not keep all parsed symbols tree item in memory.