Code completion using LSP and clangd

[kipade]

It works fine for my first try. Its so cool.
My local environment:
wx 3.1.6
gcc 11.2
newest cb
Thanks your GREAT job!

[Miguel Gimenez]

The changes in commit r12689 may apply to this plugin.

[AndrewCot]

Thanks. I created a ticket in the SF clangd_client project so that it gets tested and checked when time permits.

[Pecan]

The changes in commit r12689 may apply to this plugin.

Applied to clangd_client rev 39/40; Thanks Miguel and Andrew

[ollydbg]

Hi, Pecan, I see some code snippet:

Code

        ConfigManager *cfgApp = Manager::Get()->GetConfigManager(_T("app"));
        bool i18n = cfgApp->ReadBool(_T("/locale/enable"), false);

I think the i18n is not a correct variable to check whether the later "(unsigned int)invChar" should be casted.
I think the i18n is used to identify whether a custom language translation is used for the GUI. For me, this value is false, because I use the default English version.

[ollydbg]

One issue I see when using this plugin to parse the "Code::Blocks wx3.1.x (64bit)" cbp file.

In my computer, it takes more than 15 minutes, but from the "Code::Blocks Debug" log, I still see there are more than 280 more files need to be parsed. I see the "clangd.exe" eat about 75% of my cpu usage.

Is there option that we can only parse the active cpp files? Or active target source files. It looks like parsing the whole projects is really slow. 

Thanks.

[Pecan]

One issue I see when using this plugin to parse the "Code::Blocks wx3.1.x (64bit)" cbp file.

In my computer, it takes more than 15 minutes, but from the "Code::Blocks Debug" log, I still see there are more than 280 more files need to be parsed. I see the "clangd.exe" eat about 75% of my cpu usage.

Is there option that we can only parse the active cpp files? Or active target source files. It looks like parsing the whole projects is really slow. 

Thanks.

You do not have to wait for all files to parse to use clangd.

Re: cpu usage. Set concurrently parsing threads to 2 at
   Setting->Editor->Clangd_Client->C.C++ parser(tab)->Concurrently parsing threads.

You should not worry about parsing going on in the background when restricting the number of parsing threads. My cpu usage is only 34% when I restrict concurrent parsing to 2 threads. And clangd response is quite fast.
(Intel i7 with 8 threads), only half of the threads are allowed to be used by clangd (hard coded), and only 2 to be used for background parsing (user option).

Active editors always get parsed first, then inactive editors, then background (non-editor) files. Newly opened editors go to the head of the queue and get parsed ahead of background files. Code completion requests also go to the head of queue.

There's no need to wait after the active editor is parsed (3 to 7 secs).
 
When I open CodeBlocks workspace (31_64), the active editor is indexed by clangd in a max of 7 seconds. All open editors have been indexed in less than 30 seconds. But I never have to wait more then 7 seconds to use code completion or other clangd features.

All those files being parsed in the background are used to fill the symbols tree. They're parsed in order of last-changed-time, then last-opened-time then all others so that the symbols tree gets filled with the most likely used files and symbols.
 
However, I'll put your suggestion in the ToDo list with an option to only parse files in editors. I don't know why you'd want to do that though, if you'll restrict the number of concurrent parsing threads.

[ollydbg]

One issue I see when using this plugin to parse the "Code::Blocks wx3.1.x (64bit)" cbp file.

In my computer, it takes more than 15 minutes, but from the "Code::Blocks Debug" log, I still see there are more than 280 more files need to be parsed. I see the "clangd.exe" eat about 75% of my cpu usage.

Is there option that we can only parse the active cpp files? Or active target source files. It looks like parsing the whole projects is really slow. 

Thanks.

You do not have to wait for all files to parse to use clangd.

Re: cpu usage. Set concurrently parsing threads to 2 at
   Setting->Editor->Clangd_Client->C.C++ parser(tab)->Concurrently parsing threads.

You should not worry about parsing going on in the background when restricting the number of parsing threads. My cpu usage is only 34% when I restrict concurrent parsing to 2 threads. And clangd response is quite fast.
(Intel i7 with 8 threads), only half of the threads are allowed to be used by clangd (hard coded), and only 2 to be used for background parsing (user option).

Active editors always get parsed first, then inactive editors, then background (non-editor) files. Newly opened editors go to the head of the queue and get parsed ahead of background files. Code completion requests also go to the head of queue.

There's no need to wait after the active editor is parsed (3 to 7 secs).
 
When I open CodeBlocks workspace (31_64), the active editor is indexed by clangd in a max of 7 seconds. All open editors have been indexed in less than 30 seconds. But I never have to wait more then 7 seconds to use code completion or other clangd features.

All those files being parsed in the background are used to fill the symbols tree. They're parsed in order of last-changed-time, then last-opened-time then all others so that the symbols tree gets filled with the most likely used files and symbols.
 
However, I'll put your suggestion in the ToDo list with an option to only parse files in editors. I don't know why you'd want to do that though, if you'll restrict the number of concurrent parsing threads.

Hi, thanks for the explanation.

I see in my setting: Setting->Editor->Clangd_Client->C.C++ parser(tab)->Concurrently parsing threads.

This value is 1. (I haven't change this value, so the default value is 1)

By reading the source code, I see that if the value is 1, it means we only let the indexer use 1 thread of the total CPUs. In my PC, I have a CPU with 4 threads.

So, the expected cpu usage should be 25%, am I correct?

Here is a test:
I set the value:  Setting->Editor->Clangd_Client->C.C++ parser(tab)->Concurrently parsing threads.  to 2.
Now, I see that the cpu usage of clangd.exe is about 50%.

This is interesting, as 2 means 2/4 = 50%, which is correct. 

But why the cpu usage is 75% if I set the value to 1?, I did test again, and see it is 25%, so sorry about that wrong report  . I may looked wrongly at the the "System Idle process" cpu usage, because when the clangd.exe use 25% cpu, and the 75% cpu is from the Idle process.

[ollydbg]

Maybe, the background index can be saved when we close the project, and reopen it when we load the project.

Here are some reference:

Qt Creator and clangd: An Introduction
It said the index can be persistent.

Some from clangd and llvm github and forum
Ability to move a project's clangd background index without triggering reindexing  Issue #847  clangd/clangd
and Offline background index preparation  Issue #587 clangd/clangd
and Using background and static indexes simultaneously for large codebases - Clang Frontend / clangd - LLVM Discussion Forums

[Pecan]

Maybe, the background index can be saved when we close the project, and reopen it when we load the project.

Here are some reference:

Qt Creator and clangd: An Introduction
It said the index can be persistent.

Some from clangd and llvm github and forum
Ability to move a project's clangd background index without triggering reindexing  Issue #847  clangd/clangd
and Offline background index preparation  Issue #587 clangd/clangd
and Using background and static indexes simultaneously for large codebases - Clang Frontend / clangd - LLVM Discussion Forums

Thanks for that information. I've often thought it would be nice to persist the clangd .cache and the symbols tree.
I'll put it on the ToDo list.

[ollydbg]

I see an annoying issue that the focus changes of the log message.

Here is the steps to reproduce:

1, I hit the "save" (or the "save all") toolbar botton
2, I hit the "build" toolbar botton
3, The building process starts, and the active log is switched to the build log
4, after for a while, the active log is switched to the LSP message.


In the step 4, it is a bit annoying. Can we stop the focus switch in the step 4?
Or we can only switch the LSP message if the compiler(build process) is not running.

Thanks.


EDIT:

I see such code snippet about the log switch(focus)

Code

        // If last request was anything but "textDocument/didSave", don't steal the log focus.
        if ( not popupActive ) switch(1)
        {
            default:
            // switch to LSP messages only when user used "save"
            if (not GetLSPClient()->GetSaveFileEventOccured()) break;
            wxWindow* pFocusedWin = wxWindow::FindFocus();
            if (not GetLSPClient()->LSP_GetLog()) break;

            CodeBlocksLogEvent evtSwitch(cbEVT_SWITCH_TO_LOG_WINDOW, GetLSPClient()->LSP_GetLog());
            CodeBlocksLogEvent evtShow(cbEVT_SHOW_LOG_MANAGER);
            Manager::Get()->ProcessEvent(evtSwitch);
            Manager::Get()->ProcessEvent(evtShow);
            if (pFocusedWin) pFocusedWin->SetFocus();
        }

I'm not sure why you combine the if and switch statement in the same line?

[ollydbg]

I see some bugs here:

BUG1:

1, I have 5 cbp files in the same folder, and I also have a workspace file which contains the 5 cbp files.

2, I open the workskpace, and switch the active project between those 5 cbps

3, When I close the C::B or close the workspace, I got 4 error messages:

Code

19:17:44: File 'D:\code\plugin\wxModularApp\build\.cache\Clangd-cache.lock' couldn't be removed (error 2: 系统找不到指定的文件。)
19:17:45: File 'D:\code\plugin\wxModularApp\build\.cache\Clangd-cache.lock' couldn't be removed (error 2: 系统找不到指定的文件。)
19:17:45: File 'D:\code\plugin\wxModularApp\build\.cache\Clangd-cache.lock' couldn't be removed (error 2: 系统找不到指定的文件。)
19:17:45: File 'D:\code\plugin\wxModularApp\build\.cache\Clangd-cache.lock' couldn't be removed (error 2: 系统找不到指定的文件。)

The Chinese words means "System cannot find the file specified.".

So, I think that this plugin use the same .lock file name for all the cbp projects?



BUG2:

1, I have 5 cbp files in the same folder, and I also have a workspace file which contains the 5 cbp files.

2, I open the workskpace, and switch the active project between those 5 cbps.

3, Now, I use the git tool to update those 5 cbp files, and C::B asked that whether I need to reload the cbp files, I press "all".
Now, I see that my CPU usage is from the 25% CPU (which is the 1 thread in my clang_client setting) to 100%, and I see 4 clangd.exe running in the task manager window.
Each process of clangd eat about 25% of the cpu.

I guess that when 5 cbp files get reloaded in the same time, it will trigger several clangd.exe to run.

My testing project is:
asmwarrior/wxModularApp: Cross-Platform Modular Application (Main app + plugins) example for C++/wxWidgets

[ollydbg]

There is another issue to show the LSP messages.

Suppose you have several cpp files in the cbp, when you first open the cbp, you get all the LSP diagnostics from all the cpp files.

But once you edit only one cpp file, and press "save" button, the whole "LSP message" will be cleaned, and only the recent saved cpp file's diagnostics will be shown.

So, I suggest if we can only update the updated files' message, but keep the old ones, thanks.

[ollydbg]

Hi, guys, I think I found an issue when using clangd under msys2.

In the instruction:

CB Clangd Client / Code / [r41] /trunk/clangd_client/documentation-install/Windows-LLVM-ClangD-Install-Readme.txt

It said:

Code

MSYS2 Compiler - MinGW64
-------------------------
There are two main options to install the clangd.exe as follows:
    1) The first option in order to  minimise disk space is to install the Clang extra tools using one of the following packages:
        +------------------------------------------+------------------------+
        |               Package                    | Clangd executable      |
        +------------------------------------------+------------------------+
        | mingw-w64-clang-x86_64-clang-tools-extra | clang64/bin/clangd.exe |
        | mingw-w64-x86_64-clang-tools-extra       | mingw64/bin/clangd.exe |
        +------------------------------------------+------------------------+

       To intall the package do the following:
        a)  Open the msys2.exe bash shell 
        b) Run the following command:
               pacman -S <Package name in the table above>
    
 OR
    2) The second option is to intall the full Clang tool chain as follows:
        a)  Open the msys2.exe bash shell 
        b) Run the following command:
               pacman -S mingw-w64-clang-x86_64-toolchain

If you are using the gcc from msys2, I mean the compilers in the folder "msys64\mingw64\bin", you should use "mingw-w64-x86_64-clang-tools-extra", which means the clangd.exe is under the folder "msys64\mingw64\bin" (the same folder in your gcc.exe).


If you are using the clang tool chain, I mean you use the compiler from the folder "msys64\clang64\bin", you should use "mingw-w64-clang-x86_64-clang-tools-extra".


I found that I just make a big mistake in one of my PC, that is I use the gcc toolchain from "msys64\mingw64\bin", but I use the clangd.exe from "msys64\clang64\bin", the result is I got a lot of LSP diagnostics messages about the errors.     Luckily I found the reason, and fix this issue. Hope this will help other guys. 

[AndrewCot]

Can you raise a ticket for this so I do not forget about it as it may not be as simple as changing the docs as the code may need some changes to ensure that end users with GCC and CLANG do not have issues like you found.