Code completion using LSP and clangd

[killerbot]

and what does it do with :
using TypeA = TypeB;

[ollydbg]

and what does it do with :
using TypeA = TypeB;

The same issue as the typedef.

Here is a simple test:

The cbp only contains main.cpp

Code

#include "TypeA.h"

int main()
{
    TypeA a;

    return 0;
}

And here are the TypeA.h

Code

#include "TypeB.h"


// typedef TypeB TypeA;


using TypeA = TypeB;

Here is TypeB.h

Code

class TypeB
{
public:
    int abc;

};

For testing, you can put the TypeA.h and TypeB.h in the same folder as main.cpp.

No, I can see that find declaration only goes from a file in cbp file to external file(a file not belong to cbp), such as main.cpp -> TypeA.h.

But find declaration will NOT go from an external file to an external file, so it failed from TypeA.h ->TypeB.h.

This is really an annoying issue. (BTW: our old code completion plugin works OK on this issue  )

[ollydbg]

Code

// ----------------------------------------------------------------------------
void CodeCompletion::OnGotoDeclaration(wxCommandEvent& event)
// ----------------------------------------------------------------------------
{
    ProjectManager* pPrjMgr = Manager::Get()->GetProjectManager();
    cbProject* pActiveProject = pPrjMgr->GetActiveProject();
    if (not GetLSPclient(pActiveProject)) return;

    EditorManager* pEdMgr  = Manager::Get()->GetEditorManager();
    cbEditor*      pActiveEditor = pEdMgr->GetBuiltinActiveEditor();
    if (!pActiveEditor)
        return;

    TRACE(_T("OnGotoDeclaration"));

    const int pos      = pActiveEditor->GetControl()->GetCurrentPos();
    const int startPos = pActiveEditor->GetControl()->WordStartPosition(pos, true);
    const int endPos   = pActiveEditor->GetControl()->WordEndPosition(pos, true);

    wxString targetText;
    targetText << pActiveEditor->GetControl()->GetTextRange(startPos, endPos);
    if (targetText.IsEmpty())
        return;

    // prepare a boolean filter for declaration/implementation
    bool isDecl = event.GetId() == idGotoDeclaration    || event.GetId() == idMenuGotoDeclaration;
    bool isImpl = event.GetId() == idGotoImplementation || event.GetId() == idMenuGotoImplementation;
   // ----------------------------------------------------------------------------
   // LSP Goto Declaration/definition                //(ph 2020/10/12)
   // ----------------------------------------------------------------------------
    bool usingLSP_client = true;
    if (usingLSP_client)
    {
        // Assure editors file belongs to the active project (else it's not parsed yet).
        ProjectFile* pProjectFile = pActiveEditor->GetProjectFile();
        cbProject* pEdProject = pProjectFile ? pProjectFile->GetParentProject() : nullptr;
        wxString filename = pActiveEditor->GetFilename();
        if ( (not pEdProject)
             //?or (not (pEdProject == pActiveProject)) //(ph 2022/02/15)
             //?or (not pActiveProject->GetFileByFilename(filename,false))  //(ph 2022/02/15)
                or (not GetLSPclient(pEdProject))
            )
        {
            //? InfoWindow::Display("LSP " + wxString(__FUNCTION__), "Editor's file is not contained in the active project.", 6000); //(ph 2022/02/15)
            wxString msg = _("The editor's file does not have an associated ");
            if (not pEdProject)
                msg << _("project.") << _("\nPerhaps add the file to a project ?");
            else if (not GetLSPclient(pEdProject))
                msg << "clangd_client." << _("\nPerhaps the project needs to be reparsed ?");
            cbMessageBox(msg, "LSP " + wxString(__FUNCTION__));
            return;
        }

If I looked at the comment "// Assure editors file belongs to the active project (else it's not parsed yet)."

I think it is not correct, I'm not sure clangd has some feature to query if a file is parsed or not, any one know the research direction?  Because I can only find some information about clangd in this page: What is clangd?, I'm not sure where is the official document of clangd, maybe I need to read the document about LSP?

EDIT:
Maybe, the LSP document is here:
language-server-protocol/protocol-2-x.md at main microsoft/language-server-protocol

[Pecan]

If I looked at the comment "// Assure editors file belongs to the active project (else it's not parsed yet)."

I think it is not correct, I'm not sure clangd has some feature to query if a file is parsed or not, any one know the research direction?-server-protocol[/url]

I apologize for not responding. But I'm in the middle of the US tax season. I'll respond as soon as I can get my head out  of the tax instructions.

This is not a clangd problem.
It's a "this  programmer failed to implement sending non-project files to clangd" problem.

[ollydbg]

If I looked at the comment "// Assure editors file belongs to the active project (else it's not parsed yet)."

I think it is not correct, I'm not sure clangd has some feature to query if a file is parsed or not, any one know the research direction?-server-protocol[/url]

I apologize for not responding. But I'm in the middle of the US tax season. I'll respond as soon as I can get my head out  of the tax instructions.

This is not a clangd problem.
It's a "this  programmer failed to implement sending non-project files to clangd" problem.

Hi, Pecan, thanks for the reply.

Maybe, we can have a mechanism that if a non-project file is opened in the editor, we can dynamically add(send) it to the clangd's database, and once the editor is closed, its content(the file) can be dynamically removed from the clangd's database.

This is only a guess.

Because, for my previous example, if I put the "TypeA.h", "TypeB.h" in the cbp file, then I see the find declaration works correctly in either header files. 


EDIT:
This issue maybe is related to this github issue:
Support non-self-contained files  Issue #45  clangd/clangd

[Pecan]

The current rev 50 of clangd_client adds support to load and parse non-project files (some project must be active), allow macros in clangd installation path and fixes assorted crash and logic errors.

[ollydbg]

The current rev 50 of clangd_client adds support to load and parse non-project files (some project must be active), allow macros in clangd installation path and fixes assorted crash and logic errors.

Thanks, great work!

But I got build errors, see below build logs:

Code

-------------- Build: Clangd_Client-wx31_64 in Clangd_Client-wx31_64 (compiler: GNU GCC Compiler)---------------

[ 33.3%] Running target pre-build steps
[ 66.7%] cmd /c @echo TARGET_OUTPUT_DIR: devel31_64\
TARGET_OUTPUT_DIR: devel31_64\
[100.0%] cmd /c @echo TARGET_OUTPUT_FILENAME: clangd_client.dll
TARGET_OUTPUT_FILENAME: clangd_client.dll
cmd /c @ECHO TARGET_DEVEL_DIR: D:\code\cb\cb_sf_git\cccrash2019
TARGET_DEVEL_DIR: D:\code\cb\cb_sf_git\cccrash2019
[  3.4%] g++.exe -Wall -std=gnu++17 -m64 -g -g -pipe -mthreads -fmessage-length=0 -fexceptions -DHAVE_W32API_H -DBUILDING_PLUGIN -D__WXMSW__ -DWXUSINGDLL -DcbDEBUG -DNOPCH -DwxUSE_UNICODE -D_WIN64 -DCC_NO_COLLAPSE_ITEM -DLOGGING -DDONT_SHOW_SERVER_CONSOLE -ID:\code\cb\cb_sf_git\cccrash2019\src\include -ID:\code\cb\cb_sf_git\cccrash2019\src\sdk\wxscintilla\include\ -ID:\code\cb\cb_sf_git\cccrash2019\src\include\tinyxml\ -ID:\code\cb\clangd_plugin\trunk\clangd_client\src -Isrc -Isrc\LSPclient\include -Isrc\winprocess -Isrc\winprocess\asyncprocess -Isrc\winprocess\misc -IF:\code\wxWidgets-3.1.6\include -IF:\code\wxWidgets-3.1.6\lib\gcc_dll\mswu -Isdk\wxscintilla\include -Iinclude\tinyxml -Isrc\LSPclient -c D:\code\cb\clangd_plugin\trunk\clangd_client\src\codecompletion\codecompletion.cpp -o .obj\clangd_client\src\codecompletion\codecompletion.o
D:\code\cb\clangd_plugin\trunk\clangd_client\src\codecompletion\codecompletion.cpp: In member function 'ProcessLanguageClient* CodeCompletion::CreateNewLanguageServiceProcess(cbProject*)':
D:\code\cb\clangd_plugin\trunk\clangd_client\src\codecompletion\codecompletion.cpp:2995:25: error: 'class ProcessLanguageClient' has no member named 'SetParser'
 2995 |             pLSPclient->SetParser( static_cast<Parser*>(pParser));
      |                         ^~~~~~~~~
D:\code\cb\clangd_plugin\trunk\clangd_client\src\codecompletion\codecompletion.cpp: In member function 'void CodeCompletion::DoParseOpenedProjectAndActiveEditor(wxTimerEvent&)':
D:\code\cb\clangd_plugin\trunk\clangd_client\src\codecompletion\codecompletion.cpp:4747:23: error: 'class ProcessLanguageClient' has no member named 'SetParser'
 4747 |         pProxyClient->SetParser((Parser*)pProxyParser);
      |                       ^~~~~~~~~
Process terminated with status 1 (0 minute(s), 13 second(s))
2 error(s), 0 warning(s) (0 minute(s), 13 second(s))

The code snippet is around codecompletion.cpp line 2979, see below:

Code

        if (pParser)
        {
            pParser->SetLSP_Client(pLSPclient);

            // Create ProxyProject move to OnAppDoneStartup() //(ph 2022/04/16)
////            // Create a ProxyProject to use for non-project files (if not already existent )
////            GetParseManager()->SetProxyProject(pcbProject);
////            // Set the ProxyProject to share this clangd client.
////            cbProject* pProxyProject = GetParseManager()->GetProxyProject();
////            if (pProxyProject)
////            {
////                m_LSP_Clients[GetParseManager()->GetProxyProject()] = pLSPclient;
////                ParserBase* pProxyParser = GetParseManager()->GetParserByProject(pProxyProject);
////                pProxyParser->SetLSP_Client(pLSPclient);
////            }

            pLSPclient->SetParser( static_cast<Parser*>(pParser));
        }

        pLSPclient->LSP_Initialize(pcbProject);
    }

[BlueHazzard]

With this rev i get the following compiler errors:

Code

cb_clang\clangd_client\src\codecompletion\parser\parser.cpp|1405|error: 'class ProcessLanguageClient' has no member named 'GetClientsCBProject'; did you mean 'GetClientObject'?|
cb_clang\clangd_client\src\codecompletion\codecompletion.cpp|2995|error: 'class ProcessLanguageClient' has no member named 'SetParser'|
cb_clang\clangd_client\src\codecompletion\codecompletion.cpp|4747|error: 'class ProcessLanguageClient' has no member named 'SetParser'|

[edit:] ollydbg was faster....

[Pecan]

I have no idea why Tortoise svn hates me so much.

Rev 52 should now have the correct files.
I downloaded and compiled it ok.

Thanks guys !!

[ollydbg]

I have no idea why Tortoise svn hates me so much.

Rev 52 should now have the correct files.
I downloaded and compiled it ok.

Thanks guys !!

Hi, Pecan, thanks for the fix.

In rev 52, why do you add a clangd_client.zip file to the repo?

BTW: I don't use TortoiseSVN, instead, I use TortoiseGit, and the git-svn bridge works OK in TortoiseGit.

EDIT:

CC_ProxyProject.cbp, what does this cbp file used for?

[Pecan]

In rev 52, why do you add a clangd_client.zip file to the repo?
...snip...
CC_ProxyProject.cbp, what does this cbp file used for?

That zip file got there out of frustration with Tortoise svn.
I gave up and just updated with everything from my local trunk.
I'll delete it for the next update. Thanks for the heads-up.

Re: CC_ProxyProject.cbp.
 
It's a hidden cbProject used to contain files that are not associated with a project. To send a file to clangd, we need info containing it's location and compile commands.
It's used to avoid making any changes to user workspaces and  projects.

A file (not belonging to a project but needing to be parsed) is added to this "~ProxyProject~" and then sent through the code that prepares the info that clangd needs.

For Example, To solve the problem described by https://forums.codeblocks.org/index.php/topic,24357.msg169687.html#msg169687
"TypeA.h" and "TypeB.h" must be scanned by clangd in order to ask for its symbols to use in a GoToDecl/Impl request (or any other request to clangd).
Since the files don't belong to a project, they're just added to a hidden project and sent off to clangd via the usual process.

CB requires a filename in order to create a project.
We can't even issue a "pointer = new cbProject()" without a filename.

So the clangd_client code puts an empty project in the users' appdata/codeblocks folder and loads it, clones it, closes it, then hides the clone.
Loading and closing it via ProjectManager is necessary to keep from screwing up the order of plugin notifications.

This one-and-only hidden ~ProxyProject~ is created at OnAppStartupDone() and removed at CB clangd_client shutdown. It's the equivalent of the old CodeCompletion hidden temp parser.

Thanks for testing !

[MaxGaspa]

I'm using the latest nightly in a Windows 7 pro 64 bit OS and for the first time I tested your implementation of the code completion using Clangd. Overall I like it and it's working but I have the following comments.

1) Using a medium sized projects (75 .cpp files for 20000 lines of code) the time needed for a complete parsing is 5 minutes (4 seconds for each file). I was expecting that at the first time the full parsing is needed. Then I closed the project and reloaded it (without closing CB). Then a full re-parsing is done by CB  for other 5 minutes. Because I didn't modify any source file I was expecting that no re-parsing is required. Waiting 5 minutes, before having a "ready" code completion, each time I open the project seems too much considering that no modifications was applied. Is there a way to avoid a full re-parsing each time I open a project? May be no but I'm wondering if some optimization is possible just re-paring modified files.

2) I typed some code in a file to have a test of the parser making some bad indentation and syntax error like typing "elset" instead of typing "else". During typing the LSP messages window is reporting various issues and a red rectangle appeared at the involved line. Then I corrected the errors but nothing happened. The red rectangle is still there! Saving the file the red rectangle disappeared. It seems that saving the file  is required to refresh the parser errors/warnings despite I selected the "Update parser when typing".  Is this behavior expected?  Have I to save the file to refresh the parser status anyway?

3) During typing it's easy to get messages related to indentation like

LSP diagnostics: AteReaders.cpp|:|----Time: 23:19:23.784---- (1 diagnostic)|
C:\Development\Projects\AteSfp28BiDi\AteReaders.cpp|56|warning:Different indentation for 'if' and corresponding 'else'|

Because it's not a major issue, considering also the topic 2), a good idea could be to have an option to ignore indentation issues. Moreover I'm wondering which indentation rules are used. Different teams may have different indentation rules for their projects. So it's possible to ignore the indentation warnings as an option?

Thank you in advance for your attention.

Max

[Pecan]

I'm using the latest nightly in a Windows 7 pro 64 bit OS and for the first time I tested your implementation of the code completion using Clangd. Overall I like it and it's working but I have the following comments.

@Max
Firstly, thanks for testing.

Regarding 1)
You do not have to wait for the background parsing to finish.
Active editors always get parsed first, then inactive editors, then background (non-editor) files. Newly opened editors go to the head of the queue and get parsed ahead of background files. Code completion requests also go to the head of queue.

There's no need to wait after the active editor is parsed (3 to 7 secs).
 
When I open CodeBlocks workspace (31_64), the active editor is indexed by clangd in a max of 7 seconds. All open editors have been indexed in less than 30 seconds. But I never have to wait more then 3-7 seconds to use code completion or other clangd features from within the active editor.

All those files being parsed in the background are used to fill the symbols tree. They're parsed in order of last-changed-time, then last-opened-time then all others so that the symbols tree gets filled with the most likely used files and symbols.

Regarding 2)
Currently, changes on a single line need to be saved manually. Changes on multiple lines usually are recognized and clangd reparses the file. This is how the older codecompletion worked but I'll see if I can do a little better in a furture fix.

Regarding 3)
CodeBlocks clangd_client has no control over what clangd reports as a warning. However you can hide the warning as follows:
With the warning in the LSP messages log, right click on the "LSP messages" tab and click on "Show/set ignore messages". In the resulting dialog, check the box for the messages you want to ignore.

Regarding Indentation:
I have no idea what indentation rules clang/clangd is applying to source files.

[ollydbg]

I'm building this plugin with the latest wx 3.2.0 release.

The source code I use is the latest rev: [r67]

It looks like I see "Requested token Not found" MessageBox when I use the mouse right context menu-> find declaration.

I switch back to an old rev [r66], which I build against wx 3.1.7, and I don't see this issue there.

Does any one notice the same issue?

Thanks.

EDIT:

I just build the revision 66 against wx 3.2.0, and it works OK.
So, my guess is some regression in this plugin's source code in revision 67.

[AndrewCot]

I updated during the week and have been updating my scripts for building both wx 3.1.7 and 3.2. and have not finished all of the combinations. Once I do I will see if I get the same results for r67, unless it is resolved by then. I hope it does not take more than another 2 days to finish.