Code completion using LSP and clangd

[ollydbg]

@ollydbg

When you run the debugger, CB  ordinary uses the personality "default" which, for you, I'm guessing has CodeCompletion plugin enabled.

You should use a separate personality for debugging so that you can disable CodeCompletion in the debugged CB without disabling it in your production CB.

For me, in CB main menu/project/SetProgramArgument I have:
--debug-log --no-dde --no-check-associations --multiple-instance --no-splash-screen --verbose /p cbDebug315

Then I can disable CodeCompletion in the debugged CB MainMenu/plugins/managePlugins without it affecting the main CB personality "default".

I'd still like to see the "manage plugins" status of both clangd_client and Code Completion plugins to determine why your m_CodeCompletionEnabled variable is false.

I'm using the same CB personality cbDebug315 as yours. Because I'm running the debugger from the clangd_client_wx31_64.cbp.

[Pecan]

@ollydbg

m_CodeCompletionEnabled variable refers to Settings/Editor/Code completion checkbox.

For your debugged CB, it must be currently unchecked which means neither Code completion nor clangd_client can run.

I have screen shots, see below as in attachment.

I have Settings/Editor/Code completion checkbox checked. If I member correctly this checkbox does not mean we should "disable" the plugin, instead, it means we don't show the suggestion prompt when we are editing.

The other screen shot shows the version latest version I'm using.

I'll look into the historical usage of the Settings/Editor/Code completion box.

Are these snapshots form the debugger or the debuggee ?
I think your debugged CB (the debuggee) has that box unchecked.
Would you give us a snapshot of your debugged CB Settings/Editor/Code completion box while the debugger is running?

Thanks for testing. It helps.
 

[ollydbg]

@ollydbg

m_CodeCompletionEnabled variable refers to Settings/Editor/Code completion checkbox.

For your debugged CB, it must be currently unchecked which means neither Code completion nor clangd_client can run.

I have screen shots, see below as in attachment.

I have Settings/Editor/Code completion checkbox checked. If I member correctly this checkbox does not mean we should "disable" the plugin, instead, it means we don't show the suggestion prompt when we are editing.

The other screen shot shows the version latest version I'm using.

I'll look into the historical usage of the Settings/Editor/Code completion box.

Are these snapshots form the debugger or the debuggee ?

debugee.

[ollydbg]

This is the whole patch I'm using:

Code

 clangd_client/clangd_client_wx31_64.cbp | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/clangd_client/clangd_client_wx31_64.cbp b/clangd_client/clangd_client_wx31_64.cbp
index 1af073c..eb3e6d4 100644
--- a/clangd_client/clangd_client_wx31_64.cbp
+++ b/clangd_client/clangd_client_wx31_64.cbp
@@ -6,8 +6,8 @@
 		<Option compiler="gcc" />
 		<Build>
 			<Target title="Clangd_Client-uw">
-				<Option output="bin/Clangd_Client" prefix_auto="1" extension_auto="1" />
-				<Option working_dir="bin31_64" />
+				<Option output="$(TARGET_DEVEL_DIR)/src/devel31_64/share/CodeBlocks/plugins/clangd_client" prefix_auto="1" extension_auto="1" />
+				<Option working_dir="$(TARGET_DEVEL_DIR)/src/devel31_64" />
 				<Option object_output=".objs31_64" />
 				<Option external_deps="$(CODEBLOCKS)/libcodeblocks.a;" />
 				<Option type="3" />
@@ -50,12 +50,13 @@
 					<Add before="cmd /c @echo CODEBLOCKS: $(CODEBLOCKS)" />
 					<Add before="cmd /c @echo TARGET_DEVEL_DIR: $(TARGET_DEVEL_DIR)" />
 					<Add before="g++ --version" />
-					<Add after="cmd /c MakeRepoUpload.bat" />
+					<Add after="cmd /c if not exist $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks mkdir $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks" />
+					<Add after="zip -jq9 $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/manifest.xml src/resources/*.xrc" />
+					<Add after="zip -r9  $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/images" />
 					<Mode after="always" />
 				</ExtraCommands>
 				<Environment>
-					<Variable name="TARGET_DEVEL_DIR" value="$(CODEBLOCKS)\..\.." />
-					<Variable name="TARGET_DEVEL_DIR_AC" value="D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller" />
+					<Variable name="TARGET_DEVEL_DIR" value="D:\code\cb\cb_sf_git\cccrash2019" />
 				</Environment>
 			</Target>
 			<Environment>

Note that the post build command has changed, I just removed the last " > nul", which causes some warning.

Code

zip -r9  $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/images

[Pecan]

@ollydbg.

Well, I'm at a loss to figure out why that variable is false for you.

Somehow, clangd_client is seeing Setting/Editor/Code completion as unchecked in your  CB .conf file.

This has nothing to do with your .cbp file. It's the contents of the .conf file that the debugger is using that's causing this problem.

[ollydbg]

@ollydbg.

Well, I'm at a loss to figure out why that variable is false for you.

Somehow, clangd_client is seeing Setting/Editor/Code completion as unchecked in your  CB .conf file.

This has nothing to do with your .cbp file. It's the contents of the .conf file that the debugger is using that's causing this problem.

Hi, Pecan, thanks for the help, I will try to debug this issue and see what happens in the code.

[ollydbg]

I see some code:

Code

    wxString fileURI = fileUtils.FilePathToURI(infilename); //(ph 2022/01/5)
    fileURI.Replace("\\", "/");
    DocumentUri docuri = DocumentUri(fileURI.c_str());
    cbStyledTextCtrl* pCntl = pcbEd->GetControl();
    if (not pCntl) return false;

    // save current length of the file
    m_FileLinesHistory[pcbEd] = pCntl->GetLineCount();

    wxString strText = pCntl->GetText();
    const char* pText = strText.mb_str();           //works

    writeClientLog(wxString::Format("<<< LSP_DidOpen:%s", docuri.c_str()) );

    try { DidOpen(docuri, string_ref(pText, strText.Length()) ); }
    catch(std::exception &err)
    {
        //printf("read error -> %s\nread -> %s\n ", e.what(), read.c_str());
        wxString errMsg(wxString::Format("\nLSP_DidOpen() error: %s\n%s", err.what(), docuri.c_str()) );
        writeClientLog(errMsg);
        cbMessageBox(errMsg);
        return false;
    }

My guess is that:

Code

const char* pText = strText.mb_str(); 

Is this correct?

From my point of view, it should be:

Final word of caution: most of these functions may return either directly the pointer to internal string buffer or a temporary wxCharBuffer or wxWCharBuffer object. Such objects are implicitly convertible to char and wchar_t pointers, respectively, and so the result of, for example, wxString::ToUTF8() can always be passed directly to a function taking const char*. However code such as

Code

const char *p = s.ToUTF8();
...
puts(p); // or call any other function taking const char *

This is from wx document here:  https://docs.wxwidgets.org/trunk/overview_unicode.html

[ollydbg]

I make a clean git svn clone, and it looks like the clangd_client can be active now. I'm not sure why.

I see that the changes in my previous post did solve the error message issue. Here is the whole patch I use.

Code

 clangd_client/clangd_client_wx31_64.cbp    | 11 ++++++-----
 clangd_client/src/LSPclient/src/client.cpp |  4 ++--
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/clangd_client/clangd_client_wx31_64.cbp b/clangd_client/clangd_client_wx31_64.cbp
index 1af073c..eb3e6d4 100644
--- a/clangd_client/clangd_client_wx31_64.cbp
+++ b/clangd_client/clangd_client_wx31_64.cbp
@@ -6,8 +6,8 @@
 		<Option compiler="gcc" />
 		<Build>
 			<Target title="Clangd_Client-uw">
-				<Option output="bin/Clangd_Client" prefix_auto="1" extension_auto="1" />
-				<Option working_dir="bin31_64" />
+				<Option output="$(TARGET_DEVEL_DIR)/src/devel31_64/share/CodeBlocks/plugins/clangd_client" prefix_auto="1" extension_auto="1" />
+				<Option working_dir="$(TARGET_DEVEL_DIR)/src/devel31_64" />
 				<Option object_output=".objs31_64" />
 				<Option external_deps="$(CODEBLOCKS)/libcodeblocks.a;" />
 				<Option type="3" />
@@ -50,12 +50,13 @@
 					<Add before="cmd /c @echo CODEBLOCKS: $(CODEBLOCKS)" />
 					<Add before="cmd /c @echo TARGET_DEVEL_DIR: $(TARGET_DEVEL_DIR)" />
 					<Add before="g++ --version" />
-					<Add after="cmd /c MakeRepoUpload.bat" />
+					<Add after="cmd /c if not exist $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks mkdir $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks" />
+					<Add after="zip -jq9 $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/manifest.xml src/resources/*.xrc" />
+					<Add after="zip -r9  $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/images" />
 					<Mode after="always" />
 				</ExtraCommands>
 				<Environment>
-					<Variable name="TARGET_DEVEL_DIR" value="$(CODEBLOCKS)\..\.." />
-					<Variable name="TARGET_DEVEL_DIR_AC" value="D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller" />
+					<Variable name="TARGET_DEVEL_DIR" value="D:\code\cb\cb_sf_git\cccrash2019" />
 				</Environment>
 			</Target>
 			<Environment>
diff --git a/clangd_client/src/LSPclient/src/client.cpp b/clangd_client/src/LSPclient/src/client.cpp
index 3aa6956..6ae6afa 100644
--- a/clangd_client/src/LSPclient/src/client.cpp
+++ b/clangd_client/src/LSPclient/src/client.cpp
@@ -1053,7 +1053,7 @@ bool ProcessLanguageClient::DoValidateUTF8data(std::string& data)
             char invChar = data[invloc]; // **debugging**
             data[invloc] = ' '; //clear the invalid utf8 char
             wxString msg = "Error: Removed clangd response invalid utf8 char:";
-            msg << "\'" << invChar << "\'";
+            msg << "\'" << wxString::Format("%x", invChar ) << "\'";
             Manager::Get()->GetLogManager()->DebugLog(msg);
         }
     }
@@ -1562,7 +1562,7 @@ bool ProcessLanguageClient::LSP_DidOpen(cbEditor* pcbEd)
     m_FileLinesHistory[pcbEd] = pCntl->GetLineCount();
 
     wxString strText = pCntl->GetText();
-    const char* pText = strText.mb_str();           //works
+    const char* pText = strText.ToUTF8();           // ollydbg 2022-01-16
 
     writeClientLog(wxString::Format("<<< LSP_DidOpen:%s", docuri.c_str()) );
 

Note the change in :

Code

msg << "\'" << wxString::Format("%x", invChar ) << "\'";

is also try to suppress a message alert box(an ascii char to wxString convert, but the invChar value is bigger than 127)

[Pecan]

@ollydbg

Thanks for working on that. I'll give those changes a test tomorrow.

[ollydbg]

I have another patch, it looks like this code should be enabled:

Code

 clangd_client/src/LSPclient/src/client.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/clangd_client/src/LSPclient/src/client.cpp b/clangd_client/src/LSPclient/src/client.cpp
index 6ae6afa..88f5f8f 100644
--- a/clangd_client/src/LSPclient/src/client.cpp
+++ b/clangd_client/src/LSPclient/src/client.cpp
@@ -2443,7 +2443,7 @@ void ProcessLanguageClient::LSP_DidChange(cbEditor* pEd)
 
     // Assure text is UTF8 before handing to DidChange()
     didChangeEvent.text = edText;
-    // didChangeEvent.text = edText.ToUTF8(); Trying to find bad utf8 problem
+    didChangeEvent.text = edText.ToUTF8(); //Trying to find bad utf8 problem
     std::vector<TextDocumentContentChangeEvent> tdcce{didChangeEvent};
     DocumentUri docuri = DocumentUri(fileURI.c_str());
     // **debugging**

[ollydbg]

I see there are many pch files in the folder such as:

C:\Users\[myusername]\AppData\Local\Temp\preamble-c7460b.pch

I think those files is created by clangd, and are there any way to automatically delete them when exit C::B?

EDIT：

clangd writes too much disk : CPP-19402

This discussion looks like the pch can keep in "memory".

[ollydbg]

I see some error message comes from this function call, but I'm not sure why, this happens when I'm editing source file in the C::B editor.

Code

int ProcessLanguageClient::ReadLSPinputLength()
// ----------------------------------------------------------------------------
{
    // this function is driven by readJson() via the thread in transport::loop()
    // and the incoming data is locked.
    // thread was started in constructor and called from readJason() after locking the buffer

    // "Content-Length: <digits>\r\n\r\n"

    if (Has_LSPServerProcess() and m_std_LSP_IncomingStr.length())
    {
        // search for LSP header
        size_t hdrPosn = m_std_LSP_IncomingStr.find("Content-Length: ");
        if (hdrPosn == std::string::npos)
            return wxNOT_FOUND;
        else //have incoming text
        {
            if (hdrPosn != 0)   // verify LSP header is at beginning of buffer
            {
                // Error: header is not at beginning of buffer. Try to fix it.
                // usually caused by clangd invalid utf8 sequence
                wxString msg(wxString::Format("ERROR:%s(): buffLength (%d): Position of content header %d.\n",
                             __FUNCTION__, int(m_std_LSP_IncomingStr.length()), int(hdrPosn)) );
                msg += "Buffer contents written to client log.";
                #if defined(cb_DEBUG)
                    wxSafeShowMessage("Input Buffer error",msg);
                #endif
                msg += "LSP_IncomingStrBuf:\n" + m_std_LSP_IncomingStr + "\n";
                writeClientLog(msg);
                // adjust the data buf to get clangd header at buff beginning
                m_std_LSP_IncomingStr = m_std_LSP_IncomingStr.substr(hdrPosn);
            }

I just enabled the log file, and see the log message(in C:\Users\[myusername]\AppData\Local\Temp\CBclangd_client-1332.log

And the content of the log file are:

Code

19:45:09.758 ERROR:ReadLSPinputLength(): buffLength (549): Position of content header 112.
Buffer contents written to client log.LSP_IncomingStrBuf:
":{"newText":"wxConvAuto($0)","range":{"end":{"character":16,"line":94},"start":{"character":12,"line":94}}}}]}}Content-Length: 414



{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[{"category":"Parse Issue","code":"expected_unqualified_id","codeActions":[],"message":"Expected unqualified-id","range":{"end":{"character":14,"line":97},"start":{"character":12,"line":97}},"relatedInformation":[],"severity":1,"source":"clang"}],"uri":"file:///F:/test/Config.cpp","version":1}}

[ollydbg]

I see there are many pch files in the folder such as:

C:\Users\[myusername]\AppData\Local\Temp\preamble-c7460b.pch

I think those files is created by clangd, and are there any way to automatically delete them when exit C::B?

EDIT：

clangd writes too much disk : CPP-19402

This discussion looks like the pch can keep in "memory".

Here is the patch to fix this pch file issue:

Code

 clangd_client/src/LSPclient/src/client.cpp | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/clangd_client/src/LSPclient/src/client.cpp b/clangd_client/src/LSPclient/src/client.cpp
index 88f5f8f..f6b5eb9 100644
--- a/clangd_client/src/LSPclient/src/client.cpp
+++ b/clangd_client/src/LSPclient/src/client.cpp
@@ -266,6 +266,10 @@ ProcessLanguageClient::ProcessLanguageClient(const cbProject* pProject, const ch
 
     command += " --limit-results=20";              // Limit the number of results returned by clangd. 0 means no limit (default=100)
 
+    // clangd writes too much disk : CPP-19402 https://youtrack.jetbrains.com/issue/CPP-19402
+    // "-pch-storage=memory"
+    command += " -pch-storage=memory";
+
     if (wxDirExists(clangResourceDir))
         command += " --resource-dir=" + clangResourceDir;  // Directory for system includes
 

[ollydbg]

I see there are some code snippet like:

Code

m_MutexInputBufGuard.Lock;

m_MutexInputBufGuard.Unlock();

But in the code, we have to carefully handle the unlocking the wxMutex when return the function body, especially when there are multiply returns.

Is it possible to use the wxMutexLocker, and check the IsOK() function for checking whether it get locked or not.

[Pecan]

I see some error message comes from this function call, but I'm not sure why, this happens when I'm editing source file in the C::B editor.

Code

int ProcessLanguageClient::ReadLSPinputLength()
// ----------------------------------------------------------------------------
{
    // this function is driven by readJson() via the thread in transport::loop()
    // and the incoming data is locked.
    // thread was started in constructor and called from readJason() after locking the buffer

    // "Content-Length: <digits>\r\n\r\n"

    if (Has_LSPServerProcess() and m_std_LSP_IncomingStr.length())
    {
        // search for LSP header
        size_t hdrPosn = m_std_LSP_IncomingStr.find("Content-Length: ");
        if (hdrPosn == std::string::npos)
            return wxNOT_FOUND;
        else //have incoming text
        {
            if (hdrPosn != 0)   // verify LSP header is at beginning of buffer
            {
                // Error: header is not at beginning of buffer. Try to fix it.
                // usually caused by clangd invalid utf8 sequence
                wxString msg(wxString::Format("ERROR:%s(): buffLength (%d): Position of content header %d.\n",
                             __FUNCTION__, int(m_std_LSP_IncomingStr.length()), int(hdrPosn)) );
                msg += "Buffer contents written to client log.";
                #if defined(cb_DEBUG)
                    wxSafeShowMessage("Input Buffer error",msg);
                #endif
                msg += "LSP_IncomingStrBuf:\n" + m_std_LSP_IncomingStr + "\n";
                writeClientLog(msg);
                // adjust the data buf to get clangd header at buff beginning
                m_std_LSP_IncomingStr = m_std_LSP_IncomingStr.substr(hdrPosn);
            }

I just enabled the log file, and see the log message(in C:\Users\[myusername]\AppData\Local\Temp\CBclangd_client-1332.log

And the content of the log file are:

Code

19:45:09.758 ERROR:ReadLSPinputLength(): buffLength (549): Position of content header 112.
Buffer contents written to client log.LSP_IncomingStrBuf:
":{"newText":"wxConvAuto($0)","range":{"end":{"character":16,"line":94},"start":{"character":12,"line":94}}}}]}}Content-Length: 414



{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[{"category":"Parse Issue","code":"expected_unqualified_id","codeActions":[],"message":"Expected unqualified-id","range":{"end":{"character":14,"line":97},"start":{"character":12,"line":97}},"relatedInformation":[],"severity":1,"source":"clang"}],"uri":"file:///F:/test/Config.cpp","version":1}}

This is caused by clangd sending a message (usually a code completion response) with an illegal utf8 char. Clangd sets the message with a header specifying the wrong length which eats up the content-header for the following message.


I have to correct the situation by searching for the content-header.
I have now commented out that message box. The code works to correct the situation so we don't need that anymore.

Thanks for testing and finding these "gotchas" I've forgotten about.