Author Topic: Code completion using LSP and clangd  (Read 299168 times)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Code completion using LSP and clangd
« Reply #135 on: July 16, 2022, 04:41:59 pm »
There are some other issues in rev67, which I never see in rev66.

When editing the comments, I got this message box:

Code
[Window Title]
codeblocks.exe

[Content]
LSP: trying to get AST for non-added document

[OK]

I really don't know why this will happen.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2809
Re: Code completion using LSP and clangd
« Reply #136 on: July 16, 2022, 06:08:55 pm »
There are some other issues in rev67, which I never see in rev66.

When editing the comments, I got this message box:

Code
[Window Title]
codeblocks.exe

[Content]
LSP: trying to get AST for non-added document

[OK]

I really don't know why this will happen.

Acknowledge. I get this when adding a new file to the project. Specifically (for me) MainMenu/File/new/file...

It's caused because CB invokes OnEditorActivated() before setting the editor's ProjectFile member (pointing to the parent project) and ~proxyProject~ grabs it thinking it's a non-project file.

It'll be fixed in the next commit.

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2809
Re: Code completion using LSP and clangd
« Reply #137 on: July 16, 2022, 06:16:54 pm »
OK, it looks like the change in rev67 is here:

Code
@@ -1664,9 +1683,13 @@ bool ProcessLanguageClient::LSP_DidOpen(cbEditor* pcbEd)
     // save current length of the file
     m_FileLinesHistory[pcbEd] = pCntl->GetLineCount();
 
-    wxString strText = pCntl->GetText();
-    //-const char* pText = strText.mb_str();        //works //(ph 2022/01/17)
-    const char* pText = strText.ToUTF8();           //ollydbg  220115 did not solve illegal utf8char
+    #if wxCHECK_VERSION(3,1,5) //3.1.5 or higher
+    wxString strText = pCntl->GetText().utf8_string(); //solves most illegal utf8chars
+    #else
+    //const char* pText = strText.mb_str();         //works //(ph 2022/01/17)
+    wxString strText = pCntl->GetText().ToUTF8();  //ollydbg  220115 did not solve illegal utf8chars
+    #endif
+    const char* pText = strText.c_str();

This code change looks wrong (cause my issue) here.

I just revert this changes in rev67, and rebuild the clangd_client plugin, and my issue is gone!  :)

Thanks, I'll revert that change for the next commit.

Question: Do you NOT use utf8 in CB editors?
The line containing " wxString strText = pCntl->GetText().utf8_string();" was advertised as solving non-utf8 chars in a buffer. And does so for me.

What editor encoding do you use?
« Last Edit: July 16, 2022, 09:34:17 pm by Pecan »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Code completion using LSP and clangd
« Reply #138 on: July 17, 2022, 01:31:17 am »


Question: Do you NOT use utf8 in CB editors?
The line containing " wxString strText = pCntl->GetText().utf8_string();" was advertised as solving non-utf8 chars in a buffer. And does so for me.

What editor encoding do you use?

Yes, I use UTF8 in the C::B editors. My source code is also in UTF8 format.

My question: The function wxString::utf8_string() returned a std::string type, but why you still use a wxString to hold the returned value? Maybe there is a conversion from std::string to wxString?

My OS: Windows7 64bit, but my OS' local language setting is Simplified Chinese. My guess is that some conversion is wrong here, maybe, a std::string to wxString is using some local language setting, and its not necessary. 
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Code completion using LSP and clangd
« Reply #139 on: July 17, 2022, 01:35:19 am »
There are some other issues in rev67, which I never see in rev66.

When editing the comments, I got this message box:

Code
[Window Title]
codeblocks.exe

[Content]
LSP: trying to get AST for non-added document

[OK]

I really don't know why this will happen.

Acknowledge. I get this when adding a new file to the project. Specifically (for me) MainMenu/File/new/file...

It's caused because CB invokes OnEditorActivated() before setting the editor's ProjectFile member (pointing to the parent project) and ~proxyProject~ grabs it thinking it's a non-project file.

It'll be fixed in the next commit.

OK, thanks for the explanation.

Sometimes, when a new editor get opened. (mostly happens when I use find declaration to open a new editor), there is a message shown in the right bottom corner of the C::B's main frame immediately after the editor opened, and it said this file is not parsed yet. I also believe this is a wrong event sequence handling issue.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Code completion using LSP and clangd
« Reply #140 on: July 17, 2022, 09:40:34 am »
Code
[Window Title]
codeblocks.exe

[Content]

LSP_DidChange error: [json.exception.type_error.316] invalid UTF-8 byte at index 4: 0xB4
file:///D:/project/abc.cpp

[OK]

I also got some message like the above in the rev67.

I checked the code change and it looks like:

Code
@@ -2585,7 +2609,12 @@ void ProcessLanguageClient::LSP_DidChange(cbEditor* pEd)
         ///- hasChangedLineCount = true; //(ph 2021/07/26) //(ph 2021/10/11) clangd v13 looks ok
     if ( (hasChangedLineCount) or (lineChangedNbr >= oldLineCount-1) )
         // send the whole editor text to the server.
-        edText = pCtrl->GetText();
+        // Assure text is UTF8 before handing to DidChange()
+        #if wxCHECK_VERSION(3,1,5) //3.1.5 or higher
+        edText = pCtrl->GetText().utf8_string();    //(ph 2022/06/22)
+        #else
+        edText = pCtrl->GetText().ToUTF8();
+        #endif
     else
     {
         // send only the one line that changed. //(send previous, current, and next line maybe??)
@@ -2599,9 +2628,7 @@ void ProcessLanguageClient::LSP_DidChange(cbEditor* pEd)
         didChangeEvent.range = range;
     }
 
-    // Assure text is UTF8 before handing to DidChange()
     didChangeEvent.text = edText;
-    // didChangeEvent.text = edText.ToUTF8(); Trying to find bad utf8 problem
     std::vector<TextDocumentContentChangeEvent> tdcce{didChangeEvent};
     DocumentUri docuri = DocumentUri(fileURI.c_str());
     // **debugging**


This may cause the issue.

EDIT: code in rev67.

Code
    wxString edText;
    // If line count has changed, send full text, else send changed line.
    // Also, special handling for last line of text
        ///- hasChangedLineCount = true; //(ph 2021/07/26) //(ph 2021/10/11) clangd v13 looks ok
    if ( (hasChangedLineCount) or (lineChangedNbr >= oldLineCount-1) )
        // send the whole editor text to the server.
        // Assure text is UTF8 before handing to DidChange()
        #if wxCHECK_VERSION(3,1,5) //3.1.5 or higher
        edText = pCtrl->GetText().utf8_string();    //(ph 2022/06/22)
        #else
        edText = pCtrl->GetText().ToUTF8();
        #endif
    else
    {
        // send only the one line that changed. //(send previous, current, and next line maybe??)
        edText = lineText;
        Range range;
        range.start.line = lineChangedNbr;
        range.start.character = lineBeginCol;
        range.end.line = lineEndNbr+1;
        range.end.character = 0;
        //didChangeEvent.rangeLength = lineText.Length(); dont use. it's been deprecated
        didChangeEvent.range = range;
    }

    didChangeEvent.text = edText;

I think the variable "edText" should have the std::string type, not the wxString.

Also, if possible this line

edText = lineText;

Should also be adjusted because lineText is wxString, while edText is changed to std::string.

Maybe, you should also put the wxString to utf8 format at the last stage as:

didChangeEvent.text = edText.ToUTF8();

« Last Edit: July 17, 2022, 09:50:08 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2809
Re: Code completion using LSP and clangd
« Reply #141 on: July 18, 2022, 07:29:09 pm »
...snipped...
I think the variable "edText" should have the std::string type, not the wxString.

Also, if possible this line

edText = lineText;

Should also be adjusted because lineText is wxString, while edText is changed to std::string.

Maybe, you should also put the wxString to utf8 format at the last stage as:

didChangeEvent.text = edText.ToUTF8();

I will revert DidChange() and DidOpen() to rev66 since that code worked ok.
Then (after the revert commit) I'll make an experimental branch with your suggestions and have you test the changes.

Thanks for the suggestions and thanks for testing.
« Last Edit: July 18, 2022, 07:32:02 pm by Pecan »

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2809
Re: Code completion using LSP and clangd
« Reply #142 on: July 18, 2022, 07:44:37 pm »
Sometimes, when a new editor get opened. (mostly happens when I use find declaration to open a new editor), there is a message shown in the right bottom corner of the C::B's main frame immediately after the editor opened, and it said this file is not parsed yet. I also believe this is a wrong event sequence handling issue.

Yes, I know.
There needs to be some wait set between the time a closed file is opened and sent to clangd and when the code for GoTo Decl/Impl is executed. This probably is also true of FindReferences etc.

I'll enter this comment into the clangd_client repo ticket system.
(Ticket #54 in Clangd_Client repo)
« Last Edit: July 18, 2022, 07:52:05 pm by Pecan »

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: Code completion using LSP and clangd
« Reply #143 on: July 19, 2022, 12:05:14 am »
Pecan, You can blow away my old experimental branch so that there is only one experimental branch.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Code completion using LSP and clangd
« Reply #144 on: July 22, 2022, 07:12:37 am »
Hi, Pecan, I think this patch is still needed to handle the wxString to char* convertion, the char* should be UTF-8 format.

This is against rev69.


Code
-------------------- clangd_client/src/LSPclient/client.cpp --------------------
index c1d5d59..c6f8c22 100644
@@ -2649,7 +2649,7 @@ void ProcessLanguageClient::LSP_DidChange(cbEditor* pEd)
         didChangeEvent.range = range;
     }
 
-    didChangeEvent.text = edText;
+    didChangeEvent.text = edText.ToStdString(wxConvUTF8);
     std::vector<TextDocumentContentChangeEvent> tdcce{didChangeEvent};
     DocumentUri docuri = DocumentUri(fileURI.c_str());
     // **debugging**
« Last Edit: July 22, 2022, 07:15:14 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2809
Re: Code completion using LSP and clangd
« Reply #145 on: July 22, 2022, 05:45:03 pm »
Hi, Pecan, I think this patch is still needed to handle the wxString to char* convertion, the char* should be UTF-8 format.

This is against rev69.


Code
-------------------- clangd_client/src/LSPclient/client.cpp --------------------
index c1d5d59..c6f8c22 100644
@@ -2649,7 +2649,7 @@ void ProcessLanguageClient::LSP_DidChange(cbEditor* pEd)
         didChangeEvent.range = range;
     }
 
-    didChangeEvent.text = edText;
+    didChangeEvent.text = edText.ToStdString(wxConvUTF8);
     std::vector<TextDocumentContentChangeEvent> tdcce{didChangeEvent};
     DocumentUri docuri = DocumentUri(fileURI.c_str());
     // **debugging**

Applied locally. Will test it for awhile.
If ok, it'll be in rev 70.

Thanks

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Code completion using LSP and clangd
« Reply #146 on: July 24, 2022, 06:15:38 am »
I see one bug, I'm not sure it is a clangd bug or our client plugin bug.

When I do code refactoring, I mean I want to rename a variable aaa to bbb.

If there is a line:

Quote
wxLogMessage("aaa", aaa);

Then I will get this:

Quote
wxLogMessage("bbb", aaa);

maybe, clangd does not give us the column information of a variable? It just tell use the line information?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2809
Re: Code completion using LSP and clangd
« Reply #147 on: July 24, 2022, 06:58:20 am »
I see one bug, I'm not sure it is a clangd bug or our client plugin bug.

When I do code refactoring, I mean I want to rename a variable aaa to bbb.

If there is a line:

Quote
wxLogMessage("aaa", aaa);

Then I will get this:

Quote
wxLogMessage("bbb", aaa);

maybe, clangd does not give us the column information of a variable? It just tell use the line information?

Do you mean that the unquoted aaa should have been changed to the unquoted bbb (the second parameter) ?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Code completion using LSP and clangd
« Reply #148 on: July 24, 2022, 07:23:36 am »
I see one bug, I'm not sure it is a clangd bug or our client plugin bug.

When I do code refactoring, I mean I want to rename a variable aaa to bbb.

If there is a line:

Quote
wxLogMessage("aaa", aaa);

Then I will get this:

Quote
wxLogMessage("bbb", aaa);

maybe, clangd does not give us the column information of a variable? It just tell use the line information?

Do you mean that the unquoted aaa should have been changed to the unquoted bbb (the second parameter) ?

Yes.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3352
Re: Code completion using LSP and clangd
« Reply #149 on: July 29, 2022, 10:04:45 pm »
Hi, not quite sure what is happening, and i did not investigate, but i got a crash and this backtrace, just wanted to note it here:
Code
AddrPC           Params
00007FFC87E084EB 00000158724B49E0 0000002690BFE4F0 00000000FFFF86AD  clangd_client.dll!Parser::LSP_ParseSemanticTokens
00007FFC87E13403 00000158724B49E0 00000158771FF080 0000000000000000  clangd_client.dll!Parser::OnLSP_RequestedSemanticTokensResponse
00007FFC87DB64F1 000001586D6EB3F0 00000158771FF080 0000002690BFEED0  clangd_client.dll!ClgdCompletion::OnLSP_Event
00007FFC2F35398C 000001586CB04130 000001586D6EB3F0 00007FFC2A605B5E  wxmsw315ud_gcc_custom.dll!wxAppConsoleBase::HandleEvent
00007FFC2F353A0D 000001586CB04130 000001586D6EB3F0 000001586D50E880  wxmsw315ud_gcc_custom.dll!wxAppConsoleBase::CallEventHandler
00007FFC2F42DBF4 000001586D50E8E0 000001586D6EB3F0 00000158771FF080  wxmsw315ud_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatchesId
00007FFC2F42E90A 000001586D6EB3F0 00000158771FF080 00000158771FF080  wxmsw315ud_gcc_custom.dll!wxEvtHandler::SearchDynamicEventTable
00007FFC2F42E07A 000001586D6EB3F0 00000158771FF080 0000002690BFF0C0  wxmsw315ud_gcc_custom.dll!wxEvtHandler::TryHereOnly
00007FFC2FC31040 000001586D6EB3F0 00000158771FF080 0000015870E46080  wxmsw315ud_gcc_custom.dll!wxEvtHandler::TryBeforeAndHere
00007FFC2F42DE83 000001586D6EB3F0 00000158771FF080 000001586D6EB3F0  wxmsw315ud_gcc_custom.dll!wxEvtHandler::ProcessEvent
00007FFC2F42DF7F 000001587244F920 00000158771FF080 0000002690BFF120  wxmsw315ud_gcc_custom.dll!wxEvtHandler::DoTryChain
00007FFC2F42DF10 000001587244F920 00000158771FF080 000001586CAB5FB0  wxmsw315ud_gcc_custom.dll!wxEvtHandler::ProcessEventLocally
00007FFC2F42DE95 000001587244F920 00000158771FF080 00000158773507D0  wxmsw315ud_gcc_custom.dll!wxEvtHandler::ProcessEvent
00007FFC2F42DA98 000001587244F920 0000000000000000 000001586CB042F8  wxmsw315ud_gcc_custom.dll!wxEvtHandler::ProcessPendingEvents
00007FFC2F3535ED 000001586CB04130 00007FFC2F4FFA99 000001586CB04130  wxmsw315ud_gcc_custom.dll!wxAppConsoleBase::ProcessPendingEvents
00007FFC2F37666E 000001587479F1C0 0000000000000040 0000002690BFF380  wxmsw315ud_gcc_custom.dll!wxEventLoopManual::ProcessEvents
00007FFC2F37677D 000001587479F1C0 000001587479F1D4 000001587479F100  wxmsw315ud_gcc_custom.dll!wxEventLoopManual::DoRun
00007FFC2F376234 000001587479F1C0 000001586CB042F0 000001587479F1C0  wxmsw315ud_gcc_custom.dll!wxEventLoopBase::Run
00007FFC2F352E88 000001586CB04130 0000000000000055 000001586CAB5FB0  wxmsw315ud_gcc_custom.dll!wxAppConsoleBase::MainLoop
00007FFC2F352BB2 000001586CB04130 00007FFCDC9A03E0 00007FFCDC980000  wxmsw315ud_gcc_custom.dll!wxAppConsoleBase::OnRun
00007FFC2F59BA17 000001586CB04130 000001586CB00000 0000000000000000  wxmsw315ud_gcc_custom.dll!wxAppBase::OnRun
00007FF742BF5C8A 000001586CB04130 00007FFC3028D7B0 000001586B0CC050  codeblocks.exe!CodeBlocksApp::OnRun
00007FFC2F3A0FA2 00007FFC3028D7B0 000001586B0CC050 000001586CAAF898  wxmsw315ud_gcc_custom.dll!wxEntryReal
00007FFC2F436B60 00007FFC3028D7B0 000001586B0CC050 0000000000000000  wxmsw315ud_gcc_custom.dll!wxEntry
00007FFC2F436C45 00007FF742BF0000 0000000000000000 000001586B0C3987  wxmsw315ud_gcc_custom.dll!wxEntry
00007FF742BF2544 00007FF742BF0000 0000000000000000 000001586B0C3987  codeblocks.exe!WinMain
00007FF742BF13B1 0000000000000000 0000000000000000 0000000000000000  codeblocks.exe!__tmainCRTStartup
00007FF742BF14C6 0000000000000000 0000000000000000 0000000000000000  codeblocks.exe!WinMainCRTStartup
00007FFCDC997034 0000000000000000 0000000000000000 0000000000000000  KERNEL32.DLL!BaseThreadInitThunk
00007FFCDD9E2651 0000000000000000 0000000000000000 0000000000000000  ntdll.dll!RtlUserThreadStart