Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Plugins development => Topic started by: Pecan on February 20, 2021, 09:13:50 pm

Title: Code completion using LSP and clangd
Post by: Pecan on February 20, 2021, 09:13:50 pm
I have developed a clangd CB plugin client using Language Server Protocol (LSP).

It's working well.
However, it requires clangd version 12 to work even better.

What is the CodeBlocks policy of distributing an executable, say, clangd.exe v12 with the usual CodeBlock distribution.

CodeLite also distributes clangd with its LSP plugin.

Request for Comments.
Title: Re: Code completion using LSP and clangd
Post by: oBFusCATed on February 20, 2021, 11:33:10 pm
Shipping clangd is a windows thing. I don't think we should do it. Just detect the LLVM installation and use it. Or prompt the user to install it.
Title: Re: Code completion using LSP and clangd
Post by: oBFusCATed on February 20, 2021, 11:34:07 pm
What is the purpose of this LSP plugin? Just provide faster error messages?
In my experience implementing a CC plugin with LSP doesn't work well (don't remember the details).
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on February 21, 2021, 01:24:00 am
Hi, Pecan, good work!

For me, I think distribute a clangd.dll is OK, or we can distribute a separate plugin package(include the plugin and the clangd.dll). Then the user can just download and unpack it.

If I remember correctly, there is an option in CodeLite IDE, which can let user select which clangd.dll it will use.

Do you have a code repo which others can build/test this plugin?

What is the purpose of this LSP plugin? Just provide faster error messages?
In my experience implementing a CC plugin with LSP doesn't work well (don't remember the details).
Using the LSP, I think at least code completion list will be more precise than the native CC.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on February 22, 2021, 05:22:56 pm
Shipping clangd is a windows thing. I don't think we should do it. Just detect the LLVM installation and use it. Or prompt the user to install it.

Just detecting the installation and using its clangd does not work. I've already tried that. The llvm's versions up to version 8 have no reliable clangd for a CB plugin.

The clangd in llvm versions 8 through 11 are so different from  version 12 that the plugin using them would require two separate plugins or spaghetti code to support them.

CB will have to pick some version between 10 through 12 and program towards that.

I've chosen to work with clangd v12. It can reload the compile commands database without having to kill/restart the LSP server when a user opens a new file.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on February 22, 2021, 05:27:11 pm
What is the purpose of this LSP plugin? Just provide faster error messages?
In my experience implementing a CC plugin with LSP doesn't work well (don't remember the details).

Using  LSP "workspace/publishDiagnostics" with clangd is working well for me. I'ts halved my development time. I no longer have to compile the code to find the errors. They're presented to me as I enter the code or save it.

Clangd supports all the LSP calls plus it's own extensions. So far, I've implemented all CB CC's features via LSP calls except the Symbols browser. Though that's possible to do.

LSP/Clangd is primarily focused on the translation unit (TU). But its still possible to provide a bigger picture by asking for workspace symbols. I haven't tried that (yet).
Title: Re: Code completion using LSP and clangd
Post by: Pecan on February 22, 2021, 05:38:30 pm
...<snip>...
If I remember correctly, there is an option in CodeLite IDE, which can let user select which clangd.dll it will use.

Do you have a code repo which others can build/test this plugin?
...<snip>...

CodeLite allows the user to choose the clang to use, but it prioritizes using clangd v10, which it distributes in dir Runtime/lsp.
Later versions of clangd are very independent of llvm/clang. For example clangd v12 can use many llvm/clang versions.

I don't have a repository yet. Things are still very experimental. But I'll work on that.
Title: Re: Code completion using LSP and clangd
Post by: oBFusCATed on February 22, 2021, 06:05:26 pm
For the record the latest version of clang is 11.1. clang/llvm 12 is some night build. This would fail dramatically on linux if you base your development on this version at least until clang/llvm 12 is actually released in the summer.

Clangd supports all the LSP calls plus it's own extensions. So far, I've implemented all CB CC's features via LSP calls except the Symbols browser. Though that's possible to do.
I know that, CC features required changes to the CCManager API last time I've tried to do it. At least if you want to provide good experience.
I don't think implementing the diagnostics in another info pane tab is a good idea, but this requires another CCManager change which would allow in-editor-notifications.

Also I don't know how is using a mix of clang/gcc these days, but it was a problem in the past.
Also I'm not sure if it is a good idea to base your LSP plugin on only one implementation. If you're going to do it name it clangd-lsp. :)
Title: Re: Code completion using LSP and clangd
Post by: Pecan on February 22, 2021, 06:21:33 pm
For the record the latest version of clang is 11.1. clang/llvm 12 is some night build. This would fail dramatically on linux if you base your development on this version at least until clang/llvm 12 is actually released in the summer.

I know that, CC features required changes to the CCManager API last time I've tried to do it. At least if you want to provide good experience.
I don't think implementing the diagnostics in another info pane tab is a good idea, but this requires another CCManager change which would allow in-editor-notifications.

Also I don't know how is using a mix of clang/gcc these days, but it was a problem in the past.
Also I'm not sure if it is a good idea to base your LSP plugin on only one implementation. If you're going to do it name it clangd-lsp. :)

In the furture, I'm sure we can generalize to a LSP UI. For the time being, I'm coding a plugin to concentrate learning and experimentation. I'll be happy to name it clangd-lsp. That's a good name.
 
Good to know clangd v12 rc1 may be out in the summer. I don't expect this plugin will be ready to publish any sooner.
Title: Re: Code completion using LSP and clangd
Post by: Lazauya on May 02, 2021, 05:02:42 am
Do you have a public repo? I'm super interested in contributing to this plugin.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on May 02, 2021, 06:34:47 am
Do you have a public repo? I'm super interested in contributing to this plugin.

I do not yet have a public repo.
The current code is very experimental and in a raw combination of both used and deprecated code, full of unreleasable log writes.

I will make a repo when I can clean up the code  and after I do more debugging.
I'm working hard at it.

But thanks for the offer. 
Title: Re: Code completion using LSP and clangd
Post by: Lazauya on May 02, 2021, 06:46:02 pm
The current code is very experimental and in a raw combination of both used and deprecated code, full of unreleasable log writes.

I will make a repo when I can clean up the code  and after I do more debugging.

I don't mind. I could definitely help in debugging, test writing, code cleaning, etc. This is the kind of thing I've been looking for, and I'd very much like to accelerate development of it as much as possible, if you would let me.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 07, 2022, 06:58:50 am
@ AndrewCot:

What is your concurrently processing threads set to at
Settings->clangd_client-> C/C++parser ->Concurrently parsing threads.

It might not be there. I don't know when I implementd that.

Clangd seems to need 3-4 seconds to index a 2-3k line file, but it's very fast at the user response level once a file is indexed

In the new implementation (not uploaded yet) I set clangd parsing the active editor first, then the other open editors, then recently modified files, then all the others.

That way, you do not have to wait to begin using clangd. Just open the editor you want to work in. It gets sent to the front of the line, and it's should be parsed and indexed in 3 or 4 seconds.

Clangd can parse, Goto (decls and defs), find references, rename symbols etc. etc. all at the same time. No need to wait. Just open the file.

You do not need to wait for files to parse like the older CodeCompletion. Just activate the file you want to work in. The non-opened files parse in another address space at their own pace. If you switch editors, clangd-client knows whether it's been parsed or not. If it hasn't been, it's send immediately to clangd which parses it ahead of any background files.

This works great as long as at least one clangd thread is available.

The trick I found was to set the clangd threads to no more than 1/2 of the system (cpu) threads (hard coded in client allocation). I've given clangd 4 threads of my i7's 8 , and set concurrent parsing threads to 2. That usually guarantees that a thread will be available for priority (active editor) parses.  I'm able to compile, background parse, GoTo, Find references, code complete, show function, etc all at the same time.

My parsing times look 1/2 of yours.
What's your hardware look like? What cpu, how many threads?

But all of this will come to light with testing and discussion (as you are doing now). Thanks.

Go easy on me for a little longer. I'm running into problems.
Clangd is passing back 3 byte UTF8 chars during code completions, crashing json parses. I'm stalled trying to fix that. I found the culprit today, Hopefully I can fix it tomorrow .

@pecan
It appears that processing the files is slow as per the logged info shown below:

Opening D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\CodeBlocks_wx31_64.cbp
Done.
cbProject::Open took: 0.725 seconds.
ProjectManager::SetProject took: 1.686 seconds.
ProjectManager::LoadProject took: 2.732 seconds.
ParseManager::CreateParser: Finish creating a new parser for project 'Code::Blocks wx3.1.x (64 bit)'
LSP opened editor parse finished for D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\plugins\clangd_client\src\LSPclient\include\protocol.h (1441 ms) (496 more)
LSP opened editor parse finished for D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\plugins\clangd_client\src\codecompletion\parser\LSP_tokenizer.h (3194 ms) (495 more)
LSP opened editor parse finished for D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\plugins\clangd_client\src\codecompletion\codecompletion.cpp (4342 ms) (494 more)
LSP opened editor parse finished for D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\plugins\clangd_client\src\LSPclient\include\client.cpp (3738 ms) (493 more)
LSP opened editor parse finished for D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\include\compilerfactory.h (660 ms) (492 more)
LSP opened editor parse finished for D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\plugins\clangd_client\src\asyncprocess\asyncprocess.cpp (2228 ms) (491 more)
LSP opened editor parse finished for D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\sdk\configmanager-revision.cpp (3543 ms) (490 more)
LSP opened editor parse finished for D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\sdk\cygwin.cpp (3534 ms) (489 more)
LSP background parsing finished for: D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller\src\src\environmentsettingsdlg.cpp (3262 ms) (488 more)

Is there a way of speeding this up? Any ideas on where to look or what to check or do?
I am using MSYS2 with the latest updates as of a few hours ago (this resulted in waht looked like a big MinGW 64 9.00 run time update) and GCC 11.2.0.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 07, 2022, 08:06:13 am
Does anyone know of a good UTF8 char validator.
Clangd is handing back (occasionally) a 3 byte utf8 char.
It's blowing nlohmann json parser out of the water.

I'd like to scan the clangd stdout responses for invalid UTF8.
They're weird. \xE2\x80\xA6 right in the middle of an empty function() param area. Also that strange period half way up the middle of a char area. Like an item list indicator. \xE2\x80\xA2.

When, in the initialization, we tell clangd to use "Only UTF8", it's supposed to mean 8 bits only, not 24 bits. And evidently they don't know it, because they set the response length as if all chars are 8bit.

Thanks
Title: Re: Code completion using LSP and clangd
Post by: AndrewCot on January 07, 2022, 08:26:32 am
@pecan.

I am not trying to be a PITA, but trying to give feedback and what I have seen when I see it so I do not forget. Even as it is now it's still a big improvement on the existing codecompletion code. The changes I have made are in the https://github.com/acotty/codeblocks_sf/tree/AC-WindowsInstaller branch if you need or want to look at what I have done.

My setup is:
    Windows 10 21H2 with latest updates (checked last night and it was up to date)
    32GB DDR4 3600Mhz ram (2x16gb sticks)
    AMD Ryzen 7 5700G (8 core plus 8 HT for a total of 16, base 3800 Mhz boost 4650)
    No separate GPU
    C: & D: are on a Samsung SSD 980 pro.
    MSYS2 MinGW64 GCC 11.2.0
   C: has the MSYS 2 compiler on it.
   D: has the CB and other source code on it.
 
I had 1 concurrent parsing thread setup, but changed this about 2 hours ago to 8 and it speed up the overall time, but the time shown per file was still about the same.

As for the UTF8 I have no idea, but will do some searching after dinner (OZ Sydney time).
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 07, 2022, 04:35:18 pm
@pecan.

I am not trying to be a PITA, but trying to give feedback and what I have seen when I see it so I do not forget....snipped...

I understand. And I really appreciate your doing all this work. I really need to set up some sort of bug tracking.
I'll work on it.
Thanks.
Title: Re: Code completion using LSP and clangd
Post by: AndrewCot on January 09, 2022, 01:55:05 am
@pecan
I have updated to the r8 code and it works better than the r5 as expected. As expected it's working allot better than the r5.  It's probably worth putting up on the SF repo the plugin binary/zip file so people can test it with the nightly build. As for bug tracking can you enable the tickets tab in SF?

Below is my updated list of things (queries, potential to do items, work I did ) for the plugin

Queries I have:
1) What is the clangd_client toolbar for? It is greyed out when I enable it and neither drop down is populated/enabled.


Changes that still need to be done that I spotted are:
1) Fix one compiler warning left (when built as a core plugin):
    src\plugins\clangd_client\src\LSPclient\lspdiagresultslog.cpp:10:25:
        warning: .objs31_64\include/sdk_precomp.h.gch: not used because `EXPORT_LIB' not defined [-Winvalid-pch]
        10 | #include "sdk_precomp.h"
2) Add support for building under Linux, Mac and Windows x64 & x86 as a core plugin.
3) Add plugin files to the Debain, Mac and windows installer scripts/process.
4) Test installers works and the plugin works after installing using installer
5) With r8 the setup requires you to specify the C:\msys64\clang64\bin\clangd.exe , not the direxctory it is in.
    The wxwidget group text needs to change to match this change.
6) The LSP plugin column widths do not take up the full width of the lopg dialog on my FDH screen (1920 x 1080). The
    other logs also have this issue, so it may not be easy to fix.

The changes done to incorporate plug as a core plugin follow that may/will need to be incorporated at some stage:
1. Included the SF src in the src\plugins\clangd_client directory (same level as the codecompletion core plugin)
2. Incorporate the windows cpb into the src\CodeBlocks_wx31_64.cbp with changes
3. Update code for use with PCH as per src\CodeBlocks_wx31_64.cbp (client.cpp and codecompletion.cpp changed)
4. Updated my unofficial NSIS installer to add the plugin

The following is what is needed to get the plugin working with MSYS2:
1) Install MSYS 2  clang packages:
    pacman -S mingw-w64-clang-x86_64-toolchain
2) Configure for use:
    a) In the Settings->editor select Clangd_Client settings.
    b) In the C/C++ parser tab change the "LLVM directory" to specify the clangd.exe you have installed:
            C:\msys64\clang64\bin\clangd.exe
Title: Re: Code completion using LSP and clangd
Post by: AndrewCot on January 09, 2022, 11:21:31 am
FYI. The following tickets have a codecompletion tag and are currently open, so when finished hopefully a heck of allot of these will get resolved:

Ticket#    Summary
1057   Code completion header search path disrespect build target/platform
1028   Tab Code Completion bug
859   "Implementation of" is searching in all projects instead of current one
820   "Find references of" doesn't always work
793   Code completion stop working with enum defined inside function body
790   Code Completion lacks cstdint typedefs when prefixed with std::
788   Code completion doesn't account for shared_ptr, unique_ptr, etc
716   Code::Blocks 17.12 CodeCompletion slows down the editor dramatically
692   Intellisense not working unless include paths provided plus CB slash auto-conversion in paths
679   Right-click on <header.h>: CB looks in GCC path first regardless of chosen compiler
576   Add case sensitivity for header/source search under case-sensitive systems
518   Source parser error (SVN 11087
498   Parsing of the doxygen comment "see also" command error
474   Autocompletion does not work by using const pointer
388   Code completition doesn't work if used C++11 raw string literal
298   Code completion only works in gcc projects
286   CC crash when closing workspace
283   Bug e Outros 15.xx Brasil C++ Previso Erros
281   Let's CC support Conditional Code :: Blocks. highlighting in the editor
237   CC will not show all members in the namespace std
236   No code completion in array-of-struct
224   Duplication in documentation tooltip
213   ParserThread::GetBaseArgs function need to be refactored
212   space in Tokenizer::ReadParentheses issue
197   thread safty issue of CodeCompletion plugin
193   running cctest is too slow
177   Code completion fails with enum variables defined/declared along with the enum types
172   code completion doesn-t work with auto key word
156   CC failed parsing the char (*varname)[7]; (a pointer to an array of characters) definition
145   Code completion ignores parameters of catch-clauses
127   Code completion fails with complex declarations/definitions
88   Code completion mouse hover tooltip is always active
83   codecompletion's setting not saved correctly
47   Show call tip of the constructor function
43   Code complete - keyword final
31   restrict keyword and auto-completion
159   Code Completion fails for C++ functions marked "override"
158   Code completion fails with the "using Alias = Type" syntax in C++11
33   C++11 Initializer lists fool code completion
211   CCTest for the parser logic
313   CB hangs when loading a project for the first time after new launch
256   CC's Toolbar does not show function correctly inside the namespace scope
312   CC's symbol tree does not show correctly unless collapse and expand again
438   Popup hint for std::vector::end() causes Codeblocks to freeze
311   C::B hangs when exits
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 09, 2022, 05:49:04 pm
@pecan
I have updated to the r8 code and it works better than the r5 as expected. As expected it's working allot better than the r5.  It's probably worth putting up on the SF repo the plugin binary/zip file so people can test it with the nightly build. As for bug tracking can you enable the tickets tab in SF?
FYI. The following tickets have a codecompletion tag and are currently open, so when finished hopefully a heck of allot of these will get resolved:

@AndrewCot

Damn !! Thanks for all this hard work.
I'll start working on this today.

I'm not sure right now, but I think I have to enable tickets user by user. I seem to remember a lot of ads showing up in the tickets if it's opened for everyone. I could be wrong.

Would you send me a private message with your SourceForge login ID. (not your password). I'll find a way to authorize you for tickets. (i"m still not very adept at "SorceForging".

Title: Re: Code completion using LSP and clangd
Post by: killerbot on January 09, 2022, 07:51:02 pm
I would suggest to add it to the opfficial Cb repo at location next to the current cc.
Title: Re: Code completion using LSP and clangd
Post by: sodev on January 10, 2022, 06:13:05 pm
Does anyone know of a good UTF8 char validator.
Clangd is handing back (occasionally) a 3 byte utf8 char.
It's blowing nlohmann json parser out of the water.

I'd like to scan the clangd stdout responses for invalid UTF8.
They're weird. \xE2\x80\xA6 right in the middle of an empty function() param area. Also that strange period half way up the middle of a char area. Like an item list indicator. \xE2\x80\xA2.

When, in the initialization, we tell clangd to use "Only UTF8", it's supposed to mean 8 bits only, not 24 bits. And evidently they don't know it, because they set the response length as if all chars are 8bit.

Thanks

I'm not sure if i fully understand what you have written, but it seems you are mixing some things. UTF8 is a variable length encoding, while each of its code units is 8 bit, a valid encoding can be comprised of 1 to 6 code units. Such a sequence gets decoded to a 32 bit code point, which i think you refer to as character. Both your mentioned sequences above are totally valid UTF8.
Title: Re: Code completion using LSP and clangd
Post by: BlueHazzard on January 10, 2022, 08:04:49 pm
i was curious what symbol it is:
https://www.compart.com/en/unicode/U+2026

ps.: I can not post the symbol here, it seems the forum software is not utf8 conform :)
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 10, 2022, 09:44:51 pm
A clangd_client plugin cbPlugin package is available for the 31 December nightly at:
https://sourceforge.net/projects/cb-clangd-client/files/

Download the clangd_client.cbplugin file.

In CodeBlocks MainMenu=>plugins=>Manage plugin=> disable CodeCompletion .

Navigate to the downloaded file (clangd_client_plugin.zip) and unzip it to a temporary folder.
Copy the clangd_client.zip file to the CodeBlocks nightly ...\share\CodeBlocks\ folder.
Copy the clangd_client.dll file to the CodeBlocks nightly ...\share\Codeblocks\plugins\ folder
Restart the nightly CodeBlocks.

To uninstall, delete the above files from the CodeBlocks Nightly folders.

If you uninstall it using the "Manage plugins" menu , a crash may occur if a workspace is active.
Close the workspace first before uninstalling to avoid the crash.
We'll apply a fix for that soonest.

If you previously installed the clangd_client.cbplugin version, delete it with the commands:
 del %APPDATA%\CodeBlocks\share\codeblocks\plugins\clangd_client.dll
 del %APPDATA%\CodeBlocks\share\codeblocks\clangd_client.zip
The .cbplugin version became entirely too invasive.

Thanks to AndrewCot.
Title: Re: Code completion using LSP and clangd
Post by: AndrewCot on January 12, 2022, 06:52:25 am
Due to incompatibilities between plugins built with the latest MSYS2 and the nightly releases that cause then plugin to not load (unless you update the Mingw64 DLL's) there is an update of the plugin available that has been built using Mingw64 8.1.0. This update works with the nightly builds and also locally build C::B built with MSYS2 at:
    https://sourceforge.net/projects/cb-clangd-client/files/

The install docs have been updated with a bunch of options for both the plugin and the clangd.exe.
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 12, 2022, 12:01:57 pm
Due to incompatibilities between plugins built with the latest MSYS2 and the nightly releases that cause then plugin to not load (unless you update the Mingw64 DLL's) there is an update of the plugin available that has been built using Mingw64 8.1.0. This update works with the nightly builds and also locally build C::B built with MSYS2 at:
    https://sourceforge.net/projects/cb-clangd-client/files/

The install docs have been updated with a bunch of options for both the plugin and the clangd.exe.

There is a typo in the file: https://sourceforge.net/projects/cb-clangd-client/files/Plugin_Install_Package/Windows_x64/0.2.5%20-%2012JAN2022/Windows-LLVM-ClangD-Install-Readme.txt/download

Quote
instructions below for the specific compiler you haev installed:



Title: Re: Code completion using LSP and clangd
Post by: AndrewCot on January 12, 2022, 12:20:11 pm
File in SVN has been updated, so the next update will include the typo fixed.

Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 12, 2022, 04:07:20 pm
File in SVN has been updated, so the next update will include the typo fixed.

There is another typo:

Quote
Windows Clangd-Client Plugin install process:
============================================
1) Install the LLVM or Clangd.exe as documented in the following file:
    Windows-LLVM-ClangD-Install-Readme.txt
2) Disable the Code completion plugin as follows:
    a) Open the Plugin manager via the Code::Blocks "MainMenu=>Plugins=>Manage plugins..." menu
    b) In the Manage Plugin dialog do the following:
       i) Find and select the "Code completion" plugin via it's title
       ii) Press the "Disable" button on the right near the top
       iii) If you get any errors please try again.
3) Install the Clangd-Client Plugin using one of the following options, which are documneted later in this readme file:
    a) Install via the Plugin Manager
    b) Manaully install the plugin files
3) Configure the Clangd-Client Plugin for use as follows:
    a) Select the "MainMenu=>Settings->Editor..." menu
    b) In the list on the left click/select the "clangd_client" option.
    c) In the "C/C++ parser" tab change the "Specify clangd executable to use" to reference the clangd.exe you installed via step 1) above.
         Some examples of this could be:
            C:\msys64\clang64\bin\clangd.exe
            C:\msys64\clang32\bin\clangd.exe
            C:\LLVM\bin\clangd.exe
            C:\comilers\cmang\clangd.exe

You can see there are two "3)" sub-sections.

BTW:

I download the package: clangd_client.zip (https://sourceforge.net/projects/cb-clangd-client/files/Plugin_Install_Package/Windows_x64/0.2.5%20-%2012JAN2022/clangd_client.zip/download), but when I try to "install" the plugin by the plugin manager, it need a file "*.cbplugin", while I see only this zip file, so I need to rename the file extension?
Title: Re: Code completion using LSP and clangd
Post by: Miguel Gimenez on January 12, 2022, 04:22:10 pm
File clangd_client.zip is a mess, it contains two levels of older copies of itself.

You just need the DLL and a zip containing the resources.
Title: Re: Code completion using LSP and clangd
Post by: Miguel Gimenez on January 12, 2022, 05:56:44 pm
I have just created a wiki entry for the .cbplugin file format: https://wiki.codeblocks.org/index.php/Code::Blocks_Plugins#Format_of_.cbplugin_files (https://wiki.codeblocks.org/index.php/Code::Blocks_Plugins#Format_of_.cbplugin_files)

It is a draft based on the source code, if somebody has more information or spots an error please edit it.
Title: Re: Code completion using LSP and clangd
Post by: AndrewCot on January 13, 2022, 07:58:01 am
@ollydbg In the plugin manager you specify the zip file instead of a .cbplugin and it works

@Miguel Thanks for the info. I need mod the project file to check and delete the zip file otherwise you get the zip file stored in it multiple times (probably recursive until I delete the zip file).

As for the .cbplugin format please can you modify the page you did and remove the info as it is duplicated on the following page along with allot more info for a developer:
http://wiki.codeblocks.org/index.php/Managing_Plug-in_Resources

I would suggest on the page you modify the General section where the If you are developing plugins to the following as is has a few issues:
If you are developing plugins:
* Please read the "http://wiki.codeblocks.org/index.php/Managing_Plug-in_Resources" page for creating and producing a plugin.
* You will need to store the source code for the plugin for other developers to have access to it. You can do this on SourceForge or Github or your favorite source code repository web site.
* Setup your own webpage on a file sharing platform that can be used for documenting and distribution of the plugin. This will allow users to download the plugin to install on their Code::Blocks installation.

Once you have a working and production ready plugin for other developers to use then you can do the following:
* Read the "https://wiki.codeblocks.org/index.php/Announcement_for_plugins/patches" page for information on how to announce you have a plugin on the Code::Blocks forum.
* In your announcement in the "https://forums.codeblocks.org/index.php/board,14.0.html" forum please also ask for your plugin to be added to the Code::Blocks "https://wiki.codeblocks.org/index.php?title=Code::Blocks_Plugins" wiki page with the details you would like to see on the page.



One issue with this is that the referenced pages are not up to date either, so when do you stop..... Also it seems that some of the info is on multiple pages or it may be very similar info which is not right as later if one page gets updated and not the other then there may be issues.
Title: Re: Code completion using LSP and clangd
Post by: AndrewCot on January 13, 2022, 10:49:17 am
Repackaged plugin is available from the following page:
https://sourceforge.net/projects/cb-clangd-client/files/Plugin_Install_Package/Windows_x64/0.2.6%20-%2013JAN2022/ (https://sourceforge.net/projects/cb-clangd-client/files/Plugin_Install_Package/Windows_x64/0.2.6%20-%2013JAN2022/)
Title: Re: Code completion using LSP and clangd
Post by: Miguel Gimenez on January 13, 2022, 11:47:05 am
I have updated the Wiki with your suggestions
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 13, 2022, 12:21:43 pm
@ollydbg In the plugin manager you specify the zip file instead of a .cbplugin and it works

I have rename the .zip file to .cbplugin, because I can't select a zip file in the plugin manager to install the new plugin(NO zip files will be shown in the file select dialog).


In the Windows-LLVM-ClangD-Install-Readme.txt

Quote
MSYS2 Compiler - MinGW64
-------------------------
a) Install MSYS 2 clang packages via the msys2.exe bash shell:
    pacman -S mingw-w64-clang-x86_64-toolchain

Run this command, I got this:

Code
# pacman -S mingw-w64-clang-x86_64-toolchain
:: There are 22 members in group mingw-w64-clang-x86_64-toolchain:
:: Repository clang64
   1) mingw-w64-clang-x86_64-clang  2) mingw-w64-clang-x86_64-clang-analyzer
   3) mingw-w64-clang-x86_64-clang-tools-extra
   4) mingw-w64-clang-x86_64-compiler-rt  5) mingw-w64-clang-x86_64-crt-git
   6) mingw-w64-clang-x86_64-headers-git  7) mingw-w64-clang-x86_64-libc++
   8) mingw-w64-clang-x86_64-libc++abi  9) mingw-w64-clang-x86_64-libmangle-git
   10) mingw-w64-clang-x86_64-libunwind
   11) mingw-w64-clang-x86_64-libwinpthread-git  12) mingw-w64-clang-x86_64-lld
   13) mingw-w64-clang-x86_64-lldb  14) mingw-w64-clang-x86_64-llvm
   15) mingw-w64-clang-x86_64-make  16) mingw-w64-clang-x86_64-mlir
   17) mingw-w64-clang-x86_64-openmp  18) mingw-w64-clang-x86_64-pkgconf
   19) mingw-w64-clang-x86_64-polly  20) mingw-w64-clang-x86_64-tools-git
   21) mingw-w64-clang-x86_64-winpthreads-git
   22) mingw-w64-clang-x86_64-winstorecompat-git

Enter a selection (default=all):


Do I need to only select the option 1?
Or do I need to use the all?


EDIT:

clangd is contained in mingw-w64-clang-x86_64-clang-tools-extra, so normally we should use option 3.
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 13, 2022, 01:16:30 pm
I also see the "LSP_DidOpen() error: [json.exception.type_error.316] invalid UTF-8 byte at index 1019: 0xB5" message box.

A quick search gives this related discussion:

[json.exception.type_error.316] invalid UTF-8 byte at index 1: 0xC3 Issue #1383 nlohmann/json (https://github.com/nlohmann/json/issues/1383)

Invalid UTF-8 byte at index 1: 0x65 Issue #1831 nlohmann/json (https://github.com/nlohmann/json/issues/1831)

I'm not sure how we pass the source file to clangd, maybe the file is not encoded in UTF8, but you pass the file to clangd as a UTF8 format file, and in this case, clangd return a string which is not encoded in UTF8.  And finally we get the exception message box.
Title: Re: Code completion using LSP and clangd
Post by: BlueHazzard on January 13, 2022, 06:10:45 pm
What is the encoding of your source file?
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 13, 2022, 08:57:58 pm
What is the encoding of your source file?

The data sent to clangd comes directly from the editors' wxTextctrl.
The encoding in the status bar for my CB shows windows-1252.
This problem only occurs on the incoming textDocument/completion response.
The data going out never gets a json exception. Only the data coming in.

How can I determine what the encoding is for the outgoing data?

Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 13, 2022, 10:11:47 pm
For the time being, I've coded a hack for the illegal utf8 problem.

It clears the illegal utf8 char and continues with the code completion.
The completion suggestions are correct so I'm ok with it.
I'll figure it out some other time.
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 15, 2022, 04:08:33 am
I see your latest commit here in rev 27

* clangd_client - Code to remove invalid utf8 chars from clangd responses; cf., client.cpp/h DoValidateUTF8data() (https://sourceforge.net/p/cb-clangd-client/code/27/)

I will test it, thanks!

EDIT updated:

The illegal utf8 problem still exits in rev 27.   :(
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 15, 2022, 06:53:26 am
I see your latest commit here in rev 27

* clangd_client - Code to remove invalid utf8 chars from clangd responses; cf., client.cpp/h DoValidateUTF8data() (https://sourceforge.net/p/cb-clangd-client/code/27/)

I will test it, thanks!

EDIT updated:

The illegal utf8 problem still exits in rev 27.   :(

Can you give me some guidance as to how you created the problem. I can't make it happen here.

What source are you using that got a utf8 invalid char?

What symbol were you trying to code complete.

What indication are you getting that there was an invalid utf8 char. A wxMessageBox or an error shown in the cb debug log?

If a message was in the CB debug log, what did it say?

Did you use the repo source or the download file ?
In plugin management, what is the version you see for clangd_client?

I don't think the download files have been updated yet.

Thanks for testing.
Title: Re: Code completion using LSP and clangd
Post by: AndrewCot on January 15, 2022, 07:29:43 am
The plugin download does not have the fix in it.
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 15, 2022, 02:15:44 pm
Hi, I try to debug this plugin, but I have always have this line hit:

Code
// ----------------------------------------------------------------------------
void CodeCompletion::OnAttach()
// ----------------------------------------------------------------------------
{
    AppVersion appVersion;
    appVersion.m_AppName = "clangd_client";
    // Set current plugin version
PluginInfo* pInfo = (PluginInfo*)(Manager::Get()->GetPluginManager()->GetPluginInfo(this));
pInfo->version = appVersion.GetVersion();

    // ccmanager's config obtained from Menu=>Settings=>Editor=>Code Completion (sub panel)
    // Get the CB config item that enables CodeCompletion
    ConfigManager* ccmcfg = Manager::Get()->GetConfigManager(_T("ccmanager"));
    m_CodeCompletionEnabled = ccmcfg->ReadBool(_T("/code_completion"), false);
    if (not m_CodeCompletionEnabled)
    {
        pInfo->version = pInfo->version.BeforeFirst(' ') + " Inactive";
        return;
    }


I have always get the m_CodeCompletionEnabled is false, and I have no idea how to enable it.

What I did is like below:

1, I have build a normal cbp:

Code
D:\code\cb\cb_sf_git\cccrash2019\src\CodeBlocks_wx31_64.cbp

I just build it.

2, I just open the downloaded clangd_client project, and I changed some build options:

In the build option custom variables, I set the variable:  TARGET_DEVEL_DIR
as the value: D:\code\cb\cb_sf_git\cccrash2019, which is the root folder of my local C::B git/svn

3, I changed the output filename as:
Code
$(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\plugins\clangd_client.dll

which means the build target(clangd_client.dll) will be put in the same folder as the other core plugins.

4,  I set the Execution working dir as:

Code
$(TARGET_DEVEL_DIR)\src\devel31_64

Which is the folder where codeblocks.exe locates.

5,  I change the post build script as:

Code
cmd /c if not exist $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks mkdir $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks
zip -jq9 $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/manifest.xml src/resources/*.xrc
zip -r9  $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/images > nul

This means I make a clangd_client.zip, and put it in the destination devel31_64 folder.




Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 15, 2022, 06:59:46 pm
Hi, I try to debug this plugin, but I have always have this line hit:


I have always get the m_CodeCompletionEnabled is false, and I have no idea how to enable it.

What I did is like below:

1, I have build a normal cbp:

Code
D:\code\cb\cb_sf_git\cccrash2019\src\CodeBlocks_wx31_64.cbp

I just build it.

2, I just open the downloaded clangd_client project, and I changed some build options:

In the build option custom variables, I set the variable:  TARGET_DEVEL_DIR
as the value: D:\code\cb\cb_sf_git\cccrash2019, which is the root folder of my local C::B git/svn

3, I changed the output filename as:
Code
$(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\plugins\clangd_client.dll

which means the build target(clangd_client.dll) will be put in the same folder as the other core plugins.

4,  I set the Execution working dir as:

Code
$(TARGET_DEVEL_DIR)\src\devel31_64

Which is the folder where codeblocks.exe locates.

5,  I change the post build script as:

Code
cmd /c if not exist $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks mkdir $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks
zip -jq9 $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/manifest.xml src/resources/*.xrc
zip -r9  $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/images > nul

This means I make a clangd_client.zip, and put it in the destination devel31_64 folder.

The variable m_CodeCompletionEnabled should,in fact, be false.
Note that the line:     m_CodeCompletionEnabled = ccmcfg->ReadBool(_T("/code_completion"), false);

is asking if the old code_completion is enabled. It not asking if "/clangd_client" is enabled.
"/code_completion" must be disabled for clangd_client to run.
I'll change the comments to make that clearer.

Edit: 2022/01/15 10:30
I might be wrong about this. I'll do more checking.

The .cbp you've created look good.

Just a little more information.

Run or debug the CB you created the plugin for.
In the menu for the running debugged/run CB
Go to MainMenu=>Plugins=>Manage Plugins and tell us what the clangd_client plugin version is:

(https://forums.codeblocks.org/index.php?action=dlattach;topic=24357.0;attach=11062;image)
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 15, 2022, 09:51:15 pm
@ollydbg

When you run the debugger, CB  ordinary uses the personality "default" which, for you, I'm guessing has CodeCompletion plugin enabled.

You should use a separate personality for debugging so that you can disable CodeCompletion in the debugged CB without disabling it in your production CB.

For me, in CB main menu/project/SetProgramArgument I have:
--debug-log --no-dde --no-check-associations --multiple-instance --no-splash-screen --verbose /p cbDebug315

Then I can disable CodeCompletion in the debugged CB MainMenu/plugins/managePlugins without it affecting the main CB personality "default".

I'd still like to see the "manage plugins" status of both clangd_client and Code Completion plugins to determine why your m_CodeCompletionEnabled variable is false.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 15, 2022, 10:32:40 pm
@ollydbg

m_CodeCompletionEnabled variable refers to Settings/Editor/Code completion checkbox.

For your debugged CB, it must be currently unchecked which means neither Code completion nor clangd_client can run.


Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 16, 2022, 02:10:21 am
@ollydbg

m_CodeCompletionEnabled variable refers to Settings/Editor/Code completion checkbox.

For your debugged CB, it must be currently unchecked which means neither Code completion nor clangd_client can run.

I have screen shots, see below as in attachment.

I have Settings/Editor/Code completion checkbox checked. If I member correctly this checkbox does not mean we should "disable" the plugin, instead, it means we don't show the suggestion prompt when we are editing.

The other screen shot shows the version latest version I'm using.

Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 16, 2022, 02:24:36 am
@ollydbg

When you run the debugger, CB  ordinary uses the personality "default" which, for you, I'm guessing has CodeCompletion plugin enabled.

You should use a separate personality for debugging so that you can disable CodeCompletion in the debugged CB without disabling it in your production CB.

For me, in CB main menu/project/SetProgramArgument I have:
--debug-log --no-dde --no-check-associations --multiple-instance --no-splash-screen --verbose /p cbDebug315

Then I can disable CodeCompletion in the debugged CB MainMenu/plugins/managePlugins without it affecting the main CB personality "default".

I'd still like to see the "manage plugins" status of both clangd_client and Code Completion plugins to determine why your m_CodeCompletionEnabled variable is false.

I'm using the same CB personality cbDebug315 as yours. Because I'm running the debugger from the clangd_client_wx31_64.cbp.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 16, 2022, 02:34:34 am
@ollydbg

m_CodeCompletionEnabled variable refers to Settings/Editor/Code completion checkbox.

For your debugged CB, it must be currently unchecked which means neither Code completion nor clangd_client can run.

I have screen shots, see below as in attachment.

I have Settings/Editor/Code completion checkbox checked. If I member correctly this checkbox does not mean we should "disable" the plugin, instead, it means we don't show the suggestion prompt when we are editing.

The other screen shot shows the version latest version I'm using.

I'll look into the historical usage of the Settings/Editor/Code completion box.

Are these snapshots form the debugger or the debuggee ?
I think your debugged CB (the debuggee) has that box unchecked.
Would you give us a snapshot of your debugged CB Settings/Editor/Code completion box while the debugger is running?

Thanks for testing. It helps.
 
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 16, 2022, 02:39:10 am
@ollydbg

m_CodeCompletionEnabled variable refers to Settings/Editor/Code completion checkbox.

For your debugged CB, it must be currently unchecked which means neither Code completion nor clangd_client can run.

I have screen shots, see below as in attachment.

I have Settings/Editor/Code completion checkbox checked. If I member correctly this checkbox does not mean we should "disable" the plugin, instead, it means we don't show the suggestion prompt when we are editing.

The other screen shot shows the version latest version I'm using.

I'll look into the historical usage of the Settings/Editor/Code completion box.

Are these snapshots form the debugger or the debuggee ?

debugee.






Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 16, 2022, 03:08:27 am
This is the whole patch I'm using:

Code
 clangd_client/clangd_client_wx31_64.cbp | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/clangd_client/clangd_client_wx31_64.cbp b/clangd_client/clangd_client_wx31_64.cbp
index 1af073c..eb3e6d4 100644
--- a/clangd_client/clangd_client_wx31_64.cbp
+++ b/clangd_client/clangd_client_wx31_64.cbp
@@ -6,8 +6,8 @@
  <Option compiler="gcc" />
  <Build>
  <Target title="Clangd_Client-uw">
- <Option output="bin/Clangd_Client" prefix_auto="1" extension_auto="1" />
- <Option working_dir="bin31_64" />
+ <Option output="$(TARGET_DEVEL_DIR)/src/devel31_64/share/CodeBlocks/plugins/clangd_client" prefix_auto="1" extension_auto="1" />
+ <Option working_dir="$(TARGET_DEVEL_DIR)/src/devel31_64" />
  <Option object_output=".objs31_64" />
  <Option external_deps="$(CODEBLOCKS)/libcodeblocks.a;" />
  <Option type="3" />
@@ -50,12 +50,13 @@
  <Add before="cmd /c @echo CODEBLOCKS: $(CODEBLOCKS)" />
  <Add before="cmd /c @echo TARGET_DEVEL_DIR: $(TARGET_DEVEL_DIR)" />
  <Add before="g++ --version" />
- <Add after="cmd /c MakeRepoUpload.bat" />
+ <Add after="cmd /c if not exist $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks mkdir $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks" />
+ <Add after="zip -jq9 $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/manifest.xml src/resources/*.xrc" />
+ <Add after="zip -r9  $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/images" />
  <Mode after="always" />
  </ExtraCommands>
  <Environment>
- <Variable name="TARGET_DEVEL_DIR" value="$(CODEBLOCKS)\..\.." />
- <Variable name="TARGET_DEVEL_DIR_AC" value="D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller" />
+ <Variable name="TARGET_DEVEL_DIR" value="D:\code\cb\cb_sf_git\cccrash2019" />
  </Environment>
  </Target>
  <Environment>


Note that the post build command has changed, I just removed the last " > nul", which causes some warning.

Code
zip -r9  $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/images
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 16, 2022, 03:17:19 am
@ollydbg.

Well, I'm at a loss to figure out why that variable is false for you.

Somehow, clangd_client is seeing Setting/Editor/Code completion as unchecked in your  CB .conf file.

This has nothing to do with your .cbp file. It's the contents of the .conf file that the debugger is using that's causing this problem.

Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 16, 2022, 03:24:31 am
@ollydbg.

Well, I'm at a loss to figure out why that variable is false for you.

Somehow, clangd_client is seeing Setting/Editor/Code completion as unchecked in your  CB .conf file.

This has nothing to do with your .cbp file. It's the contents of the .conf file that the debugger is using that's causing this problem.

Hi, Pecan, thanks for the help, I will try to debug this issue and see what happens in the code.
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 16, 2022, 03:45:19 am
I see some code:

Code
    wxString fileURI = fileUtils.FilePathToURI(infilename); //(ph 2022/01/5)
    fileURI.Replace("\\", "/");
    DocumentUri docuri = DocumentUri(fileURI.c_str());
    cbStyledTextCtrl* pCntl = pcbEd->GetControl();
    if (not pCntl) return false;

    // save current length of the file
    m_FileLinesHistory[pcbEd] = pCntl->GetLineCount();

    wxString strText = pCntl->GetText();
    const char* pText = strText.mb_str();           //works

    writeClientLog(wxString::Format("<<< LSP_DidOpen:%s", docuri.c_str()) );

    try { DidOpen(docuri, string_ref(pText, strText.Length()) ); }
    catch(std::exception &err)
    {
        //printf("read error -> %s\nread -> %s\n ", e.what(), read.c_str());
        wxString errMsg(wxString::Format("\nLSP_DidOpen() error: %s\n%s", err.what(), docuri.c_str()) );
        writeClientLog(errMsg);
        cbMessageBox(errMsg);
        return false;
    }

My guess is that:

Code
const char* pText = strText.mb_str(); 

Is this correct?

From my point of view, it should be:

Quote
Final word of caution: most of these functions may return either directly the pointer to internal string buffer or a temporary wxCharBuffer or wxWCharBuffer object. Such objects are implicitly convertible to char and wchar_t pointers, respectively, and so the result of, for example, wxString::ToUTF8() can always be passed directly to a function taking const char*. However code such as

Code
const char *p = s.ToUTF8();
...
puts(p); // or call any other function taking const char *


This is from wx document here:  https://docs.wxwidgets.org/trunk/overview_unicode.html

Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 16, 2022, 04:53:58 am
I make a clean git svn clone, and it looks like the clangd_client can be active now. I'm not sure why.

I see that the changes in my previous post did solve the error message issue. Here is the whole patch I use.

Code
 clangd_client/clangd_client_wx31_64.cbp    | 11 ++++++-----
 clangd_client/src/LSPclient/src/client.cpp |  4 ++--
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/clangd_client/clangd_client_wx31_64.cbp b/clangd_client/clangd_client_wx31_64.cbp
index 1af073c..eb3e6d4 100644
--- a/clangd_client/clangd_client_wx31_64.cbp
+++ b/clangd_client/clangd_client_wx31_64.cbp
@@ -6,8 +6,8 @@
  <Option compiler="gcc" />
  <Build>
  <Target title="Clangd_Client-uw">
- <Option output="bin/Clangd_Client" prefix_auto="1" extension_auto="1" />
- <Option working_dir="bin31_64" />
+ <Option output="$(TARGET_DEVEL_DIR)/src/devel31_64/share/CodeBlocks/plugins/clangd_client" prefix_auto="1" extension_auto="1" />
+ <Option working_dir="$(TARGET_DEVEL_DIR)/src/devel31_64" />
  <Option object_output=".objs31_64" />
  <Option external_deps="$(CODEBLOCKS)/libcodeblocks.a;" />
  <Option type="3" />
@@ -50,12 +50,13 @@
  <Add before="cmd /c @echo CODEBLOCKS: $(CODEBLOCKS)" />
  <Add before="cmd /c @echo TARGET_DEVEL_DIR: $(TARGET_DEVEL_DIR)" />
  <Add before="g++ --version" />
- <Add after="cmd /c MakeRepoUpload.bat" />
+ <Add after="cmd /c if not exist $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks mkdir $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks" />
+ <Add after="zip -jq9 $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/manifest.xml src/resources/*.xrc" />
+ <Add after="zip -r9  $(TARGET_DEVEL_DIR)\src\devel31_64\share\CodeBlocks\clangd_client.zip src/resources/images" />
  <Mode after="always" />
  </ExtraCommands>
  <Environment>
- <Variable name="TARGET_DEVEL_DIR" value="$(CODEBLOCKS)\..\.." />
- <Variable name="TARGET_DEVEL_DIR_AC" value="D:\Andrew_Development\WorkingOnThese\AC-WindowsInstaller" />
+ <Variable name="TARGET_DEVEL_DIR" value="D:\code\cb\cb_sf_git\cccrash2019" />
  </Environment>
  </Target>
  <Environment>
diff --git a/clangd_client/src/LSPclient/src/client.cpp b/clangd_client/src/LSPclient/src/client.cpp
index 3aa6956..6ae6afa 100644
--- a/clangd_client/src/LSPclient/src/client.cpp
+++ b/clangd_client/src/LSPclient/src/client.cpp
@@ -1053,7 +1053,7 @@ bool ProcessLanguageClient::DoValidateUTF8data(std::string& data)
             char invChar = data[invloc]; // **debugging**
             data[invloc] = ' '; //clear the invalid utf8 char
             wxString msg = "Error: Removed clangd response invalid utf8 char:";
-            msg << "\'" << invChar << "\'";
+            msg << "\'" << wxString::Format("%x", invChar ) << "\'";
             Manager::Get()->GetLogManager()->DebugLog(msg);
         }
     }
@@ -1562,7 +1562,7 @@ bool ProcessLanguageClient::LSP_DidOpen(cbEditor* pcbEd)
     m_FileLinesHistory[pcbEd] = pCntl->GetLineCount();
 
     wxString strText = pCntl->GetText();
-    const char* pText = strText.mb_str();           //works
+    const char* pText = strText.ToUTF8();           // ollydbg 2022-01-16
 
     writeClientLog(wxString::Format("<<< LSP_DidOpen:%s", docuri.c_str()) );
 


Note the change in :

Code
msg << "\'" << wxString::Format("%x", invChar ) << "\'";

is also try to suppress a message alert box(an ascii char to wxString convert, but the invChar value is bigger than 127)
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 16, 2022, 07:14:00 am
@ollydbg

Thanks for working on that. I'll give those changes a test tomorrow.
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 16, 2022, 12:09:08 pm
I have another patch, it looks like this code should be enabled:
Code
 clangd_client/src/LSPclient/src/client.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/clangd_client/src/LSPclient/src/client.cpp b/clangd_client/src/LSPclient/src/client.cpp
index 6ae6afa..88f5f8f 100644
--- a/clangd_client/src/LSPclient/src/client.cpp
+++ b/clangd_client/src/LSPclient/src/client.cpp
@@ -2443,7 +2443,7 @@ void ProcessLanguageClient::LSP_DidChange(cbEditor* pEd)
 
     // Assure text is UTF8 before handing to DidChange()
     didChangeEvent.text = edText;
-    // didChangeEvent.text = edText.ToUTF8(); Trying to find bad utf8 problem
+    didChangeEvent.text = edText.ToUTF8(); //Trying to find bad utf8 problem
     std::vector<TextDocumentContentChangeEvent> tdcce{didChangeEvent};
     DocumentUri docuri = DocumentUri(fileURI.c_str());
     // **debugging**
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 17, 2022, 12:32:16 pm
I see there are many pch files in the folder such as:

C:\Users\[myusername]\AppData\Local\Temp\preamble-c7460b.pch

I think those files is created by clangd, and are there any way to automatically delete them when exit C::B?

EDIT:

clangd writes too much disk : CPP-19402 (https://youtrack.jetbrains.com/issue/CPP-19402)

This discussion looks like the pch can keep in "memory". :)
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 17, 2022, 01:14:47 pm
I see some error message comes from this function call, but I'm not sure why, this happens when I'm editing source file in the C::B editor.

Code
int ProcessLanguageClient::ReadLSPinputLength()
// ----------------------------------------------------------------------------
{
    // this function is driven by readJson() via the thread in transport::loop()
    // and the incoming data is locked.
    // thread was started in constructor and called from readJason() after locking the buffer

    // "Content-Length: <digits>\r\n\r\n"

    if (Has_LSPServerProcess() and m_std_LSP_IncomingStr.length())
    {
        // search for LSP header
        size_t hdrPosn = m_std_LSP_IncomingStr.find("Content-Length: ");
        if (hdrPosn == std::string::npos)
            return wxNOT_FOUND;
        else //have incoming text
        {
            if (hdrPosn != 0)   // verify LSP header is at beginning of buffer
            {
                // Error: header is not at beginning of buffer. Try to fix it.
                // usually caused by clangd invalid utf8 sequence
                wxString msg(wxString::Format("ERROR:%s(): buffLength (%d): Position of content header %d.\n",
                             __FUNCTION__, int(m_std_LSP_IncomingStr.length()), int(hdrPosn)) );
                msg += "Buffer contents written to client log.";
                #if defined(cb_DEBUG)
                    wxSafeShowMessage("Input Buffer error",msg);
                #endif
                msg += "LSP_IncomingStrBuf:\n" + m_std_LSP_IncomingStr + "\n";
                writeClientLog(msg);
                // adjust the data buf to get clangd header at buff beginning
                m_std_LSP_IncomingStr = m_std_LSP_IncomingStr.substr(hdrPosn);
            }

I just enabled the log file, and see the log message(in C:\Users\[myusername]\AppData\Local\Temp\CBclangd_client-1332.log

And the content of the log file are:

Code
19:45:09.758 ERROR:ReadLSPinputLength(): buffLength (549): Position of content header 112.
Buffer contents written to client log.LSP_IncomingStrBuf:
":{"newText":"wxConvAuto($0)","range":{"end":{"character":16,"line":94},"start":{"character":12,"line":94}}}}]}}Content-Length: 414



{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[{"category":"Parse Issue","code":"expected_unqualified_id","codeActions":[],"message":"Expected unqualified-id","range":{"end":{"character":14,"line":97},"start":{"character":12,"line":97}},"relatedInformation":[],"severity":1,"source":"clang"}],"uri":"file:///F:/test/Config.cpp","version":1}}
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 17, 2022, 02:56:17 pm
I see there are many pch files in the folder such as:

C:\Users\[myusername]\AppData\Local\Temp\preamble-c7460b.pch

I think those files is created by clangd, and are there any way to automatically delete them when exit C::B?

EDIT:

clangd writes too much disk : CPP-19402 (https://youtrack.jetbrains.com/issue/CPP-19402)

This discussion looks like the pch can keep in "memory". :)

Here is the patch to fix this pch file issue:

Code
 clangd_client/src/LSPclient/src/client.cpp | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/clangd_client/src/LSPclient/src/client.cpp b/clangd_client/src/LSPclient/src/client.cpp
index 88f5f8f..f6b5eb9 100644
--- a/clangd_client/src/LSPclient/src/client.cpp
+++ b/clangd_client/src/LSPclient/src/client.cpp
@@ -266,6 +266,10 @@ ProcessLanguageClient::ProcessLanguageClient(const cbProject* pProject, const ch
 
     command += " --limit-results=20";              // Limit the number of results returned by clangd. 0 means no limit (default=100)
 
+    // clangd writes too much disk : CPP-19402 https://youtrack.jetbrains.com/issue/CPP-19402
+    // "-pch-storage=memory"
+    command += " -pch-storage=memory";
+
     if (wxDirExists(clangResourceDir))
         command += " --resource-dir=" + clangResourceDir;  // Directory for system includes
 
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 18, 2022, 12:35:08 pm
I see there are some code snippet like:

Code
m_MutexInputBufGuard.Lock;

m_MutexInputBufGuard.Unlock();

But in the code, we have to carefully handle the unlocking the wxMutex when return the function body, especially when there are multiply returns.

Is it possible to use the wxMutexLocker, and check the IsOK() function for checking whether it get locked or not.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 18, 2022, 08:23:19 pm
I see some error message comes from this function call, but I'm not sure why, this happens when I'm editing source file in the C::B editor.

Code
int ProcessLanguageClient::ReadLSPinputLength()
// ----------------------------------------------------------------------------
{
    // this function is driven by readJson() via the thread in transport::loop()
    // and the incoming data is locked.
    // thread was started in constructor and called from readJason() after locking the buffer

    // "Content-Length: <digits>\r\n\r\n"

    if (Has_LSPServerProcess() and m_std_LSP_IncomingStr.length())
    {
        // search for LSP header
        size_t hdrPosn = m_std_LSP_IncomingStr.find("Content-Length: ");
        if (hdrPosn == std::string::npos)
            return wxNOT_FOUND;
        else //have incoming text
        {
            if (hdrPosn != 0)   // verify LSP header is at beginning of buffer
            {
                // Error: header is not at beginning of buffer. Try to fix it.
                // usually caused by clangd invalid utf8 sequence
                wxString msg(wxString::Format("ERROR:%s(): buffLength (%d): Position of content header %d.\n",
                             __FUNCTION__, int(m_std_LSP_IncomingStr.length()), int(hdrPosn)) );
                msg += "Buffer contents written to client log.";
                #if defined(cb_DEBUG)
                    wxSafeShowMessage("Input Buffer error",msg);
                #endif
                msg += "LSP_IncomingStrBuf:\n" + m_std_LSP_IncomingStr + "\n";
                writeClientLog(msg);
                // adjust the data buf to get clangd header at buff beginning
                m_std_LSP_IncomingStr = m_std_LSP_IncomingStr.substr(hdrPosn);
            }

I just enabled the log file, and see the log message(in C:\Users\[myusername]\AppData\Local\Temp\CBclangd_client-1332.log

And the content of the log file are:

Code
19:45:09.758 ERROR:ReadLSPinputLength(): buffLength (549): Position of content header 112.
Buffer contents written to client log.LSP_IncomingStrBuf:
":{"newText":"wxConvAuto($0)","range":{"end":{"character":16,"line":94},"start":{"character":12,"line":94}}}}]}}Content-Length: 414



{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[{"category":"Parse Issue","code":"expected_unqualified_id","codeActions":[],"message":"Expected unqualified-id","range":{"end":{"character":14,"line":97},"start":{"character":12,"line":97}},"relatedInformation":[],"severity":1,"source":"clang"}],"uri":"file:///F:/test/Config.cpp","version":1}}

This is caused by clangd sending a message (usually a code completion response) with an illegal utf8 char. Clangd sets the message with a header specifying the wrong length which eats up the content-header for the following message.


I have to correct the situation by searching for the content-header.
I have now commented out that message box. The code works to correct the situation so we don't need that anymore.

Thanks for testing and finding these "gotchas" I've forgotten about.
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 18, 2022, 08:30:01 pm
I see there are many pch files in the folder such as:

C:\Users\[myusername]\AppData\Local\Temp\preamble-c7460b.pch

I think those files is created by clangd, and are there any way to automatically delete them when exit C::B?

EDIT:

clangd writes too much disk : CPP-19402 (https://youtrack.jetbrains.com/issue/CPP-19402)

This discussion looks like the pch can keep in "memory". :)

Here is the patch to fix this pch file issue:

Code
 clangd_client/src/LSPclient/src/client.cpp | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/clangd_client/src/LSPclient/src/client.cpp b/clangd_client/src/LSPclient/src/client.cpp
index 88f5f8f..f6b5eb9 100644
--- a/clangd_client/src/LSPclient/src/client.cpp
+++ b/clangd_client/src/LSPclient/src/client.cpp
@@ -266,6 +266,10 @@ ProcessLanguageClient::ProcessLanguageClient(const cbProject* pProject, const ch
 
     command += " --limit-results=20";              // Limit the number of results returned by clangd. 0 means no limit (default=100)
 
+    // clangd writes too much disk : CPP-19402 https://youtrack.jetbrains.com/issue/CPP-19402
+    // "-pch-storage=memory"
+    command += " -pch-storage=memory";
+
     if (wxDirExists(clangResourceDir))
         command += " --resource-dir=" + clangResourceDir;  // Directory for system includes
 

Thanks for that.

I've added code to remove preamble-*.tmp and preamble-*.pch files when the project closes.

I'll later add an option to keep the .pch in memory. (For me, I'd rather keep them on disk and remove them at project close time).

As an aside: Windows does not allow me to remove files that are open and being used. But Linux does. Do you know of a way to tell Linux NOT to allow me to delete open files?
Title: Re: Code completion using LSP and clangd
Post by: Pecan on January 18, 2022, 08:54:19 pm
I see there are some code snippet like:

Code
m_MutexInputBufGuard.Lock;

m_MutexInputBufGuard.Unlock();

But in the code, we have to carefully handle the unlocking the wxMutex when return the function body, especially when there are multiply returns.

Is it possible to use the wxMutexLocker, and check the IsOK() function for checking whether it get locked or not.

There are only 2 locks in the code that can cause any trouble. The lock on the input buffer. One to write to the buffer, and one to get the next clangd response out of the buffer. And neither affect the UI thread.

I tried wxMutexLocker first before giving up on it.
I want to be able to unlock the input buffer and then do more work in the function.
When I unlocked the mutex before the function ended,  wxWidgets gave me errors about the mutex. I lost confidence that I could mix a wxMutexLocker and manual unlocks.

In fact, I removed all locks on the main UI thread and used idle time callbacks instead.
Title: Re: Code completion using LSP and clangd
Post by: ollydbg on January 20, 2022, 08:07:39 am
I see there are some code snippet like:

Code
m_MutexInputBufGuard.Lock;

m_MutexInputBufGuard.Unlock();

But in the code, we have to carefully handle the unlocking the wxMutex when return the function body, especially when there are multiply returns.

Is it possible to use the wxMutexLocker, and check the IsOK() function for checking whether it get locked or not.

There are only 2 locks in the code that can cause any trouble. The lock on the input buffer. One to write to the buffer, and one to get the next clangd response out of the buffer. And neither affect the UI thread.

I tried wxMutexLocker first before giving up on it.
I want to be able to unlock the input buffer and then do more work in the function.
When I unlocked the mutex before the function ended,  wxWidgets gave me errors about the mutex. I lost confidence that I could mix a wxMutexLocker and manual unlocks.

In fact, I removed all locks on the main UI thread and used idle time callbacks instead.

Yes, there are only There are only 2 locks in the code. Sometimes, I got error message about lock failed, I'm not sure why.

BTW: since the console pipe is connected with clangd.exe, I'm not sure why the locker is needed. Since the content is from a thread to the main GUI thread by the Event, when you got the Event, you were already in the main GUI thread, and the content in the Event(wxThreadEvent) is already deep copied. So, I think the locker is not necessary, am I correct?