Author Topic: LLDB debugger low level interface investigation results  (Read 13157 times)

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
LLDB debugger low level interface investigation results
« on: April 15, 2022, 08:50:43 am »
I have come to the conclusion that the LLDB debugger docs suck big time compared to the GDB manual that has all of the info in the one spot and at least it explains how things work and gives examples of inputs and some outputs. I cannot find anything like this for LLDB.

I have investigated the different ways of "controlling" the LLDB debugger and they are:
  • Direct via the command line like the GDB annotations does
  • Via a LLDB/MI executable that is no longer part of the LLVM project, but a third party product now.
  • Via the LLDB Scripting Bridge API (LLDB SB API)
This is what I have learnt from my investigations:
  • The LLDB command line is not documented enough to write a plugin easily and it also NOT guaranteed to be stable (aka not change). As such this is not viable IMHO.
  • The LLDB/MI has a number of issues and is not fully compatible with the GDB/MI interface that it says it is. I could not get a s simple break point did not work that worked via GDB or via the LLDB command line. As such this has too many issues to consider being viable.
  • The LLDB SB API is used by CodeLite and other IDE's. as per the docs "The SB APIs constitute the stable C++ API that LLDB presents to external clients....", so it should not change too much, if at all.  Below is a simple example I found on the web and have been able to build and run to test the LLDB SB API.
Code
// https://github.com/llvm/llvm-project/issues/52769#include <lldb/API/LLDB.h>
#include <windows.h>
#include <iostream>

using namespace lldb;

int main()
{
    SBDebugger::Initialize();
    auto debugger = SBDebugger::Create(true);
    debugger.SetAsync(false);

    auto const appPath = R"(D:\\Andrew_Development\\Z_Testing_Apps\\Clang_printf\\bin\\clang_Printf.exe)";  // Path to the program being debugged
    auto target = debugger.CreateTarget(appPath);

    auto breakpoint = target.BreakpointCreateByLocation("main.cpp", 45);  // Breakpoint location in the program being debugged
    auto breakpointExit = target.BreakpointCreateByLocation("main.cpp", 153);  // Breakpoint location in the program being debugged

    auto process = target.LaunchSimple(nullptr, nullptr, nullptr);

    while (true)
    {
        process.Continue();
        auto const count = breakpoint.GetHitCount();
        if (count % 10 == 0)
        {
            std::cout << count << " ";  // We just print the number of times the breakpoint was hit every 100 times
        }
        auto const countExit = breakpointExit.GetHitCount();
        if (countExit != 0)
        {
            break;
        }

    }

    SBDebugger::Terminate();

printf("EXITING!!!!!!!");
    return 0;
}


« Last Edit: April 18, 2022, 01:54:06 am by AndrewCot »

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: LLDB debugger low level interface investigation results
« Reply #1 on: April 15, 2022, 10:40:57 am »
Thats interesting.
Strange the way they take with  LLDB/MI... but probably there is a good reason for it.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: LLDB debugger low level interface investigation results
« Reply #2 on: April 15, 2022, 04:44:03 pm »
Interesting. Thank you for reporting.
Are the calls (for example process.Continue();)  blocking or not blocking?

Offline eranif

  • Regular
  • ***
  • Posts: 256
Re: LLDB debugger low level interface investigation results
« Reply #3 on: April 15, 2022, 06:42:45 pm »
Hi C::B team,

I rarely post on this forum, but I couldn't help noticing the discussion about LLDB (and the path that I took with CodeLite)
However, I would not take this path today. In fact the future of the debuggers, is similar to code completion: via dedicated small servers.

Today, most of the IDEs (including CodeLite) is using LSP (Language-Server-Protocol) for getting code completion via various implementations (clangd, rust-analyzer, pylsp etc)

The same revolution is taking place for the debuggers using the DAP protocol (Debug-Adapter-Protocol)

https://microsoft.github.io/debug-adapter-protocol/

I would recommend your to consider this path before investing time in using LLDB API directly


Eran

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #4 on: April 16, 2022, 02:13:54 am »
Eran,

Thanks for the info.

I did not investigate the DAP, but will now have a look.

Thanks,Andrew

 

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #5 on: April 16, 2022, 05:10:31 am »
I have looked at the DAP and it looks like the way to go as once implemented will then allow C::B to support in theory all of the debuggers listed on the following page:   https://microsoft.github.io/debug-adapter-protocol/implementors/adapters/
Now comes the hard part, where to start over the next week and then restart and hopefully get a handle on the work I hope.


Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: LLDB debugger low level interface investigation results
« Reply #6 on: April 17, 2022, 02:20:51 pm »
Hi Eran

many thanks for the information
very much appreciated  :)

Cheers

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: LLDB debugger low level interface investigation results
« Reply #7 on: April 17, 2022, 09:07:32 pm »
I have looked at the DAP and it looks like the way to go as once implemented will then allow C::B to support in theory all of the debuggers listed on the following page:   https://microsoft.github.io/debug-adapter-protocol/implementors/adapters/
Now comes the hard part, where to start over the next week and then restart and hopefully get a handle on the work I hope.

Have you found an gdb server? I am quite confused with the architecture or more, where to get the debugger servers. Are thy embedded in VS code and not stand alone?

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #8 on: April 18, 2022, 01:55:30 am »
I am playing with the following library/libraries to plug the DAP interface code hole:
https://github.com/google/cppdap
https://github.com/stoianmihail/cppdap-debugger

I am using the following DAP LLDB adapter for testing:
https://github.com/vadimcn/vscode-lldb

You will need to extract the https://github.com/vadimcn/vscode-lldb/releases/download/v1.7.0/codelldb-x86_64-windows.vsix file using 7zip. I run the adapter using the following command:
Code
adapter\codelldb --port 12345

I have not looked at how VSCode starts the different DAP adapters as I want to get something "working" first. "Working" is load an exe, set a break point on a line, run and hit the break point then run to the end of the hello world type app.

Attached in the 7z is my sample test project inc source that has a a hacked source code sample (LLDB_Client_Test.cpp) that uses bits from both github "cppdap" repos, but the launch step does not return or show anything. It does show the DAP adapter initialize request results, so at least I know the app and DAP adapter are communicating.

The following is a quick guide to build the cppdap library using MSYS2:

The Code::Blocks Debugger Application Protocol debugger plugin uses the following libraries:
    * https://github.com/google/cppdap

Use the following steps to build the cppdap library using MSYS2:

1. Clone the repo using the following command:
  * git clone --recurse-submodules https://github.com/google/cppdap

2. Generate the makefile:

bash
cd <path-to-cppdap>
mkdir build
cd build

cmake .. -G "MSYS Makefiles"

3. Finally, build the project:

make


P.S. One thing I spotted yesterday that I need to get my head around is that the DAP protocol launch request spec as it has the following in the spec, so then it makes the IDE only able to launch DAP adapters that it knows about as the launch arguments are specific to the DAP adapter.... I hope this is not the case, but this definitely needs more investigation to see how the launch request is implemented in IDE's.

"Since launching is debugger/runtime specific, the arguments for this request are not part of this specification."

Hope this is helpful.

Offline eranif

  • Regular
  • ***
  • Posts: 256
Re: LLDB debugger low level interface investigation results
« Reply #9 on: June 21, 2022, 01:11:35 am »
A quick update on this subject:

Since our chat here, I revived my 2 years old project "dbgd"

My goal was to create dap-gdb and dap-client, due to lack of time I paused my work on that.
Recently, I moved to develop full time on Mac and I needed a better lldb support (replacing my current lldb implementation which uses liblldb.dylib directly)

So with small effort I made dapcxx library that it built for wxWidgets.

All the interaction is done via events, i.e. whenever a an event / response arrives from the dap server, the GUI is notified via event (all the reads are done in a background thread)

You can check it out here:
https://github.com/eranif/dbgd -> it is a complete standalone library with zero dependencies (other than wxWidgets)
It comes with a small UI that demonstrate how to use it

The DAP server initialization is done here:
https://github.com/eranif/dbgd/blob/master/dbgcli/MainFrame.cpp#L59

PS:
You can install lldb-vscode from MSYS2 to try it out

Hope you will find it useful for your purposes

Eran

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #10 on: June 21, 2022, 05:02:22 am »
Thanks for the post as it will help.

I looked at your repo's a long time ago and seen it was a few years old and did not spend allot of time figuring out how it worked. I will have another look and spend time figuring it out.
I did not see that MSYS2 has the lldb-vscode as I downloaded the official release for it from the src git repo and started it. I will switch to the MSYS2 version.

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #11 on: June 24, 2022, 06:04:22 am »
Eran,

I have just successfully modified my local C::B for the following:a) Created a debugger_WXDAP.dll that is the dap dll and right now is in the same directory as the codeblocks.exe.
b) Created a new debugger plugin debugger_dap.dll that is wired in like the other debuggers (GDB and GDB/MI).
c) The changes worked enough and I have wired up enough so that when I run a program to debug it in the C::B GUI the connection through the debugger_WXDAP.dll works and I get debug output on the lldb-vscode.exe. I have only wired up the connection and the events only log to the debug log on the C:B gui.

There is still a huge amount of work to do and it will take a while to go from where I am to

If I find anything I will create issues and/or PR's against the github repo.

Thanks very much for your help and advice on the DAP and the DLL.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: LLDB debugger low level interface investigation results
« Reply #12 on: June 25, 2022, 12:53:12 am »
Hi!
this looks really nice, great work.
Also thank you for sharing here in the forum!



Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #13 on: June 26, 2022, 02:42:47 am »
I now have some rudimentary debugging on an exe in C::B working as follows:
  • The exe is loaded via the DAP dll using the new DAP debugger plugin.
  • The exe can be started and run with a breakpoint set on main() as default in the debugger plugin for the time being (I do not have any persistent breakpoint or watches etc)
  • I can use step to the next line. I have wired up most of the other step features, but have not tested them.
  • I can set one line break point in the file and then run to the break point. The DAP break point API works on a file basis with array of breakpoints for the file, which is different to the GDB/MI code I started with and as such I need to refactor the way the breakpoints are stored in the debugger plugin.
  • I have the call stack partially working in that it shows data and some of it looks right, but some does not.
  • I can stop debugging and start debugging again.
Things not in the list above are not working. 


I will be working on the following in the order listed, unless I hit something or need a break on the work:
  • Add support for multiple line break points in a file
  • Wire up simple watches. Aka add to watch dialog only and update as you step / run the debugee
  • Enable break point and watchs to be persistent from the GDB/MI debugger plugin.
  • Get the call stack working correctly
  • Wire up starting the LLDB debugger exe when you start debugging. (commented out at the moment so I can see what is happening on a command prompt)
  • Migrate/test on Linux
  • Migrate/test on MacOS
  • Use in anger to fix C::B issue(s) on MacOS. In other words use it in the real world.
  • Add features required to make debugging better.
  • Update ticket 1114 to add support for the CLANG compiler to detect debugger and add it as a debugger automatically.
« Last Edit: June 26, 2022, 03:10:19 am by AndrewCot »

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2750
Re: LLDB debugger low level interface investigation results
« Reply #14 on: June 26, 2022, 07:25:02 am »
Wow!! It nice to see this happening.

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #15 on: July 01, 2022, 10:58:35 am »
Update:
  • Done - Add support for multiple line break points in a file
  • Done - Wire up simple watches. Aka add to watch dialog only and update as you step / run the debugee
  • Done - Enable break point and watchs to be persistent from the GDB/MI debugger plugin.
  • Done - Get the call stack working correctly
  • Done - Wire up starting the LLDB debugger exe when you start debugging. (commented out at the moment so I can see what is happening on a command prompt)
  • Build issues Migrate/test on Linux
  • WIP - Migrate/test on MacOS
  • Use in anger to fix C::B issue(s) on MacOS. In other words use it in the real world.
  • Add features required to make debugging better.
  • Update ticket 1114 to add support for the CLANG compiler to detect debugger and add it as a debugger automatically.
I have finally got the code building and the plugin loading on Mac. Next day or two I will see about configuring the plugin and testing it on the Mac.

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #16 on: July 02, 2022, 05:17:16 am »
Great news...
  • DAP debugging working on Linux
  • DAP debugging working on MacOS - see attachment
Now the not so great news. There are some issues on the Mac:
  • The code to run the following does not work, so I need to manually run it and comment out the code:
                  /usr/local/Cellar/llvm/14.0.6/bin/lldb-vscode -port 12345
  • I have had C::B crash when doing some debugging
The source code for the DLL and plugin is available from the following repo, but be aware that it is only for the hard core developers and WILL change and is NOT production quality or even beta quality:
    https://github.com/acotty/CB_Debugger_DAP_Plugin         
In the next few hours I will do a major update on the following repo to include all of the changes so you can use it to build C::B with the plugin:   https://github.com/acotty/CodeBlocks_Unofficial_Testing
BTW There is an issue with the main workspace I am using where I have specified the plugin depends on the DAP DLL, but the DAP DLL does not get built before the plugin so the plugin link fails.
Note: Working is used very very very loosely above.



Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: LLDB debugger low level interface investigation results
« Reply #17 on: July 03, 2022, 02:06:16 am »
Great work!
Nice list of features you have there!

>The debugger is NOT able to debug itself
I am quite curios why this is so? What is the problem? Network port collision?

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #18 on: July 03, 2022, 02:21:32 am »
>The debugger is NOT able to debug itself
I am quite curios why this is so? What is the problem? Network port collision?
I have not got that far yet as it's part of the following dot point, but I will be giving it a go later today:
  • Use in anger to fix C::B issue(s) on MacOS. In other words use it in the real world.
Can you let me know where I put the info that it cannot debug itself or inferred this so I can correct it or try to correct it.

I will need to use two different network ports for the DAP debugger, one for the "master" C::B to use to debug the C::B debuggee and then another one for the C::B debuggee to debug a hello world app.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: LLDB debugger low level interface investigation results
« Reply #19 on: July 03, 2022, 10:20:41 am »
Quote
CODE::BLOCKS DAP DEBUGGER PLUGIN
Description

This GitHub repo contains the source code for a Code::Blocks DAP debugger plugin. The debugger is NOT able to debug itself and is a NOT viable replacement for the existing GDB plugin at this point in time.
in the read me. I was asking this, because an earlier version of codeblocks had the same problem. You could not debug codeblocks running gdb. I think this was because some fancy PID detection. But it was fixed some time....

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #20 on: July 03, 2022, 11:22:39 am »
Thanks. I have updated the readme.md

I got side tracked onto the debugger detection on startup and it has taken allot longer than I expected and have not got to using the plugin to debug itself.

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #21 on: July 04, 2022, 10:45:19 am »
The DAP debugger can debug itself, so long as you use different network ports. I just finished testing this.

If you are one of the people who have cloned the https://github.com/acotty/CB_Debugger_DAP_Plugin repo then please pull the changes as I fixed a nasty issue in the code today where I commented the OnIdle(..) from the GDB/MI as I thought the code was non needed, but the event.Skip(); was as it causes compilation issues. Go figure!!!!






Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2750
Re: LLDB debugger low level interface investigation results
« Reply #22 on: July 04, 2022, 10:50:27 pm »
...where I commented the OnIdle(..) from the GDB/MI as I thought the code was not needed, but the "event.Skip();" was as it causes compilation issues. Go figure!!!!

That's caught me a couple of times also.

wxWidgets ends processing any event if there is no Skip() called. Events are pass on to other users only when a call to Skip() is done.

It's especially important to issue a call to Skip() from within an OnIdle() event. Else plugins (etal) will never receive their OnIdle event hooked onto async I/O etc.

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #23 on: July 06, 2022, 01:00:43 pm »
Update:
  • Done - Add support for multiple line break points in a file
  • Done - Wire up simple watches. Aka add to watch dialog only and update as you step / run the debugee
  • Done - Enable break point and watchs to be persistent from the GDB/MI debugger plugin.
  • Done - Get the call stack working correctly
  • Done - Wire up starting the LLDB debugger exe when you start debugging.
  • Done - Mac Project files now build DLL and plugin
  • Done - Linux Project files now builds DLL and plugin
  • Done - Linux bootstrap/configure/make process now builds DLL and plugin
  • Done - MacOS Project files now builds DLL and plugin
  • Done - Macos bootstrap/configure/make process now builds DLL and plugin
  • Done - add support for the CLANG compiler to detect debugger and add it as a debugger automatically (compiler config XML changes
  • Done - MAcOS update DMG file creation - needed to easily install updates
  • Working - by default it captures exceptions on Linux and Mac and if you use Clang64 on Windows (Mingw64 does not work and 32 bit not tested)
Real world usage:
  • Use in anger to fix C::B issue(s) on Mac to go through the Mac Tickets. Most seem to be codecompletion issues and
  • Add features required to make debugging better
Future:
  • Remove the codecompletion from the builds as clangd_client is working better than codecompletion and I have it in my builds. This will resolve a number of tickets.
  • Future - Linux update DEB file creation and test
I have started looking a the Mac tickets and found that there is not allot of info on how to reproduce the issues or logs that can be used to analyze the problem when it occurred.... Something devs need to be aware of that even tough you may not look at the ticket you need to ensure that enough info is captured for ad different dev to look at it in a few months/years.
« Last Edit: July 06, 2022, 01:04:15 pm by AndrewCot »

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #24 on: July 24, 2022, 04:33:20 am »
Quick update:
I can now successfully debug C::B from C::B using the DAP debugger on Windows.

The code is available from the following repo:https://github.com/acotty/CodeBlocks_Unofficial_Testing

If you want to try it then you will need to start with the following directories and add the CBP to your workspace and configure your environment to build the two projects:
  • src\plugins\contrib-wip\DebugAdapterProtocol
  • src\plugins\contrib-wip\Debugger_DAP
To use the LLDB debugger you need to configure your C++ to build with -dwarf-4 instead of -ggdb. GDB can use the -ddwarf-4 debug information.

Next for me to try the code on Mac to see how if it also works w.r.t. C::B debugging C::B.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: LLDB debugger low level interface investigation results
« Reply #25 on: July 24, 2022, 05:37:31 am »
Nice work.

One question(newbie question)

Under Windows, do you mean DAP debugger == LLVM debugger ?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #26 on: July 24, 2022, 07:18:22 am »
Simple answer : Yes

Additional info: Currently the debugger plugin code auto detects the DAP adapter for LLDB (the LLVM debugger). There is code in the repo to also auto setup the debugger config from the options*.xml file, which is an outstanding SF ticket, which make setting up the compiler and debugger allot easier.

You can run a number of different DAP adapters that then interface to different debuggers for different languages, so it "theory" you could use the DAP debugger plugin to debug say rust or some other language without changing the plugin or C::B code. "theory" is in quotes as in I have no idea if it will or will not work, but this s how VSCode supports debugging different languages without having to change the IDE.

The DAP adapter used for LLDB debugging is the C:\msys64\mingw64\bin\lldb-vscode.exe adapter (please do not try the official LLVM lldb-vscode.exe as it works differently for some reason and I have not looked at why, but it seems that the different LLDB adapters all seem to work differently or have issues/bugs).

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: LLDB debugger low level interface investigation results
« Reply #27 on: July 26, 2022, 11:29:48 am »
Very good news:  *I have succeeded in C::B debugging C::B on Windows and MAC!!!!

On Windows you can now use the following DAP adapters:
  • MSYS2 MingW64  C:\msys64\mingw64\bin\lldb-vscode.exe
  • MSYS2 Clang64  C:\msys64\clang64\bin\lldb-vscode.exe
  • https://github.com/vadimcn/vscode-lldb using the extension\adapter\codelldb.exe file
On the Mac Intel computers you can now use the following DAP adapters (I do not have a M1 or M2):
For details on setting up the DAP adapters and more info please read the following:
     https://github.com/acotty/CodeBlocks_Unofficial_Testing/blob/master/src/plugins/contrib-wip/Debugger_DAP/Readme_DAP_setup.md
If you have a Mac and need debugging in C::B and are NOT a newbiee then give the unofficial C::B in the following github repo a go:
https://github.com/acotty/CodeBlocks_Unofficial_Testing