Author Topic: Linux: gdb ignores removed breakpoints  (Read 1468 times)

Offline tigerbeard

  • Almost regular
  • **
  • Posts: 158
Linux: gdb ignores removed breakpoints
« on: April 07, 2022, 10:55:25 am »
I am sorry that I did not pay attention to exactly when this started, but since a few month my Linux CB builds show a bit annoying behaviour when removing a breakpoint.

Procedure
Set a two breakpoints and start in debug, reach both breakpoints with continue and eventually end the program. Remove the first breakpoint, note that the first red breakpoint icon beside the code is gone and start in debug again.

Observed:
After start the debugger still stops at that position. With "continue" the 2second breakpoint is reached as norma. The only workaround I found to remove the breakpoint is to go the the debugger tab's command line and enter "clear". After that the breakpoint is really removed.

Expected:
After start the debugger only stops at the second breakpoint

Does anyone else see this behaviour?


The last revision I have tested is rev12765. I also saw it in 12535. I built in linux Ubuntu 18.04 using the repository wx3.0 gtk2 binaries.
Also I note that any breakpoint I set while the program is running is ignored. This only works when the program is acutally interrupted elsewhere. The easy workaround clearly is to keep a breakpoint in a help menu, so not a big deal, but is this expected behaviour?

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: Linux: gdb ignores removed breakpoints
« Reply #1 on: April 07, 2022, 11:31:17 am »
Can you please supply a GDB debug log as follows:1) Enable the "Full (debug) log" in the "Settings->Debugger->Common"2) Now load the project3) Go thought your test. With the breakpoint dialog showing (see below for how to see it)
4) In the Logs and Other dialog right click in the Debugger tab and "Copy the contents to the clipboard" and save to a file5) Attach the file.
I can give pointers about where the issue is if the log has enough info in it. Be aware that the GDB plugin log does NOT show all of the communication between GDB and C::B even with the full debug log enabled.

Can you also bring up the breakpoint dialog (Debug->Debug windows->Breakpoints)  after you hit the first breakpoint and keep it up and note what it says at the different steps. The editor and brekpoints dialog are independent and may help with where the problem could be.

I would suggest checking which version of GDB you are using and include that in your followup post . BTW the latest GDB is 11.2. If you are using an old version of GDB, say below 9 then I would advise updating, but if you are using say 10 or above then I do not know if it will or will not help, but it's worth a try.
Setting a breakpoint on a running app may or may not work as the code I looked at for this today when I was checking out missing functionality in the GDB/MI debugger I am working on was not easy to understand or follow. Setting a breakpoint on a running app should show up something in the debugger log if you have full debug on.  I have not got to the stage of testing adding a breakpoint while the app is running with the GDB/MI debugger I am working on as there are lower hanging issues that I need to fix first.

Offline tigerbeard

  • Almost regular
  • **
  • Posts: 158
Re: Linux: gdb ignores removed breakpoints
« Reply #2 on: April 07, 2022, 01:18:19 pm »
Thanks for the detailed info.

Indeed gbd seems rather old with v8.1.1 (latest 18.04 repostitory version).
White trying to setup the data for you it seems that actvating the full log makes the issue disappear. I think I need to further trace down 100% reproducible steps, preferable in a smaller code base such as a wxWidget sample. A first check on another installation showed that there is issue did not come up every time.
Looks I need to do some more homework on this...
« Last Edit: April 07, 2022, 01:26:50 pm by tigerbeard »

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: Linux: gdb ignores removed breakpoints
« Reply #3 on: April 07, 2022, 01:59:36 pm »
Good luck. Try to get GDB 11.2. Post updates if you need help.

Check you GCC as well. Below is a snippet from an install shell script I use to install GCC 9,10 and 11 on Xubuntu and Mint so I can then swap between them using the update-alternatives if I need to.
echo ">>>>>>>>>>>>>>>>>>  Installing GCC10 <<<<<<<<<<<<<<<<<<"
add-apt-repository ppa:ubuntu-toolchain-r/test  --yes
apt-get update
apt-get -y  install cpp-10 gcc-10 g++-10
apt-get -y  install cpp-11 gcc-11 g++-11
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 90 --slave /usr/bin/g++ g++ /usr/bin/g++-9 --slave /usr/bin/gcov gcov /usr/bin/gcov-9
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 --slave /usr/bin/g++ g++ /usr/bin/g++-10 --slave /usr/bin/gcov gcov /usr/bin/gcov-10
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 110 --slave /usr/bin/g++ g++ /usr/bin/g++-11 --slave /usr/bin/gcov gcov /usr/bin/gcov-11
# update-alternatives --config gcc

Offline tigerbeard

  • Almost regular
  • **
  • Posts: 158
Re: Linux: gdb ignores removed breakpoints
« Reply #4 on: April 07, 2022, 03:06:58 pm »
Hi Andrew,
thanks for the script. The original is all on offline computers, so not so quick to get new stuff on them.

However, even with Debugger Full Log on I ran into the issue and had a look at the things you said. I guess it makes it a bit more clear what is hapening. See the exceprt from the log file below. I included "<======" where I think the key points are.

Basically I can say both Editor and BP Window work as expected and consistently. All BPs I add or remove in the editor are added or removed in the BP window.

The trace action was looking from a user perspective like this:
The program starts and hits one specific BP. After Continue it its a second time. There the BP is removed, the BP window also removes it from the list. Then continue is hit. GDB stops at the same position again (where both Editor and BP Window show no BP). When the BP is set again, it shows in the BP window. If you do it a few times the one can see in the Debug log that the BP numbers increment each time - it does not reused a deleted BP number..

The trace shows a precularity with two BPs when they are initially set, an additional ">>>>>>cb_gdb:". Maybe meaningless, but the second is involved in the mixup later.
The BP from the test was BP6. The trace shows hitting it twice: OK. Deleteing it, however, it deletes BP5. The BP Windows still shows BP5. Wehen dbg continues it still has BP6 and of course stops there. When the BP is added in the editor again, it added "over" the first one, generating the last message.

Maybe its connected to the fact that the application uses multiple threads. So I guess to further nail it down a simple piece of code is neeed that can reproduce this 100%.


Code
Active debugger config: GDB/CDB debugger:Default
Building to ensure sources are up-to-date
Selecting target:
DebugDll
Starting debugger: /usr/bin/gdb -fullname -quiet  -args /home/..
Setting SHELL to '/bin/sh'
done
[debug]> set prompt >>>>>>cb_gdb:

Setting breakpoints

[debug]Using terminal's PID as console PID 13700, TTY /dev/pts/5
[debug]Queued:[tty /dev/pts/5]
[debug]done.
[debug](gdb) >>>>>>cb_gdb:
Debugger name and version: GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git

[debug]Breakpoint 1 at 0x19fa02: file /home/
[debug]>>>>>>cb_gdb:Breakpoint 2 at 0x19f3c4: file /home/                    <================
[debug]Breakpoint 3 at 0x19f851: file /home/
[debug]Breakpoint 4 at 0x1a6567: file /home/
[debug]Breakpoint 5 at 0x29f59f: file /home/
[debug]>>>>>>cb_gdb:Breakpoint 6 at 0x19fe19: file /home/                    <================ this is the correct BP
[debug]> run
[debug]Starting program: /home/

[debug][New Thread 0x7fffea60f700 (LWP 13706)]
[debug][New Thread 0x7fffe9e0e700 (LWP 13707)]

[debug]Thread 1 "" hit Breakpoint 6,
[debug]/home/
[debug]>>>>>>cb_gdb:

At /home/
Continuing...

[debug]Thread 1 "" hit Breakpoint 6,                    <================ 2nd Hit. Deleting in in Editor
[debug]/home/
[debug]>>>>>>cb_gdb:

At /home/
[debug]>>>>>>cb_gdb:
[debug]> x/256xb 0x0
[debug]Cannot access memory at address 0x0
[debug]0x0: >>>>>>cb_gdb:
[debug]> delete breakpoints 5                    <================ BP6 is gone in Editor and BP Window. BP5 is still there in both windows
[debug]>>>>>>cb_gdb:
Continuing...
[debug]> cont
[debug]Continuing.
[debug]Thread 1 "" hit Breakpoint 6,                    <================ no BP here, both in Editor nor Breakpoint windows
[debug]/home/
[debug]>>>>>>cb_gdb:
At /home/
[debug]>>>>>>cb_gdb:
[debug]> x/256xb 0x0
[debug]Cannot access memory at address 0x0
[debug]0x0: >>>>>>cb_gdb:
[debug]> break "/home/
[debug]Note: breakpoint 6 also set at pc 0x5555556f3e19.                 <================
[debug]Breakpoint 7 at 0x5555556f3e19: file /home/                    <================ this is receating the same BP in the editor
[debug]>>>>>>cb_gdb:

Offline AndrewCot

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 678
Re: Linux: gdb ignores removed breakpoints
« Reply #5 on: April 07, 2022, 03:49:54 pm »
I would see if you can install a later GDB as besides the break point issue there is also a memory watch that is not working as the "x/256xb 0x0" is the GDB annotations way of reading memory (GDB/MI is different), but the address is 0x0 and as such GDB complains that it "Cannot access memory at address 0x0". I have seen this when I had not coded things up correctly and was trying to watch a variable, but the variable was converted to a long and as you can appreciate it failed so the address became 0, which once I finished the changes for supporting memory read using variables it worked.

The break point number is returned by GDB when you set a break point and this is the number reported in the log. I have also seen where GDB does not re-se them.

Unfortunately the existing GDB logging is lacking info. It could be related to threading as I have not looked at multiple threading, apart from getting the threading dialog working (or what looks like working).

The following is an example of the GDB/MI logging, which has allot more usable debugging info to show you what GDB/MI plugin and GDB are doing:

Code
[debug   ] <std::shared_ptr<cbBreakpoint> Debugger_GDB_MI::AddBreakpoint(L  1349)> D:\Andrew_Development\Z_Testing_Apps\Printf_I64\main.cpp:130
[debug   ] <   dbg_mi::GDBBreakpointAddAction::OnStart(L   122)> GDBBreakpointAddAction::m_initial_cmd = 110000000000
[cmd     ] <    dbg_mi::CommandExecutor::ExecuteSimple(L    77)> cmd==>110000000000-break-insert -f D:\Andrew_Development\Z_Testing_Apps\Printf_I64\main.cpp:130==<
[info    ] <    dbg_mi::CommandExecutor::ProcessOutput(L    93)> Receive ==>110000000000^done,bkpt={number="4",type="breakpoint",disp="keep",enabled="y",addr="0x00007ff7af9c197b",func="main()",file="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",fullname="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",line="130",thread-groups=["i1"],times="0",original-location="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130"}<==
{Receive } <              Debugger_GDB_MI::OnGDBOutput(L   274)> Ignore =>(gdb) <=
[info    ] <dbg_mi::ResultParser* dbg_mi::CommandExecutor::GetResult(L   257)> Parsing: id: 110000000000 parser ==>type: result , ClassDone  , { m_value results: {bkpt={number=4,type=breakpoint,disp=keep,enabled=y,addr=0x00007ff7af9c197b,func=main(),file=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,fullname=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,line=130,thread-groups=["i1"],times=0,original-location=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130}} }<== for ==>^done,bkpt={number="4",type="breakpoint",disp="keep",enabled="y",addr="0x00007ff7af9c197b",func="main()",file="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",fullname="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",line="130",thread-groups=["i1"],times="0",original-location="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130"}<==
[debug   ] <dbg_mi::GDBBreakpointAddAction::OnCommandOutput(L    54)> Currently disabled id: 110000000000 index is 4 for =>type: result , ClassDone  , { m_value results: {bkpt={number=4,type=breakpoint,disp=keep,enabled=y,addr=0x00007ff7af9c197b,func=main(),file=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,fullname=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,line=130,thread-groups=["i1"],times=0,original-location=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130}} }<=
[debug   ] <dbg_mi::GDBBreakpointAddAction::OnCommandOutput(L    92)> finishing for id: 110000000000 for =>type: result , ClassDone  , { m_value results: {bkpt={number=4,type=breakpoint,disp=keep,enabled=y,addr=0x00007ff7af9c197b,func=main(),file=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,fullname=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,line=130,thread-groups=["i1"],times=0,original-location=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130}} }<=
[info    ] <dbg_mi::GDBBreakpointAddAction::~GDBBreakpointAddAction(L    29)> GDBBreakpointAddAction::destructor
[debug   ] <                 Debugger_GDB_MI::Continue(L  1202)> Debugger_GDB_MI::Continue
[cmd     ] <         Debugger_GDB_MI::CommitRunCommand(L  1142)> =>-exec-continue<=
[debug   ] <dbg_mi::GDBRunAction<StopNotification>::OnStart(L   112)> GDBRunAction::OnStart -> -exec-continue
[cmd     ] <    dbg_mi::CommandExecutor::ExecuteSimple(L    77)> cmd==>120000000000-exec-continue==<
[info    ] <    dbg_mi::CommandExecutor::ProcessOutput(L    93)> Receive ==>120000000000^running<==
[info    ] <    dbg_mi::CommandExecutor::ProcessOutput(L    93)> Receive ==>*running,thread-id="all"<==
{Receive } <              Debugger_GDB_MI::OnGDBOutput(L   274)> Ignore =>(gdb) <=
[info    ] <    dbg_mi::CommandExecutor::ProcessOutput(L    93)> Receive ==>=breakpoint-modified,bkpt={number="4",type="breakpoint",disp="keep",enabled="y",addr="0x00007ff7af9c197b",func="main()",file="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",fullname="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",line="130",thread-groups=["i1"],times="1",original-location="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130"}<==
{Receive } <              Debugger_GDB_MI::OnGDBOutput(L   274)> Ignore =>~"\n"<=
{Receive } <              Debugger_GDB_MI::OnGDBOutput(L   274)> Ignore =>~"Thread 1 hit Breakpoint 4, main () at D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130\n"<=
{Receive } <              Debugger_GDB_MI::OnGDBOutput(L   274)> Ignore =>~"\032\032D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130:3113:beg:0x7ff7af9c197b\n"<=
[info    ] <    dbg_mi::CommandExecutor::ProcessOutput(L    93)> Receive ==>*stopped,reason="breakpoint-hit",disp="keep",bkptno="4",frame={addr="0x00007ff7af9c197b",func="main",args=[],file="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",fullname="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",line="130",arch="i386:x86-64"},thread-id="1",stopped-threads="all"<==
{Receive } <              Debugger_GDB_MI::OnGDBOutput(L   274)> Ignore =>(gdb) <=
---------------
{Receive } <dbg_mi::ResultParser* dbg_mi::CommandExecutor::GetResult(L   265)> Parsing : id: 120000000000 parser ==>type: result , ClassRunning<== for ==>^running<==
[debug   ] <dbg_mi::GDBRunAction<StopNotification>::OnCommandOutput(L   101)> GDBRunAction success, the debugger is !stopped!
[debug   ] <dbg_mi::GDBRunAction<StopNotification>::OnCommandOutput(L   102)> GDBRunAction::Output - type: result , ClassRunning
[debug   ] <              dbg_mi::GDBExecutor::Stopped(L   325)> Executor started
---------------
{Receive } <dbg_mi::ResultParser* dbg_mi::CommandExecutor::GetResult(L   265)> Parsing : id: -1-000000001 parser ==>type: exec-async-ouput , ClassRunning  , { m_value results: {thread-id=all} }<== for ==>*running,thread-id="all"<==
---------------
{Receive } <                 Notifications::operator()(L   479)> notification event received: ==>type: exec-async-ouput , ClassRunning  , { m_value results: {thread-id=all} }<==
[info    ] <dbg_mi::ResultParser* dbg_mi::CommandExecutor::GetResult(L   257)> Parsing: id: -1-000000001 parser ==>type: notify-async-ouput , class unknown  , { m_value results: {bkpt={number=4,type=breakpoint,disp=keep,enabled=y,addr=0x00007ff7af9c197b,func=main(),file=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,fullname=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,line=130,thread-groups=["i1"],times=1,original-location=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130}} }<== for ==>=breakpoint-modified,bkpt={number="4",type="breakpoint",disp="keep",enabled="y",addr="0x00007ff7af9c197b",func="main()",file="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",fullname="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",line="130",thread-groups=["i1"],times="1",original-location="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130"}<==
{Receive } <     Notifications::ParseNotifyAsyncOutput(L   600)> Notification for breakpoint-modified: type: notify-async-ouput , class unknown  , { m_value results: {bkpt={number=4,type=breakpoint,disp=keep,enabled=y,addr=0x00007ff7af9c197b,func=main(),file=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,fullname=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,line=130,thread-groups=["i1"],times=1,original-location=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp:130}} }
[info    ] <dbg_mi::ResultParser* dbg_mi::CommandExecutor::GetResult(L   257)> Parsing: id: -1-000000001 parser ==>type: exec-async-ouput , ClassStopped  , { m_value results: {reason=breakpoint-hit,disp=keep,bkptno=4,frame={addr=0x00007ff7af9c197b,func=main,args=[],file=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,fullname=D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp,line=130,arch=i386:x86-64},thread-id=1,stopped-threads=all} }<== for ==>*stopped,reason="breakpoint-hit",disp="keep",bkptno="4",frame={addr="0x00007ff7af9c197b",func="main",args=[],file="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",fullname="D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp",line="130",arch="i386:x86-64"},thread-id="1",stopped-threads="all"<==
           <             Notifications::ParseStateInfo(L   533)> Breakpoint hit on line#: 130 in file: D:\\Andrew_Development\\Z_Testing_Apps\\Printf_I64\\main.cpp
[debug   ] <              dbg_mi::GDBExecutor::Stopped(L   321)> Executor stopped
[debug   ] <dbg_mi::GDBRunAction<StopNotification>::~GDBRunAction(L    94)> GDBRunAction::destructor
[debug   ] <   dbg_mi::GDBWatchesUpdateAction::OnStart(L  1660)> -var-update 1 *
[cmd     ] <    dbg_mi::CommandExecutor::ExecuteSimple(L    77)> cmd==>130000000000-var-update 1 *==<
[info    ] <    dbg_mi::CommandExecutor::ProcessOutput(L    93)> Receive ==>130000000000^done,changelist=[{name="var1",value="7",in_scope="true",type_changed="false",has_more="0"}]<==
{Receive } <              Debugger_GDB_MI::OnGDBOutput(L   274)> Ignore =>(gdb) <=
[info    ] <dbg_mi::ResultParser* dbg_mi::CommandExecutor::GetResult(L   257)> Parsing: id: 130000000000 parser ==>type: result , ClassDone  , { m_value results: {changelist=[{name=var1,value=7,in_scope=true,type_changed=false,has_more=0}]} }<== for ==>^done,changelist=[{name="var1",value="7",in_scope="true",type_changed="false",has_more="0"}]<==
[debug   ] <dbg_mi::GDBWatchesUpdateAction::ParseUpdate(L  1679)> List count: 1   , result: ==>type: result , ClassDone  , { m_value results: {changelist=[{name=var1,value=7,in_scope=true,type_changed=false,has_more=0}]} }<==
[debug   ] <dbg_mi::GDBWatchesUpdateAction::ParseUpdate(L  1773)> Update ==>var1<<== = ==>7<<==
[debug   ] <dbg_mi::GDBWatchesUpdateAction::OnCommandOutput(L  1832)> WatchUpdateAction::Output - finishing at==>130000000000<<==
[debug   ] <                     dbg_mi::UpdateWatches(L  1125)> updating watches

Offline tigerbeard

  • Almost regular
  • **
  • Posts: 158
Re: Linux: gdb ignores removed breakpoints
« Reply #6 on: April 07, 2022, 04:48:55 pm »
I new nothing about GDB/MI or the plugin, but just checked out this thread.
 https://forums.codeblocks.org/index.php/topic,24694.0.html?PHPSESSID=f4cf192e67e1c69570e79e98afe402e6

The Ubuntu repository does not seem to have a special GDB/MI I only found something like a perl addon referencing it. Is the MI part of the standard gdb?
Using a better interface than CLI parsing is a great idea and hopefully makes things less complicated in the future. There are also some interesting features possible like function return values. Sound promising. And a lot happening at the moment. Thanks for your efforts!

I am aware that the log is incomplete, I thought the sections at least show there there is a BP number mixup. When I have got something more useful I will post it


Not sure if i got your point with the 0x0, though. I did have a number of invalid watches in the watch window at that time so I did not think that section was really relevant. Are you saying this section points to an error in my code that might throw off gdb or are you saying this could point to an other issue with gdb?