My configuration : custom C::B build (quite some modification), 32 bits, Win7, wx2.8.12.
I can modify/rebuild wx and C::B
I randomly (*) end in a deadlock, C::B unresponsive, not eating any processor
(*)can be 5 in a row, just after opening C::B or last for one week to appear. Seems to be related to activity - the more agressive I am on keyboard, the more it
seems to occurs. I've no easy reproducable steps.
I would nearly say that it has nearly always occured, it's possible I see it more often now.
So to debug this, I launch C::B in GDB and work with that [quite a bit slow, on codecompletion specifically]
I share my findings :
So I just deadlockeded(*), I break process in GDB :
(*) It's not my first deadlock trace, and all are consistent with following trace
main thread stack trace#0 0x7790f8b1 ntdll!ZwWaitForSingleObject() (C:\Windows\SysWOW64\ntdll.dll:??)
#1 0x7790f8b1 ntdll!ZwWaitForSingleObject() (C:\Windows\SysWOW64\ntdll.dll:??)
#2 0x75100a91 WaitForSingleObjectEx() (C:\Windows\syswow64\KernelBase.dll:??)
#3 0x3a0 ?? () (??:??)
#4 ?? ?? () (??:??)
Thread is stuck in a WaitForSingleObject (a standard windows multitasking primivitive, sort of
select with only one handle)
Stack
seems wired. But it's probably GDB that is wired and is lost between kernel/user call (or Windows that is not as GDB expect as you prefear, don't mind). Trying to force backtrace to start from specific address end in a GDB crash ... so as deadlock didn't occurs enought often, I do not follow this way
With ProcessExplorer, I catch the whole frame :
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x6c0
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x4a8
wow64.dll!Wow64SystemServiceEx+0x1ce
wow64.dll!Wow64KiUserCallbackDispatcher+0x204
wow64win.dll+0x3798c
ntdll.dll!KiUserCallbackDispatcher+0x1f
wow64win.dll+0x3fe3a
wow64win.dll+0x1aea8
wow64.dll!Wow64SystemServiceEx+0xd7
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x2d
wow64.dll!Wow64SystemServiceEx+0x1ce
wow64.dll!Wow64LdrpInitialize+0x429
ntdll.dll!RtlUniform+0x6e6
ntdll.dll!RtlCreateTagHeap+0xa7
ntdll.dll!LdrInitializeThunk+0xe
ntdll.dll!ZwWaitForSingleObject+0x15
kernel32.dll!WaitForSingleObjectEx+0x43 <=======
kernel32.dll!WaitForSingleObject+0x12 <=======
wxmsw28u_gcc_cb.dll!_ZN17wxCriticalSection5LeaveEv+0x85
codecompletion.dll+0xd1c7 <=======
codecompletion.dll+0xd54f
codecompletion.dll+0x4a071
wxmsw28u_gcc_cb.dll!_ZNK12wxAppConsole11HandleEventEP12wxEvtHandlerMS0_FvR7wxEventES3_+0x22
codecompletion.dll+0x80a8d
codecompletion.dll+0x813ae
wxmsw28u_gcc_cb.dll!_ZNK12wxAppConsole11HandleEventEP12wxEvtHandlerMS0_FvR7wxEventES3_+0x22
wxmsw28u_gcc_cb.dll!_ZN12wxEvtHandler20ProcessPendingEventsEv+0xa1
wxmsw28u_gcc_cb.dll!_ZN12wxAppConsole20ProcessPendingEventsEv+0x40
ntdll.dll!KiUserCallbackDispatcher+0x2e
wxmsw28u_gcc_cb.dll!_ZN11wxEventLoop8DispatchEv+0x7c
wxmsw28u_gcc_cb.dll!_ZN17wxEventLoopManual3RunEv+0x128
wxmsw28u_gcc_cb.dll!_Z14wxUninitializev+0x181
codeblocks.exe+0x3898
codeblocks.exe+0x10fd
ntdll.dll!RtlInitializeExceptionChain+0x63
ntdll.dll!RtlInitializeExceptionChain+0x36
The interesting point is that at this stage is that it end at codecompletion.dll. Obviously ProcessExplorer use PDB symbols where GCC use DWARF. I have a tool to convert them, but it crash on such large file
Playing with dll offset and info symbol in GDB lead me to :
ClassBrowser::OnViewScope(wxCommandEvent&) + 599 in section .text of D:\bat\data\program\gcc\codeblocks\src\devel\share\codeblocks\plugins\codecompletion.dll
[Note that I'm not 100% sure of offset calculation]
Now, the other thread involved is (GDB trace) :
#0 0x75f272b9 USER32!GetPropW() (C:\Windows\syswow64\user32.dll:??)
#1 0x75f296c5 USER32!SendMessageW() (C:\Windows\syswow64\user32.dll:??)
#2 0x6cdbf2a7 wxTreeCtrl::GetItemParam(wxTreeItemId const&) const() (D:\bat\data\program\gcc\codeblocks\src\devel\wxmsw28u_gcc_cb.dll:??)
#3 0xbe4fac8 ?? () (??:??)
#4 0x6cdbfe99 wxTreeCtrl::DoExpand(wxTreeItemId const&, int) () (D:\bat\data\program\gcc\codeblocks\src\devel\wxmsw28u_gcc_cb.dll:??)
#5 0x6cdc0047 wxTreeCtrl::CollapseAndReset(wxTreeItemId const&) () (D:\bat\data\program\gcc\codeblocks\src\devel\wxmsw28u_gcc_cb.dll:??)
#6 0xbe4fd60 ?? () (??:??)
#7 0x5828336 ClassBrowserBuilderThread::BuildTree(this=this@entry=0x7578c30) (D:\bat\data\program\gcc\codeblocks\src\plugins\codecompletion\classbrowserbuilderthread.cpp:550)
#8 0x582883d ClassBrowserBuilderThread::Entry(this=0x7578c30) (D:\bat\data\program\gcc\codeblocks\src\plugins\codecompletion\classbrowserbuilderthread.cpp:254)
#9 0x6ccd727c wxThreadInternal::DoThreadStart(wxThread*) () (D:\bat\data\program\gcc\codeblocks\src\devel\wxmsw28u_gcc_cb.dll:??)
#10 0x6ccd731f wxThreadInternal::WinThreadStart(void*)@4() (D:\bat\data\program\gcc\codeblocks\src\devel\wxmsw28u_gcc_cb.dll:??)
#11 0xbe4ff10 ?? () (??:??)
So it's ClassBrowserBuilderThread that is involved. It's currently refreshing the ClassBrowser wxTree, that end to a "SendMessage" in windows. SendMessage in a thread will wait that message pump that handle the specific windows process the message
But the message pump is in main thread ... and it's waiting something else, my guess is that it's waiting ClassBrowser to process something, ending in a classical deadlock.
At this point, it can be :
-a specific Windows issue (or not)
-a wx issue (lock/unlock) issue
-a usage issue
I've added 2
invasive trace
1)One in
bool Manager::ProcessEvent(CodeBlocksEvent& event)
to track if issue is during a specific C::B event process.
Answer : No
Over 7 recorded deadlock it's nearly always after a call to JumpTracker plugin (so it's related to some input/editor move)
353188229 0 CALL 10364:M11JumpTrackerFvR15CodeBlocksEventE
353188229 0 END CALL 10364:M11JumpTrackerFvR15CodeBlocksEventE
One time it after a call to debugger plugin
1195030 0 CALL 10361:M16cbDebuggerPluginFvR15CodeBlocksEventE
1195030 0 END CALL 10361:M16cbDebuggerPluginFvR15CodeBlocksEventE
2)One for logging all wx Event
int CodeBlocksApp::FilterEvent(wxEvent& event)
I had to remove some of them occuring very often from log (or it end to hundred MB files in one minute)
Here, it always end up on an event "10183", that is "wxEVT_COMMAND_MENU_SELECTED" on my wx
Searching a bit tell me that this event seems to be send when something is clicked in a menu, but also as a mean to send inter thread event. In sdk_events.h
// Thread event, this is basically a derived wxCommandEvent but enforce a deep copy of its
// m_cmdString member. wxEVT_COMMAND_MENU_SELECTED is reused and event handlers are matched by
// ids. This is just to conserve the old code, an alternative is use some
// new event type like: cbEVT_THREAD_LOG_MESSAGE, cbEVT_THREAD_LOGDEBUG_MESSAGE
// cbEVT_THREAD_SYSTEM_HEADER_UPDATE.
For now I just catched the id associated with event one time :
EVT 10183 id:1463 ref:0x00000000
I will follow my way, but remarks and idea are welcomed to be more effective.