Author Topic: CC makes C::B hang in ExpandBackticks (Read 73208 times)

Jenna · « **on:** January 02, 2013, 01:09:01 pm »

I just opened the C::B worksapece and started a rebuild.
C::B hangs after the first cleaning message.
This seems to happen due to the timer-triggered kick-in of the nativeparser.

I attach a backtrace of my attached (hanging) C::B process.
It's without debugging symbols (stripped version of my repo).
C::B is r8710.

On the first look it looks like a deadlock after the call to Manager::Yield(); in void CompilerGCC::BuildStateManagement() .

No time to digg into it deeper at the moment.

MortenMacFly · « **Reply #1 on:** January 02, 2013, 02:00:21 pm »

Quote from: jens on January 02, 2013, 01:09:01 pm

On the first look it looks like a deadlock after the call to Manager::Yield(); in void CompilerGCC::BuildStateManagement() .

Hmmm... again such a nice one unable to reproduce. But I wonder why calling Yield should lead to a deadlock. Could it be that instead the command issued by CC to obtain the compiler dirs hangs? Can you verify this in the task manager somehow? Especially what command is it? Is this (compiler) executable present?

Jenna · « **Reply #2 on:** January 02, 2013, 02:43:39 pm »

Quote from: jens on January 02, 2013, 01:09:01 pm

On the first look it looks like a deadlock after the call to Manager::Yield(); in void CompilerGCC::BuildStateManagement() .

Quote from: MortenMacFly on January 02, 2013, 02:00:21 pm

But I wonder why calling Yield should lead to a deadlock.

I did not say, that the call to Yield leads to the dealock, but it appears right after it.
I guess Yield allows the pending events to be called (timer-event in this case).
Athe the same moment the BuildManagement has started and CompilerCommandGenerator::Init is called, which clears all its member-variables and reinitializes them.
Yes the compiler exists and the hang is in SetupLinkerOptions, so the call must be either in `wx-config --libs`, `wx-config --libs aui` or `pkg-config --libs gtk+-2.0`.
There might be something done in compilergcc, what blocks the wxExecute for some reason.

I will try to reproduce with a debug-build, so I get a little more information.

MortenMacFly · « **Reply #3 on:** January 02, 2013, 02:53:20 pm »

Quote from: jens on January 02, 2013, 02:43:39 pm

There might be something done in compilergcc, what blocks the wxExecute for some reason.

I would guess that, too. The direct wxExecute calls are no good IMHO. Some time ago I had implemented a tiny cbProcess class that wraps direct wxExecute calls in a wxThreadHelper and returns the output event based. The class is ready and I wanted to replace the calls in the compiler plugin with that and change the logic to be event based. But its not as easy as I thought, unfortunately and due to conflicts with other changes in the compiler plugin I dropped it... but I still believe this would help.

Jenna · « **Reply #4 on:** January 03, 2013, 10:54:12 am »

A problem might be, that the parser(s) run in (a) thread(s) and indirectly call wxExecute (in ExpandBackticks for example.
Even if wxExecute is a function in wxBase it uses wxGui also.
My guess is, that this might lead to problems, if it is not called from the main-thread.
wxExecute might also call Yield (according to the docu), but this can also be a problem, if it is done recursively (at least if onlyIfNeeded is not set).
I attach a full backtrace, but I don't think it really helps here.
The best way to avoid issues would be to avoid any gui-calls from non-main threads and use events and callbacks there.

MortenMacFly · « **Reply #5 on:** January 03, 2013, 10:59:05 am »

Quote from: jens on January 03, 2013, 10:54:12 am

[...]
Even if wxExecute is a function in wxBase it uses wxGui also.
[...]
The best way to avoid issues would be to avoid any gui-calls from non-main threads and use events and callbacks there.

How shall that work? You mean CC should command the core to run wxExecute statements and wait until its signalled to expand backticks, for example?

Jenna · « **Reply #6 on:** January 03, 2013, 12:01:27 pm »

Quote from: MortenMacFly on January 03, 2013, 10:59:05 am

How shall that work? You mean CC should command the core to run wxExecute statements and wait until its signalled to expand backticks, for example?

That would be a solution.
Something like issuing a cbEVENT with the commandline as parameter and a pointer to a callback-function.

This must not necessarily done by the core (but this might be useful for others also), but can also be done by the not threaded part of CC of course.
Something like this might be necessary for the clasbrowserbuilderthread also, but here it would probably lead to tons of messages (large trees), which needs many resources.

oBFusCATed · « **Reply #7 on:** January 04, 2013, 12:53:58 am »

Jens: Is there any chance that you are using gtk+ 3.6.

For me it is almost impossible to rebuild cb's workspace at the moment.

Code

(gdb) thread apply all bt

Thread 7 (Thread 0x7fc561d04700 (LWP 14207)):
#0  0x00007fc56e32d59c in pthread_cond_wait () from /lib64/libpthread.so.0
#1  0x00007fc56fa7d893 in wxConditionInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#2  0x00007fc56fa7dda0 in wxSemaphoreInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#3  0x00007fc56f3da87e in ?? ()
#4  0x0000000000f5d701 in ?? ()
#5  0x0000000000f5dd28 in ?? ()
#6  0x00000001df148a98 in ?? ()
#7  0x00007fc54ad69fd0 in ?? ()
#8  0x00000000007894a0 in ?? ()
#9  0x00007fc56fa7e991 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#10 0x00007fc56afab344 in ?? () from /usr/lib64/libGL.so.1
#11 0x00007fc56e328d96 in start_thread () from /lib64/libpthread.so.0
#12 0x00007fc56e06246d in clone () from /lib64/libc.so.6

Thread 6 (Thread 0x7fc561503700 (LWP 14208)):
#0  0x00007fc56e32d59c in pthread_cond_wait () from /lib64/libpthread.so.0
#1  0x00007fc56fa7d893 in wxConditionInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#2  0x00007fc56fa7dda0 in wxSemaphoreInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#3  0x00007fc56f3da87e in ?? ()
#4  0x0000000000000001 in ?? ()
#5  0x0000000000f5dd60 in ?? ()
#6  0x0000000000f5eb60 in ?? ()
#7  0x00007fc56fa7d9d9 in wxMutexInternal::Unlock() () from /usr/lib64/libwx_baseu-2.8.so.0
#8  0x00007fc56fa7e991 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#9  0x00007fc56afab344 in ?? () from /usr/lib64/libGL.so.1
#10 0x00007fc56e328d96 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fc56e06246d in clone () from /lib64/libc.so.6

Thread 5 (Thread 0x7fc560d02700 (LWP 14209)):
#0  0x00007fc56e32d59c in pthread_cond_wait () from /lib64/libpthread.so.0
#1  0x00007fc56fa7d893 in wxConditionInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#2  0x00007fc56fa7dda0 in wxSemaphoreInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#3  0x00007fc56f3da87e in ?? ()
#4  0x0000000000000001 in ?? ()
#5  0x0000000000f5dd98 in ?? ()
#6  0x0000000000f5f500 in ?? ()
#7  0x00007fc56fa7d9d9 in wxMutexInternal::Unlock() () from /usr/lib64/libwx_baseu-2.8.so.0
#8  0x00007fc56fa7e991 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#9  0x00007fc56afab344 in ?? () from /usr/lib64/libGL.so.1
#10 0x00007fc56e328d96 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fc56e06246d in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7fc560501700 (LWP 14210)):
#0  0x00007fc56e32d59c in pthread_cond_wait () from /lib64/libpthread.so.0
#1  0x00007fc56fa7d893 in wxConditionInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#2  0x00007fc56fa7dda0 in wxSemaphoreInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#3  0x00007fc56f3da87e in ?? ()
#4  0x0000000000000001 in ?? ()
#5  0x0000000000f5ddd0 in ?? ()
#6  0x0000000000f5fc80 in ?? ()
#7  0x00007fc56fa7d9d9 in wxMutexInternal::Unlock() () from /usr/lib64/libwx_baseu-2.8.so.0
#8  0x00007fc56fa7e991 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#9  0x00007fc56afab344 in ?? () from /usr/lib64/libGL.so.1
#10 0x00007fc56e328d96 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fc56e06246d in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7fc552c29700 (LWP 14236)):
#0  0x00007fc56e32d59c in pthread_cond_wait () from /lib64/libpthread.so.0
#1  0x00007fc56fa7d893 in wxConditionInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#2  0x00007fc56fa7dda0 in wxSemaphoreInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#3  0x00007fc552d509e0 in ClassBrowserBuilderThread::Entry (this=0x1bd8210) at /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/classbrowserbuilderthread.cpp:191
#4  0x00007fc56fa7e991 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#5  0x00007fc56afab344 in ?? () from /usr/lib64/libGL.so.1
#6  0x00007fc56e328d96 in start_thread () from /lib64/libpthread.so.0
#7  0x00007fc56e06246d in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7fc55c23d700 (LWP 14237)):
#0  0x00007fc56e32d59c in pthread_cond_wait () from /lib64/libpthread.so.0
#1  0x00007fc56fa7d893 in wxConditionInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#2  0x00007fc56fa7dda0 in wxSemaphoreInternal::Wait() () from /usr/lib64/libwx_baseu-2.8.so.0
#3  0x00007fc56f35be27 in ?? ()
#4  0x000000000267a600 in ?? ()
#5  0x0000000001c8d501 in ?? ()
#6  0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7fc573842940 (LWP 14186)):
#0  0x00007fc56e06efae in __lll_lock_wait_private () from /lib64/libc.so.6
#1  0x00007fc56dff8ed9 in _L_lock_7748 () from /lib64/libc.so.6
#2  0x00007fc56dff7c5d in calloc () from /lib64/libc.so.6
#3  0x00007fc56815efc2 in ?? () from /usr/lib64/libnvidia-tls.so.310.19
#4  0x00007fc56fa48ff4 in wxStringBase::AllocBuffer(unsigned long) () from /usr/lib64/libwx_baseu-2.8.so.0
#5  0x00007fc56fa491bd in wxStringBase::AllocBeforeWrite(unsigned long) () from /usr/lib64/libwx_baseu-2.8.so.0
#6  0x00007fc56fa49fe6 in wxStringBase::AssignCopy(unsigned long, wchar_t const*) () from /usr/lib64/libwx_baseu-2.8.so.0
#7  0x00007fc56fa4a0eb in wxStringBase::operator=(wchar_t) () from /usr/lib64/libwx_baseu-2.8.so.0
#8  0x00007fc56fa1fcb8 in wxFileName::GetPathSeparators(wxPathFormat) () from /usr/lib64/libwx_baseu-2.8.so.0
---Type <return> to continue, or q <return> to quit---
#9  0x00007fc56fa1fe0a in wxFileName::GetPathTerminators(wxPathFormat) () from /usr/lib64/libwx_baseu-2.8.so.0
#10 0x00007fc56fa21436 in wxFileName::SplitPath(wxString const&, wxString*, wxString*, wxString*, wxString*, bool*, wxPathFormat) () from /usr/lib64/libwx_baseu-2.8.so.0
#11 0x00007fc56fa233ba in wxFileName::SplitPath(wxString const&, wxString*, wxString*, wxString*, wxPathFormat) () from /usr/lib64/libwx_baseu-2.8.so.0
#12 0x00007fc56fa23531 in ?? () from /usr/lib64/libwx_baseu-2.8.so.0
#13 0x00007fc56fa23a6a in ?? () from /usr/lib64/libwx_baseu-2.8.so.0
#14 0x00007fc570ac5686 in wxDebugReport::wxDebugReport() () from /usr/lib64/libwx_gtk2u_qa-2.8.so.0
#15 0x0000000000455f19 in CodeBlocksApp::OnFatalException (this=0x7c3a40) at /home/obfuscated/projects/codeblocks/git/src/src/app.cpp:819
#16 0x00007fc56fa832ec in wxFatalSignalHandler () from /usr/lib64/libwx_baseu-2.8.so.0
#17 <signal handler called>
#18 0x00007fc56dff20a3 in malloc_consolidate () from /lib64/libc.so.6
#19 0x00007fc56dff3cc4 in _int_malloc () from /lib64/libc.so.6
#20 0x00007fc56dff6b84 in malloc () from /lib64/libc.so.6
#21 0x00007fc56e02c059 in __alloc_dir () from /lib64/libc.so.6
#22 0x00007fc56afa82da in ?? () from /usr/lib64/libGL.so.1
#23 0x00007fc56afa8a43 in ?? () from /usr/lib64/libGL.so.1
#24 0x00007fc56afa8d0c in ?? () from /usr/lib64/libGL.so.1
#25 0x00007fc56afa8d6a in ?? () from /usr/lib64/libGL.so.1
#26 0x00007fc56af8486a in ?? () from /usr/lib64/libGL.so.1
#27 0x00007fc57364fafb in _dl_fini () from /lib64/ld-linux-x86-64.so.2
#28 0x00007fc56dfaf441 in __run_exit_handlers () from /lib64/libc.so.6
#29 0x00007fc56dfaf495 in exit () from /lib64/libc.so.6
#30 0x00007fc572db4861 in ?? () from /usr/lib64/libgdk-x11-2.0.so.0
#31 0x00007fc56da7a9d5 in _XError () from /usr/lib64/libX11.so.6
#32 0x00007fc56da77dd7 in ?? () from /usr/lib64/libX11.so.6
#33 0x00007fc56da77e25 in ?? () from /usr/lib64/libX11.so.6
#34 0x00007fc56da78465 in _XEventsQueued () from /usr/lib64/libX11.so.6
#35 0x00007fc56da6961d in XPending () from /usr/lib64/libX11.so.6
#36 0x00007fc572daa6b6 in ?? () from /usr/lib64/libgdk-x11-2.0.so.0
#37 0x00007fc57121d525 in g_main_context_prepare () from /usr/lib64/libglib-2.0.so.0
#38 0x00007fc57121dbf3 in ?? () from /usr/lib64/libglib-2.0.so.0
#39 0x00007fc57121e11a in g_main_loop_run () from /usr/lib64/libglib-2.0.so.0
#40 0x00007fc57313e927 in gtk_main () from /usr/lib64/libgtk-x11-2.0.so.0
#41 0x00007fc57010c528 in wxEventLoop::Run() () from /usr/lib64/libwx_gtk2u_core-2.8.so.0
#42 0x00007fc57017fa5b in wxAppBase::MainLoop() () from /usr/lib64/libwx_gtk2u_core-2.8.so.0
#43 0x0000000000455cac in CodeBlocksApp::OnRun (this=0x7c3a40) at /home/obfuscated/projects/codeblocks/git/src/src/app.cpp:787
#44 0x00007fc56fa2dcda in wxEntry(int&, wchar_t**) () from /usr/lib64/libwx_baseu-2.8.so.0
#45 0x0000000000453279 in main (argc=8, argv=0x7fffdf149de8) at /home/obfuscated/projects/codeblocks/git/src/src/app.cpp:266

Jenna · « **Reply #8 on:** January 04, 2013, 01:29:53 am »

Quote from: oBFusCATed on January 04, 2013, 12:53:58 am

Jens: Is there any chance that you are using gtk+ 3.6.

I use gtk+ 3.4 and 2.24 for C::B I use gtk+2, of course.
There is no wx2.8 with gtk3 available and C::B itself compiles and lnks against gtk+2.

I can reproduce the issue not always but quite often, it's always ExpandBackticks called by CC-timer which hangs.

MortenMacFly · « **Reply #9 on:** January 04, 2013, 06:33:37 am »

Quote from: jens on January 04, 2013, 01:29:53 am

I can reproduce the issue not always but quite often, it's always ExpandBackticks called by CC-timer which hangs.

I can't.

Must be Linux only - maybe because under Windows there is nothing to expand...?! Did you manage to find a way to reproduce it under Windows? Is it really ExpandBackticks only?

Edit: Oh, and BTW: Where does CC call ExpandBackticks? I didn't find it with a quick search...

MortenMacFly · « **Reply #10 on:** January 04, 2013, 06:37:40 am »

Quote from: oBFusCATed on January 04, 2013, 12:53:58 am

For me it is almost impossible to rebuild cb's workspace at the moment.

In your BT I don-t even see what goes wrong...

Jenna · « **Reply #11 on:** January 04, 2013, 09:30:28 am »

Quote from: MortenMacFly on January 04, 2013, 06:33:37 am

Quote from: jens on January 04, 2013, 01:29:53 am
I can reproduce the issue not always but quite often, it's always ExpandBackticks called by CC-timer which hangs.
I can't. Must be Linux only - maybe because under Windows there is nothing to expand...?! Did you manage to find a way to reproduce it under Windows? Is it really ExpandBackticks only?

Edit: Oh, and BTW: Where does CC call ExpandBackticks? I didn't find it with a quick search...

CC calls GetCommandGenerator in AddProjectToParser.
The commandgenerator get initiated and calls SetupCompilerOptions, SetupLinkerOptions etc.
And this calls ExpandBackticks.
wxExecute always hangs in wxGUIAppTraits::WaitForChild() or wxPipeInputStream::CanRead().
The latter might be the real issue for some reason I don't know.

I just saw that I forgot to attach the full bt in one of my prior posts.
I attach it here.

oBFusCATed · « **Reply #12 on:** January 04, 2013, 09:58:25 am »

Quote from: MortenMacFly on January 04, 2013, 06:37:40 am

In your BT I don-t even see what goes wrong...

The strangest thing is that it happens if I run cb from cb, if I use the run.sh script it works.

MortenMacFly · « **Reply #13 on:** January 04, 2013, 11:39:53 am »

Quote from: jens on January 04, 2013, 09:30:28 am

CC calls GetCommandGenerator in AddProjectToParser.

So it means this is the actually command that should be wrapped in an event, or is it the wxExecute call in the compiler? The latter is more hard to do, cause it would change the logic in CC as the call does immediately return w/o a result.

I wonder what's best here:
Actually calling GetCommandGenerator is done to obtain the project include dirs which will trigger wxExecute if backticked expressions are in place. Maybe a better solution is if CC does not use CommandGenerator at all, but evaluates the project / target settings itself, including executing what ever. Another option is to move the method from CommandGenerator to the core and wrap it there... maybe extend project manager.

What would be a good thing to do [tm]?

oBFusCATed · « **Reply #14 on:** January 04, 2013, 12:02:26 pm »

Quote from: MortenMacFly on January 04, 2013, 11:39:53 am

Maybe a better solution is if CC does not use CommandGenerator at all, but evaluates the project / target settings itself, including executing what ever.

Why? Duplicating code is never a good thing.

BTW: I'm looking why the no-change-build of cb is slow. It turned out that the DirectCommands object is recreated tons of times (takes 30-50% time).

Code::Blocks Forums

News:

Author Topic: CC makes C::B hang in ExpandBackticks (Read 73208 times)

Jenna

CC makes C::B hang in ExpandBackticks

MortenMacFly

Re: CC makes C::B hang in ExpandBackticks

Jenna

Re: CC makes C::B hang in ExpandBackticks

MortenMacFly

Re: CC makes C::B hang in ExpandBackticks

Jenna

Re: CC makes C::B hang in ExpandBackticks

MortenMacFly

Re: CC makes C::B hang in ExpandBackticks

Jenna

Re: CC makes C::B hang in ExpandBackticks

oBFusCATed

Re: CC makes C::B hang in ExpandBackticks

Jenna

Re: CC makes C::B hang in ExpandBackticks

MortenMacFly

Re: CC makes C::B hang in ExpandBackticks

MortenMacFly

Re: CC makes C::B hang in ExpandBackticks

Jenna

Re: CC makes C::B hang in ExpandBackticks

oBFusCATed

Re: CC makes C::B hang in ExpandBackticks

MortenMacFly

Re: CC makes C::B hang in ExpandBackticks

oBFusCATed

Re: CC makes C::B hang in ExpandBackticks