High CPU usage

[thomas]

Call it a misnomer. HasInput() indeed polls input and posts messages to the attached event handler. As a side effect, it also returns true when it had input

Patches, humm... ok, but I'd better checkout a pristine copy first, after I went through the code like a berserker.  :lol:

So... I'll change CompilerGCC, DebuggerGDB, and PipedProcess to avoid using OnIdle, but leave the
ProjectManager::OnIdle function in place and only comment the event table macro .

To wxScintilla, I will only add a "NOTE" comment, as you cannot just modify the idle function there without affecting line breaking. Maybe someone wants to look into that another time, but for the time being, this is probably not so much of an issue. The "typing" peak of the graph looks intimidating, but I was really hitting keys as fast as I could. If someone really experiences the fan winding up due to typing, one can still look at that.

[thomas]

Humm, interesting...  But did you try benchmarking the code with a profiler? This would be objective measurements instead of your subjectives ones. I remember that the windows task manager is not accurate for anything except for detecting the most resource consuming process

I did not profile the code, unluckily. Ow... rebuild everything with profiling info? Painful But I might still do that later.
I guess the graphs are telling enough, though. Using lowest update speed, task manager averages total cpu usages for about 4-5 seconds per tick, given that nothing else ran on this machine, that should give a quite comparable measure. Hopefully, we may assume that the Windows scheduler knows how much time it has spent in user code. The peaks are on a consistent level for > 30 seconds every time, too, so as a general comparison (without knowing the actual function) I guess it will do.

On my computer, the most consuming part is the wx event loop dispatch (see call stack)

That, too, confirms my hypothesis. The exact thing I am trying to do by removing OnIdle and WakeupIdle (and RequestMore) is reduce the number of messages that get fed into the message queue.

[rickg22]

Hmmmmmmmmm if this is the case (posting messages), what are the chances of reentrancy in the code? Shouldn't we add some kind of flag to prevent reentrancy in the timer function? i.e. if the message polling takes too long, the ontimer would get triggered again, leading to crashes etc etc.

[thomas]

For all I know wxWindows takes care of that, so we need not fear that.
But it is nevertheless inefficient to send that many messages.
To give an example, wxScintilla has code like this:

Code

void wxScintilla::OnIdle (wxIdleEvent& evt)
{
m_swx->DoOnIdle (evt);
}
...
void ScintillaWX::DoOnIdle(wxIdleEvent& evt)
{
if ( Idle() )
    evt.RequestMore();
else
    SetIdle(false);
}
...
bool Editor::Idle() 
{
bool wrappingDone = (wrapState == eWrapNone) || (!backgroundWrapEnabled);
...
return !idleDone;
}

In other words, this code does "post idle messages as long as there is text left to wrap" and "do the actual work when the messages come back in".
It is not surprising that the profiler shows the event loop dispatcher as the most CPU intensive section. It's being used

[rickg22]

Yes, it might sound inefficient, but take into account that each idle cycle is handle separately.

In other words, if you have 500 objects doing idle processing, each one gets its own timeslice. You could call it "cooperative multithreading".

I think this idle handling of scintilla is precisely what makes it so responsive. (Believe me, I've used other editors, and word wrapping a 300K file can be QUITE unresponsive! )

[thomas]

And luckily, Scintilla does not need to word wrap all the time, too. That's why I said the Scintilla bit is probably not an issue (unless some complains about the fan winding up when he presses keys).

Anyway, for the compiler, it seems to make sense, there is no noticeable difference when compiling a file, just the overall load is lower.

[takeshimiya]

thomas: if you don't want to profile, I recommend you the freeware Sysinternal Process Explorer, it's like the Task Manager, but you can set the refresh time, you can see the CPU/MEM graphs per process, can see the stack or CPU for each thread, support multiple CPUs, etc.

I'm using it as a Task Manager replace. 

http://www.sysinternals.com/Utilities/ProcessExplorer.html

[thomas]

This is great, thanks

I would love to profile all of my code, it is just so painful, especially since gprof is the only profiler I have on my hard drive :s  (I shall look into that LTProf during the weekend).

Can't someone write a one-click profiler plugin for Code::Blocks so my life is not that miserable? :lol:

[Ceniza]

Can't someone write a one-click profiler plugin for Code::Blocks so my life is not that miserable?

What about the one being developed? link

I haven't tested it yet but if it's already usable and compiles without problems with Code::Blocks CVS I could provide it in the binary snapshots

[mandrav]

Hi Thomas et al
I just wanted to say why the compiler's and debugger's input polling uses OnIdle().

If we remove the Has Input() from OnIdle() and move it to OnTimer(), everything might work fine for you but it breaks the non-windows builds (at least it used too). In non-windows platforms, any deviation from the current strategy, will lock the process being checked for input (and subsequently C::B). If I remember correctly from the tests I had performed back then to see what was wrong, it looked like the stream wouldn't correctly return eof...
If someone wishes to debug this, be my guest
The current way things are done, is the only way I could find to correctly launch a process and read its input/output without freezing.

Maybe by simply changing OnTimer() to check for valid process before calling wxWakeUpIdle(), could satisfy all sides...

[zieQ]

thomas, with LTProf, you don't need to recompile the code, that's why I'm happy with it

Also note that, on my screenshot, the idle event processing is at the second place

[thomas]

lock the process being checked for input (and subsequently C::B).

I did experience that problem with the svn/cvs plugin in very early versions. As an "optimization", I only polled output once in OnTerminate. That way, you would get all output in one chunk, but that was fine.
The nasty thing about this approach is that it works perfectly for output less than the pipe size (4k). However, when the invoked program outputs more than 4k, the pipe will block, and you wonder why suddenly everything freezes. Checking out Code::Blocks suddenly froze my PC  (by coincidence, as the project grew bigger, cvs output just crossed the 4k border) when all was fine the day before.
With regular polling via OnTimer, I haven't experienced a similar problem yet.

If I remember correctly [...] stream wouldn't correctly return eof...

Oh yes, had that too, that's when you don't poll in OnTerminate. When polling via OnTimer, I thought "why poll again in OnTerminate then!". OnIdle 99.9% certainly polled just before the process dies, but OnTimer only polls on fixed time intervals. After OnTerminate, m_process is null, so EOF is not correct.
PipedProcess does clean up in OnTerminate, so it should actually work. Did you actually have such a problem recently, or are these only the heirs of long forgotten nightmares from the dark ages?

Maybe by simply changing OnTimer() to check for valid process before calling wxWakeUpIdle(), could satisfy all sides...

Sounds like a good compromise. That will eliminate idle messages as long as no compile is running. While a compile is running, CPU usage is ~99% anyway, so the cpu fan will spin up either way

What about the one being developed?

:oops: Was completely unaware of that.

[rickg22]

Cool! So all we need (read: we = Yiannis :lol: ) is to add OnTerminate stuff and we'll get rid of the Idle stuff?

[thomas]

No, don't need to add anything, OnTerminate is already good the way it is. It always was.

But like I pointed out, polling inside OnTerminate alone is not enough, that could lead to filling the pipe until it blocks in some scenarios ( >4k of compiler warnings).
The reason OnTimer (and possibly OnIdle) is needed is that you want to poll before that happens.

I strongly believe that OnIdle is unnecessary when polling is done both in OnTimer and OnTerminate, though (which is done). The former prevents the pipe from overfilling, and the latter catches the last few lines of output that would otherwise slip through. Adding OnIdle on top of that only polls a few times more, but should not make a difference really.

[grv575]

Sound like the right way to go about it (OnIdle + OnTerminate).  I'd be willing to test on windows and linux (wx2.4.2 & wx2.6.1 which are already built for both platforms) to make sure it doesn't freeze up.