High CPU usage

[thomas]

This maybe?

Code

EVT_IDLE( CompilerGCC::OnIdle )
EVT_IDLE( DebuggerGDB::OnIdle )

These plugins read stdout from processes they launch on a regular base triggered by a wxTimer (100ms). OnTimer() does not do anything except for calling wxWakeUpIdle(), and OnIdle() does the actual polling.

While moving the mouse, the system goes idle quite a few times, so it may be that OnIdle() is indeed called many, many times per second. At the very least, this means a lot of calls to wxProcess::IsInputAvailable() which calls wxStream::CanRead(). If there are actually processes running, two wxTextInputStreams  are opened in addition, and one line is read from each.

I am not saying this *is* the actual cause, but it might just be. One should run gprof to see how often OnIdle() is really called. If it is 50 or 100 times per second, we have a culprit. In that case, have OnTimer() poll for input, which is better anyway, and remove OnIdle().
Even if it is not that, gprof will likely find some function that gets called amazingly often, providing a starting point for the search.

[Michael]

Hello,

I have remarked the "same" high CPU usage with my configuration:

CodeBlocks 1.0-RC1-1 (all plugins + cache activated)
Windows XP Pro SP2
Pentium M 1,8 GHz
1024 MB RAM

Simply moving the mouse cursor on the CB window, the CPU usage rises around 20-30%, sometimes until more than 40%, other time around 10-20%.

Ad-Aware SE Personal rises the CPU usage until 11% and Windows Media Player until 10% approximately (just moving the mouse cursor at different speed and in/out the application window).

Therefore, CB increases the CPU usage around 2-4 times more (to confirm this result, it would be necessary to perform more comparisons and tests).

Anyway, I have not experienced particular problems using it and all seems to work well.

Best wishes,
Michael

[rickg22]

OnIdle is safe to use... it's called only once per event cycle. In fact, it does what its name says: It's only activated when the system is IDLE.

But I'm not sure if the wakeupidle should be used...

In that case, I think OnIdle should use a counter or an internal timer to assure a MAXIMUM of calls per second.

Instead, if you use a timer instead of idle, you specify the EXACT number of times the routine will be called.

[thomas]

The point I am having about OnIdle is that if you move your mouse, the OS generates a message, wakes up some thread to move the pointer, does some rectangle hit tests, possibly some MouseOver or MouseEnter/Leave messages are fired and processed, which alltogether goes rather quick (few dozen microseconds?), and then the system *goes idle* again.
If you move your mouse over a window and dump all WM_MOUSEMOVE notifications to a console, you get an idea about how many messages Windows(R) creates... 30-50 per second is not unusual. That means that you call OnIdle quite a few times per second (unless wxWindows filters these).

That might indeed possibly be an issue. First, message queues are usually thread-safe (I don't actually know about wxWindows, but I strongly assume they are!), so handling 50 messages means acquiring and releasing 50 mutexes or semaphores (or locked atomic writes, whatever) as a basic overhead. This is particularly bad on a system with NUM_CPU > 1.
And second, while that may still not be an awful lot, there are still several library calls (IsInputAvailable, CanRead, ...) and allocations/deallocations of complex objects (wxTextInputStream, wxEvent), which are not free. All of these derive from wxObject, they run non-trivial constructors and have a chain of virtual destructors. This is not noticeable if you create a few objects once, but if you allocate/destroy a couple of them several dozen of times every second, it adds up, so the resulting payload is not necessarily trivial.
At the very least, it is worth looking at.

WakeupIdle, like the docs say, forces idle events to be fired, even if the system is idle already. However, this function is only called twice every 100 ms, and only while either a compile or the debugger is running, too, so that should not do much harm. Still, polling for input directly inside OnTimer would be more efficient. That would remove the necessity to have OnIdle at all, and it would remove posting and handling of one message per timer event, as well as the creation/destruction of half a dozen objects each time.
I believe that calling WakeupIdle is necessary with the current design, because otherwise you could not run the compiler and debugger plugin at the same time. Whichever comes first would steal the idle events from the respective other (unless explicitely calling Skip), and the other would never poll for input.

[rickg22]

Is there a way to check the ellapsed time in milliseconds since the last idle event handled?

Also, something makes me think. Is there a point on doing these checks if NO compilation or debug session has been started yet? I think we need a couple of if's  in there.

Hmmm....

Code

void CompilerGCC::OnIdle(wxIdleEvent& event)
{
    if (m_Process && ((PipedProcess*)m_Process)->HasInput())
		event.RequestMore();
	else
		event.Skip();
}

void CompilerGCC::OnTimer(wxTimerEvent& event)
{
	wxWakeUpIdle();
}

I think we should add an "if(m_Process)" before doing the wxWakeUpIdle, this would save us from creating an idle events chain. The same code applies for debuggergdb.

[thomas]

Or better, you could simply copy the first line from OnIdle into OnTimer, and delete OnIdle alltogether.
That should work the same (I am pretty sure it does because I poll stdout from inside OnTimer in the svn plugin, too - it works fine).

If my theory is correct, you have solved the problem with this.

If my theory is not correct, you don't lose anything - the only difference is that the system has to push one less message through its queue and you save a few calls.

[rickg22]

Why don't you change the code and try it out?

[grv575]

Maybe profile the debugger polling as well?  It takes quite a long time to step through a program as is (time between pressing the step/step over button and the actual update is very unresponsive...).  I'm thinking the way the gdb output is polled and read could be optimized to make this more instantaneous.

[rickg22]

HMmmmmmmmmmmmmmmmmmmmmmmmmm

Did you try modifying the code and moving the mouse around?

[grv575]

No I've just looked at the code so far.  Haven't tried changing delays or logging events or anything yet.

[thomas]

It gets more surprising...

Before modifying and recompiling anything, I started doing some timings on my own machine this morning.
a) RC1-1 as downloaded from the Code::Blocks site

• With all standard plugins (plus my own), it uses 36-63% CPU (average ~55%) while moving the mouse over the editor. The spread (42%) is simply amazing, and interestingly, the average does not lie in the middle of the interval, either. This is weird.
• Someone complained about a similar thing on RC1 a while ago, and I know for sure that I could not reproduce any such behaviour on RC1 (tested that at that time).
• The same configuration only uses 20-25% while the mouse moves over the project manager.
• The messages notebook is similar to the project manager.
• CPU load is roughly proportional to the number of plugins, even plugins that do *nothing* (code completion wizard, Windows XP manifest) add to it
• Debugger and Compiler are the rule-breakers, these affect CPU by far more than any other plugin
• All plugins except Debugger and Compiler: 13-28% (average ~15%)
• Only Debugger and Compiler: 22-50% (average ~45%)

b) custom build from CVS, sources are maybe 5-6 days old

• 13-24% (avg ~20%) with all plugins
• 9-20% (avg ~16%) with all plugins except Debugger
• 10-16% (avg ~13%) with all plugins except Compiler
• 3-9% (avg ~6%) with all plugins except Debugger and Compiler
• 3-9% (avg ~6%) with no plugins at all

New working hypothesis:

• RC1-1, in particular the editor component, sends a lot of plugin notifications or UI messages to every plugin.
• HEAD does not do that, or at least not to the same extent.
• The debugger and compiler plugins are CPU hogs for some reason, even when being idle.
• The theory that the latter is related to OnIdle has to be tested next (will do that this afternoon).

[thomas]

Could not wait until afternoon, too interesting...

Unluckily, the overall result is a bit disappointing, the changes in CPU time are not really significant:

13-24% (average ~20%) all plugins (including debugger and compiler)  original version
9-22% (average ~16%) all plugins (including debugger and compiler)  after removing OnIdle

It appears to be slightly better, but the two intervals overlap quite a bit, so it may well be that the observed difference is due to inaccurate measurement.

The code was changed in the following manner:

Code

//void CompilerGCC::OnIdle(wxIdleEvent& event)
//{
//    if (m_Process && ((PipedProcess*)m_Process)->HasInput())
//		event.RequestMore();
//	else
//		event.Skip();
//}

void CompilerGCC::OnTimer(wxTimerEvent& event)
{
if(m_Process)
while (((PipedProcess*)m_Process)->HasInput())
	;
}

(the respective declarations and event table entries were deleted)

These changes were done for CompilerGCC, DebuggerGDB, and PipedProcess, which internally all use this strategy. Another possible optimization would be to move the while into PipedProcess::HasInput, so the two TextInputStreams are not opened and closed for every single line of output read, but that will only affect performance while a compile is running, it should make no difference when idle (probably not an issue, so best leave as is).
Also, I deleted the OnIdle function from ProjectManager because all it did was call Skip() - this is quite useless, we get the same result if we don't insert that function into the event table in the first place.

wxScintilla does some very peculiar idle processing, too. Apparently, the line wrapping is done by calling RequestMore() for each line (?). Using RequestMore in a loop is titled as "not usually a good idea" on comp.soft-sys.wxwindows, hmm... anyway. I removed this idle code to check what difference it makes, but since my test setting has no document open, the event table entry is not generated anyway, so of course there was no difference at all (it might while editing, maybe... but nobody complained so far?).

NotifyPlugins is not guilty of taking up CPU time. I removed each and every call to NotifyPlugins and did not see any difference in CPU load at all.

To summarise:
It is not OnIdle (it could be accused of maybe 5% CPU).
It is not NotifyPlugins (not noticeable at all).
Neither compiler nor debugger spawn threads secretly.
Whatever eats those 15% of CPU time, must consquently be related to UI events.

Or... any other ideas?

[thomas]

I have to revoke the above post. The difference is significant. Very much, you could say.

My previous "measurement" was this:
- open Taskmanager, go to process list
- perform some action (move mouse etc.) for about a minute
- remember the highest and lowest CPU load during that time
- remember what appears to be "the mean" load
- write that down

As the perceived difference was not too big and the "measurement" was obviously subjective, there could very well have been a Type II Error. Therefore, I planned some more "scientific" measurement:
- open Taskmanager, go to CPU usage chart
- set Refresh frequency to lowest, so peak artefacts are averaged out by Taskmanger
- perform some action
- take a screenshot
- compute the area under the curve over a unit interval
- compare AUCs

Looking at the resulting graphs, it is obvious that calculating the AUC is not necessary at all:


Peaks in order of their appearance: application startup, moving mouse over empty editor area, opening document, moving mouse over editor, typing and scrolling, application shutdown.

This result is particularly stunning not only because of the clear difference shown, but also by the fact how much perception and objective data can deviate.

The correct conclusion must therefore be: Removing OnIdle() reduces CPU time by about one half.

[rickg22]

The ProjectManager's OnIdle was added for future expansions. In any case, please submit the patches for the plugins so we can get them fixed

Just something that catches my attention. Why does the plugin's OnIdle just check HasInput() and not doing anything to retrieve such input? Is it just discarding the data?

[zieQ]

Humm, interesting...

But did you try benchmarking the code with a profiler? This would be objective measurements instead of your subjectives ones. I remember that the windows task manager is not accurate for anything except for detecting the most resource consuming process You can try LTProf (the demo version, not free !) which do not instrument the code. On my computer, the most consuming part is the wx event loop dispatch (see call stack)



[attachment deleted by admin]