Author Topic: The 17 May 2012 build (7966) is out.  (Read 87085 times)

Offline cacb

  • Lives here!
  • ****
  • Posts: 536
Re: The 17 May 2012 build (7966) is out.
« Reply #60 on: June 03, 2012, 12:26:50 am »
If you where on unix you can have used the sort utility to sort both files :)
Is there a chance to try to bisect the nightly which is causing the slowness.
We have several nightlies in between 7452 and 7966.

Well, I have a Kubuntu machine in my network, so if you can tell me how use "the sort utility", I could try.

Regarding which nightly is slow, I already reported the following in April
http://forums.codeblocks.org/index.php/topic,16199.msg109543.html#msg109543

See also
http://forums.codeblocks.org/index.php/topic,16199.msg109568.html#msg109568
Where it is stated (by someone else) that 7918 is the first build with lags
« Last Edit: June 03, 2012, 12:29:18 am by cacb »

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 17 May 2012 build (7966) is out.
« Reply #61 on: June 03, 2012, 01:41:39 am »
"cat myfile | sort > mysortedfile"

If the nightly is slow/lags in r7919 only, this should be fixed or you could work around it by disabling the todo plugin.

Slow building is caused by something else, so if you could pinpoint the revision where it started it could help.
Also are you sure that C::B correctly recognises the thread option?
Set it to something big and look in the task manager if you see many instances of cl.exe.
What happens if you tell it to build the code on a single processor?
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline cacb

  • Lives here!
  • ****
  • Posts: 536
Re: The 17 May 2012 build (7966) is out.
« Reply #62 on: June 03, 2012, 02:03:57 pm »
I have tested the "thread option" if you mean Global compiler settings | Build options | Number of processes for parallel builds
I get the expected number of cl.exe instances in Windows Task Manager.

If I set the number to 1, the build takes 5 minutes, i.e. slower than using the 2 processors I have. I get one cl.exe instance. If I set the number to 6, (3 times more than my 2 processors), I get 6 instances of cl.exe, the build then takes 2 minutes 42 seconds, about the same as when using 2 processors (no surprise).

So the problem is unrelated to the number of processor-setting. The problem is that, in 7966, something is preventing the compiler from using available CPU. During build with 7966, CPU use behaves like a yo-yo in Windows Task Manager, see attachment. The processors work at 100% for a while, then pauses for a significant period, where none of the cl.exe processes consume any CPU. This is totally different in 7452, where such pausing is not observed, compilation runs at full speed almost continuously. This is the reason why it is so much faster. The important question is what is causing the pausing in 7966, and I don't know the answer to that.

Another clue is perhaps this: When tweaking the setting Global compiler settings | Build options | Number of processes for parallel builds and press OK on 7452, the dialog disappears immediately. Again, it is different in 7966, where it takes almost 10 seconds from the time OK is pressed until the dialog disappears.

Btw. there is no antivirus or other "known nasties" running on the system. If that had been the problem, it shouldn't help to run an older C::B, but it does.

PS: Todo plugin (and many other plugins) disabled long time ago.
« Last Edit: June 03, 2012, 02:28:55 pm by cacb »

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 17 May 2012 build (7966) is out.
« Reply #63 on: June 03, 2012, 03:24:45 pm »
So 7966 is slower than 7452 in single-thread mode?

Also please try to find the first release which is broken. Keep in mind that this issue is not related to the lag depicted in the old nighly's topic.
Also can you try with a clean settings?

And finally: If I understand correctly you see cl.exe doing nothing in the task manager? Or do you see periods of time where there are no cl.exe running?
The first one is not a problem of C::B, it is miss configuration. The second is a problem of C::B.
« Last Edit: June 03, 2012, 03:26:17 pm by oBFusCATed »
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline cacb

  • Lives here!
  • ****
  • Posts: 536
Re: The 17 May 2012 build (7966) is out.
« Reply #64 on: June 04, 2012, 12:12:16 am »
So 7966 is slower than 7452 in single-thread mode?

Also please try to find the first release which is broken. Keep in mind that this issue is not related to the lag depicted in the old nighly's topic.
Also can you try with a clean settings?

And finally: If I understand correctly you see cl.exe doing nothing in the task manager? Or do you see periods of time where there are no cl.exe running?
The first one is not a problem of C::B, it is miss configuration. The second is a problem of C::B.

What kind of misconfiguration are you referring to? The configuration is exactly the same with the two C::B nightlies. I wish someone would believe my words.


Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 17 May 2012 build (7966) is out.
« Reply #65 on: June 04, 2012, 09:55:07 am »
How would I know?
Why do you think I don't believe you?
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline cacb

  • Lives here!
  • ****
  • Posts: 536
Re: The 17 May 2012 build (7966) is out.
« Reply #66 on: June 04, 2012, 11:16:47 pm »
How would I know?
Why do you think I don't believe you?

I don't want to be polemic, but it seems to me as you seem to be sure it isn't the same problem as the lag issue I reported earlier, and you also seem to be sure it is an unidentified "misconfiguration" issue. I am happy to be proven wrong, but the info provided seems to contradict such assertions. I have provided much information that seems quite adequate in characterising this very real issue.  I hope it is understood that I am trying to provide the necessary information.

Some main bullets

1. The current build (7966) lags significantly, also when not building, ref the reported issue with Global compiler settings | Build options | Number of processes for parallel builds here http://forums.codeblocks.org/index.php/topic,16352.msg111221.html#msg111221
The same behaviour is apparent when closing a workspace. So to me it looks like the lag issue is not fixed and could very well be what I am seeing also during build (ok, I speculate, but anyway). 7452 does not have the same lag. I am not convinced now the lag issue reported in April is a different thing, but I am always open for new info.

2. I provided 2 clear screenshots showing the behaviour as seen in Task Manager. I am told this is not a C::B problem even though the only difference is different C::B versions. I am also told it is a "misconfiguration", and I am happy to correct any misconfiguration if it can be identified, but you say you wouldn't know, so it looks like my information is not believed. The configuration is identical in the two cases. How can you be so sure it isn't a C::B problem? I am not saying it is, but I all evidence points in that direction as I see it. Please tell me why I am wrong, as it might help sort things out.

3. Please tell me how to test with clean settings. Is it just a matter of deleting
C:\Documents and Settings\<user>\Application Data\codeblocks\default.conf ?

Thanks.



Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 17 May 2012 build (7966) is out.
« Reply #67 on: June 04, 2012, 11:27:10 pm »
1. There was an identified lag that was fixed (related to the to-do plugin).
2. OK, you have not told me if the cl.exe-s are there doing nothing or c::b is blocking somewhere between running
3. Yes, but you can rename it. I'm asking this just in case, it isn't because I don't believe you.

And I've asked you to try to find the first nightly with this problem, you have not! And you're comparing 500 revisions.

p.s. Can you provide an example project, which build slow?
p.p.s Does this happen with vc2008 or older compilers?
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline cacb

  • Lives here!
  • ****
  • Posts: 536
Re: The 17 May 2012 build (7966) is out.
« Reply #68 on: June 05, 2012, 12:28:12 am »
1. There was an identified lag that was fixed (related to the to-do plugin).
2. OK, you have not told me if the cl.exe-s are there doing nothing or c::b is blocking somewhere between running
3. Yes, but you can rename it. I'm asking this just in case, it isn't because I don't believe you.

And I've asked you to try to find the first nightly with this problem, you have not! And you're comparing 500 revisions.

p.s. Can you provide an example project, which build slow?
p.p.s Does this happen with vc2008 or older compilers?

1. Ok, but as you can see, I still observe issues.
2. The cl.exe are there all the time as stated before, but they don't always consume CPU, as shown in the Task manager screenshots. That means, they all sometimes do work, then sometimes they don't. My guess is that C::B is blocking, but I cannot be sure, I see the expected number of cl.exe, sometimes they they are listed with using significant CPU, somtetimes they are listed using 0 CPU.
3. Thanks, I shall try it with 7966.

Actually, I have referred you to this post
http://forums.codeblocks.org/index.php/topic,16199.msg109543.html#msg109543
You are telling me I haven't told you, because you believe it is irrelevant, but I am not convinced. The symptoms I reported then
http://forums.codeblocks.org/index.php/topic,16199.msg109481.html#msg109481
appears at least rather similar. If some issues were fixed, maybe some remain.

I recently switched from using VC2008 to VC2010 (both express), I have both installed still and my projects can with some tweaking be built against both. I just tried compiling with 7966 using VC2008. The exact same problem occurs, t looks to be unrelated to using VC2008 or VC2010 compiler (checked also 7452+VC2008).

I don't have a ready workspace to share unfortunately, much as I'd like to do it.

I am going to try renaming the *.conf file now.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 17 May 2012 build (7966) is out.
« Reply #69 on: June 05, 2012, 12:55:26 am »
Quoting myself:
cacb: What about 7789, 7678, 7550?
If we are going to talk with old links. This post was never answered by you!

If the cl.exe are there all the time then they are blocking on some file read or global mutex/semaphore, etc.
cl.exe does some writing to common files (pdbs for example) and this may cause performance problems.
The thing is that C::B is starting them as fast as it can, but there is something slowing them down.
Thus it is not a problem in C::B, but some configuration problem or the cl.exe options are not correct.
(all this analysis is just a speculation)

What is the time difference between the working and the non-working version,
when you set the number of process to 1 in both versions? (I've ask this before, but you've never answer this too).

Also please try to build with mingw as this is know to be good performer, when used in multi-core machines.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline cacb

  • Lives here!
  • ****
  • Posts: 536
Re: The 17 May 2012 build (7966) is out.
« Reply #70 on: June 05, 2012, 09:55:35 pm »
Unfortunately, I don't have all all the time in the world to run all thinkable and unthinkable combinations of C::B versions, different MS compilers and mingw compiler, plus different settings on each. That is too bad, but it is the truth. I have spent many hours with this issue, and tomorrow I have an important conference to attend to. So I will have to let this rest for a few days.

The cl.exe blocking issue sounds more like what I thought was going on. How this can be a configuration issue I honestly don't understand, I repeat it is the same configuration for the versions I have tested, giving the different result.

I'll check the time difference with one processor, but then I am running out of my own time.

EDIT: Running with 1 processor in both cases below
7452: 2 minutes, 42 seconds
7966: 5 minutes, 7 seconds  (same problem with pausing as always)
« Last Edit: June 05, 2012, 10:08:24 pm by cacb »

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 17 May 2012 build (7966) is out.
« Reply #71 on: June 05, 2012, 10:17:43 pm »
The cl.exe blocking issue sounds more like what I thought was going on. How this can be a configuration issue I honestly don't understand, I repeat it is the same configuration for the versions I have tested, giving the different result.
Do project directories look the same after a successful build?
Are you willing to send me both log files for inspection?
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]