Author Topic: cc-branch crashes on linux  (Read 22236 times)

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: cc-branch crashes on linux
« Reply #15 on: August 24, 2010, 11:04:17 am »
It's at least much slower.

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: cc-branch crashes on linux
« Reply #16 on: August 24, 2010, 02:10:27 pm »
It's at least much slower.
Here is the test log on linux.
Quote
10.05:
Parsing stage done (1531 total parsed files, 58540 tokens in 0 minute(s), 7.808 seconds).

CC branch -O0
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64146 tokens in 0 minute(s), 14.026 seconds).

CC branch -O2
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64147 tokens in 0 minute(s), 12.685 seconds).

We can see that, cc branch get more tokens, and handle more, like condition preprocessor handle, function-like macro handle...e.g.
You can open the "-O2" option, can make cc faster.

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: cc-branch crashes on linux
« Reply #17 on: August 24, 2010, 02:21:54 pm »
Hopefully this can be fixed.

EDIT : I found a way to reproduce this in my working environment (didn't try with a small project).
So I have a workspace with several projects, some files of these projects are open. I drag and drop a cpp file to CB (this file doesn't belong to any of the projects). CB shows the file.
Then I select the tab of one of those other open files (which are part of the project). All still is ok, then I select the tab of the "open non project file" --> crash.

EDIT 2 : tried this with a small project (generated by CB), and same recipe as in the EDIT above.
I will fix it.

I also have the *feeling* that cc works less good then before. Several times nothing comes up, where I think in the past it did. As said : a *feeling*.
Could you send me a test project?

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13235
    • Travis build status
Re: cc-branch crashes on linux
« Reply #18 on: August 24, 2010, 02:23:29 pm »
Why the number of parsed files is less that the trunk version 1250 vs 1531?
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: cc-branch crashes on linux
« Reply #19 on: August 24, 2010, 02:27:59 pm »
Why the number of parsed files is less that the trunk version 1250 vs 1531?
Because the trunk version also has non sources and headers in the files-list (xrc and others).

I do not know the cause, but I think it parses all project-files that can be opened (just a wild guess).

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: cc-branch crashes on linux
« Reply #20 on: August 24, 2010, 02:29:14 pm »
Why the number of parsed files is less that the trunk version 1250 vs 1531?
Because there are many parsed header files is wrong!
Code
#if 0
#include <a.h>
#include <b.h>
#include <c.h>
#include <d.h>
#include <e.h>
#else
#include <f.h>
#endif

In trunk(or 10.05), cc will parse the #if condition preprocessor:
#if 0
#include <a.h>
...

But in cc branch, will parse the #else condition preprocessor:
#else
#include <f.h>

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: cc-branch crashes on linux
« Reply #21 on: August 24, 2010, 02:35:20 pm »
It's at least much slower.
Here is the test log on linux.
Quote
10.05:
Parsing stage done (1531 total parsed files, 58540 tokens in 0 minute(s), 7.808 seconds).

CC branch -O0
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64146 tokens in 0 minute(s), 14.026 seconds).

CC branch -O2
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64147 tokens in 0 minute(s), 12.685 seconds).

We can see that, cc branch get more tokens, and handle more, like condition preprocessor handle, function-like macro handle...e.g.
You can open the "-O2" option, can make cc faster.
Another reason: In current cc branch, we need get system headers list in another thread.
So, It's need some time.

When i open the CB project again, it's make faster.
Quote
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64147 tokens in 0 minute(s), 6.390 seconds).

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: cc-branch crashes on linux
« Reply #22 on: August 24, 2010, 02:58:15 pm »
Hopefully this can be fixed.

EDIT : I found a way to reproduce this in my working environment (didn't try with a small project).
So I have a workspace with several projects, some files of these projects are open. I drag and drop a cpp file to CB (this file doesn't belong to any of the projects). CB shows the file.
Then I select the tab of one of those other open files (which are part of the project). All still is ok, then I select the tab of the "open non project file" --> crash.

EDIT 2 : tried this with a small project (generated by CB), and same recipe as in the EDIT above.
I will fix it.
Fixed!

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5341
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: cc-branch crashes on linux
« Reply #23 on: August 24, 2010, 03:00:45 pm »
It's at least much slower.
compared with which?
As loaden said:
The conditional preprocessor and macro expansion takes a lot of time, also make cc get more precise tokens.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5341
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: cc-branch crashes on linux
« Reply #24 on: August 24, 2010, 03:12:13 pm »
Btw:
About the performance issue, I have discussed with loaden, currently, we can't use "multi-thread" of parserthread, because wxString is not thread-safe. so you can see, the memory pool is force to run only one thread.

Code
Parser::Parser(wxEvtHandler* parent) :
    m_pParent(parent),
    m_UsingCache(false),
    m_Pool(this, wxNewId(), 1), // in the meanwhile it'll have to be forced to 1
So, this way, CC can only run on a single CPU, my experience is my dualcore CPU is under only 50% usage when CC do a batch parsing.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: cc-branch crashes on linux
« Reply #25 on: August 24, 2010, 03:26:26 pm »
It's at least much slower.
compared with which?
As loaden said:
The conditional preprocessor and macro expansion takes a lot of time, also make cc get more precise tokens.

trunk (svn r6525 compiled with -O2)
not first time parsing, but several reparses to have as much as possible in OS-buffer (debian 64-bit with 4 GB memory)
Code
Parsing stage done (1521 total parsed files, 50655 tokens in 0 minute(s), 2.924 seconds).
...
Parsing stage done (1521 total parsed files, 50655 tokens in 0 minute(s), 2.680 seconds).

cc-branch (svnr6524)
Code
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 6.697 seconds).
...
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 6.659 seconds).

cc-branch (svnr6524 codecompletion-plugin compiled with -O2)
Code
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 3.012 seconds).
...
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 3.019 seconds).

It looks like the optimization does the trick.
It's turned off by default in C::B project file (at least on linux), but turned on in automake-system.

But it still feels slower, the cause might be the much greater amount of tokens that have to be added to symbols-browser.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5280
Re: cc-branch crashes on linux
« Reply #26 on: August 24, 2010, 08:25:42 pm »
hi, the patch has fixed the crash, but now I get a crash whe nCB closes (again when we having been switching between a non project file).
This is a stack trace :
Code
  <stack>
    <frame level="0"/>
    <frame level="1" function="TextCtrlLogger::Append(wxString const&amp;, Logger::level)" offset="00000064"/>
    <frame level="2" function="NativeParser::ClearParsers()" offset="000000b3"/>
    <frame level="3" function="CodeCompletion::OnRelease(bool)" offset="0000003f"/>
    <frame level="4" function="cbPlugin::Release(bool)" offset="00000065"/>
    <frame level="5" function="PluginManager::DetachPlugin(cbPlugin*)" offset="00000043"/>
    <frame level="6" function="PluginManager::UnloadPlugin(cbPlugin*)" offset="0000001a"/>
    <frame level="7" function="PluginManager::UnloadAllPlugins()" offset="00000023"/>
    <frame level="8" function="PluginManager::~PluginManager()" offset="00000022"/>
    <frame level="9" function="PluginManager::~PluginManager()" offset="00000009"/>
    <frame level="10" function="Manager::Shutdown()" offset="00000076"/>
  </stack>

EDIT : I tried this 3 times, each time crash, but only once the rpt xml file contained a stack trace, the other times only a list of modules.
« Last Edit: August 24, 2010, 08:29:42 pm by killerbot »

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9614
Re: cc-branch crashes on linux
« Reply #27 on: August 24, 2010, 08:31:06 pm »
hi, the patch has fixed the crash, but now I get a crash whe nCB closes (again when we having been switching between a non project file).
This is a stack trace :
Code
  <stack>
    <frame level="0"/>
    <frame level="1" function="TextCtrlLogger::Append(wxString const&amp;, Logger::level)" offset="00000064"/>
    <frame level="2" function="NativeParser::ClearParsers()" offset="000000b3"/>
    <frame level="3" function="CodeCompletion::OnRelease(bool)" offset="0000003f"/>
    <frame level="4" function="cbPlugin::Release(bool)" offset="00000065"/>
    <frame level="5" function="PluginManager::DetachPlugin(cbPlugin*)" offset="00000043"/>
    <frame level="6" function="PluginManager::UnloadPlugin(cbPlugin*)" offset="0000001a"/>
    <frame level="7" function="PluginManager::UnloadAllPlugins()" offset="00000023"/>
    <frame level="8" function="PluginManager::~PluginManager()" offset="00000022"/>
    <frame level="9" function="PluginManager::~PluginManager()" offset="00000009"/>
    <frame level="10" function="Manager::Shutdown()" offset="00000076"/>
  </stack>
In ClearParsers() which is also called from the destructor there is an logger output but no check if the app is shutting down. So the output to the log fails. This can be fixed by either just removing this log message or checking if C::B shuts down and if so, not posting anything to the logger.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9614
Re: cc-branch crashes on linux
« Reply #28 on: August 24, 2010, 08:39:39 pm »
In ClearParsers() which is also called from the destructor there is an logger output but no check if the app is shutting down. So the output to the log fails. This can be fixed by either just removing this log message or checking if C::B shuts down and if so, not posting anything to the logger.
Wait a sec... even worse! TextCtrlLogger derives from Loggers which has a virtual destructor, but TextCtrlLogger does not implement a destructor. Thus the control of the TextCtrlLogger does not get cleared properly (most likely). I wonder why the compiler does not complain about the missing implementation of the destructor... or am I missing something (it's late and I had a long day...).

BTW: All sanity checks in the TextCtrlLogger are useless therefore...?!
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: cc-branch crashes on linux
« Reply #29 on: August 25, 2010, 04:28:56 am »
hi, the patch has fixed the crash, but now I get a crash whe nCB closes (again when we having been switching between a non project file).
This is a stack trace :
Code
  <stack>
    <frame level="0"/>
    <frame level="1" function="TextCtrlLogger::Append(wxString const&amp;, Logger::level)" offset="00000064"/>
    <frame level="2" function="NativeParser::ClearParsers()" offset="000000b3"/>
    <frame level="3" function="CodeCompletion::OnRelease(bool)" offset="0000003f"/>
    <frame level="4" function="cbPlugin::Release(bool)" offset="00000065"/>
    <frame level="5" function="PluginManager::DetachPlugin(cbPlugin*)" offset="00000043"/>
    <frame level="6" function="PluginManager::UnloadPlugin(cbPlugin*)" offset="0000001a"/>
    <frame level="7" function="PluginManager::UnloadAllPlugins()" offset="00000023"/>
    <frame level="8" function="PluginManager::~PluginManager()" offset="00000022"/>
    <frame level="9" function="PluginManager::~PluginManager()" offset="00000009"/>
    <frame level="10" function="Manager::Shutdown()" offset="00000076"/>
  </stack>
In ClearParsers() which is also called from the destructor there is an logger output but no check if the app is shutting down. So the output to the log fails. This can be fixed by either just removing this log message or checking if C::B shuts down and if so, not posting anything to the logger.
I personally think that not CC bug.
We need find the reason?