Code::Blocks
Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: jens on August 22, 2010, 10:27:12 pm
-
The cc-branch crashes reproucible on linux, if <wx/hashmap.h> is parsed.
Ther crash happens in wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded) when #define _WX_DECLARE_HASHTABLE( [...] is parsed.
After many debugging I found the cause.
Normally if we reach the backslash before the EOL, the space(s) before the backslash and the backslash itself are removed from buffer by decrementing the pointer p and after that moving to the next char:
while (*(--p) <= _T(' ') && p > buffer)
In some rare cases we just appended the buffer to the return-string and set the temporary pointer p to the beginning of buffer. If we now decrement p and look if the character it points to is less or equal ' ' we get a segfault.
The same might happen some lines later.
This can only happen if the buffer is smaller than the line and if the next character (after appending buffer to str) is the backslash before the EOL.
I attach a project wher this happens (at least on linux 64-bit), I just copied the define of _WX_DECLARE_HASHTABLE into main.cpp and removed everything else.
The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).
Another approach would be to leave everything as it is, but do it only if p is greater than buffer and if it is not (buffer has already been appended to str), remove the trailing spaces from str.
It's also easy to do, but I am not sure whether it is needed, or if we can leave the spaces.
It would be nice if the cc-gurus can have alook at the patch:
Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (Revision 6508)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (Arbeitskopie)
@@ -410,13 +410,13 @@
break;
else
{
- while (*(--p) <= _T(' ') && p > buffer)
+ while ((p > buffer) && *(--p) <= _T(' '))
;
MoveToNextChar();
}
}
- while (*(p - 1) <= _T(' ') && --p > buffer)
+ while (p > buffer && *(--p) <= _T(' '))
;
str.Append(buffer, p - buffer);
The attached patch is the same as above, but with abuild-fix and a correction of slashes in the unix project-file.
-
Thank Jens!
-
Hi, Jens, can you give me a favor?
In Ubuntu 10.04, if apply this patch, will lead C::B can not load libcodecompletion.so.
I check it more times, but i can not find the reason.
I need you help. ^_^
-
WOW, I got the reason.
THAT IS: We must add "classnavigator.cpp/h" to project!!
Solved now!
-
The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).
Ooops - sounds weired. You mean for a single file "Hello World" project?! :shock:
@Loaden: What could be the reason for this?!
-
The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).
Ooops - sounds weired. You mean for a single file "Hello World" project?! :shock:
@Loaden: What could be the reason for this?!
I am a little busy, will comment later.
-
Another approach would be to leave everything as it is, but do it only if p is greater than buffer and if it is not (buffer has already been appended to str), remove the trailing spaces from str.
Next patch, this time it generates the exactly same tokens-tree, -list, include-dirs and file-list:
Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (Revision 6508)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (Arbeitskopie)
@@ -410,14 +410,34 @@
break;
else
{
- while (*(--p) <= _T(' ') && p > buffer)
- ;
+ // only work with p if the buffer is not just appended (p==buffer)
+ // otherwise remove the traling spaces from str (avoid segfault on linux in rare cases)
+ if(p > buffer)
+ {
+ while (*(--p) <= _T(' ') && p > buffer)
+ ;
+ }
+ else
+ {
+ while(!str.IsEmpty() && str.Last() <= _T(' '))
+ str.RemoveLast();
+ }
MoveToNextChar();
}
}
- while (*(p - 1) <= _T(' ') && --p > buffer)
- ;
+ // only work with p if the buffer is not just appended (p==buffer)
+ // otherwise remove the traling spaces from str (avoid segfault on linux in rare cases)
+ if(p > buffer)
+ {
+ while (*(p - 1) <= _T(' ') && --p > buffer)
+ ;
+ }
+ else
+ {
+ while(!str.IsEmpty() && str.Last() <= _T(' '))
+ str.RemoveLast();
+ }
str.Append(buffer, p - buffer);
TRACE(_T("ReadToEOL(): (END) We are now at line %d, CurrentChar='%c', PreviousChar='%c', NextChar='%c'"),
[attachment deleted by admin]
-
The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).
I'm back.
Need changed to:
- while (*(p - 1) <= _T(' ') && --p > buffer)
- ;
+ while (p > buffer && *(p - 1) <= _T(' '))
+ --p;
This patch fixed crash in ReadParentheses too.
And do some simple optimization.
-
Tested and confirmed, no crash and identical tokens-list etc. for test-project.
Thank you for the quick fix !
-
I don't know if it is related, or already fixed by above mentioned patch, but for me CB is crashing also a lot.
Whenever I have method call for example, and just before the closing ')' of the call I press comma (,) CB dies on me (well never comes back and I have to kill it).
x.foo(1,2,3); --> before the ) and after the 3 press comma and it crashes.
Other way to reproduce. Have CB create a console application,
Go to the line of void main(), and try to type "int argc," , once the "," has been typed --> crash.
Can the above path be committed to cc branch.
PS : I am on linux
-
I just svn updated (cc branch and rev 6522) and the problem still occurs. Anyone else can reproduce this ?
EDIT : I did even a make clean && make : no luck, issue remains.
-
I don't know if it is related, or already fixed by above mentioned patch, but for me CB is crashing also a lot.
Whenever I have method call for example, and just before the closing ')' of the call I press comma (,) CB dies on me (well never comes back and I have to kill it).
x.foo(1,2,3); --> before the ) and after the 3 press comma and it crashes.
Other way to reproduce. Have CB create a console application,
Go to the line of void main(), and try to type "int argc," , once the "," has been typed --> crash.
Can the above path be committed to cc branch.
PS : I am on linux
This is another issue.
I will fix it soon, please waiting.
-
Fixed!
-
I can confirm this patch fixes the issue. Bring it on the cc branch ;-)
-
today, already had a lot of other crashes.
From the log xml, this is the stack part (all crashes resulted in the same dump like this) :
<stack>
<frame level="0"/>
<frame level="1" function="NativeParser::OnEditorActivatedTimer(wxTimerEvent&)" offset="0000024d"/>
<frame level="2" function="wxEvtHandler::ProcessEventIfMatches(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)" offset="00000055"/>
<frame level="3" function="wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*)" offset="000000ac"/>
<frame level="4" function="wxEvtHandler::ProcessEvent(wxEvent&)" offset="000000b4"/>
<frame level="5" function="wxTimerBase::Notify()" offset="0000006e"/>
<frame level="6"/>
<frame level="7"/>
<frame level="8" function="g_main_context_dispatch" offset="000001f3"/>
<frame level="9"/>
<frame level="10" function="g_main_loop_run" offset="00000195"/>
<frame level="11" function="gtk_main" offset="000000a7"/>
<frame level="12" function="wxEventLoop::Run()" offset="00000048"/>
<frame level="13" function="wxAppBase::MainLoop()" offset="0000004b"/>
</stack>
Hopefully this can be fixed.
I also have the *feeling* that cc works less good then before. Several times nothing comes up, where I think in the past it did. As said : a *feeling*.
EDIT : I found a way to reproduce this in my working environment (didn't try with a small project).
So I have a workspace with several projects, some files of these projects are open. I drag and drop a cpp file to CB (this file doesn't belong to any of the projects). CB shows the file.
Then I select the tab of one of those other open files (which are part of the project). All still is ok, then I select the tab of the "open non project file" --> crash.
EDIT 2 : tried this with a small project (generated by CB), and same recipe as in the EDIT above.
-
It's at least much slower.
-
It's at least much slower.
Here is the test log on linux.
10.05:
Parsing stage done (1531 total parsed files, 58540 tokens in 0 minute(s), 7.808 seconds).
CC branch -O0
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64146 tokens in 0 minute(s), 14.026 seconds).
CC branch -O2
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64147 tokens in 0 minute(s), 12.685 seconds).
We can see that, cc branch get more tokens, and handle more, like condition preprocessor handle, function-like macro handle...e.g.
You can open the "-O2" option, can make cc faster.
-
Hopefully this can be fixed.
EDIT : I found a way to reproduce this in my working environment (didn't try with a small project).
So I have a workspace with several projects, some files of these projects are open. I drag and drop a cpp file to CB (this file doesn't belong to any of the projects). CB shows the file.
Then I select the tab of one of those other open files (which are part of the project). All still is ok, then I select the tab of the "open non project file" --> crash.
EDIT 2 : tried this with a small project (generated by CB), and same recipe as in the EDIT above.
I will fix it.
I also have the *feeling* that cc works less good then before. Several times nothing comes up, where I think in the past it did. As said : a *feeling*.
Could you send me a test project?
-
Why the number of parsed files is less that the trunk version 1250 vs 1531?
-
Why the number of parsed files is less that the trunk version 1250 vs 1531?
Because the trunk version also has non sources and headers in the files-list (xrc and others).
I do not know the cause, but I think it parses all project-files that can be opened (just a wild guess).
-
Why the number of parsed files is less that the trunk version 1250 vs 1531?
Because there are many parsed header files is wrong!
#if 0
#include <a.h>
#include <b.h>
#include <c.h>
#include <d.h>
#include <e.h>
#else
#include <f.h>
#endif
In trunk(or 10.05), cc will parse the #if condition preprocessor:
#if 0
#include <a.h>
...
But in cc branch, will parse the #else condition preprocessor:
#else
#include <f.h>
-
It's at least much slower.
Here is the test log on linux.
10.05:
Parsing stage done (1531 total parsed files, 58540 tokens in 0 minute(s), 7.808 seconds).
CC branch -O0
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64146 tokens in 0 minute(s), 14.026 seconds).
CC branch -O2
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64147 tokens in 0 minute(s), 12.685 seconds).
We can see that, cc branch get more tokens, and handle more, like condition preprocessor handle, function-like macro handle...e.g.
You can open the "-O2" option, can make cc faster.
Another reason: In current cc branch, we need get system headers list in another thread.
So, It's need some time.
When i open the CB project again, it's make faster.
Project 'Code::Blocks - Unix' parsing stage done (1250 total parsed files, 64147 tokens in 0 minute(s), 6.390 seconds).
-
Hopefully this can be fixed.
EDIT : I found a way to reproduce this in my working environment (didn't try with a small project).
So I have a workspace with several projects, some files of these projects are open. I drag and drop a cpp file to CB (this file doesn't belong to any of the projects). CB shows the file.
Then I select the tab of one of those other open files (which are part of the project). All still is ok, then I select the tab of the "open non project file" --> crash.
EDIT 2 : tried this with a small project (generated by CB), and same recipe as in the EDIT above.
I will fix it.
Fixed!
-
It's at least much slower.
compared with which?
As loaden said:
The conditional preprocessor and macro expansion takes a lot of time, also make cc get more precise tokens.
-
Btw:
About the performance issue, I have discussed with loaden, currently, we can't use "multi-thread" of parserthread, because wxString is not thread-safe. so you can see, the memory pool is force to run only one thread.
Parser::Parser(wxEvtHandler* parent) :
m_pParent(parent),
m_UsingCache(false),
m_Pool(this, wxNewId(), 1), // in the meanwhile it'll have to be forced to 1So, this way, CC can only run on a single CPU, my experience is my dualcore CPU is under only 50% usage when CC do a batch parsing.
-
It's at least much slower.
compared with which?
As loaden said:
The conditional preprocessor and macro expansion takes a lot of time, also make cc get more precise tokens.
trunk (svn r6525 compiled with -O2)
not first time parsing, but several reparses to have as much as possible in OS-buffer (debian 64-bit with 4 GB memory)
Parsing stage done (1521 total parsed files, 50655 tokens in 0 minute(s), 2.924 seconds).
...
Parsing stage done (1521 total parsed files, 50655 tokens in 0 minute(s), 2.680 seconds).
cc-branch (svnr6524)
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 6.697 seconds).
...
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 6.659 seconds).
cc-branch (svnr6524 codecompletion-plugin compiled with -O2)
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 3.012 seconds).
...
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 3.019 seconds).
It looks like the optimization does the trick.
It's turned off by default in C::B project file (at least on linux), but turned on in automake-system.
But it still feels slower, the cause might be the much greater amount of tokens that have to be added to symbols-browser.
-
hi, the patch has fixed the crash, but now I get a crash whe nCB closes (again when we having been switching between a non project file).
This is a stack trace :
<stack>
<frame level="0"/>
<frame level="1" function="TextCtrlLogger::Append(wxString const&, Logger::level)" offset="00000064"/>
<frame level="2" function="NativeParser::ClearParsers()" offset="000000b3"/>
<frame level="3" function="CodeCompletion::OnRelease(bool)" offset="0000003f"/>
<frame level="4" function="cbPlugin::Release(bool)" offset="00000065"/>
<frame level="5" function="PluginManager::DetachPlugin(cbPlugin*)" offset="00000043"/>
<frame level="6" function="PluginManager::UnloadPlugin(cbPlugin*)" offset="0000001a"/>
<frame level="7" function="PluginManager::UnloadAllPlugins()" offset="00000023"/>
<frame level="8" function="PluginManager::~PluginManager()" offset="00000022"/>
<frame level="9" function="PluginManager::~PluginManager()" offset="00000009"/>
<frame level="10" function="Manager::Shutdown()" offset="00000076"/>
</stack>
EDIT : I tried this 3 times, each time crash, but only once the rpt xml file contained a stack trace, the other times only a list of modules.
-
hi, the patch has fixed the crash, but now I get a crash whe nCB closes (again when we having been switching between a non project file).
This is a stack trace :
<stack>
<frame level="0"/>
<frame level="1" function="TextCtrlLogger::Append(wxString const&, Logger::level)" offset="00000064"/>
<frame level="2" function="NativeParser::ClearParsers()" offset="000000b3"/>
<frame level="3" function="CodeCompletion::OnRelease(bool)" offset="0000003f"/>
<frame level="4" function="cbPlugin::Release(bool)" offset="00000065"/>
<frame level="5" function="PluginManager::DetachPlugin(cbPlugin*)" offset="00000043"/>
<frame level="6" function="PluginManager::UnloadPlugin(cbPlugin*)" offset="0000001a"/>
<frame level="7" function="PluginManager::UnloadAllPlugins()" offset="00000023"/>
<frame level="8" function="PluginManager::~PluginManager()" offset="00000022"/>
<frame level="9" function="PluginManager::~PluginManager()" offset="00000009"/>
<frame level="10" function="Manager::Shutdown()" offset="00000076"/>
</stack>
In ClearParsers() which is also called from the destructor there is an logger output but no check if the app is shutting down. So the output to the log fails. This can be fixed by either just removing this log message or checking if C::B shuts down and if so, not posting anything to the logger.
-
In ClearParsers() which is also called from the destructor there is an logger output but no check if the app is shutting down. So the output to the log fails. This can be fixed by either just removing this log message or checking if C::B shuts down and if so, not posting anything to the logger.
Wait a sec... even worse! TextCtrlLogger derives from Loggers which has a virtual destructor, but TextCtrlLogger does not implement a destructor. Thus the control of the TextCtrlLogger does not get cleared properly (most likely). I wonder why the compiler does not complain about the missing implementation of the destructor... or am I missing something (it's late and I had a long day...).
BTW: All sanity checks in the TextCtrlLogger are useless therefore...?!
-
hi, the patch has fixed the crash, but now I get a crash whe nCB closes (again when we having been switching between a non project file).
This is a stack trace :
<stack>
<frame level="0"/>
<frame level="1" function="TextCtrlLogger::Append(wxString const&, Logger::level)" offset="00000064"/>
<frame level="2" function="NativeParser::ClearParsers()" offset="000000b3"/>
<frame level="3" function="CodeCompletion::OnRelease(bool)" offset="0000003f"/>
<frame level="4" function="cbPlugin::Release(bool)" offset="00000065"/>
<frame level="5" function="PluginManager::DetachPlugin(cbPlugin*)" offset="00000043"/>
<frame level="6" function="PluginManager::UnloadPlugin(cbPlugin*)" offset="0000001a"/>
<frame level="7" function="PluginManager::UnloadAllPlugins()" offset="00000023"/>
<frame level="8" function="PluginManager::~PluginManager()" offset="00000022"/>
<frame level="9" function="PluginManager::~PluginManager()" offset="00000009"/>
<frame level="10" function="Manager::Shutdown()" offset="00000076"/>
</stack>
In ClearParsers() which is also called from the destructor there is an logger output but no check if the app is shutting down. So the output to the log fails. This can be fixed by either just removing this log message or checking if C::B shuts down and if so, not posting anything to the logger.
I personally think that not CC bug.
We need find the reason?
-
I personally think that not CC bug.
We need find the reason?
True - it's not as CC bug, but CC could avoid it.
-
Hm, Looks the same as the one in the debugger branch.
I think this should be fixed in C::B's sdk, not in every plugin, it will be easier I suppose :)
-
but rather urgently ;-)
-
but rather urgently ;-)
Fixed in CC.
-
I think this should be fixed in C::B's sdk, not in every plugin, it will be easier I suppose :)
True, If what I said is correct. As the loggers is an implementation of Thomas, I wonder what he believes is the right thing to do.
I would implement the destructor for the relevant loggers (including TextCtrlLogger) and set the control pointer to zero (NOT deleting it, as deletion will be done by the parent component in wxwidgets).
Any objections / thoughts / hints on that topic?!
-
but rather urgently ;-)
Fixed in CC.
confirmed this fixes the issue.
-
Hi, all!
I personally hope we can publish a CC BRANCH build in the "Nightly builds" board.
Because we need more extensive testing in order to perfect the CC.
Any comments?
-
confirmed this fixes the issue.
On second thoughts:
The loggers are one of the components destroyed at last. So in fact they should be valid during the shutdown/destruction of ANY plugin.
However, logging/UI access in operations called during shutdown (of plugins) should ALWAYS check if the app is shutting down -> just as Loaden did it in the patch for the CC branch. Basically that's what this method is for!
I have (however) attached a patch concerning the loggers for testing. It shouldn't harm but may introduce some "safety"... although it's not really a "good way".
I would welcome testing and feedback (@oBFusCATed: e.g. if this "fixes" the crash in the debugger branch).
I personally hope we can publish a CC BRANCH build in the "Nightly builds" board.
I had asked killerbot to do so, I'm afraid he didn't find the time or wanted to wait until the show-stoppers are solved.
-
wanted to wait until the show-stoppers are solved.
But this weekend I will create one.
I could do all 3 of them :
regular
cc
debugger (if there are any updates ?)
-
wanted to wait until the show-stoppers are solved.
But this weekend I will create one.
I could do all 3 of them :
regular
cc
debugger (if there are any updates ?)
Thanks a lot!
-
wanted to wait until the show-stoppers are solved.
But this weekend I will create one.
I could do all 3 of them :
regular
cc
debugger (if there are any updates ?)
Nice, glad to hear! thank you!!!
-
I would welcome testing and feedback (@oBFusCATed: e.g. if this "fixes" the crash in the debugger branch).
No, it doesn't, it's the same as before, the logger has control pointer pointing to the dead gui object...
Steps to reprocude:
1. start cb
2. start some app
3. attach to the app
4. close cb