The cc-branch crashes reproucible on linux, if <wx/hashmap.h> is parsed.
Ther crash happens in wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded) when #define _WX_DECLARE_HASHTABLE( [...] is parsed.
After many debugging I found the cause.
Normally if we reach the backslash before the EOL, the space(s) before the backslash and the backslash itself are removed from buffer by decrementing the pointer p and after that moving to the next char:
while (*(--p) <= _T(' ') && p > buffer)
In some rare cases we just appended the buffer to the return-string and set the temporary pointer p to the beginning of buffer. If we now decrement p and look if the character it points to is less or equal ' ' we get a segfault.
The same might happen some lines later.
This can only happen if the buffer is smaller than the line and if the next character (after appending buffer to str) is the backslash before the EOL.
I attach a project wher this happens (at least on linux 64-bit), I just copied the define of _WX_DECLARE_HASHTABLE into main.cpp and removed everything else.
The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).
Another approach would be to leave everything as it is, but do it only if p is greater than buffer and if it is not (buffer has already been appended to str), remove the trailing spaces from str.
It's also easy to do, but I am not sure whether it is needed, or if we can leave the spaces.
It would be nice if the cc-gurus can have alook at the patch:
Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (Revision 6508)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (Arbeitskopie)
@@ -410,13 +410,13 @@
break;
else
{
- while (*(--p) <= _T(' ') && p > buffer)
+ while ((p > buffer) && *(--p) <= _T(' '))
;
MoveToNextChar();
}
}
- while (*(p - 1) <= _T(' ') && --p > buffer)
+ while (p > buffer && *(--p) <= _T(' '))
;
str.Append(buffer, p - buffer);
The attached patch is the same as above, but with abuild-fix and a correction of slashes in the unix project-file.
Another approach would be to leave everything as it is, but do it only if p is greater than buffer and if it is not (buffer has already been appended to str), remove the trailing spaces from str.
Next patch, this time it generates the exactly same tokens-tree, -list, include-dirs and file-list:
Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (Revision 6508)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (Arbeitskopie)
@@ -410,14 +410,34 @@
break;
else
{
- while (*(--p) <= _T(' ') && p > buffer)
- ;
+ // only work with p if the buffer is not just appended (p==buffer)
+ // otherwise remove the traling spaces from str (avoid segfault on linux in rare cases)
+ if(p > buffer)
+ {
+ while (*(--p) <= _T(' ') && p > buffer)
+ ;
+ }
+ else
+ {
+ while(!str.IsEmpty() && str.Last() <= _T(' '))
+ str.RemoveLast();
+ }
MoveToNextChar();
}
}
- while (*(p - 1) <= _T(' ') && --p > buffer)
- ;
+ // only work with p if the buffer is not just appended (p==buffer)
+ // otherwise remove the traling spaces from str (avoid segfault on linux in rare cases)
+ if(p > buffer)
+ {
+ while (*(p - 1) <= _T(' ') && --p > buffer)
+ ;
+ }
+ else
+ {
+ while(!str.IsEmpty() && str.Last() <= _T(' '))
+ str.RemoveLast();
+ }
str.Append(buffer, p - buffer);
TRACE(_T("ReadToEOL(): (END) We are now at line %d, CurrentChar='%c', PreviousChar='%c', NextChar='%c'"),
[attachment deleted by admin]
today, already had a lot of other crashes.
From the log xml, this is the stack part (all crashes resulted in the same dump like this) :
<stack>
<frame level="0"/>
<frame level="1" function="NativeParser::OnEditorActivatedTimer(wxTimerEvent&)" offset="0000024d"/>
<frame level="2" function="wxEvtHandler::ProcessEventIfMatches(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)" offset="00000055"/>
<frame level="3" function="wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*)" offset="000000ac"/>
<frame level="4" function="wxEvtHandler::ProcessEvent(wxEvent&)" offset="000000b4"/>
<frame level="5" function="wxTimerBase::Notify()" offset="0000006e"/>
<frame level="6"/>
<frame level="7"/>
<frame level="8" function="g_main_context_dispatch" offset="000001f3"/>
<frame level="9"/>
<frame level="10" function="g_main_loop_run" offset="00000195"/>
<frame level="11" function="gtk_main" offset="000000a7"/>
<frame level="12" function="wxEventLoop::Run()" offset="00000048"/>
<frame level="13" function="wxAppBase::MainLoop()" offset="0000004b"/>
</stack>
Hopefully this can be fixed.
I also have the *feeling* that cc works less good then before. Several times nothing comes up, where I think in the past it did. As said : a *feeling*.
EDIT : I found a way to reproduce this in my working environment (didn't try with a small project).
So I have a workspace with several projects, some files of these projects are open. I drag and drop a cpp file to CB (this file doesn't belong to any of the projects). CB shows the file.
Then I select the tab of one of those other open files (which are part of the project). All still is ok, then I select the tab of the "open non project file" --> crash.
EDIT 2 : tried this with a small project (generated by CB), and same recipe as in the EDIT above.
Why the number of parsed files is less that the trunk version 1250 vs 1531?
Because there are many parsed header files is wrong!
#if 0
#include <a.h>
#include <b.h>
#include <c.h>
#include <d.h>
#include <e.h>
#else
#include <f.h>
#endif
In trunk(or 10.05), cc will parse the #if condition preprocessor:
#if 0
#include <a.h>
...
But in cc branch, will parse the #else condition preprocessor:
#else
#include <f.h>
Btw:
About the performance issue, I have discussed with loaden, currently, we can't use "multi-thread" of parserthread, because wxString is not thread-safe. so you can see, the memory pool is force to run only one thread.
Parser::Parser(wxEvtHandler* parent) :
m_pParent(parent),
m_UsingCache(false),
m_Pool(this, wxNewId(), 1), // in the meanwhile it'll have to be forced to 1
So, this way, CC can only run on a single CPU, my experience is my dualcore CPU is under only 50% usage when CC do a batch parsing.
It's at least much slower.
compared with which?
As loaden said:
The conditional preprocessor and macro expansion takes a lot of time, also make cc get more precise tokens.
trunk (svn r6525 compiled with -O2)
not first time parsing, but several reparses to have as much as possible in OS-buffer (debian 64-bit with 4 GB memory)
Parsing stage done (1521 total parsed files, 50655 tokens in 0 minute(s), 2.924 seconds).
...
Parsing stage done (1521 total parsed files, 50655 tokens in 0 minute(s), 2.680 seconds).
cc-branch (svnr6524)
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 6.697 seconds).
...
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 6.659 seconds).
cc-branch (svnr6524 codecompletion-plugin compiled with -O2)
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 3.012 seconds).
...
Project 'Code::Blocks - Unix' parsing stage done (1249 total parsed files, 66373 tokens in 0 minute(s), 3.019 seconds).
It looks like the optimization does the trick.
It's turned off by default in C::B project file (at least on linux), but turned on in automake-system.
But it still feels slower, the cause might be the much greater amount of tokens that have to be added to symbols-browser.
hi, the patch has fixed the crash, but now I get a crash whe nCB closes (again when we having been switching between a non project file).
This is a stack trace :
<stack>
<frame level="0"/>
<frame level="1" function="TextCtrlLogger::Append(wxString const&, Logger::level)" offset="00000064"/>
<frame level="2" function="NativeParser::ClearParsers()" offset="000000b3"/>
<frame level="3" function="CodeCompletion::OnRelease(bool)" offset="0000003f"/>
<frame level="4" function="cbPlugin::Release(bool)" offset="00000065"/>
<frame level="5" function="PluginManager::DetachPlugin(cbPlugin*)" offset="00000043"/>
<frame level="6" function="PluginManager::UnloadPlugin(cbPlugin*)" offset="0000001a"/>
<frame level="7" function="PluginManager::UnloadAllPlugins()" offset="00000023"/>
<frame level="8" function="PluginManager::~PluginManager()" offset="00000022"/>
<frame level="9" function="PluginManager::~PluginManager()" offset="00000009"/>
<frame level="10" function="Manager::Shutdown()" offset="00000076"/>
</stack>
EDIT : I tried this 3 times, each time crash, but only once the rpt xml file contained a stack trace, the other times only a list of modules.