- Updated the code for detecting whether the mouse pointer is over a selected text. The original code was just a simple approximation, this one does a proper test.
Can you provide the steps, where the old one failed?
When a line is not fully selected, the old test would detect the unselected parts of the line as selected and display tooltip on it (we only want to display debugger tooltip on the selected text).
Additional info: the old test simply takes a rect from selection start to selection end, so it gives incorrect results when selecting multiple lines. I've attached some screenshots below - the red rectangle shows the incorrectly detected selection block. It's something that bothered me when I was selecting text (to copy, etc) and noticed the tooltip appearing behavior was not consistent.
I also checked if any of my other patches were in need of explanation:
scintilla_fixes: In LexerCPP::Fold(), fixed folding glitch in case of hanging braces in a preprocessor block. Eg,
#ifdef _DEFINE
if (condition1) {
#else
if (condition2) {
#endif
...
}
jump_tracker: More consistent behavior for Jump Tracker (cursor history navigation) feature (View -> Jump -> Jump Back / Frwd - I set Ctrl - Left / Right shortcuts for it as it's a very useful feature for me :) ).
incremental_search: Fixed search history flooding. Eg, when typing "codecomp" in the search box, it used to record in history:
The rest of patches are CC related...
cc_fix_tokenizer: In Tokenizer::ReadToEOL(), which is used by HandleDefines(),
//FIX(huki) if (ch <= _T(' ') && (p == buffer || *(p - 1) == ch))
if (ch <= _T(' ') && p > buffer && *(p - 1) == ch)
The fix in Tokenizer::ReadParantheses() support '==' (previously it was taken as two assignments, i.e., ' = = '.
Rest should be commented already in the patch.
Additional info: the old test simply takes a rect from selection start to selection end, so it gives incorrect results when selecting multiple lines. I've attached some screenshots below - the red rectangle shows the incorrectly detected selection block. It's something that bothered me when I was selecting text (to copy, etc) and noticed the tooltip appearing behavior was not consistent.
Understood and I was able to reproduce it. Also I was able to implement it in a simpler way:
int startPos = stc->GetSelectionStart();
int endPos = stc->GetSelectionEnd();
int mousePos = stc->PositionFromPointClose(mousePosition->x, mousePosition->y);
if (mousePos == wxSCI_INVALID_POSITION)
return wxEmptyString;
else if (startPos <= mousePos && mousePos <= endPos)
return selected_text;
else
return wxEmptyString;
BTW: Thank you for you contribution, it is appreciated.
Yes, your code does the job nicely.
In svn...
The following is more succinct (but perhaps less readable?), and has the same execution.
Index: src/sdk/cbplugin.cpp
===================================================================
--- src/sdk/cbplugin.cpp (revision 9299)
+++ src/sdk/cbplugin.cpp (working copy)
@@ -207,12 +207,10 @@
int startPos = stc->GetSelectionStart();
int endPos = stc->GetSelectionEnd();
int mousePos = stc->PositionFromPointClose(mousePosition->x, mousePosition->y);
- if (mousePos == wxSCI_INVALID_POSITION)
+ if (startPos > mousePos || endPos < mousePos)
return wxEmptyString;
- else if (startPos <= mousePos && mousePos <= endPos)
+ else
return selected_text;
- else
- return wxEmptyString;
}
else
return selected_text;
$ patch <../patches/CodeBlocks_patch_huki_130906/cc_enum_values.patch -p0
patching file `src/plugins/codecompletion/parser/expression.cpp'
patching file `src/plugins/codecompletion/parser/expression.h'
patch: **** malformed patch at line 36: Index: src/plugins/codecompletion/parser
/parserthread.cpp
EDIT: problem solved, I found that an empty line is missing, that is mostly caused by manually editing/spliting the patch file
@@ -88,7 +88,7 @@
PostfixVector m_PostfixExpression;
InfixVector m_InfixExpression;
- bool m_Result;
+ long m_Result; //MOD(huki), return type was (bool)
bool m_Status;
};
Index: src/plugins/codecompletion/parser/parserthread.cpp
About other patches:
cc_build_target_handling.patch
If a build target is changed, reparse the whole project, I think it is not necessory. E.g. If we change the build target in Codeblocks.cbp, the whole project get reparsed? That's not necessory I think. If the user do want to a reparse, he can right click on the context menu of the project manager, and select "reparse this project" sub menu.
cc_includes_parsing.patch
I think this is OK, priority header parsing is not needed here. The only issue is that some standalone header files included in a project, but those include files were not used/included in any cpp files, so they will be skipped by the parser.
Hm...
Can you post thorough descriptions for the problems you try to solve with every patch?
Steps to reproduce will be useful, too.
About the CC part: I'm interested on the patch: cc_fix_tokenizer.patch, cc_fix_ctor_dtor.patch, thanks. Maybe other cc patches, but I'm not viewing them yet. ;)
For cc_fix_tokenizer.patch I already explained in a previous post. I'm quoting it again for convenience:
cc_fix_tokenizer: In Tokenizer::ReadToEOL(), which is used by HandleDefines(),
//FIX(huki) if (ch <= _T(' ') && (p == buffer || *(p - 1) == ch))
if (ch <= _T(' ') && p > buffer && *(p - 1) == ch)
The fix in Tokenizer::ReadParantheses() support '==' (previously it was taken as two assignments, i.e., ' = = '.
Rest should be commented already in the patch.
I'll discuss the others later tonight.
I think I need to update the comments in either HandleDefines and ReadToEOL function now. :)
Ok, I extract a patch dedicated to ReadToEOL with some of comments added, see below:
From 276d59ccf760ee85a16170db86efa9e362368a0a Mon Sep 17 00:00:00 2001
From: asmwarrior <asmwarrior@gmail.com>
Date: Tue, 17 Sep 2013 11:10:15 +0800
Subject: [PATCH] handle macro handling correctly, distinguish between function
like macro definition and variable like definition
---
src/plugins/codecompletion/parser/tokenizer.cpp | 15 ++++++++++++---
src/plugins/codecompletion/parser/tokenizer.h | 3 +++
2 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/src/plugins/codecompletion/parser/tokenizer.cpp b/src/plugins/codecompletion/parser/tokenizer.cpp
index 221d5ac..c0c5bbe 100644
--- a/src/plugins/codecompletion/parser/tokenizer.cpp
+++ b/src/plugins/codecompletion/parser/tokenizer.cpp
@@ -438,8 +438,10 @@ wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded)
wxChar* p = buffer;
wxString str;
+ // loop all the physical lines in reading macro definition
for (;;)
{
+ // this while statement end up in a physical EOL '\n'
while (NotEOF() && CurrentChar() != _T('\n'))
{
while (SkipComment())
@@ -449,7 +451,12 @@ wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded)
if (ch == _T('\n'))
break;
- if (ch <= _T(' ') && (p == buffer || *(p - 1) == ch))
+ // if we see two spaces in the buffer, we should drop the second one. Note, if the
+ // first char is space, we should always save it to buffer, this is to distinguish
+ // a function/variable like macro definition, e.g.
+ // #define MYMACRO(A) ... -> function like macro definition
+ // #define MYMACRO (A) ... -> variable like macro definition, note a space before '('
+ if (ch <= _T(' ') && p > buffer && *(p - 1) == ch)
{
MoveToNextChar();
continue;
@@ -475,16 +482,18 @@ wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded)
MoveToNextChar();
}
+ // check to see it is a logical EOL, some long macro definition contains a backslash-newline
if (!IsBackslashBeforeEOL() || IsEOF())
- break;
+ break; //break the outer for loop
else
{
+ //remove the backslash-newline and goto next physical line
while (p > buffer && *(--p) <= _T(' '))
;
MoveToNextChar();
}
}
-
+ // remove the extra spaces in the end of buffer
while (p > buffer && *(p - 1) <= _T(' '))
--p;
diff --git a/src/plugins/codecompletion/parser/tokenizer.h b/src/plugins/codecompletion/parser/tokenizer.h
index 3999252..dbc8d1b 100644
--- a/src/plugins/codecompletion/parser/tokenizer.h
+++ b/src/plugins/codecompletion/parser/tokenizer.h
@@ -188,6 +188,9 @@ public:
/** return the string from the current position to the end of current line, in most case, this
* function is used in handling #define, use with care outside this class!
+ * @param nestBraces true if you still need to count the '{' and '}' levels
+ * @param stripUnneeded true if you are going to remove comments and compression spaces(two or
+ * more spaces should become one space)
*/
wxString ReadToEOL(bool nestBraces = true, bool stripUnneeded = true);
--
1.8.4.msysgit.0
PS: git is vary good at split commit. :)
EDIT: Here is the test case
Parsing code:
#define FUNCTION_LIKE_DEFINE(A) (A+B)
#define VARIABLE_LIKE_DEFINE (A) (A+B)
000001. --------------M-a-i-n--L-o-g--------------
000002. -----------I-n-t-e-r-i-m--L-o-g-----------
000003. InitTokenizer() : m_Filename='C:\DOCUME~1\zyh23\LOCALS~2\Temp\cc20.h', m_FileSize=85.
000004. Init() : m_Filename='C:\DOCUME~1\zyh23\LOCALS~2\Temp\cc20.h'
000005. C:\DOCUME~1\zyh23\LOCALS~2\Temp\cc20.h
000006. Parse() : Parsing 'C:\DOCUME~1\zyh23\LOCALS~2\Temp\cc20.h'
000007. DoParse() : Loop:m_Str='', token='#'
000008. wxString Tokenizer::ReadToEOL(bool, bool) : line=2, CurrentChar='(', PreviousChar='E', NextChar='A', nestBrace(0)
000009. ReadToEOL(): (END) We are now at line 2, CurrentChar='\n', PreviousChar='\r', NextChar='#'
000010. ReadToEOL(): (A) (A+B)
000011. DoAddToken() : Created token='FUNCTION_LIKE_DEFINE', file_idx=1, line=2, ticket=258
000012. GetTokenBaseType() : Searching within m_Str='(A+B)'
000013. GetTokenBaseType() : Compensated m_Str='(A+B)'
000014. GetTokenBaseType() : Found ''
000015. DoAddToken() : Prepending ''
000016. DoAddToken() : Added/updated token 'FUNCTION_LIKE_DEFINE' (0), kind 'preprocessor', type '(A+B)', actual ''. Parent is (-1)
000017. DoParse() : Loop:m_Str='', token='#'
000018. wxString Tokenizer::ReadToEOL(bool, bool) : line=3, CurrentChar=' ', PreviousChar='E', NextChar='(', nestBrace(0)
000019. ReadToEOL(): (END) We are now at line 3, CurrentChar='\n', PreviousChar='\r', NextChar='\r'
000020. ReadToEOL(): (A) (A+B)
000021. DoAddToken() : Created token='VARIABLE_LIKE_DEFINE', file_idx=1, line=3, ticket=259
000022. GetTokenBaseType() : Searching within m_Str=' (A) (A+B)'
000023. GetTokenBaseType() : Compensated m_Str=' (A) (A+B)'
000024. GetTokenBaseType() : Found ''
000025. DoAddToken() : Prepending ''
000026. DoAddToken() : Added/updated token 'VARIABLE_LIKE_DEFINE' (1), kind 'preprocessor', type ' (A) (A+B)', actual ''. Parent is (-1)
********************************************************
Looks good to me.
About this code snippet:
// Here, we are in the comment body
while (true)
{
if (cstyle) // C style comment
{
//FIX(huki), Moved this from below to avoid taking the same '*' for comment begin and end
// eg, /*// ... */
if (!MoveToNextChar())
break;
SkipToChar('/');
if (PreviousChar() == '*') // end of a C style comment
{
MoveToNextChar();
break;
}
}
else // C++ style comment
{
TRACE(_T("SkipComment() : Need to call SkipToInlineCommentEnd() here at line = %u"), m_LineNumber);
SkipToInlineCommentEnd();
break;
}
}
When begin, we have the index:
If you run MoveToNextChar() first, you are now deliberately skip one character, and you are here in:
Although this code did work correctly, but I think we can have a better method.
Method1: SkipToChar('*'); and see whether there is a '/' after '*'
Method2: if (PreviousChar() == '*' && CheckWeReallyMovedAfterSkipToChar)
I think method1 may be better, what's your opinion.
I do not understand what this code snippet trying to solve, can you explain?
wxString Tokenizer::PeekToken()
{
if (!m_PeekAvailable)
{
m_PeekAvailable = true;
unsigned int savedTokenIndex = m_TokenIndex;
unsigned int savedLineNumber = m_LineNumber;
unsigned int savedNestLevel = m_NestLevel;
if (SkipUnwanted())
m_PeekToken = DoGetToken();
else
m_PeekToken.Clear();
m_PeekTokenIndex = m_TokenIndex;
m_PeekLineNumber = m_LineNumber;
m_PeekNestLevel = m_NestLevel;
//FIX(huki), check whether m_TokenIndex has decreased which implies a ReplaceBufferForReparse() was done.
// We can also check for change in m_IsReplaceParsing and m_RepeatReplaceCount before and after DoGetToken().
if (m_IsReplaceParsing && savedTokenIndex > m_TokenIndex)
savedTokenIndex = m_TokenIndex;
m_TokenIndex = savedTokenIndex;
m_LineNumber = savedLineNumber;
m_NestLevel = savedNestLevel;
}
return m_PeekToken;
}
I have try to let the cctest project parsing a simple buffer:
/********************************/
/********************************/
/********************************/
/********************************/
_STD_BEGIN
int a;
int b;
_STD_END
With the replacement rule:
Tokenizer::SetReplacementString(_T("_STD_BEGIN"), _T("namespace std {"));
Tokenizer::SetReplacementString(_T("_STD_END"), _T("}"));
But I don't see the if(m_IsReplaceParsing && savedTokenIndex > m_TokenIndex) condition is true.
Ok, I extract a patch dedicated to ReadToEOL with some of comments added, see below:
From 276d59ccf760ee85a16170db86efa9e362368a0a Mon Sep 17 00:00:00 2001
From: asmwarrior <asmwarrior@gmail.com>
Date: Tue, 17 Sep 2013 11:10:15 +0800
Subject: [PATCH] handle macro handling correctly, distinguish between function
like macro definition and variable like definition
---
src/plugins/codecompletion/parser/tokenizer.cpp | 15 ++++++++++++---
src/plugins/codecompletion/parser/tokenizer.h | 3 +++
2 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/src/plugins/codecompletion/parser/tokenizer.cpp b/src/plugins/codecompletion/parser/tokenizer.cpp
index 221d5ac..c0c5bbe 100644
--- a/src/plugins/codecompletion/parser/tokenizer.cpp
+++ b/src/plugins/codecompletion/parser/tokenizer.cpp
@@ -438,8 +438,10 @@ wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded)
wxChar* p = buffer;
wxString str;
+ // loop all the physical lines in reading macro definition
for (;;)
{
+ // this while statement end up in a physical EOL '\n'
while (NotEOF() && CurrentChar() != _T('\n'))
{
while (SkipComment())
@@ -449,7 +451,12 @@ wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded)
if (ch == _T('\n'))
break;
- if (ch <= _T(' ') && (p == buffer || *(p - 1) == ch))
+ // if we see two spaces in the buffer, we should drop the second one. Note, if the
+ // first char is space, we should always save it to buffer, this is to distinguish
+ // a function/variable like macro definition, e.g.
+ // #define MYMACRO(A) ... -> function like macro definition
+ // #define MYMACRO (A) ... -> variable like macro definition, note a space before '('
+ if (ch <= _T(' ') && p > buffer && *(p - 1) == ch)
{
MoveToNextChar();
continue;
@@ -475,16 +482,18 @@ wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded)
MoveToNextChar();
}
+ // check to see it is a logical EOL, some long macro definition contains a backslash-newline
if (!IsBackslashBeforeEOL() || IsEOF())
- break;
+ break; //break the outer for loop
else
{
+ //remove the backslash-newline and goto next physical line
while (p > buffer && *(--p) <= _T(' '))
;
MoveToNextChar();
}
}
-
+ // remove the extra spaces in the end of buffer
while (p > buffer && *(p - 1) <= _T(' '))
--p;
diff --git a/src/plugins/codecompletion/parser/tokenizer.h b/src/plugins/codecompletion/parser/tokenizer.h
index 3999252..dbc8d1b 100644
--- a/src/plugins/codecompletion/parser/tokenizer.h
+++ b/src/plugins/codecompletion/parser/tokenizer.h
@@ -188,6 +188,9 @@ public:
/** return the string from the current position to the end of current line, in most case, this
* function is used in handling #define, use with care outside this class!
+ * @param nestBraces true if you still need to count the '{' and '}' levels
+ * @param stripUnneeded true if you are going to remove comments and compression spaces(two or
+ * more spaces should become one space)
*/
wxString ReadToEOL(bool nestBraces = true, bool stripUnneeded = true);
--
1.8.4.msysgit.0
Yes, that looks good.
About your test case:
000026. DoAddToken() : Added/updated token 'VARIABLE_LIKE_DEFINE' (1), kind 'preprocessor', type ' (A) (A+B)', actual ''. Parent is (-1)
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 9271)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1294,7 +1345,7 @@
Token* newToken = 0;
wxString newname(name);
- m_Str.Trim();
+ m_Str.Trim(true).Trim(false);
if (kind == tkDestructor)
{
// special class destructors case
For this subject, I suggest committing your ReadToEOL() patch and this DoAddToken() patch above.
I do not understand what this code snippet trying to solve, can you explain?
//FIX(huki), check whether m_TokenIndex has decreased which implies a ReplaceBufferForReparse() was done.
// We can also check for change in m_IsReplaceParsing and m_RepeatReplaceCount before and after DoGetToken().
if (m_IsReplaceParsing && savedTokenIndex > m_TokenIndex)
savedTokenIndex = m_TokenIndex;
I have try to let the cctest project parsing a simple buffer:
With the replacement rule:
But I don't see the if(m_IsReplaceParsing && savedTokenIndex > m_TokenIndex) condition is true.
But with your test buffer, the replacement doesn't happen within PeekToken(). So of course, my fix won't be reached.
If we look at how PeekToken() works, we do the following:
// 1) First we save existing token index:
unsigned int savedTokenIndex = m_TokenIndex;
unsigned int savedLineNumber = m_LineNumber;
unsigned int savedNestLevel = m_NestLevel;
// 2) Then we get new token. Within DoGetToken(), a ReplaceBufferForReparse() may occur,
// either to expand an actual macro, or because of user replacement token.
// This will rewind m_TokenIndex, so the savedTokenIndex above will become incorrect.
if (SkipUnwanted())
m_PeekToken = DoGetToken();
else
m_PeekToken.Clear();
// 3) Restore saved index. Because of above mentioned issue, this could end up restoring m_TokenIndex to wrong value.
// This will cause bad problems if UngetToken() is called after this PeekToken().
m_TokenIndex = savedTokenIndex;
m_LineNumber = savedLineNumber;
m_NestLevel = savedNestLevel;
That is a PeekToken() specific fix. In general, we also need to reset m_UndoTokenIndex after a ReplaceBufferForReparse(). This is also done in the cc_fix_tokenizer.patch.
Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (revision 9271)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (working copy)
@@ -1817,6 +1865,7 @@
// Fix token index
m_TokenIndex -= bufferLen;
+ m_UndoTokenIndex = m_TokenIndex; //ADD(huki)
// Update the peek token
if (m_PeekAvailable && updatePeekToken)
case _T('='):
{
if (*(p - 1) <= _T(' '))
{
*p = _T('=');
// Don't add a space after '=' sign, in case another '=' follows it
// (see how the 'else' block below works).
//*++p = _T(' ');
++p;
}
else
{
switch (*(p - 1))
{
case _T('='):
case _T('!'):
case _T('>'):
case _T('<'):
{
*p = _T('=');
*++p = _T(' ');
++p;
}
break;
default:
{
*p = _T(' ');
*++p = _T('=');
*++p = _T(' ');
++p;
}
break;
}
}
}
break;
[...]
// '<', '>' and '!' are handled here.
default:
{
*p = ch;
++p;
}
break;
And here is the patch:
Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (revision 9271)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (working copy)
@@ -713,7 +718,6 @@
if (*(p - 1) <= _T(' '))
{
*p = _T('=');
- *++p = _T(' ');
++p;
}
else
@huki: Can I ask you to provide a patch which adds you to the "help -> about -> thanks to" dialog?
Sure, here you go:
Index: src/src/dlgabout.cpp
===================================================================
--- src/src/dlgabout.cpp (revision 9271)
+++ src/src/dlgabout.cpp (working copy)
@@ -139,6 +139,7 @@
"Sylvain Prat : Initial MSVC workspace and project importers\n"
"Chris Raschko : Design of the 3D logo for Code::Blocks\n"
"J.A. Ortega : 3D Icon based on the above\n"
+ "Huki : Patches for Code Completion plugin\n"
"\n"
"All contributors that provided patches.\n"
"The wxWidgets project (http://www.wxwidgets.org).\n"
The goal is to get UngetToken() to work reliably when macro expansions are involved.
As an example, we have this code:
#define MYMACRO X=0 /*Alternatively, user might have a replacement rule for MYMACRO*/
int X;
MYMACRO;
X = 1;
^ denotes the token index.
_ underlines the current token string.
EOL char is ignored for our example.
CASE I: when ReplaceBufferForReparse() is called in GetToken() / used directly before calling GetToken().
GetToken() Begin:
int X;MYMACRO;X = 1;
Token: _^
Undo: ^
int X;MYMACRO;X = 1;
Token: _______^
Undo: ^
BUG HERE: Undo token is still pointing to bad location!
int X;MYMAX=0;X = 1;
Token: ^
Undo: ^
int X;MYMAX=0;X = 1;
Token: ^
Undo: ^
int X;MYMAX=0;X = 1;
Token: _^
Undo: ^
CASE II: when ReplaceBufferForReparse() is called in PeekToken(), we have an additional problem.
PeekToken() Begin:
Step 1: Save the token index.
int X;MYMACRO;X = 1;
Token: _^
Save: ^
Step 2: DoGetToken():
- Get next token.
int X;MYMACRO;X = 1;
Token: _______^
Save: ^
int X;MYMAX=0;X = 1;
Token: ^
Save: ^
int X;MYMAX=0;X = 1;
Token: _^
Save: ^
Step 3: Assign Token to Peek, then restore Token from Save.
BUG HERE: Save is still pointing to bad location!
int X;MYMAX=0;X = 1;
Peek: _^
Token: _^
int X;MYMAX=0;X = 1;
Peek: _^
Token: _ ^
Here is an updated patch with fixes for both cases.
Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (revision 9271)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (working copy)
@@ -1135,6 +1146,8 @@
unsigned int savedLineNumber = m_LineNumber;
unsigned int savedNestLevel = m_NestLevel;
+ int savedReplaceCount = m_IsReplaceParsing ? m_RepeatReplaceCount : -1;
+
if (SkipUnwanted())
m_PeekToken = DoGetToken();
else
@@ -1144,9 +1157,20 @@
m_PeekLineNumber = m_LineNumber;
m_PeekNestLevel = m_NestLevel;
- m_TokenIndex = savedTokenIndex;
- m_LineNumber = savedLineNumber;
- m_NestLevel = savedNestLevel;
+ // Check whether a ReplaceBufferForReparse() was done in DoGetToken().
+ // We assume m_Undo... have already been reset in ReplaceBufferForReparse().
+ if (m_IsReplaceParsing && savedReplaceCount != (int)m_RepeatReplaceCount)
+ {
+ m_TokenIndex = m_UndoTokenIndex;
+ m_LineNumber = m_UndoLineNumber;
+ m_NestLevel = m_UndoNestLevel;
+ }
+ else
+ {
+ m_TokenIndex = savedTokenIndex;
+ m_LineNumber = savedLineNumber;
+ m_NestLevel = savedNestLevel;
+ }
}
return m_PeekToken;
@@ -1818,6 +1871,11 @@
// Fix token index
m_TokenIndex -= bufferLen;
+ // Fix undo position
+ m_UndoTokenIndex = m_TokenIndex;
+ m_UndoLineNumber = m_LineNumber;
+ m_UndoNestLevel = m_NestLevel;
+
// Update the peek token
if (m_PeekAvailable && updatePeekToken)
{
Ok.. about the issue with ReplaceBufferForReparse(), I think it will be more clear with an example.
The goal is to get UngetToken() to work reliably when macro expansions are involved.
...
Hi, thanks for the explanation, your patch is OK from my review. :D
Besides that, I think an improved UngetToken() can be:
/* peek is always available when we run UngetToken() once, actually the m_TokenIndex is moved
* backward one step. Note that the m_UndoTokenIndex value is not updated in this function, which
* means if you are not allowed to run this function twice.
*/
void Tokenizer::UngetToken()
{
if(m_TokenIndex == m_UndoTokenIndex) //this means we have already run a UngetToken() before.
return;
m_PeekTokenIndex = m_TokenIndex;
m_PeekLineNumber = m_LineNumber;
m_PeekNestLevel = m_NestLevel;
m_TokenIndex = m_UndoTokenIndex;
m_LineNumber = m_UndoLineNumber;
m_NestLevel = m_UndoNestLevel;
m_PeekToken = m_Token;
m_PeekAvailable = true;
}
But I found it cause some GUI hang problem, see my report question about running parser in thread (http://forums.codeblocks.org/index.php/topic,18742.msg128410.html#msg128410), I guess the reason is:
If you recursively parse the #include directive.
if (!locked)
CC_LOCKER_TRACK_TT_MTX_LOCK(s_TokenTreeMutex)
//this is a recursive call to Parse() function????
result = thread->Parse();
delete thread;
if (!locked)
CC_LOCKER_TRACK_TT_MTX_UNLOCK(s_TokenTreeMutex)
I have this code in one of my projects :
SDL_mutexP (rsd->RenderingMutex);
rgd = rsd->rgd;
SDL_mutexV (rsd->RenderingMutex);
What is approximatively the size of the data that you lock ?
The tokentree, which holds all the tokens(symbols), you can regard it as symbol database.
When we parse a file, you mainly do syntax analysis and add symbols one by one.
In the current implementation, we lock the tokentree when starting a single file, after a file is parsed, the locker is released. Something like this:
while (file queue is not empty)
{
get one file from the queue
lock the tokentree
while (parsing file not finished)
{
syntax analysis
add symbol to tokentree
if (meet a #include directive)
{
add file to the queue(for later parsing)
}
}
release the locker of tokentree
remove the file from the queue
}
So, I think it is not good idea to put the locker around "add symbol to tokentree".
while (file queue is not empty)
{
get one file from the queue
while (parsing file not finished)
{
syntax analysis
lock the tokentree
add symbol to tokentree
release the locker of tokentree
if (meet a #include directive)
{
add file to the queue(for later parsing)
}
}
remove the file from the queue
}
With Huki's patch, the code becomes:
while (file queue is not empty)
{
get one file from the queue
lock the tokentree
while (parsing file not finished)
{
syntax analysis
add symbol to tokentree
if (meet a #include directive)
{
recursive parse on the included source file
}
}
release the locker of tokentree
remove the file from the queue
}
So, you can get the correct token, as the #include directive is correctly expanded(it does the same thing as C preprocessor do), the time of the locker becomes longer.
I have no idea what is the correct idea, I think the GUI should not access to the tokentree when the parsing is not finished.
And something like that :
lock the tokentree
copy the token tree
release the tokentree
while (file queue not empty)
{
analysis / add symbol / fill queue with include directives, modifying copied_tokentree
lock the tokentree, update tokentree with copied_tokentree, then release it
}
Note that the token tree should contain a var saying "I am (not) in a writable state/copiable state". I don't know what use it.
This should have 2 consequences :
- the token tree is locked during short peroid of time, without risk of infinite loop (recursive inclusion or other funky trick)
- nobody has to wait for the parser to finish its task. Even if a parser would crash, the GUI would keep responsive.
Is there some technique to make Code Completion interpret defines?
for example:
((Some_struct*) someAddr)->
#define AVAR ((Some_struct*) someAddr)
AVAR->
It is rely annoying with embedded programming.
Any news on my cc_enum_values.patch? I'm attaching it again and the supported test cases in cc_enum_test.patch.
- Uses expression solver to calculate the enum value, expand macro assignments.
- Supports enum assignment to a previous enum (checks under the correct parent).
- If the expression cannot be evaluated (eg, unknown macro), leave it and the next enums blank and don't reset it to zero.
Thanks, patches won't be forgotten ! :) But time is limited.
Applied in my local repo and works quite well. (when testing, I also found an bug introduced in rev 9601).
One question remains, what does this means:
//FIX: Moved above.
/*if (peek==ParserConsts::colon)
{
// bit specifier (eg, xxx:1)
// -> walk to , or }
SkipToOneOfChars(ParserConsts::commaclbrace);
}*/
From my point of view, it should be removed, right? Because it was already handled before (in a if condition)
if (peek==ParserConsts::colon)
{
peek = SkipToOneOfChars(ParserConsts::equals + ParserConsts::commaclbrace);
}
About the include file expansion patch:
Hi, been busy for a while...
Today, I have tested the patch: cc_includes_parsing.patch
But I found it cause some GUI hang problem, see my report question about running parser in thread (http://forums.codeblocks.org/index.php/topic,18742.msg128410.html#msg128410), I guess the reason is:
If you recursively parse the #include directive.
if (!locked)
CC_LOCKER_TRACK_TT_MTX_LOCK(s_TokenTreeMutex)
//this is a recursive call to Parse() function????
result = thread->Parse();
delete thread;
if (!locked)
CC_LOCKER_TRACK_TT_MTX_UNLOCK(s_TokenTreeMutex)
I have a improvement patch to handle this (see attachment). The parser only parses cpp files in the project, recursive call the parse function when it meet the #include directive, the result is quite good.
Yes, my cc_includes_parsing patch did cause the UI to hang-up a bit because of the recursive parsing. I just tried your patch which frees up the mutex for a while before recursing, and the result is indeed quite good. Thanks!
I'm not sure the change of my improvement patch was good enough, I just add a small sleep function, any comments about such hack??
Thanks again for your contribution.
...
...
It's actually working very well - I have no more hangs now during batch parsing. I don't have a problem with the use of sleep function, it achieves the purpose, i.e., release the mutex before each include file, (very similar to the old behavior before the includes patch when the files were simply added to the end of the queue).
Thanks, yes, it works well (on my winXP system), so if no objections, I will commit this change.
I'm ok with committing this patch, can confirm it works fine on Win 7 / 8.
The proper solution would probably be to ensure that the tokentree mutex is not accessed by the GUI while parsing, but this is not very easy.. lot of things the user does - mouse hover over a symbol, clicking into a function block, etc., will trigger the GUI to try access the mutex.. those are the times when I noticed the hangs actually.
The old behavior, when the parser is working, then gui access to tokentree was forbidden until parser finishes batch parsing, user will see some tips like "Parser is running, so no tip available". And later Loaden (one of C::B dev, but he was not active for years now) added many lockers to let the tree access from multiply thread.
So, do you think we need to change back to the old behavior? I hate those lockers. :)
We can try changing back to the old behavior, I don't see any reason to access the tokentree before parsing is finished (we don't display tooltips anyway). Is it this kind of check that we find in NativeParser::MarkItemsByAI()?
if (!m_Parser->Done())
{
wxString msg(_("The Parser is still parsing files."));
msg += m_Parser->NotDoneReason();
CCLogger::Get()->DebugLog(msg);
return 0;
}
I'm ok with committing this patch, can confirm it works fine on Win 7 / 8.
Thanks for the test.
In SVN trunk now(r9665), it also includes a lot of code clean up, thanks for contribution.
Thanks for the commit.. just one question about this change in Parser::OnAllThreadsDone()
Index: src/plugins/codecompletion/parser/parser.cpp
===================================================================
--- src/plugins/codecompletion/parser/parser.cpp (revision 9664)
+++ src/plugins/codecompletion/parser/parser.cpp (revision 9665)
@@ -931,9 +888,7 @@
// Do next task
if ( !m_PoolTask.empty()
- || !m_BatchParseFiles.empty()
- || !m_PriorityHeaders.empty()
- || !m_PredefinedMacros.IsEmpty() )
+ || !m_BatchParseFiles.empty() )
{
TRACE(_T("Parser::OnAllThreadsDone(): Still some tasks left, starting m_BatchTimer."));
m_BatchTimer.Start(ParserCommon::PARSER_BATCHPARSE_TIMER_RUN_IMMEDIATELY, wxTIMER_ONE_SHOT);
Was the "|| !m_PredefinedMacros.IsEmpty()" removed on purpose?
Two short patches:
1) A fix for tilde operator: make sure if it's really a destructor function, or just a variable with the bitwise NOT operation.
Index: src/plugins/codecompletion/codecompletion.cpp
===================================================================
--- src/plugins/codecompletion/codecompletion.cpp (revision 9271)
+++ src/plugins/codecompletion/codecompletion.cpp (working copy)
@@ -249,8 +249,9 @@
if (!word.IsEmpty())
{
NameUnderCursor.Clear();
- if (GetLastNonWhitespaceChar(control, start) == _T('~'))
- NameUnderCursor << _T('~');
+ //FIX(huki), don't prepend '~' to search term (could be bitwise operator)
+ //if (GetLastNonWhitespaceChar(control, start) == _T('~'))
+ // NameUnderCursor << _T('~');
NameUnderCursor << word;
ReturnValue = true;
IsInclude = false;
@@ -2182,8 +2233,9 @@
const int startPos = editor->GetControl()->WordStartPosition(pos, true);
const int endPos = editor->GetControl()->WordEndPosition(pos, true);
wxString target;
+ bool isDestructor = false; //NOTE(huki), don't prepend '~' in cases of bitwise operator: eg: flag &= ~bits; and use "go to decl" on 'bits'.
if (CodeCompletionHelper::GetLastNonWhitespaceChar(editor->GetControl(), startPos) == _T('~'))
- target << _T('~');
+ isDestructor = true; //target << _T('~');
target << editor->GetControl()->GetTextRange(startPos, endPos);
if (target.IsEmpty())
return;
@@ -2201,7 +2253,7 @@
CC_LOCKER_TRACK_TT_MTX_LOCK(s_TokenTreeMutex)
// special handle destructor function
- if (target[0] == _T('~'))
+ if (isDestructor) //if (target[0] == _T('~'))
{
TokenIdxSet tmp = result;
result.clear();
@@ -2211,7 +2263,7 @@
const Token* token = tree->at(*it);
if (token && token->m_TokenKind == tkClass)
{
- token = tree->at(tree->TokenExists(target, token->m_Index, tkDestructor));
+ token = tree->at(tree->TokenExists(_T("~") + target, token->m_Index, tkDestructor));
if (token)
result.insert(token->m_Index);
}
2) "Go to implementation" fails on constructor declaration. Upon "go to impl", the old code assumes it's a constructor only if the ':' prefix is found. This means user can't go to the ctor implementation from the class declaration for example. i.e.,
class CodeCompletion
{
[...]
/** Constructor */
CodeCompletion(); <-- Use "go to implementation" on this line
[...]
}
Fix:
Index: src/plugins/codecompletion/codecompletion.cpp
===================================================================
--- src/plugins/codecompletion/codecompletion.cpp (revision 9271)
+++ src/plugins/codecompletion/codecompletion.cpp (working copy)
@@ -2232,8 +2284,9 @@
}
if (isClassOrConstructor)
{
- const bool isConstructor = CodeCompletionHelper::GetNextNonWhitespaceChar(editor->GetControl(), endPos) == _T('(')
- && CodeCompletionHelper::GetLastNonWhitespaceChar(editor->GetControl(), startPos) == _T(':');
+ //NOTE(huki), Assume ctor if isImpl (user may want to go to ctor implementation from the class declaration).
+ const bool isConstructor = CodeCompletionHelper::GetNextNonWhitespaceChar(editor->GetControl(), endPos) == _T('(')
+ && (isImpl || CodeCompletionHelper::GetLastNonWhitespaceChar(editor->GetControl(), startPos) == _T(':'));
for (TokenIdxSet::const_iterator it = result.begin(); it != result.end();)
{
const Token* token = tree->at(*it);
In case a token is prefixed with the ~ operator, and the user performs "go to decl / impl" on it, we just assume it's a destructor and there is no fallback if there is no such destructor. So the operation would fail in case of normal variables. eg:
int token, result;
[...]
result = ~token;
^^^^^------------ right-click on 'token' and use go to decl / impl
My patch makes the following changes:
- Don't display the tilde in the right-click context menu, eg., Find declaration of: 'token' rather than Find declaration of: '~token'. (i.e., don't prepend ~ to "NameUnderCursor" string)
- Also don't display the tilde in "symbol not found" messages after go to decl / impl (i.e., don't prepend ~ to "target" string).
- If the tilde prefix is there, first look for a class destructor as usual but if one is not found, fallback to accept any variable.
Index: src/plugins/codecompletion/codecompletion.cpp
===================================================================
--- src/plugins/codecompletion/codecompletion.cpp (revision 9271)
+++ src/plugins/codecompletion/codecompletion.cpp (working copy)
@@ -249,8 +249,9 @@
if (!word.IsEmpty())
{
NameUnderCursor.Clear();
- if (GetLastNonWhitespaceChar(control, start) == _T('~'))
- NameUnderCursor << _T('~');
+ //FIX(huki), don't prepend '~' to NameUnderCursor (only used for displaying in right-click popup menu)
+ //if (GetLastNonWhitespaceChar(control, start) == _T('~'))
+ // NameUnderCursor << _T('~');
NameUnderCursor << word;
ReturnValue = true;
IsInclude = false;
@@ -2182,8 +2233,9 @@
const int startPos = editor->GetControl()->WordStartPosition(pos, true);
const int endPos = editor->GetControl()->WordEndPosition(pos, true);
wxString target;
+ bool isDestructor = false; //NOTE(huki), don't prepend '~' to target, use isDestructor flag instead
if (CodeCompletionHelper::GetLastNonWhitespaceChar(editor->GetControl(), startPos) == _T('~'))
- target << _T('~');
+ isDestructor = true; //target << _T('~');
target << editor->GetControl()->GetTextRange(startPos, endPos);
if (target.IsEmpty())
return;
@@ -2201,7 +2253,7 @@
CC_LOCKER_TRACK_TT_MTX_LOCK(s_TokenTreeMutex)
// special handle destructor function
- if (target[0] == _T('~'))
+ if (isDestructor) //if (target[0] == _T('~'))
{
TokenIdxSet tmp = result;
result.clear();
@@ -2211,11 +2263,15 @@
const Token* token = tree->at(*it);
if (token && token->m_TokenKind == tkClass)
{
- token = tree->at(tree->TokenExists(target, token->m_Index, tkDestructor));
+ token = tree->at(tree->TokenExists(_T("~") + target, token->m_Index, tkDestructor));
if (token)
result.insert(token->m_Index);
}
}
+
+ // no destructor found, but could be a variable.
+ if (result.empty())
+ result = tmp;
}
// special handle constructor function
else
Let me know if it's ok for commit.
Thanks for commiting the tilde patch.
A small bugfix in nativeparser.cpp, NativeParser::ParseLocalBlock():
I think it's only supposed to be run for function blocks, but it's actually run for any kind of code block (classes, etc). So for example if the user clicks on a class declaration, the entire class block will be parsed as if it's a local block and several token info (such as the line index) will be overwritten. To fix it I've added a check, here's the patch:
Index: src/plugins/codecompletion/nativeparser.cpp
===================================================================
--- src/plugins/codecompletion/nativeparser.cpp (revision 9271)
+++ src/plugins/codecompletion/nativeparser.cpp (working copy)
@@ -1877,7 +1892,8 @@
CC_LOCKER_TRACK_TT_MTX_UNLOCK(s_TokenTreeMutex)
- if (!parent)
+ //FIX(huki), not for tkClass, etc.
+ if (!parent || !(parent->m_TokenKind & tkAnyFunction))
return false;
}
Index: src/plugins/codecompletion/parser/parser.cpp
===================================================================
--- src/plugins/codecompletion/parser/parser.cpp (revision 9271)
+++ src/plugins/codecompletion/parser/parser.cpp (working copy)
@@ -385,11 +385,14 @@
Parser::~Parser()
{
- CC_LOCKER_TRACK_P_MTX_LOCK(ParserCommon::s_ParserMutex)
+ //FIX: There was a deadlock in TerminateAllThreads() when calling DeleteParser() before
+ // parsing has finished. Moved s_ParserMutex lock below and updated TerminateAllThreads().
DisconnectEvents();
TerminateAllThreads();
+ CC_LOCKER_TRACK_P_MTX_LOCK(ParserCommon::s_ParserMutex)
+
if (ParserCommon::s_CurrentParser == this)
ParserCommon::s_CurrentParser = nullptr;
@@ -851,6 +867,8 @@
void Parser::TerminateAllThreads()
{
+ CC_LOCKER_TRACK_P_MTX_LOCK(ParserCommon::s_ParserMutex)
+
while (!m_PoolTask.empty())
{
PTVector& v = m_PoolTask.front();
@@ -859,6 +877,11 @@
m_PoolTask.pop();
}
+ CC_LOCKER_TRACK_P_MTX_UNLOCK(ParserCommon::s_ParserMutex)
+
+ //NOTE: This should not be locked with s_ParserMutex, otherwise we'll be stuck in an
+ // infinite loop below since the worker thread also enters s_ParserMutex.
+ // In fact cbThreadPool maintains it's own mutex, so m_Pool is probably threadsafe.
m_Pool.AbortAllTasks();
while (!m_Pool.Done())
wxMilliSleep(1);
And another patch to speed up canceling the ongoing parser:
Index: src/plugins/codecompletion/nativeparser.cpp
===================================================================
--- src/plugins/codecompletion/nativeparser.cpp (revision 9271)
+++ src/plugins/codecompletion/nativeparser.cpp (working copy)
@@ -616,14 +616,20 @@
if (m_ParsedProjects.empty())
{
- if (it->second == m_Parser)
- SetParser(m_TempParser); // Also updates class browser
-
wxString log(F(_("NativeParser::DeleteParser(): Deleting parser for project '%s'!"), prj.wx_str()));
CCLogger::Get()->Log(log);
CCLogger::Get()->DebugLog(log);
delete it->second;
+
+ //NOTE: Moved here as RemoveLastFunctionChildren() in SetParser() takes quite a while if
+ // parsing hasn't finished yet. Just delete the entire parser (above), then change to m_TempParser.
+ if (it->second == m_Parser)
+ {
+ m_Parser = nullptr;
+ SetParser(m_TempParser); // Also updates class browser
+ }
+
m_ParserList.erase(it);
return true;
@@ -1253,7 +1275,9 @@
if (m_Parser == parser)
return;
- RemoveLastFunctionChildren(m_Parser->GetTokenTree(), m_LastFuncTokenIdx);
+ if (m_Parser) //ADD, in case parser deleted before calling SetParser().
+ RemoveLastFunctionChildren(m_Parser->GetTokenTree(), m_LastFuncTokenIdx);
+
InitCCSearchVariables();
m_Parser = parser;
In the latest revision (rev9807), I have already add a comments here in the function.
void Parser::TerminateAllThreads()
{
// FIXME (ollydbg#1#): Do we need to use a mutex to protect the m_PoolTask accessing?
while (!m_PoolTask.empty())
{
PTVector& v = m_PoolTask.front();
for (PTVector::iterator it = v.begin(); it != v.end(); ++it)
delete *it;
m_PoolTask.pop();
}
BTW, in one testing branch of my local copy, I have try to remove all the lockers in CC (locker of the Parser and locker of the TokenTree), also, I remove the m_PoolTask, it was introduced to handle the priority header files, but now, we can already recursive to parse the included files, so I don't think m_PoolTask is needed any longer. Also, my local branch has many other issues I need to fix, so it will take a long time before I publish the patches ;)
EDIT:
A bugfix for crash when trying to cancel the ongoing project parsing (to reparse, quit CB, etc). Easiest way to reproduce is to select "Project -> Reparse current project" twice successively (or once before the initial parsing has finished). It's in fact a mutex deadlock, so here is the patch:
...
I try to reproduce this bug, but sorry I can't reproduce the dead lock in rev9807, under Window XP. I try the two method:
method one:
1, load a project, so the parsing started.
2, right click on the project, and select Peparse current project
3, no crash happened
method two:
1, right click on the project, and select Peparse current project
2, right click on the project, and select Peparse current project again
3, no crash happened
Can you give some more detailed explanation? Thanks.
The parser handling is a mass, see a belief draft diagram
CodeCompletion receive Workspace changed (mostly because project loaded finishes)
->NativeParser::CreateParser for the active project
->new Parser
->DoFullParsing
->Fill Parser's macro definition(from compiler and from project setting)
->Fill Parser's file list need to parse, kick the batch timer
Parser receive batch timer event
->new ParserThreadedTask (this task will executed in thread pool)
->send Parse Start Event!
ParserThreadedTask is executed
->parse the macro definition
->new ParserThread for each file in file list
->put ParserThread in thread queue
Thread pool finish running tasks:
->do one of below
*CASE1: if thread queue is not empty, copy to thread pool, run
*CASE2: if file list is not empty, kick the batch timer
*CASE3: if macro definition is not empty, kick the batch timer
*CASE4: non of the above cases, send Parse Finish Event!
Hi, Huki, thanks, would you mind to update your local copy of C::B to the latest SVN? I have some tiny issue to apply your patch, though I can manually applied it, but it looks like some of the code I have already changed.
Oh, sorry about that. I'm in fact planning on moving some of my projects to git very soon and if everything goes well I'll be updating to obf's repo here https://github.com/obfuscated/codeblocks_sf. In the meantime I'm trying to get some of the small patches in (just a few more actually).
BTW, in one testing branch of my local copy, I have try to remove all the lockers in CC (locker of the Parser and locker of the TokenTree), also, I remove the m_PoolTask, it was introduced to handle the priority header files, but now, we can already recursive to parse the included files, so I don't think m_PoolTask is needed any longer. Also, my local branch has many other issues I need to fix, so it will take a long time before I publish the patches
The CC code indeed needs some cleanup since we don't use the priority headers. It's good if the m_PoolTask can be removed, and while it's here I think it does needs to be protected since the last time I checked it was accessed from different threads..
Btw, though I'm using an old revision I do check the latest revisions of files which I've modified, to make sure my fix is still relevant. For this one (i.e., the mutex deadlock) I noticed no changes so I was quite sure the bug still exists in the newer revisions. It's very strange that you and White-Tiger are unable to reproduce it. Let me explain:
When calling DeleteParser() we lock the parser mutex before calling TerminateAllThreads():
Parser::~Parser()
{
CC_LOCKER_TRACK_P_MTX_LOCK(ParserCommon::s_ParserMutex)
DisconnectEvents();
TerminateAllThreads();
[...]
CC_LOCKER_TRACK_P_MTX_UNLOCK(ParserCommon::s_ParserMutex)
}
In TerminateAllThreads(), we send the request to abort all threads, and wait in a while loop till all threads have aborted:
m_Pool.AbortAllTasks();
while (!m_Pool.Done())
wxMilliSleep(1);
At this time if one of the worker threads has already been executed, you can see in the Execute call that it would enter and leave the parser mutex many times during the course of the call:
int ParserThreadedTask::Execute()
{
TRACE(_T("ParserThreadedTask::Execute(): Enter"));
if (!m_Parser) return 0;
CC_LOCKER_TRACK_P_MTX_LOCK(m_ParserMutex)
[...]
CC_LOCKER_TRACK_P_MTX_UNLOCK(m_ParserMutex);
[...]
CC_LOCKER_TRACK_P_MTX_LOCK(m_ParserMutex)
[...]
And if Execute() is waiting to enter this mutex while we are doing the while loop in TerminateAllThreads(), we would never return. FYI, I had confirmed this condition when checking this bug under debugger.
Now, the m_Pool object happens to keep it's own mutex to protect calls to it from multiple threads, so there doesn't seem to be any reason to use the parser mutex around the m_Pool object. So in my patch I had removed the lock around the while loop, and also around the DisconnectEvents() call in ~Parser().
Hi, Huki, thanks for the detailed explanation, I debugged this issue, the interesting thing is that the ParserthreadedTask runs very fast after creating the parser, so when I close the parser, I always see ParserthreadedTask already finishes its job. :)
I just find a way to reproduce the dead lock, I add a long sleep. (during the long sleep, I just close the project)
src/plugins/codecompletion/parser/parserthreadedtask.cpp | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/plugins/codecompletion/parser/parserthreadedtask.cpp b/src/plugins/codecompletion/parser/parserthreadedtask.cpp
index d3c98b6..ac11852 100644
--- a/src/plugins/codecompletion/parser/parserthreadedtask.cpp
+++ b/src/plugins/codecompletion/parser/parserthreadedtask.cpp
@@ -66,6 +66,8 @@ int ParserThreadedTask::Execute()
TRACE(_T("ParserThreadedTask::Execute(): Enter"));
if (!m_Parser) return 0;
+ wxSleep(15);
+
CC_LOCKER_TRACK_P_MTX_LOCK(m_ParserMutex)
wxString preDefs(m_Parser->m_PredefinedMacros);
Two patches for build targets parsing (i.e., project defined macros):
1) When a virtual target is selected, collect the defines for all child targets. Eg., in the CB project, if the "All" virtual target is selected we should collect defines from all sub-targets (scintilla, etc).
Index: src/plugins/codecompletion/nativeparser.cpp
===================================================================
--- src/plugins/codecompletion/nativeparser.cpp (revision 9271)
+++ src/plugins/codecompletion/nativeparser.cpp (working copy)
@@ -2342,6 +2367,18 @@
for (size_t i = 0; i < targetOpts.GetCount(); ++i)
opts.Add(targetOpts[i]);
}
+ // In case of virtual targets, collect the defines from all child targets.
+ wxArrayString targets = project->GetExpandedVirtualBuildTargetGroup(project->GetActiveBuildTarget());
+ for (size_t i = 0; i < targets.GetCount(); ++i)
+ {
+ target = project->GetBuildTarget(targets[i]);
+ if (target != NULL)
+ {
+ wxArrayString targetOpts = target->GetCompilerOptions();
+ for (size_t j = 0; j < targetOpts.GetCount(); ++j)
+ opts.Add(targetOpts[j]);
+ }
+ }
for (size_t i = 0; i < opts.GetCount(); ++i)
{
2) When the build target is changed, trigger a whole project reparse.
Index: src/plugins/codecompletion/codecompletion.cpp
===================================================================
--- src/plugins/codecompletion/codecompletion.cpp (revision 9271)
+++ src/plugins/codecompletion/codecompletion.cpp (working copy)
@@ -647,6 +695,8 @@
pm->RegisterEventSink(cbEVT_WORKSPACE_CHANGED, new cbEventFunctor<CodeCompletion, CodeBlocksEvent>(this, &CodeCompletion::OnWorkspaceChanged));
+ pm->RegisterEventSink(cbEVT_BUILDTARGET_SELECTED, new cbEventFunctor<CodeCompletion, CodeBlocksEvent>(this, &CodeCompletion::OnBuildTargetSelected));
+
pm->RegisterEventSink(cbEVT_PROJECT_ACTIVATE, new cbEventFunctor<CodeCompletion, CodeBlocksEvent>(this, &CodeCompletion::OnProjectActivated));
pm->RegisterEventSink(cbEVT_PROJECT_CLOSE, new cbEventFunctor<CodeCompletion, CodeBlocksEvent>(this, &CodeCompletion::OnProjectClosed));
pm->RegisterEventSink(cbEVT_PROJECT_SAVE, new cbEventFunctor<CodeCompletion, CodeBlocksEvent>(this, &CodeCompletion::OnProjectSaved));
@@ -2552,6 +2609,29 @@
event.Skip();
}
+void CodeCompletion::OnBuildTargetSelected(CodeBlocksEvent& event)
+{
+ if (!ProjectManager::IsBusy() && IsAttached() && m_InitDone)
+ {
+ cbProject* project = Manager::Get()->GetProjectManager()->GetActiveProject();
+ if (project)
+ {
+ TRACE(_T("CodeCompletion::OnBuildTargetSelected(): Reparsing entire project."));
+
+ ReparsingMap::iterator it = m_ReparsingMap.find(project);
+ if (it != m_ReparsingMap.end())
+ m_ReparsingMap.erase(it);
+ if (m_NativeParser.DeleteParser(project))
+ m_NativeParser.CreateParser(project);
+
+ m_AutocompNameIdx.clear();
+ m_LastAutocompIndex = -1;
+ }
+ }
+
+ event.Skip();
+}
+
void CodeCompletion::OnProjectActivated(CodeBlocksEvent& event)
{
// The Class browser shouldn't be updated if we're in the middle of loading/closing
Index: src/plugins/codecompletion/codecompletion.h
===================================================================
--- src/plugins/codecompletion/codecompletion.h (revision 9271)
+++ src/plugins/codecompletion/codecompletion.h (working copy)
@@ -196,6 +200,7 @@
void OnProjectFileAdded(CodeBlocksEvent& event);
void OnProjectFileRemoved(CodeBlocksEvent& event);
void OnProjectFileChanged(CodeBlocksEvent& event);
+ void OnBuildTargetSelected(CodeBlocksEvent& event);
/** SDK editor related events */
void OnEditorSaveOrModified(CodeBlocksEvent& event);
void OnEditorOpen(CodeBlocksEvent& event);
It looks like
m_AutocompNameIdx.clear();
m_LastAutocompIndex = -1;
The second patch is more of a performance concern. Most projects have a Debug and Release target (where a DEBUG macro is defined for the Debug target). In this case when the target is changed, the user expects that the DEBUG code is enabled or disabled accordingly, which means the full project needs to be reparsed with the new macro state. Personally I don't mind the time taken for full project reparse, so I'd suggest committing the second patch too. :)
I even thought that we can switch the macro definition set when parsing different targets' source files, not sure this is doable. I do care about the performance issue. When developing C::B, I ways switch the build target from time to time. (I use thread search plugin, I need the search only limited to the current active target). So, I personally don't like the idea that switching the target cause a reparse. But I would like to see all the comments about this.
I understand about that.. I'm going to keep this feature aside for now, we'll see if we are able to find a better solution in the future.
Btw, I think you remember our fix for reliable UngetToken() behavior (see here (http://forums.codeblocks.org/index.php/topic,18315.msg125579.html#msg125579) for the original discussion). To recap, in PeekToken(): we are checking the value of m_IsReplaceParsing before and after DoGetToken() to see if a ReplaceBufferText() was done, and if it was done, we should reset the savedTokenIndex, etc.
Now, this way of checking m_IsReplaceParsing can fail in some cases:
1) We are already in replace mode (eg, m_IsReplaceParsing == 1) before DoGetToken(), and inside DoGetToken() m_IsReplaceParsing is reset to zero and a new macro replacement is started (once again m_IsReplaceParsing == 1). No change will be detected and we will fail to reset the savedToken position.
2) A preprocessor condition is hit in SkipUnwanted() and we run CalcConditionExpression(). Now, CalcConditionExpression() will not only expand the macro, but also parse the entire expanded macro and compute the result. By the time we exit SkipUnwanted(), m_IsReplaceParsing is still non-zero, but we are already at the end of the expanded macro. Now, the DoGetToken() will reset m_IsReplaceParsing to zero, and once again no change will be detected. i.e.,
savedReplaceParsing = m_IsReplaceParsing; // m_IsReplaceParsing == 0
if (SkipUnwanted()) // ReplaceBufferText() was run and expanded macro parsed, m_IsReplaceParsing == 1
DoGetToken(); // out of replace buffer, m_IsReplaceParsing reset to 0
// check if ReplaceBufferText() was run
if (savedReplaceParsing < m_IsReplaceParsing) // condition failed
[...]
Yesterday I was unlucky enough to hit both these cases 1) and 2) in the same code, so we definitely need some improvement.. ;)
I think the proper way of doing it is to make the saved___ variables class members of Tokenizer, so they can be accessed from ReplaceBufferText() and reset when required (just like we are resetting the m_Undo___ variables). We also need to reset Undo and Saved positions to the proper place after CalcConditionExpression() (because as I've said before it's not only expanding the macro but also parsing it fully, so we need to reset right after the preprocessor line has ended).
I'll post the patch with these changes and some test cases below.
I'm attaching the patch below: (git patch against very recently synced master)
- Moved local variables saved___ to Tokenizer class, m_Saved___.
- Reset m_Undo and m_Saved vars to proper position in ReplaceBufferText().
- Reset m_Undo and m_Saved vars to proper position in CalcConditionExpression().
- Added missing line in GetPreprocessorType().
- Reverted your changes in UnGetToken() and left a note.
Now for some test cases:
1) When ReplaceBufferText() is done in DoGetToken(), (the usual case):
Our code:
#define MYMACRO X=0
int X;
MYMACRO;
X = 1;
PeekToken() Begin:
int X;MYMACRO;X=1;
Token: _^
Undo : ^
Peek :
Update m_SavedTokenIndex and run DoGetToken():
int X;MYMACRO;X=1;
Token: _ ^
Save : ^
Undo : ^
Peek :
ReplaceBufferText() and reset m_TokenIndex:
int X;MYMAX=0;X=1;
Token: _ ^
Save : ^
Undo : ^
Peek :
(Added by patch) Reset m_Undo and m_Save position:
int X;MYMAX=0;X=1;
Token: _ ^
Save : ^
Undo : ^
Peek :
Get the next token:
(Note that the string will be saved in m_PeekToken,
m_Token will be untouched)
int X;MYMAX=0;X=1;
Token: _ ^
Save : ^
Undo : ^
Peek : _
PeekToken() End: update m_PeekTokenIndex and restore m_TokenIndex from m_SaveTokenIndex:
int X;MYMAX=0;X=1;
Token: _ ^
Undo : ^
Peek : _^
Now, user calls UnGetToken(): (m_PeekAvailable becomes true)
(Note that even though m_TokenIndex == m_UndoTokenIndex, we should allow UnGetToken()
to run anyway. Of course, we should only call it once after each GetToken())
int X;MYMAX=0;X=1;
Token: ^
Undo : ^
Peek : _ ^
GetToken() again: (take from peek and reset m_PeekAvailable to false)
int X;MYMAX=0;X=1;
Token: _ ^
Undo : ^
Peek :
2) When CalcConditionExpression() is called in SkipUnwanted() immediately before DoGetToken():
Note that I have modified the example code to remove the pointer (it was originally "typedef RGBQUAD *LPRGBQUAD;"). But I don't think the current CC typedef support works well with pointers (and I'm yet to submit my patch for it).
2a) A simple case when there is no macro replacement involved:
// just some code :)
typedef struct tagRGBQUAD {
char rgbBlue;
char rgbGreen;
char rgbRed;
char rgbReserved;
} RGBQUAD;
#if 1
typedef RGBQUAD LPRGBQUAD;
#endif
PeekToken() Begin:
RGBQUAD;#if 1 typedef RGBQUAD LPRGBQUAD;
Token _^
Undo ^
Peek
PeekToken(): SkipUnwanted():
Note: It will run CalcConditionExpression(), evaluate the result to
true and skip to the first character after the condition expression.
RGBQUAD;#if 1 typedef RGBQUAD LPRGBQUAD;
Token _ ^
Save ^
Undo ^
Peek
PeekToken(): SkipUnwanted(): CalcConditionExpression() End:
We reset Save and Undo:
RGBQUAD;#if 1 typedef RGBQUAD LPRGBQUAD;
Token _ ^
Save ^
Undo ^
Peek
PeekToken(): DoGetToken():
RGBQUAD;#if 1 typedef RGBQUAD LPRGBQUAD;
Token _ ^
Save ^
Undo ^
Peek _______
PeekToken() End: Reset Token from Save:
RGBQUAD;#if 1 typedef RGBQUAD LPRGBQUAD;
Token _ ^
Undo ^
Peek _______^
Now user runs UnGetToken(): m_PeekAvailable becomes true:
Note again as in our first example, TokenIndex == UndoIndex,
but we should run UnGetToken() anyway.
RGBQUAD;#if 1 typedef RGBQUAD LPRGBQUAD;
Token ^
Undo ^
Peek _ ^
GetToken() again: Use peek and reset m_PeekAvailable:
RGBQUAD;#if 1 typedef RGBQUAD LPRGBQUAD;
Token _ ^
Undo ^
Peek
Of course, in this simple case there was no macro replacement, so although resetting the undo and save tokens was good, it was not really needed for proper parsing. In case of macro replacement it becomes necessary.
2b) A (lot) more complex case of previous example, taken from the mingw-builds winapi code which gave me trouble yesterday:
Our code:
#define WINAPI_PARTITION_DESKTOP 0x1
#define WINAPI_PARTITION_APP 0x2
#define WINAPI_FAMILY_APP WINAPI_PARTITION_APP
#define WINAPI_FAMILY_DESKTOP_APP (WINAPI_PARTITION_DESKTOP \
| WINAPI_PARTITION_APP)
/* WINAPI_FAMILY can be either desktop + App, or App. */
#ifndef WINAPI_FAMILY
#define WINAPI_FAMILY WINAPI_FAMILY_DESKTOP_APP
#endif
#define WINAPI_FAMILY_PARTITION(v) ((WINAPI_FAMILY & v) == v)
#define WINAPI_FAMILY_ONE_PARTITION(vset, v) ((WINAPI_FAMILY & vset) == v)
typedef struct tagRGBQUAD {
char rgbBlue;
char rgbGreen;
char rgbRed;
char rgbReserved;
} RGBQUAD;
#if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)
typedef RGBQUAD LPRGBQUAD; // removed *
#endif
Parsing:
PeekToken() Begin:
RGBQUAD;#if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token _^
Undo ^
Peek
PeekToken(): SkipUnwanted(): GetPreprocessorType():
RGBQUAD;#if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token _ ^
Save ^
Undo ^
Peek
PeekToken(): SkipUnwanted(): CalcConditionExpression():
Note: This will expand the macro using ReplaceBufferText() which will
already reset Undo and Save positions. But is it enough?
RGBQUAD;#if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token _ ^
Save ^
Undo ^
Peek
((WINAPI_FAMILY & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ^ ;
Save ^
Undo ^
Peek
WINAPI_FAMILY_DESKTOP_APP & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ^ ;
Save ^
Undo ^
Peek
(WINAPI_PARTITION_DESKTOP|WINAPI_PARTITION_APP) & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ^ ;
Save ^
Undo ^
Peek
PeekToken(): SkipUnwanted(): CalcConditionExpression():
Now the entire expanded macro will be parsed, so we are left at the end of it:
(WINAPI_PARTITION_DESKTOP|WINAPI_PARTITION_APP) & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ; ^
Save ^
Undo ^
Peek
PeekToken(): SkipUnwanted(): CalcConditionExpression(): End:
The condition macro has lost its context (we lost the #if),
so we must reset undo and save to the end of the macro statement.
(WINAPI_PARTITION_DESKTOP|WINAPI_PARTITION_APP) & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ; ^
Save ^
Undo ^
Peek
PeekToken(): DoGetToken():
(WINAPI_PARTITION_DESKTOP|WINAPI_PARTITION_APP) & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ; ^
Save ^
Undo ^
Peek _______
PeekToken(): End: Reset Token from Save:
Note that the m_Token string is untouched, it still
contain semi-colon which we started with:
(WINAPI_PARTITION_DESKTOP|WINAPI_PARTITION_APP) & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ; ^
Undo ^
Peek _______^
Now user runs UnGetToken(): m_PeekAvailable becomes true:
(WINAPI_PARTITION_DESKTOP|WINAPI_PARTITION_APP) & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ^
Undo ^
Peek ; ^
GetToken() again: Take from peek:
(WINAPI_PARTITION_DESKTOP|WINAPI_PARTITION_APP) & WINAPI_PARTITION_DESKTOP) == WINAPI_PARTITION_DESKTOP)typedef RGBQUAD LPRGBQUAD;
Token ; ^
Undo ^
Peek
AAA->BBBBBBBBBBBBBBBBBBBBBBBBBBBBB
After the replacement, the BBBBBBBBBBBBBBBBBBBBBBBBBBBBB can overwrite the m_Buffer[m_SavedTokenIndex], and m_Buffer[m_UndoTokenIndex], so both m_SavedTokenIndex and m_UndoTokenIndex should be reset. Since m_Token is still valid, we can still safely call void Tokenizer::UngetToken() once. In-fact, UngetToken() won't move any Indexs if they are reset.
I will commit your patch soon, thanks.
EDIT committed in r9834
Thanks for the commit. 2 patches for parsing bugs:
1) Don't skip successive opening or closing brackets. For example, see any occurrence of "GetConfigManager(_T("code_completion"))->XXX" in CodeCompletion.cpp,
void CodeCompletion::UpdateToolBar()
{
bool showScope = Manager::Get()->GetConfigManager(_T("code_completion"))->ReadBool(_T("/scope_filter"), true);
^^^^^^^^________________________________ // cannot use CC on ReadBool
[...]
Fix:
From dcc4ea033b398329915a900faada2d1254261313 Mon Sep 17 00:00:00 2001
From: huki <gk7huki@gmail.com>
Date: Tue, 24 Jun 2014 08:11:28 +0530
Subject: [PATCH 04/30] CC: fix parser successive brackets
---
src/plugins/codecompletion/nativeparser_base.cpp | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/src/plugins/codecompletion/nativeparser_base.cpp b/src/plugins/codecompletion/nativeparser_base.cpp
index 89e252c..3c5cf75 100644
--- a/src/plugins/codecompletion/nativeparser_base.cpp
+++ b/src/plugins/codecompletion/nativeparser_base.cpp
@@ -561,6 +561,9 @@ unsigned int NativeParserBase::FindCCTokenStart(const wxString& line)
if (IsClosingBracket(startAt, line))
++nest;
+ //NOTE: do not skip successive opening brackets.
+ if (IsOpeningBracket(startAt, line))
+ --nest;
}
startAt = BeforeToken(startAt, line);
@@ -650,6 +653,15 @@ wxString NativeParserBase::GetNextCCToken(const wxString& line,
if (IsOpeningBracket(startAt, line))
++nest;
+ //NOTE: do not skip successive closing brackets. Eg,
+ // "GetConfigManager(_T("code_completion"))->ReadBool"
+ // ^
+ if (IsClosingBracket(startAt, line))
+ {
+ --nest;
+ if (nest == 0)
+ ++startAt;
+ }
}
}
if (IsOperatorBegin(startAt, line))
--
1.9.4.msysgit.0
2) In DoParse(), when parsing "else" we try to eat the arguments. It breaks support for "else-if" (we will end up eating the "if"), and anyway there is no need to skip anything after "else".
From 900b6df9304e30455438c362f2757c88bd690494 Mon Sep 17 00:00:00 2001
From: huki <gk7huki@gmail.com>
Date: Tue, 24 Jun 2014 09:20:07 +0530
Subject: [PATCH 26/30] CC: parser: fixed 'else-if'
---
src/plugins/codecompletion/parser/parserthread.cpp | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/src/plugins/codecompletion/parser/parserthread.cpp b/src/plugins/codecompletion/parser/parserthread.cpp
index f87b1b7..938a134 100644
--- a/src/plugins/codecompletion/parser/parserthread.cpp
+++ b/src/plugins/codecompletion/parser/parserthread.cpp
@@ -714,8 +714,9 @@ void ParserThread::DoParse()
{
if (!m_Options.useBuffer || m_Options.bufferSkipBlocks)
SkipToOneOfChars(ParserConsts::semicolonclbrace, true, true);
- else
- m_Tokenizer.GetToken(); // skip arguments
+ //NOTE: this breaks support for 'else if' !
+ // else
+ // m_Tokenizer.GetToken(); // skip arguments
m_Str.Clear();
}
else if (token == ParserConsts::kw_enum)
--
1.9.4.msysgit.0
Thanks for the commit. 2 patches for parsing bugs:
...snip...
Hi, sorry for the late reply, I'm busy those days.
I tested and committed the two patches(rev9846 and rev9847), thanks for your contribution. ;)
Hi, thanks for the commit.
A small update to the undo token behavior fix (http://forums.codeblocks.org/index.php/topic,18315.msg132555.html#msg132555).
There was some code I added to CalcConditionExpression() by mistake (assuming it is called directly inside SkipUnwanted()). See my comments below:
[...] A preprocessor condition is hit in SkipUnwanted() and we run CalcConditionExpression(). [...] We need to reset Undo and Saved positions to the proper place after CalcConditionExpression() (because it's not only expanding the macro but also parsing it fully, so we need to reset right after the preprocessor line has ended).
In fact the function which is called in SkipUnwanted() is HandleConditionPreprocessor(), that's where we should reset the undo and savetoken values. If we reset them in CalcConditionExpression() we will always reset to the start of the first conditional block even if it is false.
#if 0| <---- adding the code in CalcConditionExpression() will reset undo vars to here which is wrong
// inactive code
#else| <---- adding the code in HandleConditionPreprocessor() will reset undo vars to here
// active code
#endif
I've also added a small note to avoid recursive call to PeekToken() (which shouldn't really happen).
Here's the patch:
From fd5af3421e85d9eff782c63ab07736cadf04d45c Mon Sep 17 00:00:00 2001
From: huki <gk7huki@gmail.com>
Date: Fri, 4 Jul 2014 02:32:16 +0530
Subject: CC: update to undo token fix
---
src/plugins/codecompletion/parser/tokenizer.cpp | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/src/plugins/codecompletion/parser/tokenizer.cpp b/src/plugins/codecompletion/parser/tokenizer.cpp
index 2bfb113..4a8f0cb 100644
--- a/src/plugins/codecompletion/parser/tokenizer.cpp
+++ b/src/plugins/codecompletion/parser/tokenizer.cpp
@@ -1043,7 +1043,6 @@ wxString Tokenizer::PeekToken()
{
if (!m_PeekAvailable)
{
- m_PeekAvailable = true;
//NOTE: The m_Saved... vars will be reset to the correct position
// as necessary when a ReplaceBufferText() is done.
m_SavedTokenIndex = m_TokenIndex;
@@ -1055,6 +1054,7 @@ wxString Tokenizer::PeekToken()
else
m_PeekToken.Clear();
+ m_PeekAvailable = true; //NOTE: Moved after DoGetToken() to avoid recursive PeekToken() calls.
m_PeekTokenIndex = m_TokenIndex;
m_PeekLineNumber = m_LineNumber;
m_PeekNestLevel = m_NestLevel;
@@ -1407,11 +1407,6 @@ bool Tokenizer::CalcConditionExpression(bool checkResult)
// reset tokenizer's functionality
m_State = oldState;
- // reset undo token
- m_SavedTokenIndex = m_UndoTokenIndex = m_TokenIndex;
- m_SavedLineNumber = m_UndoLineNumber = m_LineNumber;
- m_SavedNestingLevel = m_UndoNestLevel = m_NestLevel;
-
exp.ConvertInfixToPostfix();
if (exp.CalcPostfix())
{
@@ -1697,6 +1692,11 @@ void Tokenizer::HandleConditionPreprocessor(const PreprocessorType type)
default:
break;
}
+
+ // reset undo token
+ m_SavedTokenIndex = m_UndoTokenIndex = m_TokenIndex;
+ m_SavedLineNumber = m_UndoLineNumber = m_LineNumber;
+ m_SavedNestingLevel = m_UndoNestLevel = m_NestLevel;
}
void Tokenizer::SplitArguments(wxArrayString& results)
@@ -1826,6 +1826,7 @@ bool Tokenizer::ReplaceBufferText(const wxString& target, bool updatePeekToken)
{
m_PeekAvailable = false;
PeekToken(); // NOTE (ollydbg#1#): chance of recursive call of this function
+ // NOTE (huki): updated PeekToken() to prevent unnecessary recursive call
}
return true;
--
1.9.4.msysgit.0
// Update the peek token
if (m_PeekAvailable && updatePeekToken)
{
m_PeekAvailable = false;
PeekToken(); // NOTE (ollydbg#1#): chance of recursive call of this function
// NOTE (huki): updated PeekToken() to prevent unnecessary recursive call
}
Hi, Huki, thanks for the contribution, today, I looked at you patch, and I think that
// Update the peek token
if (m_PeekAvailable && updatePeekToken)
{
m_PeekAvailable = false;
PeekToken(); // NOTE (ollydbg#1#): chance of recursive call of this function
// NOTE (huki): updated PeekToken() to prevent unnecessary recursive call
}
Oh, I was wrong, I just debug the code and found that the function bool Tokenizer::ReplaceBufferText(const wxString& target, bool updatePeekToken) will be called from void ParserThread::HandleMacroExpansion(int id, const wxString &peek), which is called from DoParse() function.
I understand you idea, that is the function Tokenizer::ReplaceBufferText() can also be called inside the PeekToken() function.
The old way (without your patch) of PeekToken() could be:
first set the flag: m_PeekAvailable = true;
some function will call ReplaceBufferText();
Now, in your patch, you set the flag a bit later, which are something like:
some function will call ReplaceBufferText();
set the flag: m_PeekAvailable = true;
Question: what happens about old way (without your patch), do we enter a infinite recursive call to PeekToken()?
Yes, the m_PeekAvailable may be true in some cases, eg:
In DoParse():
PeekToken(); // sets m_PeekAvailable to true
[...]
ReplaceBufferText(); // m_PeekAvailable is true, so call PeekToken() and update m_PeekToken
My patch makes sure that m_PeekAvailable is not true if we are already inside PeekToken(), so a useless recursive call is avoided. i.e.,
PeekToken():
-> DoGetToken():
-> ReplaceMacro():
-> ReplaceBufferText(); // no need for recursive call to PeekToken() as we are already going to
// update m_PeekToken after returning from DoGetToken()
-> DoGetToken(); // recurse DoGetToken()
-> m_PeekAvailable = true; // got m_PeekToken, now set this to true
Question: what happens about old way (without your patch), do we enter a infinite recursive call to PeekToken()?
I don't think there is an infinite recursive call, but generally PeekToken and GetToken are not supposed to recurse, because they update class variables (GetToken() updates Undo__ vars and PeekToken() updates Save__ vars), so a recursive call would overwrite these values, which is bad. So we should first get the peek token (i.e., m_PeekToken = DoGetToken(); ), then set m_PeekAvailable = true;
Code completion ignores code like this:
#if defined(TOUCHSCREEN)
void InputManager::ConvertTouch(sf::Event& event)
{
...
}
#endif
Parser does not understand "final" correctly?
An checkboxes in "Editor->Code complition->C++ parser" "Parse preproceccor..." "Parse macros..." are not saving their state. They are checked always.
Code completion ignores code like this:
#if defined(TOUCHSCREEN)
void InputManager::ConvertTouch(sf::Event& event)
{
...
}
#endif
I just test a minimal sample code
#define TOUCHSCREEN 1
#if defined(TOUCHSCREEN)
void InputManager::ConvertTouch(sf::Event& event)
{
}
#endif
Parser does not understand "final" correctly?
An checkboxes in "Editor->Code complition->C++ parser" "Parse preproceccor..." "Parse macros..." are not saving their state. They are checked always.
Works fine here. (WinXP, latest nightly build C::B)
Here is a patch that adds support for:
1) Nested unnamed (struct or union) within unnamed: all the members should be invokable from the parent class or struct.
2) Show tooltips for members of unnamed / enum within class invoked directly (also for nested cases).
Patch:
From ef154d4ef188d77b10bc7ff55d5bcb54b18d66a2 Mon Sep 17 00:00:00 2001
From: huki <gk7huki@gmail.com>
Date: Tue, 24 Jun 2014 08:15:00 +0530
Subject: CC: Support unnamed or enum within class, recursive
---
src/plugins/codecompletion/nativeparser_base.cpp | 13 +++++++--
src/plugins/codecompletion/nativeparser_base.h | 34 +++++++++++++++++++++++-
2 files changed, 44 insertions(+), 3 deletions(-)
diff --git a/src/plugins/codecompletion/nativeparser_base.cpp b/src/plugins/codecompletion/nativeparser_base.cpp
index e85b8be..5f04b0c 100644
--- a/src/plugins/codecompletion/nativeparser_base.cpp
+++ b/src/plugins/codecompletion/nativeparser_base.cpp
@@ -1411,7 +1411,14 @@ size_t NativeParserBase::GenerateResultSet(TokenTree* tree,
{
const Token* token = tree->at(*it);
// check whether its under the parentIdx
- if (token && (token->m_ParentIndex == parentIdx))
+ //NOTE: check for unnamed or enum inside class.
+ // eg, 'ParserCommon::ParserState::ptCreateParser' should be accessed as 'ParserCommon::ptCreateParser'.
+ // Here, token is ptCreateParser and parentIdx is ParserCommon, so 'token->m_ParentIndex == parentIdx'
+ // is false. Now, we iterate over the children of parentIdx and check if any of them is unnamed or enum
+ // and match with token->m_ParentIndex. Thus if we confirm that 'token' is a child of unnamed or enum
+ // (i.e., m_ParentIndex), we add the token to result.
+ if (token && ((token->m_ParentIndex == parentIdx)
+ || IsChildOfUnnamedOrEnum(tree, token->m_ParentIndex, parentIdx)))
result.insert(*it);
// "result" will become the search scope for the next loop, so
@@ -1437,7 +1444,9 @@ size_t NativeParserBase::GenerateResultSet(TokenTree* tree,
ancestorIterator != tokenParent->m_Ancestors.end();
++ancestorIterator )
{
- if (token->m_ParentIndex == (*ancestorIterator)) //matched
+ //NOTE: check for unnamed or enum inside class (see note above).
+ if (token && ((token->m_ParentIndex == (*ancestorIterator)) //matched
+ || IsChildOfUnnamedOrEnum(tree, token->m_ParentIndex, (*ancestorIterator))))
result.insert(*it);
}
}
diff --git a/src/plugins/codecompletion/nativeparser_base.h b/src/plugins/codecompletion/nativeparser_base.h
index 3141e61..da60fea 100644
--- a/src/plugins/codecompletion/nativeparser_base.h
+++ b/src/plugins/codecompletion/nativeparser_base.h
@@ -495,7 +495,12 @@ private:
if ( tokenChild
&& (parent->m_TokenKind == tkClass || tokenChild->m_Scope != tsPrivate) )
{
- result.insert(*it);
+ //NOTE: recurse (eg: class A contains struct contains union or enum)
+ if ( !AddChildrenOfUnnamed(tree, tokenChild, result) )
+ {
+ result.insert(*it);
+ AddChildrenOfEnum(tree, tokenChild, result);
+ }
}
}
return true;
@@ -520,6 +525,33 @@ private:
}
return false;
}
+
+ bool IsChildOfUnnamedOrEnum(TokenTree* tree, const int targetIdx, const int parentIdx)
+ {
+ if (targetIdx == parentIdx)
+ return true;
+ if (parentIdx == -1)
+ return false;
+
+ Token* parent = tree->at(parentIdx);
+ if (parent && (parent->m_TokenKind & tkClass))
+ {
+ for (TokenIdxSet::const_iterator it = parent->m_Children.begin();
+ it != parent->m_Children.end(); ++it)
+ {
+ Token* token = tree->at(*it);
+ if (token && (((token->m_TokenKind & tkClass)
+ && (token->m_IsAnonymous == true))
+ || (token->m_TokenKind & tkEnum)))
+ {
+ if ((targetIdx == (*it)) || IsChildOfUnnamedOrEnum(tree, targetIdx, (*it)))
+ return true;
+ }
+ }
+ }
+ return false;
+ }
+
/** loop on the input Token index set (source), add all its public constructors to output Token index set (dest) */
void AddConstructors(TokenTree *tree, const TokenIdxSet& source, TokenIdxSet& dest);
--
1.9.4.msysgit.0
Code to test:
class ClassA
{
public:
int a;
struct A
{
int c;
} z;
struct
{
int s;
union
{
int u;
short v;
};
enum EnumS
{
STRUCT_A,
STRUCT_B,
};
};
enum
{
UNNAMED_A,
UNNAMED_B
};
enum EnumA
{
ENUM_A,
ENUM_B
};
};
// Hovering on following member variables should
// show tooltip: a, s, u, v, all the enums like STRUCT_A, etc.
ClassA::a; // replace 'a' with other members listed above
// However ClassA::c is illegal and should not display tooltip.
ClassA::c; // no tip
ClassA::A::c; // should show tip
// Code completion should show all legal members.
ClassA:: // cursor here
Here it is, macros highlighting for inactive code is ON
This option is not related to CC plugin.
TEST defined in Compiler's global defines. Parser knows about it and about TEST2, but it can't find declaration for TEST2 and TEST3.
Confirm this bug. It looks like CC doesn't collect the macro definitions if it is defined in "Compiler's global defines".
Note that the below code works OK
#define TEST
#ifdef TEST
#define TEST2
#endif // TEST
#ifdef TEST2
#define TEST3
#endif // TEST2
Because #ifdef __ANROID__ ... #endif section is grey-inactive, when __ANDROID__ is not defined manually. As you see __ANDROID__ is defined:
/arm-linux-androideabi-g++ -dM -E - < /dev/null
#define __DBL_MIN_EXP__ (-1021)
#define __HQ_FBIT__ 15
#define __UINT_LEAST16_MAX__ 65535
#define __ARM_SIZEOF_WCHAR_T 32
#define __ATOMIC_ACQUIRE 2
#define __SFRACT_IBIT__ 0
#define __FLT_MIN__ 1.1754943508222875e-38F
#define __UFRACT_MAX__ 0XFFFFP-16UR
#define __UINT_LEAST8_TYPE__ unsigned char
#define __DQ_FBIT__ 63
#define __INTMAX_C(c) c ## LL
#define __ULFRACT_FBIT__ 32
#define __SACCUM_EPSILON__ 0x1P-7HK
#define __CHAR_BIT__ 8
#define __USQ_IBIT__ 0
#define __UINT8_MAX__ 255
#define __ACCUM_FBIT__ 15
#define __ANDROID__ 1
#define __WINT_MAX__ 4294967295U
#define __USFRACT_FBIT__ 8
#define __ORDER_LITTLE_ENDIAN__ 1234
#define __SIZE_MAX__ 4294967295U
#define __ARM_ARCH_ISA_ARM 1
#define __WCHAR_MAX__ 4294967295U
#define __LACCUM_IBIT__ 32
#define __DBL_DENORM_MIN__ ((double)4.9406564584124654e-324L)
#define __GCC_ATOMIC_CHAR_LOCK_FREE 1
#define __FLT_EVAL_METHOD__ 0
#define __unix__ 1
#define __LLACCUM_MAX__ 0X7FFFFFFFFFFFFFFFP-31LLK
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
#define __FRACT_FBIT__ 15
#define __UINT_FAST64_MAX__ 18446744073709551615ULL
#define __SIG_ATOMIC_TYPE__ int
#define __UACCUM_FBIT__ 16
#define __DBL_MIN_10_EXP__ (-307)
#define __FINITE_MATH_ONLY__ 0
#define __ARMEL__ 1
#define __LFRACT_IBIT__ 0
#define __GNUC_PATCHLEVEL__ 0
#define __LFRACT_MAX__ 0X7FFFFFFFP-31LR
#define __UINT_FAST8_MAX__ 255
#define __DEC64_MAX_EXP__ 385
#define __INT8_C(c) c
#define __UINT_LEAST64_MAX__ 18446744073709551615ULL
#define __SA_FBIT__ 15
#define __SHRT_MAX__ 32767
#define __LDBL_MAX__ 1.7976931348623157e+308L
#define __FRACT_MAX__ 0X7FFFP-15R
#define __ARM_ARCH_5TE__ 1
#define __UFRACT_FBIT__ 16
#define __ARM_FP 12
#define __UFRACT_MIN__ 0.0UR
#define __UINT_LEAST8_MAX__ 255
#define __GCC_ATOMIC_BOOL_LOCK_FREE 1
#define __UINTMAX_TYPE__ long long unsigned int
#define __LLFRACT_EPSILON__ 0x1P-63LLR
#define __linux 1
#define __DEC32_EPSILON__ 1E-6DF
#define __CHAR_UNSIGNED__ 1
#define __UINT32_MAX__ 4294967295U
#define __ULFRACT_MAX__ 0XFFFFFFFFP-32ULR
#define __TA_IBIT__ 64
#define __LDBL_MAX_EXP__ 1024
#define __WINT_MIN__ 0U
#define __linux__ 1
#define __ULLFRACT_MIN__ 0.0ULLR
#define __SCHAR_MAX__ 127
#define __WCHAR_MIN__ 0U
#define __INT64_C(c) c ## LL
#define __DBL_DIG__ 15
#define __ARM_NEON_FP 4
#define __GCC_ATOMIC_POINTER_LOCK_FREE 1
#define __LLACCUM_MIN__ (-0X1P31LLK-0X1P31LLK)
#define __SIZEOF_INT__ 4
#define __SIZEOF_POINTER__ 4
#define __USACCUM_IBIT__ 8
#define __USER_LABEL_PREFIX__
#define __STDC_HOSTED__ 1
#define __LDBL_HAS_INFINITY__ 1
#define __LFRACT_MIN__ (-0.5LR-0.5LR)
#define __HA_IBIT__ 8
#define __TQ_IBIT__ 0
#define __FLT_EPSILON__ 1.1920928955078125e-7F
#define __APCS_32__ 1
#define __USFRACT_IBIT__ 0
#define __LDBL_MIN__ 2.2250738585072014e-308L
#define __FRACT_MIN__ (-0.5R-0.5R)
#define __DEC32_MAX__ 9.999999E96DF
#define __DA_IBIT__ 32
#define __ARM_SIZEOF_MINIMAL_ENUM 4
#define __INT32_MAX__ 2147483647
#define __UQQ_FBIT__ 8
#define __SIZEOF_LONG__ 4
#define __UACCUM_MAX__ 0XFFFFFFFFP-16UK
#define __UINT16_C(c) c
#define __DECIMAL_DIG__ 17
#define __LFRACT_EPSILON__ 0x1P-31LR
#define __ULFRACT_MIN__ 0.0ULR
#define __LDBL_HAS_QUIET_NAN__ 1
#define __ULACCUM_IBIT__ 32
#define __UACCUM_EPSILON__ 0x1P-16UK
#define __GNUC__ 4
#define __ULLACCUM_MAX__ 0XFFFFFFFFFFFFFFFFP-32ULLK
#define __HQ_IBIT__ 0
#define __FLT_HAS_DENORM__ 1
#define __SIZEOF_LONG_DOUBLE__ 8
#define __BIGGEST_ALIGNMENT__ 8
#define __DQ_IBIT__ 0
#define __DBL_MAX__ ((double)1.7976931348623157e+308L)
#define __ULFRACT_IBIT__ 0
#define __INT_FAST32_MAX__ 2147483647
#define __DBL_HAS_INFINITY__ 1
#define __ACCUM_IBIT__ 16
#define __DEC32_MIN_EXP__ (-94)
#define __THUMB_INTERWORK__ 1
#define __LACCUM_MAX__ 0X7FFFFFFFFFFFFFFFP-31LK
#define __INT_FAST16_TYPE__ int
#define __LDBL_HAS_DENORM__ 1
#define __DEC128_MAX__ 9.999999999999999999999999999999999E6144DL
#define __INT_LEAST32_MAX__ 2147483647
#define __ARM_PCS 1
#define __DEC32_MIN__ 1E-95DF
#define __ACCUM_MAX__ 0X7FFFFFFFP-15K
#define __DBL_MAX_EXP__ 1024
#define __USACCUM_EPSILON__ 0x1P-8UHK
#define __DEC128_EPSILON__ 1E-33DL
#define __SFRACT_MAX__ 0X7FP-7HR
#define __FRACT_IBIT__ 0
#define __PTRDIFF_MAX__ 2147483647
#define __UACCUM_MIN__ 0.0UK
#define __UACCUM_IBIT__ 16
#define __LONG_LONG_MAX__ 9223372036854775807LL
#define __SIZEOF_SIZE_T__ 4
#define __ULACCUM_MAX__ 0XFFFFFFFFFFFFFFFFP-32ULK
#define __SIZEOF_WINT_T__ 4
#define __SA_IBIT__ 16
#define __ULLACCUM_MIN__ 0.0ULLK
#define __GXX_ABI_VERSION 1002
#define __UTA_FBIT__ 64
#define __SOFTFP__ 1
#define __FLT_MIN_EXP__ (-125)
#define __USFRACT_MAX__ 0XFFP-8UHR
#define __UFRACT_IBIT__ 0
#define __ARM_FEATURE_QBIT 1
#define __INT_FAST64_TYPE__ long long int
#define __DBL_MIN__ ((double)2.2250738585072014e-308L)
#define __LACCUM_MIN__ (-0X1P31LK-0X1P31LK)
#define __ULLACCUM_FBIT__ 32
#define __GXX_TYPEINFO_EQUALITY_INLINE 0
#define __ULLFRACT_EPSILON__ 0x1P-64ULLR
#define __DEC128_MIN__ 1E-6143DL
#define __REGISTER_PREFIX__
#define __UINT16_MAX__ 65535
#define __DBL_HAS_DENORM__ 1
#define __ACCUM_MIN__ (-0X1P15K-0X1P15K)
#define __SQ_IBIT__ 0
#define __UINT8_TYPE__ unsigned char
#define __UHA_FBIT__ 8
#define __NO_INLINE__ 1
#define __SFRACT_MIN__ (-0.5HR-0.5HR)
#define __UTQ_FBIT__ 128
#define __FLT_MANT_DIG__ 24
#define __VERSION__ "4.8"
#define __UINT64_C(c) c ## ULL
#define __ULLFRACT_FBIT__ 64
#define __FRACT_EPSILON__ 0x1P-15R
#define __ULACCUM_MIN__ 0.0ULK
#define __UDA_FBIT__ 32
#define __LLACCUM_EPSILON__ 0x1P-31LLK
#define __GCC_ATOMIC_INT_LOCK_FREE 1
#define __FLOAT_WORD_ORDER__ __ORDER_LITTLE_ENDIAN__
#define __USFRACT_MIN__ 0.0UHR
#define __UQQ_IBIT__ 0
#define __INT32_C(c) c
#define __DEC64_EPSILON__ 1E-15DD
#define __ORDER_PDP_ENDIAN__ 3412
#define __DEC128_MIN_EXP__ (-6142)
#define __UHQ_FBIT__ 16
#define __LLACCUM_FBIT__ 31
#define __INT_FAST32_TYPE__ int
#define __UINT_LEAST16_TYPE__ short unsigned int
#define unix 1
#define __INT16_MAX__ 32767
#define __SIZE_TYPE__ unsigned int
#define __UINT64_MAX__ 18446744073709551615ULL
#define __UDQ_FBIT__ 64
#define __INT8_TYPE__ signed char
#define __ELF__ 1
#define __ULFRACT_EPSILON__ 0x1P-32ULR
#define __LLFRACT_FBIT__ 63
#define __FLT_RADIX__ 2
#define __INT_LEAST16_TYPE__ short int
#define __LDBL_EPSILON__ 2.2204460492503131e-16L
#define __UINTMAX_C(c) c ## ULL
#define __SACCUM_MAX__ 0X7FFFP-7HK
#define __SIG_ATOMIC_MAX__ 2147483647
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
#define __VFP_FP__ 1
#define __SIZEOF_PTRDIFF_T__ 4
#define __LACCUM_EPSILON__ 0x1P-31LK
#define __DEC32_SUBNORMAL_MIN__ 0.000001E-95DF
#define __INT_FAST16_MAX__ 2147483647
#define __UINT_FAST32_MAX__ 4294967295U
#define __UINT_LEAST64_TYPE__ long long unsigned int
#define __USACCUM_MAX__ 0XFFFFP-8UHK
#define __SFRACT_EPSILON__ 0x1P-7HR
#define __FLT_HAS_QUIET_NAN__ 1
#define __FLT_MAX_10_EXP__ 38
#define __LONG_MAX__ 2147483647L
#define __DEC128_SUBNORMAL_MIN__ 0.000000000000000000000000000000001E-6143DL
#define __FLT_HAS_INFINITY__ 1
#define __unix 1
#define __USA_FBIT__ 16
#define __UINT_FAST16_TYPE__ unsigned int
#define __DEC64_MAX__ 9.999999999999999E384DD
#define __CHAR16_TYPE__ short unsigned int
#define __PRAGMA_REDEFINE_EXTNAME 1
#define __INT_LEAST16_MAX__ 32767
#define __DEC64_MANT_DIG__ 16
#define __INT64_MAX__ 9223372036854775807LL
#define __UINT_LEAST32_MAX__ 4294967295U
#define __SACCUM_FBIT__ 7
#define __GCC_ATOMIC_LONG_LOCK_FREE 1
#define __INT_LEAST64_TYPE__ long long int
#define __ARM_FEATURE_CLZ 1
#define __INT16_TYPE__ short int
#define __INT_LEAST8_TYPE__ signed char
#define __SQ_FBIT__ 31
#define __DEC32_MAX_EXP__ 97
#define __ARM_ARCH_ISA_THUMB 1
#define __INT_FAST8_MAX__ 127
#define __ARM_ARCH 5
#define __INTPTR_MAX__ 2147483647
#define __QQ_FBIT__ 7
#define linux 1
#define __UTA_IBIT__ 64
#define __LDBL_MANT_DIG__ 53
#define __SFRACT_FBIT__ 7
#define __SACCUM_MIN__ (-0X1P7HK-0X1P7HK)
#define __DBL_HAS_QUIET_NAN__ 1
#define __SIG_ATOMIC_MIN__ (-__SIG_ATOMIC_MAX__ - 1)
#define __INTPTR_TYPE__ int
#define __UINT16_TYPE__ short unsigned int
#define __WCHAR_TYPE__ unsigned int
#define __SIZEOF_FLOAT__ 4
#define __USQ_FBIT__ 32
#define __pic__ 1
#define __UINTPTR_MAX__ 4294967295U
#define __DEC64_MIN_EXP__ (-382)
#define __ULLACCUM_IBIT__ 32
#define __INT_FAST64_MAX__ 9223372036854775807LL
#define __GCC_ATOMIC_TEST_AND_SET_TRUEVAL 1
#define __FLT_DIG__ 6
#define __UINT_FAST64_TYPE__ long long unsigned int
#define __INT_MAX__ 2147483647
#define __LACCUM_FBIT__ 31
#define __USACCUM_MIN__ 0.0UHK
#define __UHA_IBIT__ 8
#define __INT64_TYPE__ long long int
#define __FLT_MAX_EXP__ 128
#define __UTQ_IBIT__ 0
#define __DBL_MANT_DIG__ 53
#define __INT_LEAST64_MAX__ 9223372036854775807LL
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
#define __DEC64_MIN__ 1E-383DD
#define __WINT_TYPE__ unsigned int
#define __UINT_LEAST32_TYPE__ unsigned int
#define __SIZEOF_SHORT__ 2
#define __ULLFRACT_IBIT__ 0
#define __LDBL_MIN_EXP__ (-1021)
#define __arm__ 1
#define __UDA_IBIT__ 32
#define __INT_LEAST8_MAX__ 127
#define __LFRACT_FBIT__ 31
#define __LDBL_MAX_10_EXP__ 308
#define __ATOMIC_RELAXED 0
#define __DBL_EPSILON__ ((double)2.2204460492503131e-16L)
#define __UINT8_C(c) c
#define __INT_LEAST32_TYPE__ int
#define __SIZEOF_WCHAR_T__ 4
#define __UINT64_TYPE__ long long unsigned int
#define __LLFRACT_MAX__ 0X7FFFFFFFFFFFFFFFP-63LLR
#define __TQ_FBIT__ 127
#define __INT_FAST8_TYPE__ signed char
#define __ULLACCUM_EPSILON__ 0x1P-32ULLK
#define __UHQ_IBIT__ 0
#define __LLACCUM_IBIT__ 32
#define __DBL_DECIMAL_DIG__ 17
#define __DEC_EVAL_METHOD__ 2
#define __TA_FBIT__ 63
#define __UDQ_IBIT__ 0
#define __ORDER_BIG_ENDIAN__ 4321
#define __ACCUM_EPSILON__ 0x1P-15K
#define __UINT32_C(c) c ## U
#define __INTMAX_MAX__ 9223372036854775807LL
#define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__
#define __FLT_DENORM_MIN__ 1.4012984643248171e-45F
#define __LLFRACT_IBIT__ 0
#define __INT8_MAX__ 127
#define __PIC__ 1
#define __UINT_FAST32_TYPE__ unsigned int
#define __CHAR32_TYPE__ unsigned int
#define __FLT_MAX__ 3.4028234663852886e+38F
#define __USACCUM_FBIT__ 8
#define __INT32_TYPE__ int
#define __SIZEOF_DOUBLE__ 8
#define __FLT_MIN_10_EXP__ (-37)
#define __UFRACT_EPSILON__ 0x1P-16UR
#define __INTMAX_TYPE__ long long int
#define __DEC128_MAX_EXP__ 6145
#define __ATOMIC_CONSUME 1
#define __GNUC_MINOR__ 8
#define __UINTMAX_MAX__ 18446744073709551615ULL
#define __DEC32_MANT_DIG__ 7
#define __HA_FBIT__ 7
#define __DBL_MAX_10_EXP__ 308
#define __LDBL_DENORM_MIN__ 4.9406564584124654e-324L
#define __INT16_C(c) c
#define __STDC__ 1
#define __PTRDIFF_TYPE__ int
#define __LLFRACT_MIN__ (-0.5LLR-0.5LLR)
#define __ATOMIC_SEQ_CST 5
#define __DA_FBIT__ 31
#define __UINT32_TYPE__ unsigned int
#define __UINTPTR_TYPE__ unsigned int
#define __USA_IBIT__ 16
#define __DEC64_SUBNORMAL_MIN__ 0.000000000000001E-383DD
#define __ARM_EABI__ 1
#define __DEC128_MANT_DIG__ 34
#define __LDBL_MIN_10_EXP__ (-307)
#define __SIZEOF_LONG_LONG__ 8
#define __ULACCUM_EPSILON__ 0x1P-32ULK
#define __SACCUM_IBIT__ 8
#define __GCC_ATOMIC_LLONG_LOCK_FREE 1
#define __LDBL_DIG__ 15
#define __FLT_DECIMAL_DIG__ 9
#define __UINT_FAST16_MAX__ 4294967295U
#define __GNUC_GNU_INLINE__ 1
#define __GCC_ATOMIC_SHORT_LOCK_FREE 1
#define __ULLFRACT_MAX__ 0XFFFFFFFFFFFFFFFFP-64ULLR
#define __UINT_FAST8_TYPE__ unsigned char
#define __USFRACT_EPSILON__ 0x1P-8UHR
#define __ULACCUM_FBIT__ 32
#define __ARM_FEATURE_DSP 1
#define __QQ_IBIT__ 0
#define __ATOMIC_ACQ_REL 4
#define __ATOMIC_RELEASE 3
Well did you enable macro expansion in the opTions?
If there no some hidden options so - yes, all is enabled.
Compiler's XML based on ARM Compiler.
<?xml version="1.0"?>
<!DOCTYPE CodeBlocks_compiler>
<CodeBlocks_compiler name="GNU GCC Compiler for Android"
id="android-gcc"
weight="60">
<Path type="master">
<Search envVar="PATH"
for="C"/>
<if platform="windows">
<Search path="C:\ndk*"/>
<Fallback path="C:\ndk"/>
</if>
<else>
<Fallback path="$NDK"/>
</else>
</Path>
<if platform="windows">
<Path type="include">
<Add><master/>/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/include</Add>
</Path>
<Path type="lib">
<Add><master/>/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/lib</Add>
</Path>
<Path type="extra">
<Add><master/>/tools</Add>
</Path>
</if>
<else>
<Path type="include">
<Add><master/>/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/include</Add>
</Path>
<Path type="lib">
<Add><master/>/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/lib</Add>
</Path>
</else>
</CodeBlocks_compiler>
<?xml version="1.0"?>
<!DOCTYPE CodeBlocks_compiler_options>
<CodeBlocks_compiler_options>
<if platform="windows">
<Program name="C" value="arm-linux-androideabi-gcc.exe"/>
<Program name="CPP" value="arm-linux-androideabi-g++.exe"/>
<Program name="LD" value="arm-linux-androideabi-g++.exe"/>
<Program name="DBGconfig" value=""/>
<Program name="LIB" value="arm-linux-androideabi-ar.exe"/>
<Program name="WINDRES" value=""/>
<Program name="MAKE" value="make.exe"/>
</if>
<else>
<Program name="C" value="arm-linux-androideabi-gcc"/>
<Program name="CPP" value="arm-linux-androideabi-g++"/>
<Program name="LD" value="arm-linux-androideabi-g++"/>
<Program name="DBGconfig" value=""/>
<Program name="LIB" value="arm-linux-androideabi-ar"/>
<Program name="WINDRES" value=""/>
<Program name="MAKE" value="ndk-build"/>
</else>
<Switch name="includeDirs" value="-I"/>
<Switch name="libDirs" value="-L"/>
<Switch name="linkLibs" value="-l"/>
<Switch name="defines" value="-D"/>
<Switch name="genericSwitch" value="-"/>
<Switch name="objectExtension" value="o"/>
<Switch name="needDependencies" value="true"/>
<Switch name="forceCompilerUseQuotes" value="false"/>
<Switch name="forceLinkerUseQuotes" value="false"/>
<Switch name="logging" value="default"/>
<Switch name="libPrefix" value="lib"/>
<Switch name="libExtension" value="a"/>
<Switch name="linkerNeedsLibPrefix" value="false"/>
<Switch name="linkerNeedsLibExtension" value="false"/>
<Option name="Produce debugging symbols"
option="-g"
category="Debugging"
checkAgainst="-O -O1 -O2 -O3 -Os"
checkMessage="You have optimizations enabled. This is Not A Good Thing(tm) when producing debugging symbols..."
supersedes="-s"/>
<if platform="windows">
<Option name="Profile code when executed"
option="-pg"
category="Profiling"
additionalLibs="-pg -lgmon"
supersedes="-s"/>
</if>
<else>
<Option name="Profile code when executed"
option="-pg"
category="Profiling"
additionalLibs="-pg"
supersedes="-s"/>
</else>
<Common name="warnings"/>
<Category name="Warnings">
<Option name="Enable Effective-C++ warnings (thanks Scott Meyers)"
option="-Weffc++"/>
<Option name="Warn whenever a switch statement does not have a default case"
option="-Wswitch-default"/>
<Option name="Warn whenever a switch statement has an index of enumerated type and lacks a case for one or more of the named codes of that enumeration"
option="-Wswitch-enum"/>
<Option name="Warn if a user supplied include directory does not exist"
option="-Wmissing-include-dirs"/>
<Option name="Warn if a global function is defined without a previous declaration"
option="-Wmissing-declarations"/>
<Option name="Warn if the compiler detects that code will never be executed"
option="-Wunreachable-code"/>
<Option name="Warn if a function can not be inlined and it was declared as inline"
option="-Winline"/>
<Option name="Warn if floating point values are used in equality comparisons"
option="-Wfloat-equal"/>
<Option name="Warn if an undefined identifier is evaluated in an '#if' directive"
option="-Wundef"/>
<Option name="Warn whenever a pointer is cast such that the required alignment of the target is increased"
option="-Wcast-align"/>
<Option name="Warn if anything is declared more than once in the same scope"
option="-Wredundant-decls"/>
<Option name="Warn about unitialized variables which are initialized with themselves"
option="-Winit-self"/>
<Option name="Warn whenever a local variable shadows another local variable, parameter or global variable or whenever a built-in function is shadowed"
option="-Wshadow"/>
<Option name="Warn if a class has virtual functions but no virtual destructor"
option="-Wnon-virtual-dtor"/>
<Option name="Check calls to printf and scanf, etc., to make sure that the arguments supplied have types appropriate to the format string specified, and that the conversions specified in the format string make sense, included in -Wall."
option="-Wformat"/>
<Option name="If -Wformat is specified, also warn about uses of format functions that represent possible security problems."
option="-Werror=format-security"/>
</Category>
<Common name="optimization"/>
<Category name="Optimization">
<Option name="-fno-omit-frame-pointer"
option="-fno-omit-frame-pointer"
supersedes="-fomit-frame-pointer"/>
<Option name="Don't keep the frame pointer in a register for functions that don't need one"
option="-fomit-frame-pointer"
checkAgainst="-g -ggdb"
checkMessage="You have debugging symbols enabled. This is Not A Good Thing(tm) when optimizing..."/>
</Category>
<Category name="ARM CPU architecture specific">
<Option name="Generate stack frame even if unnecessary"
option="-mapcs-frame"
supersedes="-mno-apcs-frame"/>
<Option name="-mno-apcs-frame"
option="-mno-apcs-frame"
supersedes="-mapcs-frame"/>
<Option name="-mabi=apcs-gnu"
option="-mabi=apcs-gnu"
supersedes="-mabi=atpcs -mabi=aapcs -mabi=aapcs-linux -mabi=iwmmxt"/>
<Option name="-mabi=atpcs"
option="-mabi=atpcs"
supersedes="-mabi=apcs-gnu -mabi=aapcs -mabi=aapcs-linux -mabi=iwmmxt"/>
<Option name="-mabi=aapcs"
option="-mabi=aapcs"
supersedes="-mabi=apcs-gnu -mabi=atpcs -mabi=aapcs-linux -mabi=iwmmxt"/>
<Option name="-mabi=aapcs-linux"
option="-mabi=aapcs-linux"
supersedes="-mabi=apcs-gnu -mabi=atpcs -mabi=aapcs -mabi=iwmmxt"/>
<Option name="-mabi=iwmmxt"
option="-mabi=iwmmxt"
supersedes="-mabi=apcs-gnu -mabi=atpcs -mabi=aapcs -mabi=aapcs-linux"/>
<Option name="-mapcs-stack-check"
option="-mapcs-stack-check"
supersedes="-mno-apcs-stack-check"/>
<Option name="-mno-apcs-stack-check"
option="-mno-apcs-stack-check"
supersedes="-mapcs-stack-check"/>
<Option name="-mapcs-float"
option="-mapcs-float"
supersedes="-mno-apcs-float"/>
<Option name="-mno-apcs-float"
option="-mno-apcs-float"
supersedes="-mapcs-float"/>
<Option name="-mapcs-reentrant"
option="-mapcs-reentrant"
supersedes="-mno-apcs-reentrant"/>
<Option name="-mno-apcs-reentrant"
option="-mno-apcs-reentrant"
supersedes="-mapcs-reentrant"/>
<Option name="-msched-prolog"
option="-msched-prolog"
supersedes="-mno-sched-prolog"/>
<Option name="Prevent the reordering of instructions in the function prolog"
option="-mno-sched-prolog"
supersedes="-msched-prolog"/>
<Option name="Generate code for little-endian processors"
option="-mlittle-endian"
supersedes="-mbig-endian"/>
<Option name="Generate code for big-endian processors"
option="-mbig-endian"
supersedes="-mlittle-endian"/>
<Option name="-mwords-little-endian"
option="-mwords-little-endian"/>
<Option name="-msoft-float"
option="-msoft-float"
supersedes="-mhard-float"/>
<Option name="-mhard-float"
option="-mhard-float"
supersedes="-msoft-float"/>
<Option name="-mfloat-abi=softfp"
option="-mfloat-abi=softfp"
supersedes="-msoft-float -mhard-float"/>
<Option name="-mfpe"
option="-mfpe"/>
<Option name="Generate code supporting calls between the ARM and Thumb instruction sets"
option="-mthumb-interwork"
supersedes="-mno-thumb-interwork -marm -mthumb"/>
<Option name="-mno-thumb-interwork"
option="-mno-thumb-interwork"
supersedes="-mthumb-interwork"/>
<Option name="'fpa' floating point unit"
option="-mfpu=fpa"
supersedes="-mfpu=fpe2 -mfpu=fpe3 -mfpu=maverick -mfpu=vfp -mfpu=vfpv3 -mfpu=vfpv3-d16 -mfpu=neon"/>
<Option name="'fpe2' floating point unit"
option="-mfpu=fpe2"
supersedes="-mfpu=fpa -mfpu=fpe3 -mfpu=maverick -mfpu=vfp -mfpu=vfpv3 -mfpu=vfpv3-d16 -mfpu=neon"/>
<Option name="'fpe3' floating point unit"
option="-mfpu=fpe3"
supersedes="-mfpu=fpa -mfpu=fpe2 -mfpu=maverick -mfpu=vfp -mfpu=vfpv3 -mfpu=vfpv3-d16 -mfpu=neon"/>
<Option name="'maverick' floating point unit"
option="-mfpu=maverick"
supersedes="-mfpu=fpa -mfpu=fpe2 -mfpu=fpe3 -mfpu=vfp -mfpu=vfpv3 -mfpu=vfpv3-d16 -mfpu=neon"/>
<Option name="'vfp' floating point unit"
option="-mfpu=vfp"
supersedes="-mfpu=fpa -mfpu=fpe2 -mfpu=fpe3 -mfpu=maverick -mfpu=vfpv3 -mfpu=vfpv3-d16 -mfpu=neon"/>
<Option name="'vfpv3' floating point unit"
option="-mfpu=vfpv3"
supersedes="-mfpu=fpa -mfpu=fpe2 -mfpu=fpe3 -mfpu=maverick -mfpu=vfp -mfpu=vfpv3-d16 -mfpu=neon"/>
<Option name="'vfpv3-d16' floating point unit"
option="-mfpu=vfpv3-d16"
supersedes="-mfpu=fpa -mfpu=fpe2 -mfpu=fpe3 -mfpu=maverick -mfpu=vfp -mfpu=vfpv3 -mfpu=neon"/>
<Option name="'neon' floating point unit"
option="-mfpu=neon"
supersedes="-mfpu=fpa -mfpu=fpe2 -mfpu=fpe3 -mfpu=maverick -mfpu=vfp -mfpu=vfpv3 -mfpu=vfpv3-d16"/>
<Option name="Round structure and union size up to a multiple of 8 bits"
option="-mstructure-size-boundary=8"
supersedes="-mstructure-size-boundary=32 -mstructure-size-boundary=64"/>
<Option name="Round structure and union size up to a multiple of 32 bits"
option="-mstructure-size-boundary=32"
supersedes="-mstructure-size-boundary=8 -mstructure-size-boundary=64"/>
<Option name="Round structure and union size up to a multiple of 64 bits"
option="-mstructure-size-boundary=64"
supersedes="-mstructure-size-boundary=8 -mstructure-size-boundary=32"/>
<Option name="Generate a call to the function abort at the end of a noreturn function"
option="-mabort-on-noreturn"/>
<Option name="-mlong-calls"
option="-mlong-calls"
supersedes="-mno-long-calls"/>
<Option name="-mno-long-calls"
option="-mno-long-calls"
supersedes="-mlong-calls"/>
<Option name="-mnop-fun-dllimport"
option="-mnop-fun-dllimport"/>
<Option name="-mpoke-function-name"
option="-mpoke-function-name"/>
<Option name="Generate code that executes in Thumb state"
option="-mthumb"
supersedes="-marm -mthumb-interwork"/>
<Option name="Generate code that executes in ARM state"
option="-marm"
supersedes="-mthumb -mthumb-interwork"/>
</Category>
<Category name="Other">
<Option name="-fpic"
option="-fpic"/>
<Option name="-ffunction-sections"
option="-ffunction-sections"/>
<Option name="-funwind-tables"
option="-funwind-tables"/>
<Option name="-fstack-protector"
option="-fstack-protector"/>
<Option name="-no-canonical-prefixes"
option="-no-canonical-prefixes"/>
<Option name="-fno-exceptions"
option="-fno-exceptions"/>
<Option name="-fno-rtti"
option="-fno-rtti"/>
<Option name="No exec stack"
option="-Wa,--noexecstack"/>
<Option name="Like -MD except mention only user header files, not system header files."
option="-MMD"/>
<Option name="This option instructs CPP to add a phony target for each dependency other than the main file, causing each to depend on nothing."
option="-MP"/>
<Option name="-fno-strict-aliasing"
option="-fno-strict-aliasing"/>
</Category>
<Category name="ARM CPU architecture derivatives"
exclusive="true">
<Option name="Armeabi-v7a"
option="-march=armv7-a"/>
</Category>
<Command name="CompileObject"
value="$compiler $options $includes -c $file -o $object"/>
<Command name="GenDependencies"
value="$compiler -MM $options -MF $dep_object -MT $object $includes $file"/>
<Command name="CompileResource"
value="$rescomp -i $file -J rc -o $resource_output -O coff $res_includes"/>
<Command name="LinkConsoleExe"
value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs"/>
<if platform="windows">
<Command name="LinkExe"
value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs -mwindows"/>
<Command name="LinkDynamic"
value="$linker -shared -Wl,--output-def=$def_output -Wl,--out-implib=$static_output -Wl,--dll $libdirs $link_objects $link_resobjects -o $exe_output $link_options $libs"/>
</if>
<else>
<Command name="LinkExe"
value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs"/>
<Command name="LinkDynamic"
value="$linker -shared $libdirs $link_objects $link_resobjects -o $exe_output $link_options $libs"/>
</else>
<Command name="LinkNative"
value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs"/>
<Command name="LinkStatic"
value="$lib_linker -r -s $static_output $link_objects"/>
<Common name="cmds"/>
<Common name="re"/>
<Common name="sort"/>
</CodeBlocks_compiler_options>
-PASS: ClassA:: a
-PASS: ClassA:: s
-PASS: ClassA:: u
-PASS: ClassA:: v
-PASS: ClassA:: STRUCT_A
-PASS: ClassA:: STRUCT_B
-PASS: ClassA:: UNNAMED_A
-PASS: ClassA:: UNNAMED_B
-PASS: ClassA:: ENUM_A
-PASS: ClassA:: ENUM_B
--------------------------------------------------------
Total 10 tests, 10 PASS, 0 FAIL
--------------------------------------------------------
Here is the test file:
class ClassA
{
public:
int a;
struct A
{
int c;
} z;
// unnamed struct
struct
{
int s;
union
{
int u;
short v;
};
enum EnumS
{
STRUCT_A,
STRUCT_B,
};
};
enum
{
UNNAMED_A,
UNNAMED_B
};
enum EnumA
{
ENUM_A,
ENUM_B
};
};
// Hovering on following member variables should
// show tooltip: a, s, u, v, all the enums like STRUCT_A, etc.
// not c
//ClassA:: //a,s,u,v,STRUCT_A,STRUCT_B,UNNAMED_A,UNNAMED_B,ENUM_A,ENUM_B
I still don't understand logic of inactive code's highlighting or is it bug?
Little test for example:
#ifndef _MAIN_H_
#define _MAIN_H_
#define TEST 0
#endif
#include "main.h"
//#define TEST 0
int main()
{
#if defined(TEST)
int = 0;
#else
int = 1;
#end
#ifdef TEST
int n = 0;
#else
int n = 1;
#endif
#if TEST == 1
int n = 2;
#endif
#if TEST == 0
int n = 3;
#endif
}
If TEST defined in header - it is ignored always, it just does not exist for highlighting. CC can't find declaration until opening Symbols browser - then it can. If TEST defined in the same file it has strange behaviour with defined: TEST, TEST 1, TEST 2. For example, "#if defined" works well, but next "#if TEST == 1" - ignored.
I'm thinking that the code:
bool Tokenizer::ReplaceBufferText(const wxString& target, bool updatePeekToken)
{
...
...
// Fix token index
m_TokenIndex -= bufferLen;
// Reset undo token
m_SavedTokenIndex = m_UndoTokenIndex = m_TokenIndex;
m_SavedLineNumber = m_UndoLineNumber = m_LineNumber;
m_SavedNestingLevel = m_UndoNestLevel = m_NestLevel;
// Update the peek token
if (m_PeekAvailable && updatePeekToken)
{
m_PeekAvailable = false;
// we set the m_PeekAvailable after calling DoGetToken() function inside the PeekToken()
// to prevent unnecessary recursive call of PeekToken() function.
PeekToken();
}
return true;
}
Should be changed to
bool Tokenizer::ReplaceBufferText(const wxString& target)
{
...
...
// Fix token index
m_TokenIndex -= bufferLen;
// Reset undo token
m_SavedTokenIndex = m_UndoTokenIndex = m_TokenIndex;
m_SavedLineNumber = m_UndoLineNumber = m_LineNumber;
m_SavedNestingLevel = m_UndoNestLevel = m_NestLevel;
// Update the peek token
if (m_PeekAvailable)
m_PeekAvailable = false;
return true;
}
Reason:
1, After some debugging, I see that once the buffer has changed, peek should be invalid, the following "PeekToken() or GetToken()" function call will update the peek token, but not inside the ReplaceBufferText() function.
2, I even think the last parameter of "bool ReplaceFunctionLikeMacro(const Token* tk, bool updatePeekToken = true);" should be removed.
Those kinds of functions just replace some piece of text in the buffer, no need to update peek token, the caller can explicitly call PeekToken or GetToken functions
EDIT1: when using git blame, I found that the second parameter of this function was added in
* * cc_branch: applied patch with rewritten function-like macro handle (v5)
git-svn-id: http://svn.code.sf.net/p/codeblocks/code/branches/codecompletion_refactoring@6436
EDIT2, the second parameter of ReplaceFunctionLikeMacro() as added in
* * cc_branch: improved function-like macro parsing, a little improved for avoid the endless loop
git-svn-id: http://svn.code.sf.net/p/codeblocks/code/branches/codecompletion_refactoring@6691
EDIT3
It looks like all the function call with second parameter value == true is used for handling macro expansion. (HandleMacro)
Here is my analysis:
Rev6436
The first place of ReplaceBufferForReparse() with second parameter value == true:
+wxString Tokenizer::GetActualContextForMacro(Token* tk)
+{
+ if (!tk)
+ return wxEmptyString;
+
+ // 1. break the args into substring with "," and store them in normals
+ wxArrayString normalArgs;
+ if (!tk->m_Args.IsEmpty())
+ {
+ ReplaceBufferForReparse(tk->m_Args);
+ SpliteArguments(normalArgs);
+ }
Here, not sure why this need to run peek again?
Since ReplaceBufferForReparse() just put a new piece of text in the buffer(a macro definition's formal parameter), and move back the m_TokenIndex a bit, what is the reason to internally call PeekToken() function?
The second place of ReplaceBufferForReparse() with second parameter value == true:
+ TRACE(_T("HandleMacro() : Adding token '%s' (peek='%s')"), tk->m_Name.wx_str(), peek.wx_str());
+ DoAddToken(tkMacro, tk->m_Name, m_Tokenizer.GetLineNumber(), 0, 0, peek);
+ m_Tokenizer.ReplaceBufferForReparse(m_Tokenizer.GetActualContextForMacro(tk));
Which is inside the ParserThread::HandleMacro() function (handling macro usage) body, but I still don't see a forced PeekToken() call is necessary.
---------------
Rev6691
First place of the ReplaceMacroActualContext() function call with second parameter value == true
void ParserThread::HandleMacro(int id, const wxString &peek)
{
Token* tk = m_TokensTree->at(id);
if (tk)
{
TRACE(_T("HandleMacro() : Adding token '%s' (peek='%s')"), tk->m_Name.wx_str(), peek.wx_str());
DoAddToken(tkMacro, tk->m_Name, m_Tokenizer.GetLineNumber(), 0, 0, peek);
if (m_Options.parseComplexMacros)
m_Tokenizer.ReplaceMacroActualContext(tk);
}
}
When try to fix the issue reported here: Re: The 02 March 2014 build (9673.) is out. (http://forums.codeblocks.org/index.php/topic,19016.msg130216.html#msg130216)
I found that all the identifier like token should be checked for macro replacement, inside the DoGetToken(), other wise
void ParserThread::SkipBlock()
{
// need to force the tokenizer _not_ skip anything
// or else default values for template params would cause us to miss everything (because of the '=' symbol)
TokenizerState oldState = m_Tokenizer.GetState();
m_Tokenizer.SetState(tsSkipNone);
// skip tokens until we reach }
// block nesting is taken into consideration too ;)
// this is the nesting level we start at
// we subtract 1 because we 're already inside the block
// (since we 've read the {)
unsigned int level = m_Tokenizer.GetNestingLevel() - 1;
while (IS_ALIVE)
{
wxString token = m_Tokenizer.GetToken();
if (token.IsEmpty())
break; // eof
// if we reach the initial nesting level, we are done
if (level == m_Tokenizer.GetNestingLevel())
break;
}
// reset tokenizer's functionality
m_Tokenizer.SetState(oldState);
}