Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: daniloz on November 15, 2011, 09:44:36 am

Title: Pointer/Array not recognized
Post by: daniloz on November 15, 2011, 09:44:36 am
Hi All,

I don't know exactly what's the technical name of what I'm doing, but look at the following code:
Code
float *pFloat;
float **pFloat2;
float (*pFloatPar)[3];

int main()
{
pFloat;
pFloat2;
pFloat
}
I've declared pFloatPar as a double array in a way that the compiler knows about the second dimension size and then I can write pFloatPar[2][1].

However, the actual code completion doesn't recognize the declaration (see the attached screenshot). My assumption is that CC cannot correctly parse the parenthesis...
Title: Re: Pointer/Array not recognized
Post by: ollydbg on November 15, 2011, 10:17:44 am
I have put your code in our parsertest project, and the result looks below:
Code
--------------M-a-i-n--L-o-g--------------

000001. ParserDummy() : ParserBase() : Instantiation of Parser object.
000002. InitTokenizer() : m_Filename='test.h', m_FileSize=108.
000003. Init() : m_Filename='test.h'
000004. test.h
000005. Parse() : Parsing 'test.h'
000006. DoParse() : Loop:m_Str='', token='float'
000007. DoParse() : Loop:m_Str='float ', token='*'
000008. DoParse() : Loop:m_Str='float ', token='pFloat'
000009. DoAddToken() : Created token='pFloat', file_idx=1, line=1, ticket=
000010. GetActualTokenType() : Searching within m_Str='float'
000011. GetActualTokenType() : Compensated m_Str='float'
000012. GetActualTokenType() : Found 'float'
000013. DoAddToken() : Prepending ''
000014. DoAddToken() : Added/updated token 'pFloat' (0), kind 'variable', type 'float*', actual 'float'. Parent is  (-1)
000015. DoParse() : Loop:m_Str='float', token=';'
000016. DoParse() : Loop:m_Str='', token='float'
000017. DoParse() : Loop:m_Str='float ', token='*'
000018. DoParse() : Loop:m_Str='float ', token='*'
000019. DoParse() : Loop:m_Str='float ', token='pFloat2'
000020. DoAddToken() : Created token='pFloat2', file_idx=1, line=2, ticket=
000021. GetActualTokenType() : Searching within m_Str='float'
000022. GetActualTokenType() : Compensated m_Str='float'
000023. GetActualTokenType() : Found 'float'
000024. DoAddToken() : Prepending ''
000025. DoAddToken() : Added/updated token 'pFloat2' (1), kind 'variable', type 'float**', actual 'float'. Parent is  (-1)
000026. DoParse() : Loop:m_Str='float', token=';'
000027. DoParse() : Loop:m_Str='', token='float'
000028. ReadParentheses(): (* pFloatPar), line=3
000029. DoParse() : Loop:m_Str='', token=';'
000030. DoParse() : Loop:m_Str='', token='int'
000031. DoParse() : Loop:m_Str='int ', token='main'
000032. ReadParentheses(): (), line=5
000033. HandleFunction() : Adding function 'main': m_Str='int '
000034. HandleFunction() : name='main', args='()', peek='{'
000035. HandleFunction() : !(Ctor/Dtor) 'main', m_Str='int ', localParent='<none>'
000036. HandleFunction() : Adding function 'main', ': m_Str='int ', enc_ns='nil'.
000037. HandleFunction() : Add token name='main', args='()', return type='int '
000038. GetBaseArgs() : args='()'.
000039. GetBaseArgs() : baseArgs='()'.
000040. DoAddToken() : Created token='main', file_idx=1, line=5, ticket=
000041. GetActualTokenType() : Searching within m_Str='int'
000042. GetActualTokenType() : Compensated m_Str='int'
000043. GetActualTokenType() : Found 'int'
000044. DoAddToken() : Prepending ''
000045. DoAddToken() : Added/updated token 'main' (2), kind 'function', type 'int', actual 'int'. Parent is  (-1)
000046. ParserDummy() : ~ParserBase() : Destruction of Parser object.


--------------T-r-e-e--L-o-g--------------

000047. float* pFloat [1,0]
000048. float** pFloat2 [2,0]
000049. main() [5,5]


--------------L-i-s-t--L-o-g--------------

000050. variable float* pFloat [1,0]
000051. variable float** pFloat2 [2,0]
000052. function int main() [5,5]


Yes. The parser just skip the "[3]".

The proposed fix is try to peek the token after the right parenthesis, and if it is a "[", then we can guess that is is a second level pointer.

Title: Re: Pointer/Array not recognized
Post by: daniloz on November 15, 2011, 02:20:40 pm
Yes. The parser just skip the "[3]".
Even if the parser does so, the result should be float *pFloatPar; and the parser should treat it as a pointer to a float. This would, of course, be wrong, but it seems the parser is just dropping this symbol completely.

The proposed fix is try to peek the token after the right parenthesis, and if it is a "[", then we can guess that is is a second level pointer.
That sounds good to me, however I'm not that familiar with the CC plugin to try a patch, sorry... ;-)
Title: Re: Pointer/Array not recognized
Post by: ollydbg on November 15, 2011, 02:47:40 pm
The proposed fix is try to peek the token after the right parenthesis, and if it is a "[", then we can guess that is is a second level pointer.
That sounds good to me, however I'm not that familiar with the CC plugin to try a patch, sorry... ;-)
I think I will take some time to fix it.  :D
Title: Re: Pointer/Array not recognized
Post by: daniloz on November 15, 2011, 02:52:37 pm
I think I will take some time to fix it.  :D
That'd be great... Thanks!  8)
Title: Re: Pointer/Array not recognized
Post by: ollydbg on November 16, 2011, 01:49:33 am
This is a dirty hack on this
Code
Index: E:/code/cb/cb_trunk/src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- E:/code/cb/cb_trunk/src/plugins/codecompletion/parser/parserthread.cpp (revision 7561)
+++ E:/code/cb/cb_trunk/src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -506,11 +506,11 @@
     m_LastToken.Clear();
     m_LastUnnamedTokenName.Clear();
 
-    // Notice: clears the queue "m_EncounteredTypeNamespaces"
+    // Notice: clear the queue "m_EncounteredTypeNamespaces"
     while (!m_EncounteredTypeNamespaces.empty())
         m_EncounteredTypeNamespaces.pop();
 
-    // Notice: clears the queue "m_EncounteredNamespaces"
+    // Notice: clear the queue "m_EncounteredNamespaces"
     while (!m_EncounteredNamespaces.empty())
         m_EncounteredNamespaces.pop();
 
@@ -942,7 +942,8 @@
                         int pos = peek.find(ParserConsts::ptr);
                         if (pos != wxNOT_FOUND)
                         {
-                            if (m_Tokenizer.PeekToken().GetChar(0) == ParserConsts::opbracket_chr)
+                            peek = m_Tokenizer.PeekToken();
+                            if (peek.GetChar(0) == ParserConsts::opbracket_chr)
                             {
                                 arg.Trim(true).RemoveLast();
                                 //wxString token = arg.Mid(pos+1,)
@@ -953,6 +954,18 @@
                                     HandleFunction(arg);
                                 }
                             }
+                            else if (peek.GetChar(0) == ParserConsts::semicolon_chr)
+                            {
+                                // we meet such mode:
+                                // AAA (*BBB)[ccc];
+                                // The "[ccc]" was skipped by Tokenizer, so we meet the semicolon
+                                m_Str << token<<_T("*");
+                                //strip the "*" from arg.
+                                arg.Trim(true).RemoveLast();
+                                arg.Remove(0, pos+1); //remove the heading "*"
+                                if (m_Options.handleVars)
+                                    DoAddToken(tkVariable, arg, m_Tokenizer.GetLineNumber());
+                            }
                         }
                         else // wxString arg = m_Tokenizer.GetToken(); // eat args ()
                             m_Str = token + arg;
@@ -2646,6 +2659,7 @@
         else if (tmp==ParserConsts::gt)
         {
             --nestLvl;
+            m_TemplateArgument.Trim();
             m_TemplateArgument << tmp;
         }
         else if (tmp==ParserConsts::semicolon)
@@ -2658,7 +2672,7 @@
         else if (tmp.IsEmpty())
             break;
         else
-            m_TemplateArgument << tmp;
+            m_TemplateArgument << tmp <<_T(" ");
         if (nestLvl <= 0)
             break;
     }


Well, the log is below:
Quote
--------------M-a-i-n--L-o-g--------------

000001. ParserDummy() : ParserBase() : Instantiation of Parser object.
000002. InitTokenizer() : m_Filename='test.h', m_FileSize=108.
000003. Init() : m_Filename='test.h'
000004. test.h
000005. Parse() : Parsing 'test.h'
000006. DoParse() : Loop:m_Str='', token='float'
000007. DoParse() : Loop:m_Str='float ', token='*'
000008. DoParse() : Loop:m_Str='float ', token='pFloat'
000009. DoAddToken() : Created token='pFloat', file_idx=1, line=1, ticket=
000010. GetActualTokenType() : Searching within m_Str='float'
000011. GetActualTokenType() : Compensated m_Str='float'
000012. GetActualTokenType() : Found 'float'
000013. DoAddToken() : Prepending ''
000014. DoAddToken() : Added/updated token 'pFloat' (0), kind 'variable', type 'float*', actual 'float'. Parent is  (-1)
000015. DoParse() : Loop:m_Str='float', token=';'
000016. DoParse() : Loop:m_Str='', token='float'
000017. DoParse() : Loop:m_Str='float ', token='*'
000018. DoParse() : Loop:m_Str='float ', token='*'
000019. DoParse() : Loop:m_Str='float ', token='pFloat2'
000020. DoAddToken() : Created token='pFloat2', file_idx=1, line=2, ticket=
000021. GetActualTokenType() : Searching within m_Str='float'
000022. GetActualTokenType() : Compensated m_Str='float'
000023. GetActualTokenType() : Found 'float'
000024. DoAddToken() : Prepending ''
000025. DoAddToken() : Added/updated token 'pFloat2' (1), kind 'variable', type 'float**', actual 'float'. Parent is  (-1)
000026. DoParse() : Loop:m_Str='float', token=';'
000027. DoParse() : Loop:m_Str='', token='float'
000028. ReadParentheses(): (* pFloatPar), line=3
000029. DoAddToken() : Created token=' pFloatPar', file_idx=1, line=3, ticket=
000030. GetActualTokenType() : Searching within m_Str='float*'
000031. GetActualTokenType() : Compensated m_Str='float*'
000032. GetActualTokenType() : Found 'float'
000033. DoAddToken() : Prepending ''
000034. DoAddToken() : Added/updated token ' pFloatPar' (2), kind 'variable', type 'float*', actual 'float'. Parent is  (-1)
000035. DoParse() : Loop:m_Str='float*', token=';'
000036. DoParse() : Loop:m_Str='', token='int'
000037. DoParse() : Loop:m_Str='int ', token='main'
000038. ReadParentheses(): (), line=5
000039. HandleFunction() : Adding function 'main': m_Str='int '
000040. HandleFunction() : name='main', args='()', peek='{'
000041. HandleFunction() : !(Ctor/Dtor) 'main', m_Str='int ', localParent='<none>'
000042. HandleFunction() : Adding function 'main', ': m_Str='int ', enc_ns='nil'.
000043. HandleFunction() : Add token name='main', args='()', return type='int '
000044. GetBaseArgs() : args='()'.
000045. GetBaseArgs() : baseArgs='()'.
000046. DoAddToken() : Created token='main', file_idx=1, line=5, ticket=
000047. GetActualTokenType() : Searching within m_Str='int'
000048. GetActualTokenType() : Compensated m_Str='int'
000049. GetActualTokenType() : Found 'int'
000050. DoAddToken() : Prepending ''
000051. DoAddToken() : Added/updated token 'main' (3), kind 'function', type 'int', actual 'int'. Parent is  (-1)
000052. ParserDummy() : ~ParserBase() : Destruction of Parser object.


--------------T-r-e-e--L-o-g--------------

000053. float* pFloat   [1,0]
000054. float** pFloat2   [2,0]
000055. float*  pFloatPar   [3,0]
000056. main()   [5,5]


--------------L-i-s-t--L-o-g--------------

000057. variable float* pFloat   [1,0]
000058. variable float** pFloat2   [2,0]
000059. variable float*  pFloatPar   [3,0]
000060. function int main()   [5,5]

Title: Re: Pointer/Array not recognized
Post by: MortenMacFly on November 18, 2011, 08:26:54 am
...I got another one (a vector) not working properly for me:
Code
#include <vector>

class AClass
{
public:
   AClass() {;}
  ~AClass() {;}

  void MyMethod() {;}
};

class AnotherClass
{
public:
   AnotherClass() {;}
  ~AnotherClass() {;}

  inline const std::vector<AClass*> GetAClassVec()
  {
    return mvpAClass;
  }

private:
   std::vector<AClass*> mvpAClass;
};

int main()
{
  AnotherClass a;
  a. // does not show GetAClassVec
}
Title: Re: Pointer/Array not recognized
Post by: ollydbg on November 18, 2011, 03:16:42 pm
Look, it seems the function does not parse correctly. See the screen shot below
(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2011-11-18221725.png)
Title: Re: Pointer/Array not recognized
Post by: ollydbg on November 18, 2011, 03:39:46 pm
OK, we arrived in this if clause.

Code
                else if (peek==ParserConsts::dcolon)
                {
                    if (m_Str.IsEmpty())
                        m_EncounteredTypeNamespaces.push(token); // it's a type's namespace
                    else
                        m_EncounteredNamespaces.push(token);
                    m_Tokenizer.GetToken(); // eat ::
                }
Here, I just parse the third line:
Code
class AnotherClass
{
  inline const std::vector<AClass*> GetAClassVec() {;}
};

Now:
token = "std"
peek = "::"
m_Str = "const "

The logic is:
We put the "std" as the m_EncounteredNamespaces,
but the correct way is: “std" should be put in m_EncounteredTypeNamespaces.

One possible way is: we should skip the ”const“ (do not put "const" into m_Str, as m_Str is like a returned type stack), so that when we peek a "::", we see the m_Str is empty, and ”std" will put into m_EncounteredTypeNamespaces.

OK, can you understand my explanation???
Title: Re: Pointer/Array not recognized
Post by: MortenMacFly on November 18, 2011, 09:23:30 pm
We put the "std" as the m_EncounteredNamespaces,
but the correct way is: “std" should be put in m_EncounteredTypeNamespaces.

OK, can you understand my explanation???
Yes, but I think removing const wouldn't be wise because you wanna know, don't you.

What about doing something like (untested, blind meta-code):
Code
else if (peek==ParserConsts::dcolon)
{
    if (   m_Str.IsEmpty()
        || m_Str.Trim(true).Trim(false).IsSameAs(ParserConsts::const)
        || m_Str.Trim(true).Trim(false).IsSameAs(ParserConsts::volatile) ) // what else?!
        m_EncounteredTypeNamespaces.push(token); // it's a type's namespace
    else
        m_EncounteredNamespaces.push(token);
    m_Tokenizer.GetToken(); // eat ::
}
Title: Re: Pointer/Array not recognized
Post by: MortenMacFly on November 18, 2011, 09:28:58 pm
What about doing something like (untested, blind meta-code):
(...answering myself...:)
Raises immediately the question, what about such things then:
Code
inline const mynamespace::anothernamespace::std::vector<AClass*> GetAClassVec() {;}
???
Title: Re: Pointer/Array not recognized
Post by: ollydbg on November 19, 2011, 05:03:33 am
What about doing something like (untested, blind meta-code):
(...answering myself...:)
Raises immediately the question, what about such things then:
Code
inline const mynamespace::anothernamespace::std::vector<AClass*> GetAClassVec() {;}
It works OK, because the next time we enter the if clause, the have the same m_Str == const.
Code
                else if (peek==ParserConsts::dcolon)
                {
                    if (   m_Str.IsEmpty()
                        || m_Str.Trim(true).Trim(false).IsSameAs(ParserConsts::kw_const)
                        || m_Str.Trim(true).Trim(false).IsSameAs(ParserConsts::kw_volatile) ) // what else?!
                        m_EncounteredTypeNamespaces.push(token); // it's a type's namespace
                    else
                        m_EncounteredNamespaces.push(token);
                    m_Tokenizer.GetToken(); // eat ::
                }