Author Topic: New code completion remarks/issues  (Read 157442 times)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5274
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #90 on: October 04, 2009, 08:42:03 am »
Ok, tiny fixed with ReadClsNames in parserthread.
for broken codes below:
Quote
#include <iostream>

using namespace std;
typedef class qq
{
    int x;
    int y;

}xx
int main()
{
    cout << "Hello world!" << endl;
    return 0;
}



Note:This code can't be compiled in G++.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #91 on: October 04, 2009, 09:19:17 am »
Exactly,but I refer to visual assist,its parser do what the patch do. :D
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5274
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #92 on: October 04, 2009, 09:49:29 am »
Exactly,but I refer to visual assist,its parser do what the patch do. :D
I have applied your patch in my local copy, it works. :D. Thanks for your effort!

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #93 on: October 04, 2009, 01:27:20 pm »
It seems
Code: [Select]
Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args); can not work for the test codes below.
Code: [Select]
#include <iostream>
#include <iostream>
using namespace std;
std::vector<string> ssss;
#define  xxxxxxxxx 12e
typedef class qq
{
    int x;
    int y;
}eeee;
typedef unsigned int wwwwwwwww;
www|
int main()
{

    cout << "Hello world!" << endl;
    return 0;
}

In the "|" position , the suggestion list flash and can not show.If I comment the statement
Code: [Select]
typedef unsigned int wwwwwwwww;,it works again.
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5274
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #94 on: October 04, 2009, 01:40:17 pm »
@blueshake
That's something related in the nativeparser.cpp, and the code you cited. I will exam that. :D
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #95 on: October 04, 2009, 02:10:06 pm »
@blueshake
That's something related in the nativeparser.cpp, and the code you cited. I will exam that. :D
Nice to hear that.Thanks. :D
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5274
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #96 on: October 04, 2009, 03:01:45 pm »
I can only understand a little about the FindAIMatches() function.

For example, your test code:
Code: [Select]
typedef unsigned int wwwww;

wwww|

You will see in the symbol tree, the "wwwww" is a global "typedef". which is quite different if your code is like below:
Code: [Select]
typedef class qq
{
    int x;
    int y;
}eeee;
then, "eeee" in the symbol tree is of type "class".


The piece of code in FindAIMatches()
Code: [Select]
if (local_result.size() == 1)
    {
        int id = *local_result.begin();
        Token* token = tree->at(id);

        if (token->m_TokenKind == tkTypedef)
        {
            std::queue<ParserComponent> type_components;
            BreakUpComponents(parser, token->m_ActualType, type_components);

            while(!components.empty())
            {
                ParserComponent comp = components.front();
                components.pop();
                type_components.push(comp);
            }

    #if DEBUG_CC_AI
            if (s_DebugSmartSense)
            #if wxCHECK_VERSION(2, 9, 0)
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.wx_str(), token->m_ActualType.wx_str()));
            #else
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.c_str(), token->m_ActualType.c_str()));
            #endif
    #endif
            return FindAIMatches(parser, type_components, result, parentTokenIdx, noPartialMatch, caseSensitive, use_inheritance, kindMask, search_scope);
        }

    }

this means, when you only get one match in the current context, then for example, you are entering
Code: [Select]
www
This time, the only matched string is "wwwww" will be replace with "unsigned int" and the total line will be reparsed. (which means the FindAIMatches will be called iteratively).

 :D

But I think at this test code, this typedef replacement is useless. I can't find a test case that this replacement is meaningful. :(

Can someone explain more about this? Thanks.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9606
Re: New code completion remarks/issues
« Reply #97 on: October 04, 2009, 08:07:42 pm »
Guys, I am a bit lost and trying to catch up with all the posts. Am I rioght, that these two patches have to be applied together?

Patch 1:
Code: [Select]
Index: tokenizer.cpp
===================================================================
--- tokenizer.cpp (revision 5834)
+++ tokenizer.cpp (working copy)
@@ -216,17 +216,22 @@
 
 bool Tokenizer::SkipToCharBreak()
 {
-  if (PreviousChar() != '\\')
-      return true;
-  else
-  {
-      // check for "\\"
-      if (   ((m_TokenIndex - 2) >= 0)
-          && ((m_TokenIndex - 2) <= m_BufferLen)
-          && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\') )
-          return true;
-  }
-  return false;
+    if (PreviousChar() != '\\')
+        return true;
+    else
+    {
+        // check for "\\"
+        unsigned int numBackslash = 2;
+        while(   ((m_TokenIndex - numBackslash) >= 0)
+            && ((m_TokenIndex - numBackslash) <= m_BufferLen)
+            && (m_Buffer.GetChar(m_TokenIndex - numBackslash) == '\\') )
+            ++numBackslash;
+        if(numBackslash%2==1) // number of back slash(include current char ) is even
+            return true;     // eg: "\""
+        else
+            return false;    // eg: "\\""
+    }
+    return false;
 }

Patch 2:
Code: [Select]
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5825)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1768,7 +1768,7 @@
     Manager::Get()->GetLogManager()->DebugLog(F(_("HandleTypedef() : Adding typedef: name='%s', ancestor='%s'"), components.front().c_str(), ancestor.c_str()));
 #endif
 //    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);
-    Token* tdef = DoAddToken(tkClass, components.front(), lineNr, 0, 0, args);
+    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);
     if (tdef)
     {
         if (!is_function_pointer)
@@ -1796,7 +1796,7 @@
                 m_Tokenizer.UngetToken();
                 break;
             }
-        else if (wxIsalpha(current.GetChar(0)))
+        else if (wxIsalpha(current.GetChar(0)) && (m_Tokenizer.PeekToken() == ParserConsts::semicolon || m_Tokenizer.PeekToken() == ParserConsts::comma))
         {
 #if PARSERTHREAD_DEBUG_OUTPUT
             Manager::Get()->GetLogManager()->DebugLog(F(_T("ReadClsNames() : Adding variable '%s' as '%s' to '%s'"), current.c_str(), m_Str.c_str(), (m_pLastParent?m_pLastParent->m_Name.c_str():_T("<no-parent>"))));
@@ -1811,7 +1811,10 @@
             }
 
         }
-        else // unexpected
+        else// unexpected
+        {
+            m_Tokenizer.UngetToken();
             break;
+        }
     }
 }

???

I think the function name "SkipToCharBreak" is confusion, because it doesn't really do a skip.
So, I suggest using another name like :  IsRealCharBreak or IsCharBreak or IsEscapeChar
That's right and probably was a typo. I'll change that certainly.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5274
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #98 on: October 05, 2009, 03:08:00 am »
@MortenMacFly
Yes, the two patches can both be applied.

But, blueshake's patch of changing the code :

Code: [Select]
-    Token* tdef = DoAddToken(tkClass, components.front(), lineNr, 0, 0, args);
+    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);

should be reconsidered, because a small bug he states in this post:
http://forums.codeblocks.org/index.php/topic,11187.msg76788.html#msg76788
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #99 on: October 05, 2009, 05:26:33 am »
hello,another bad news.
It seems some bugs hide in the codecompletion.In the file codecompletion.cpp,in this place:
Code: [Select]
           // Remove duplicate items
            for (size_t i=0; i<items.Count()-1; i++)
                if (items.Item(i)==items.Item(i+1))
                    items.RemoveAt(i);
Manager::ge|
when you type Mana,yes,the suggestion show,then you type ::,yes work again,but when you type ge,the issue come out,the suggestion list flash again,and not show.what is going on here,the suggestion list is try to show,but cancle by something.

Note:
I just use the latest cc,change nothing.
Edit:
Code: [Select]
Manager::Get()->|And in the "|" position, the cc not work.
« Last Edit: October 05, 2009, 05:30:41 am by blueshake »
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5274
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #100 on: October 05, 2009, 05:52:34 am »
hello,another bad news.
It seems some bugs hide in the codecompletion.In the file codecompletion.cpp,in this place:
Code: [Select]
           // Remove duplicate items
            for (size_t i=0; i<items.Count()-1; i++)
                if (items.Item(i)==items.Item(i+1))
                    items.RemoveAt(i);
Manager::ge|
when you type Mana,yes,the suggestion show,then you type ::,yes work again,but when you type ge,the issue come out,the suggestion list flash again,and not show.what is going on here,the suggestion list is try to show,but cancle by something.

Note:
I just use the latest cc,change nothing.
Edit:
Code: [Select]
Manager::Get()->|And in the "|" position, the cc not work.

I can confirm both the two bugs.
Is it possible to create a simple test case. So that I can debug it. Currently, Opening the codeblocks's whole source code is not necessary.

Also: I think it's time to create some standard test code for codecompletion now. :D Any comments?

Also 1: @morten, in the current CC code, there are many code using the preprocessor guard.like:

Code: [Select]
#if PARSERTHREAD_DEBUG_OUTPUT
            Manager::Get()->GetLogManager()->DebugLog(F(_T("HandlePreprocessorBlocks() : Special case \"#if 0\" not skipped.")));
#endif

Is there any way we can just clean up these code, and use something like this:

Code: [Select]
#if PARSERTHREAD_DEBUG_OUTPUT
#define DebugOutput(a,b,c)  Manager::Get()->GetLogManager()->DebugLog(a,b,c)
#else
#define DebugOutput(a,b,c)  
#endif

So, the the source code, we can only use this:

Code: [Select]
DebugOutput(F(_T("HandlePreprocessorBlocks() : Special case \"#if 0\" not skipped.")));
This will make the source code more clean.


Also 2:

There are many variable names which are not following the source style guard. like:

Code: [Select]
   Parser parser(this);
    parser.ParseBufferForFunctions(control->GetTextRange(0, pos));

    wxArrayString funcs;
    TokensTree* tmptree = parser.GetTempTokens();

    // look for implementation functions that enclose our current line number
    for(size_t i = 0; i < tmptree->size();i++)

the tmptree should be tmpTree.

Also, there are many other variables should be renamed. such as:

Code: [Select]
// find current function's namespace so we can include local scope's tokens
    // we ' ll get the function's token (all matches) and add its parent namespace
    TokenIdxSet scope_result;
    TokenIdxSet proc_result;
    if (FindCurrentFunctionToken(editor, proc_result) != 0)
    {
        for (TokenIdxSet::iterator it = proc_result.begin(); it != proc_result.end(); ++it)
        {
            Token* token = parser->GetTokens()->at(*it);
            if (!token)
                continue;
            scope_result.insert(token->m_ParentIndex);
#if DEBUG_CC_AI
            if (s_DebugSmartSense)
            {
                Token* parent = parser->GetTokens()->at(token->m_ParentIndex);
                Manager::Get()->GetLogManager()->DebugLog(_T("Adding search namespace: ") + (parent ? parent->m_Name : _T("Global namespace")));
            }
#endif
        }
    }

These name should be "scopeResult" and so on...




If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5274
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #101 on: October 05, 2009, 06:06:39 am »
@blueshake:

look at the Get function prototype, it is a static function.
Code: [Select]
class DLLIMPORT Manager
{
    static Manager* Get();
}
but, in the symbol tree debug dialog, you will see that the
Quote
type is Manager *
actual type is Manager

That's may be the cause of bug...

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #102 on: October 05, 2009, 06:36:23 am »
@blueshake:

look at the Get function prototype, it is a static function.
Code: [Select]
class DLLIMPORT Manager
{
    static Manager* Get();
}
but, in the symbol tree debug dialog, you will see that the
Quote
type is Manager *
actual type is Manager

That's may be the cause of bug...


But it worked before.I updated the local copy,and it don't work.
look at these codes.The static are ignored.
Code: [Select]
       else if (token==ParserConsts::kw_static ||
            token==ParserConsts::kw_virtual ||
            token==ParserConsts::kw_inline)
        {
            // do nothing (skip it)
        }
« Last Edit: October 05, 2009, 06:38:28 am by blueshake »
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #103 on: October 05, 2009, 07:08:26 am »
anther issue.
Code: [Select]
std::string ss;
ss. not work
Code: [Select]
string ss;
ss.work
parser error:
Code: [Select]
string ss("abc");
ss.
it should be a variable,but ss is parsed as function.

[attachment deleted by admin]
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9606
Re: New code completion remarks/issues
« Reply #104 on: October 05, 2009, 08:39:38 am »
Just a tiny remark: For testing if the current modification (a.k.a patch) does not break something else please make sure that after applying the patch do the following:
1.) Open Code::Blocks having the debug log enabled.
2.) Open CodeBlocks.cbp (project files of core and core plugins)
3.) Wait, until the parser has completed.
3.) Check the debug log for the number of tokens, it should look like the following:
Parsing stage done (1570 total parsed files, 92637 tokens in 1 minute(s), 8.922 seconds).
The tokens should be round about 90000 - 100000 (depending on the patch level of C::B).

Furthermore: I am still happy to collect sample projects for testing CC I already have some...
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ