Author Topic: CC scope  (Read 10829 times)

Offline Alpha

  • Developer
  • Lives here!
  • *****
  • Posts: 1513
CC scope
« on: October 29, 2011, 08:41:33 pm »
Does CC take scope into account?  If so, the code
Code: [Select]
int main()
{
    {
        int fooBar = 0;
    }
    return foo //CC suggests "fooBar" here
}
says otherwise...

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5302
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC scope
« Reply #1 on: October 30, 2011, 02:37:45 am »
As you said, CC does not handle the "scope" information. So, when you leave the
Code: [Select]
    {
        int fooBar = 0;
    }
The variable fooBar is still recognized as a local variable in main().

I think it is not easy to take scope into account.

Any ideas?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Alpha

  • Developer
  • Lives here!
  • *****
  • Posts: 1513
Re: CC scope
« Reply #2 on: October 30, 2011, 03:07:40 am »
One idea would be that when CC collects local variables, it stores them as a series of nodes (like xml).  The root node of a function would contain the local variables that are always available.  Child nodes would be created for each { } pair containing the variables declared there (child nodes would, of course, have their own child nodes for { } pairs within them).

When CC is called, it would start by looking for all the variables in the current child node, then traverse up, collecting the tokens in each parent node until it reaches the root node.

I think the hard part with this method would how to keep track of which child node the user is currently typing in.

(I am still working on reading Code::Blocks' source, so I do not yet know enough to be able to tell if implementing this method would be feasible.)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5302
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC scope
« Reply #3 on: October 30, 2011, 03:21:14 am »
One idea would be that when CC collects local variables, it stores them as a series of nodes (like xml).  The root node of a function would contain the local variables that are always available.  Child nodes would be created for each { } pair containing the variables declared there (child nodes would, of course, have their own child nodes for { } pairs within them).

When CC is called, it would start by looking for all the variables in the current child node, then traverse up, collecting the tokens in each parent node until it reaches the root node.

I think the hard part with this method would how to keep track of which child node the user is currently typing in.

(I am still working on reading Code::Blocks' source, so I do not yet know enough to be able to tell if implementing this method would be feasible.)
Yes, this is the correct way I think.
1, local variables were collected only when we are in a function body, we do not collect them when we do a batch parsing(batch parsing only collect declaration and skip all the function body)

2, we need a data structure to handle such scope information.

3, If you read CodeCompletion's source code, you can have a look at: Code Completion Design - CodeBlocks

4, If you have any question in CC's source code, you can ask me.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline eranif

  • Regular
  • ***
  • Posts: 254
Re: CC scope
« Reply #4 on: October 30, 2011, 07:15:43 am »
Quote
I think the hard part with this method would how to keep track of which child node the user is currently typing in.
Its pretty simple actually. Assuming you know the location of the current function's body you should collect locals and manage them in a stack.

A simple container like:
Code: [Select]
typedef std::map<wxString, TokenInfo> ScopedLocals; // TokenInfo represents here a class of your choice that holds into about a single local variable
typedef std::list<ScopedLocals> LocalsStack;

And then apply the following logic:

Code: [Select]
start:
    stack.push_back( new ScopedLocals );

if found '{':
    stack.push_back( new ScopedLocals );

if found '}':
    delete stack.back(); // No longer relevant, free the allocated memory
    stack.pop_back();    // Remove it from the stack

if found local:
    stack.back()->insert(std::make_pair<wxString, TokenInfo>(token.Name(), token);

When you hit the end of your parsing, whatever left in the stack are your visible locals.

Special exceptions that should be handled:

- Variable defined inside 'catch()'
- Variable defined inside 'for(..)'

Eran
« Last Edit: October 30, 2011, 07:17:55 am by eranif »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5302
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC scope
« Reply #5 on: October 30, 2011, 03:26:19 pm »
@eranif
Dear eranif, thanks for your direction. I guess that this has done in your Codelite IDE, it is in your Bison(yacc) based parser, right?
Yes, it can be implemented, I'm thinking how this mechanism can be added to our cc's parser. (c::b's parser should be extend to handle brace scope information, we don't have a separate parser to parse the function body, so it may be add code everywhere in the recent parser's code)
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Alpha

  • Developer
  • Lives here!
  • *****
  • Posts: 1513
Re: CC scope
« Reply #6 on: October 30, 2011, 06:36:33 pm »
Quote
I think the hard part with this method would how to keep track of which child node the user is currently typing in.
Its pretty simple actually. Assuming you know the location of the current function's body you should collect locals and manage them in a stack.
That makes sense (unless I have completely misunderstood you :)); I had been thinking along the lines of storing tokens, not re-parsing on cursor movements (except if made necessary by the user's edits).
(As you can tell, I still have a lot to learn about Code::Blocks' inner workings.)

Offline eranif

  • Regular
  • ***
  • Posts: 254
Re: CC scope
« Reply #7 on: October 30, 2011, 06:57:22 pm »
Quote
it is in your Bison(yacc) based parser, right?
Actually, I don't parse locals until they are needed (e.g. user is trying to code complete something).

I wrote a lexer for this purpose (bison is not really needed here, I managed to do it with flex only)
http://codelite.svn.sourceforge.net/viewvc/codelite/trunk/ScopeOptimizer/ScopeOptimizer/scope_optimizer.l?revision=5183&view=markup

What this lexer does, its actually "optimize the scope", that means that it will remove all hidden blocks so the parser won't have to deal with them,
so this:
Code: [Select]
int main(int argc,char **argv) {
 {
   int member;
 }
  int another_variable;
| // caret is here

will become this:

Code: [Select]
int main(int argc,char **argv) {
 {}
  int another_variable;

note that this code is very independent (it comes with its own codelite workspace) and it expose a very simple API:
Code: [Select]
int OptimizeScope(const std::string &inputScope, std::string &optimizedScope)The first argument is the buffer you want to optimize while the later is the result (the output).

Eran
« Last Edit: October 30, 2011, 07:00:53 pm by eranif »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5302
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC scope
« Reply #8 on: October 31, 2011, 07:34:30 am »
Quote
it is in your Bison(yacc) based parser, right?
Actually, I don't parse locals until they are needed (e.g. user is trying to code complete something).
Yes, Codeblocks' CC do the same thing as it needed(e.g. when user select context menu->find declaration of XXXXX).

Quote
I wrote a lexer for this purpose (bison is not really needed here, I managed to do it with flex only)
http://codelite.svn.sourceforge.net/viewvc/codelite/trunk/ScopeOptimizer/ScopeOptimizer/scope_optimizer.l?revision=5183&view=markup

What this lexer does, its actually "optimize the scope", that means that it will remove all hidden blocks so the parser won't have to deal with them,
so this:
Code: [Select]
int main(int argc,char **argv) {
 {
   int member;
 }
  int another_variable;
| // caret is here

will become this:

Code: [Select]
int main(int argc,char **argv) {
 {}
  int another_variable;

note that this code is very independent (it comes with its own codelite workspace) and it expose a very simple API:
Code: [Select]
int OptimizeScope(const std::string &inputScope, std::string &optimizedScope)The first argument is the buffer you want to optimize while the later is the result (the output).
Eran

Codeblocks' current implementation lacks such feature, In the currently CC, we just let the parser parse the function body(we say: buffer) until the caret. So, as your said, I think c::b's cc can improve this by modify the buffer, just like you mentioned. The, we can let our parser works in the modified buffer(optimized scope string as in your example).

That's really cool! thanks for the hint, so the last thing is to take some time to implement it. :D
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Alpha

  • Developer
  • Lives here!
  • *****
  • Posts: 1513
Re: CC scope
« Reply #9 on: November 17, 2012, 11:21:28 pm »
Here is a possible scope optimizer for CC.  If anyone has time to test, let me know how well it works.
Code: [Select]
Index: src/plugins/codecompletion/nativeparser.cpp
===================================================================
--- src/plugins/codecompletion/nativeparser.cpp (revision 8571)
+++ src/plugins/codecompletion/nativeparser.cpp (working copy)
@@ -1826,7 +1826,7 @@
     if (blockStart != -1)
     {
         ++blockStart; // skip {
-        const int pos = caretPos == -1 ? searchData->control->GetCurrentPos() : caretPos;
+        const int pos = (caretPos == -1 ? searchData->control->GetCurrentPos() : caretPos);
         const int line = searchData->control->LineFromPosition(pos);
         const int blockEnd = searchData->control->GetLineEndPosition(line);
         if (blockEnd < 0 || blockEnd > searchData->control->GetLength())
@@ -1842,7 +1842,29 @@
         if (blockStart >= blockEnd)
             blockStart = blockEnd;
 
-        wxString buffer = searchData->control->GetTextRange(blockStart, blockEnd);
+        wxString buffer; // = searchData->control->GetTextRange(blockStart, blockEnd);
+        // condense out-of-scope braces {...}
+        int scanPos = blockEnd;
+        for (int i = pos; i > blockStart; --i)
+        {
+            if (searchData->control->GetCharAt(i) != wxT('}'))
+                continue;
+            const int style = searchData->control->GetStyleAt(i);
+            if (   searchData->control->IsString(style)
+                || searchData->control->IsCharacter(style)
+                || searchData->control->IsComment(style))
+            {
+                continue;
+            }
+            const int scopeStart = searchData->control->BraceMatch(i);
+            if (scopeStart < blockStart)
+                break;
+            buffer.Prepend(searchData->control->GetTextRange(i, scanPos));
+            scanPos = scopeStart + 1;
+            i       = scopeStart;
+        }
+        buffer.Prepend(searchData->control->GetTextRange(blockStart, scanPos));
+
         buffer.Trim();
         if (   !buffer.IsEmpty()
             && !m_Parser->ParseBuffer(buffer, false, false, true, searchData->file, m_LastFuncTokenIdx, initLine) )

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5302
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC scope
« Reply #10 on: November 19, 2012, 06:36:12 am »
Here is a possible scope optimizer for CC.  If anyone has time to test, let me know how well it works.
Code: [Select]
Index: src/plugins/codecompletion/nativeparser.cpp
===================================================================
--- src/plugins/codecompletion/nativeparser.cpp (revision 8571)
+++ src/plugins/codecompletion/nativeparser.cpp (working copy)
@@ -1826,7 +1826,7 @@
     if (blockStart != -1)
     {
         ++blockStart; // skip {
-        const int pos = caretPos == -1 ? searchData->control->GetCurrentPos() : caretPos;
+        const int pos = (caretPos == -1 ? searchData->control->GetCurrentPos() : caretPos);
         const int line = searchData->control->LineFromPosition(pos);
         const int blockEnd = searchData->control->GetLineEndPosition(line);
         if (blockEnd < 0 || blockEnd > searchData->control->GetLength())
@@ -1842,7 +1842,29 @@
         if (blockStart >= blockEnd)
             blockStart = blockEnd;
 
-        wxString buffer = searchData->control->GetTextRange(blockStart, blockEnd);
+        wxString buffer; // = searchData->control->GetTextRange(blockStart, blockEnd);
+        // condense out-of-scope braces {...}
+        int scanPos = blockEnd;
+        for (int i = pos; i > blockStart; --i)
+        {
+            if (searchData->control->GetCharAt(i) != wxT('}'))
+                continue;
+            const int style = searchData->control->GetStyleAt(i);
+            if (   searchData->control->IsString(style)
+                || searchData->control->IsCharacter(style)
+                || searchData->control->IsComment(style))
+            {
+                continue;
+            }
+            const int scopeStart = searchData->control->BraceMatch(i);
+            if (scopeStart < blockStart)
+                break;
+            buffer.Prepend(searchData->control->GetTextRange(i, scanPos));
+            scanPos = scopeStart + 1;
+            i       = scopeStart;
+        }
+        buffer.Prepend(searchData->control->GetTextRange(blockStart, scanPos));
+
         buffer.Trim();
         if (   !buffer.IsEmpty()
             && !m_Parser->ParseBuffer(buffer, false, false, true, searchData->file, m_LastFuncTokenIdx, initLine) )

The logic seems good to me.

So, for a simple code:
Code: [Select]
void ffff()
{
    int a;
    {
        int b;
        {
            int c;
        }
    }
   
    for (....)
    {
        int d;     
        {
            int e;
            ////////// caret here;       
        }
    }
}


It should have only variable statement "e, d, a" in the buffer.

I'm going to test it in tonight. :)

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5302
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC scope
« Reply #11 on: November 19, 2012, 11:39:54 am »
Two issue:
1, if you remove the whole "{" block, you will remove all the EOL characters "\n", so after parsing the buffer, you variable location (line information) may be changed.

2, if you remove the "{" block directly, which means:

Code: [Select]
for (...)
{
     int a;
}

will become
Code: [Select]
for (...)
This becomes a invalid statement. What I think is right method is to convert to this:
Code: [Select]
for (...)
{
     
}
Keep the "{" and "}" and all the EOL characters, and remove all others. :)
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9612
Re: CC scope
« Reply #12 on: November 19, 2012, 12:13:57 pm »
Code: [Select]
for (...)
This becomes a invalid statement. What I think is right method is to convert to this:
Code: [Select]
for (...)
{
}
True. Otherwise our parser may get hick-ups. ;-)

Also, (ollydbg): Do you have applied that other patch concerning variable parsing of for loops and stuff in parallel? This could conflict.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5302
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC scope
« Reply #13 on: November 19, 2012, 02:09:49 pm »
Also, (ollydbg): Do you have applied that other patch concerning variable parsing of for loops and stuff in parallel? This could conflict.
Yes, this patch did conflict with the one for parsing of for loops. But sorry those days I'm a little busy, so I have no time to deeply test and merge those two patches. :(
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9612
Re: CC scope
« Reply #14 on: November 19, 2012, 02:43:25 pm »
Yes, this patch did conflict with the one for parsing of for loops. But sorry those days I'm a little busy, so I have no time to deeply test and merge those two patches. :(
...maybe ALPHA can do it - what I meant is this:
http://developer.berlios.de/patch/?func=detailpatch&patch_id=3345&group_id=5358
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ