Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: killerbot on September 15, 2009, 12:24:28 pm

Title: New code completion remarks/issues
Post by: killerbot on September 15, 2009, 12:24:28 pm
Shortly in a nightly build the new code completion engine will be available.
One can already get it from svn.

This thread is the start base to discuss issues.
Title: Re: New code completion remarks/issues
Post by: killerbot on September 15, 2009, 12:25:07 pm
I have several classes with several members and methods, and I don't get any completion at all.
All members start with "m_", none of them work, calling the other member methods from within a member method same story. In the Symbols browser the classes are shown correctly !!!

I think i has something to do with namespace.

Example :
I have class CStop in the namespace vipnt. That class has a member m_Angle (amongst other) and (amongst other) the following method "Configure", but no completion on either.
However when I start typing like this in a member method of this class, I do get completions.

Quote
   vipnt::CStop::Configure
   vipnt::CStop::m_Angle

Note when I to "ctr-space" I see a lot of things, but the moment I start typing m_ there are already no more matches in the list.
Title: Re: New code completion remarks/issues
Post by: killerbot on September 15, 2009, 12:26:10 pm
Here's a reduced testing example showing the issue. (just add the 2 files to the dummy console application you get from the CB project wizard)

1) create Stop.h with the following content :
Code
#ifndef _STOP_H_INCLUDE
#define _STOP_H_INCLUDE

namespace vipnt
{

class CStop
{
public:
void RePaint();
bool HasWork() const;

private:

int* m_Angle;
int* m_Variance;
};

} // namespace vipnt
#endif // _STOP_H_INCLUDE

2) create Stop.cpp with the following content :
Code
#include "Stop.h"

namespace vipnt
{

namespace
{
const int NumberOfZones = 6;
} // namespace


bool CStop::HasWork() const
{
return true;
} // end of HasWork

void CStop::RePaint()
{
} // end of RePaint

} // namespace vipnt

go into RePaint and try :
 m_Angle : no completion
 HasWork no completion
 ctrl-space  nothing to find in there that starts with m_

Now remove in the cpp file the anonymous namespace (declaring the constant) and everything will work.

EDIT : when I moved the anonymous namespace in front of the vipnt namespace in the cpp file then again it works.
Nesting seems to introduce issues.
Title: Re: New code completion remarks/issues
Post by: killerbot on September 15, 2009, 12:26:43 pm
other interesting things to note. Look at the code completion toolbar, the read only field at the left.

enter the vipnt namespace --> vipnt
enter the anonymous (nested) --> vipnt
enter CStop::HasWork()  --> CStop !!!!!!!!!!!!!!!!!!!
enter CStop::RePaint()  --> CStop  !!!!!!!!!!!!!!!!!!!

NOW : move the anonyous namespace between the HasWork en RePaint methods.

enter the vipnt namespace --> vipnt
enter CStop::HasWork()  --> vipnt::CStop
enter the anonymous (nested) --> vipnt
enter CStop::RePaint()  --> vipnt::CStop

This gives better things in the toolbar, and it also brings back the completion.

Strange, strange, strange.


OPEN  QUESTION : could it have to do with one namespace being immediately nested after the start of another namespace
Code
namespace vipnt
{

namespace
{
const int NumberOfZones = 6;
}

...
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on September 15, 2009, 12:47:07 pm
I have been wondering lately,
why not add some kind of functionality/application test for the code completion.
Parsing is one of the areas that allow easy testing.
The benefits of the testing (obvious to many I'm sure) are:
1. You get instant answer if your change is correct or you add new test, so other don't break you code later
2. The behavior of the code is documented through the test
3. Users are happier, because of less bugs
4. Developers are braver when they refactor, because they have a safety net.
5. There is measure of progress (5 test pass we have 10 left for example)

Best regards
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 15, 2009, 01:15:49 pm
why not add some kind of functionality/application test for the code completion.
Absolutely true!
It could be as simple as a project that contains 1..n files (classes) with special items to be resolved. We could start with Lievens class(es) provided here. :-) Just name them e.h. anonymous_namespace_test.cpp or alike... so we know by filename what's the actual test in the class.
Title: Re: New code completion remarks/issues
Post by: blueshake on September 15, 2009, 01:24:12 pm
Quote
namespace vipnt
{

namespace
{
   const in NumberOfZones = 6;
}

...
if we give the anonymous namespace a name .for example qq.
the cc work again.
and it is strange in function NativeParser::FindCurrentFunctionStart:
watch these codes and turn it on.
Code
                #if wxCHECK_VERSION(2, 9, 0)
                    Manager::Get()->GetLogManager()->DebugLog(F(_T("Current function: %s (at line %d)"), token->DisplayName().wx_str(), token->m_ImplLine));
                #else
                    Manager::Get()->GetLogManager()->DebugLog(F(_T("Current function: %s (at line %d)"), token->DisplayName().c_str(), token->m_ImplLine));
                #endif
if the anonymous namespace in the namespace vipnt,
the code::block debug print Current function: CStop::RePaint() : void (at line 19)
if the anonymous namespace out the namespace vipnt,
the code::block debug print Current function: vipnt::CStop::RePaint() : void (at line 19)
the difference is the outside  one has the vipnt:: but the inside one has nothing.
and this will affect the NativeParser::FindCurrentFunctionToken
it make the RePaint's parentscope can not be add to search socpe ,so the cc didn't work.
above is my guess.
Title: Re: New code completion remarks/issues
Post by: killerbot on September 15, 2009, 01:39:09 pm
but note that if I leave the anonymous namespace nested inside vipnt, but move it between the 2 methods, the parsing also works again.

Title: Re: New code completion remarks/issues
Post by: blueshake on September 15, 2009, 01:54:46 pm
ok,I move the the anonymous namespace to the position between the 2 methods.
yes ,it work.
and the code::block debug print Current function: vipnt::CStop::RePaint() : void (at line 19)
the output is the same to the anonymous outside the vipnt which cc work too.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 15, 2009, 02:13:09 pm
namespace
{
   const in NumberOfZones = 6;
}
Are you aware that this is NOT a legal paragraph? There is no type called "in"... I guess it should read "int". The parser gets confused by that probably. If I change the type to "int" it works pretty well here (latest CC from trunk).
Title: Re: New code completion remarks/issues
Post by: killerbot on September 15, 2009, 02:30:59 pm
should be int, yes. But still fails for me.
Updating to latest in svn right now ...
Title: Re: New code completion remarks/issues
Post by: killerbot on September 15, 2009, 02:51:30 pm
at rev 5790, and it still fails.

@Morton : how come so many cc stuff works for you and not for us ?? ;-)
Title: Re: New code completion remarks/issues
Post by: blueshake on September 15, 2009, 02:53:30 pm
fail for me too at rev 5790.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 15, 2009, 03:14:47 pm
@Morton : how come so many cc stuff works for you and not for us ?? ;-)
Dunno. It's really the same code as in trunk now. However - if it makes you "feel better" (;-)): I know at least one class that works for Jens and blueshake but not for me. :-(
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 15, 2009, 03:16:33 pm
...adding this report:
http://forums.codeblocks.org/index.php/topic,11188.0.html
Title: Re: New code completion remarks/issues
Post by: Jenna on September 15, 2009, 03:56:14 pm
This one does not work for me either in r5790.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 15, 2009, 04:01:57 pm
This one does not work for me either in r5790.
...which one?
Title: Re: New code completion remarks/issues
Post by: Jenna on September 15, 2009, 04:03:40 pm
This one does not work for me either in r5790.
...which one?
Lieven's sample-code.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 16, 2009, 01:44:32 pm
....adding this report:
http://forums.codeblocks.org/index.php/topic,11107.msg76268.html#msg76268
Title: Re: New code completion remarks/issues
Post by: blueshake on September 23, 2009, 09:34:35 am
in the latest cc,
in the function void CodeCompletion::EditorEventHook(cbEditor* editor, wxScintillaEvent& event)
inside the function body,
cbStyledTextCtrl* control = editor->GetControl();

there is variable named control,
but when I type cont, the cc don't work.(no control in suggestion list and control's member list didn't work too)
I am sure it work before,at least at ver 5713.
Title: Re: New code completion remarks/issues
Post by: blueshake on September 23, 2009, 09:37:29 am
the same to variable event
Title: Re: New code completion remarks/issues
Post by: blueshake on September 23, 2009, 03:55:39 pm
hello,did anyboby notice that codecompletion bar that miss some function in codecompletion.cpp.
see my snap3 .I can confirm that it work at rev 5731 see  the snap4 .
it seems that the codecompletion's member function were not included in this situation.what is going on ?

[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 23, 2009, 05:10:45 pm
I can confirm that it work at rev 5731 see  the snap4 .
I noticed the same (I told you and Jens...), but this info (the revision) is not really helpful as it is before the merge.

I wonder if it was working in your own test branch before the merge. Because that would be easier to compare.
Title: Re: New code completion remarks/issues
Post by: blueshake on September 24, 2009, 03:46:07 am
yes,I remember it worked in my local copy(I just modify the codecompletion.cpp file),but unluckly,my old  machine is down ,I lost everything. :(
Title: Re: New code completion remarks/issues
Post by: blueshake on September 24, 2009, 05:07:20 am
hello,Morten

good news: :D
the issue killerbot report can be solved by this patch.
bad news:
can not make a clear explaination here.
Code
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5816)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1051,7 +1051,11 @@
     if (ns == ParserConsts::opbrace)
     {
         // parse inside anonymous namespace
+        Token* lastParent = m_pLastParent;
+        TokenScope lastScope = m_LastScope;
         DoParse();
+        m_pLastParent = lastParent;
+        m_LastScope = lastScope;
     }
     else
     {
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 24, 2009, 07:42:01 am
hello,did anyboby notice that codecompletion bar that miss some function in codecompletion.cpp.
see my snap3 .I can confirm that it work at rev 5731 see  the snap4 .
it seems that the codecompletion's member function were not included in this situation.what is going on ?
Hi, I can confirm this bug in my local copy(based on rev 5816). :D
Investigating now!
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 24, 2009, 07:54:52 am
I think I have locate the bug. See the image below.

(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2009-09-24134555.png)

It seems the function before the current scope was shown.

So, the parser may fails parsing this function, especially "//" or "\\".


Code
wxArrayString GetIncludeDirs(cbProject &project)
{
    wxArrayString dirs;
    {
        wxArrayString target_dirs = project.GetIncludeDirs();

        for(size_t ii = 0; ii < target_dirs.GetCount(); ++ii)
        {
            wxFileName filename;
            NormalizePath(filename, target_dirs[ii]);

            wxString fullname = filename.GetFullPath();
            fullname.Replace(_T("\\"), _T("/"), true);
            if(dirs.Index(fullname) == wxNOT_FOUND)
                dirs.Add(fullname);
        }
    }

    wxString target_name = project.GetActiveBuildTarget();
    ProjectBuildTarget *target = project.GetBuildTarget(target_name);
    if(target)
    {
        wxArrayString target_dirs = target->GetIncludeDirs();
        for(size_t ii = 0; ii < target_dirs.GetCount(); ++ii)
        {
            wxFileName filename;
            NormalizePath(filename, target_dirs[ii]);

            wxString fullname = filename.GetFullPath();
            fullname.Replace(_T("\\"), _T("/"), true);
            if(dirs.Index(fullname) == wxNOT_FOUND)
                dirs.Add(fullname);
        }
    }
    return dirs;
}

Title: Re: New code completion remarks/issues
Post by: blueshake on September 24, 2009, 08:01:54 am
@ollydbg


can you provide the patch?
I believe there is some connection among reply #19,20,21.if this issue can be solved,
others wil too.
Title: Re: New code completion remarks/issues
Post by: killerbot on September 24, 2009, 08:29:10 am
hello,Morten

goog news: :D
the issue killerbot report can be solved by this patch.
bad news:
can not make a clear explaination here.
Code
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5816)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1051,7 +1051,11 @@
     if (ns == ParserConsts::opbrace)
     {
         // parse inside anonymous namespace
+        Token* lastParent = m_pLastParent;
+        TokenScope lastScope = m_LastScope;
         DoParse();
+        m_pLastParent = lastParent;
+        m_LastScope = lastScope;
     }
     else
     {


YES, I can confirm this works now !!!
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 24, 2009, 08:30:02 am
@ollydbg


can you provide the patch?
I believe there is some connection among reply #19,20,21.if this issue can be solved,
others wil too.

I can't figure out yet till now :(
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 24, 2009, 01:05:25 pm
YES, I can confirm this works now !!!
I've committed including another very bad parser bug I fixed. It slipped in with the last modification I did before doing the merge... bad boy... SO could your all please try again? Also: I am generally interested in code snippets to test the CC plugin. I am planning to put all of them in a "test" project.
Title: Re: New code completion remarks/issues
Post by: blueshake on September 24, 2009, 02:52:06 pm
Quote
hello,did anyboby notice that codecompletion bar that miss some function in codecompletion.cpp.
see my snap3 .I can confirm that it work at rev 5731 see  the snap4 .
it seems that the codecompletion's member function were not included in this situation.what is going on ?
hello,Morten
is this issue solved?
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 24, 2009, 03:13:36 pm
Quote
hello,did anyboby notice that codecompletion bar that miss some function in codecompletion.cpp.
see my snap3 .I can confirm that it work at rev 5731 see  the snap4 .
it seems that the codecompletion's member function were not included in this situation.what is going on ?
hello,Morten
is this issue solved?
No, the bug you reported in reply #19 is still exist in rev 5817
Title: Re: New code completion remarks/issues
Post by: killerbot on September 24, 2009, 03:19:52 pm
I can confirm my reported issues are still fixed with the latest code in svn.
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 24, 2009, 03:36:37 pm
I can confirm my reported issues are still fixed with the latest code in svn.
fixed or not?  :?
I tested your example, the bug still exists.


Edit

I can confirm it works!!!
Title: Re: New code completion remarks/issues
Post by: blueshake on September 24, 2009, 03:39:36 pm
I can confirm it work for killerbot's issue.
Title: Re: New code completion remarks/issues
Post by: blueshake on September 24, 2009, 03:41:50 pm
but the issue on reply#19  still exists
the parser stop in the end of wxArrayString GetIncludeDirs(cbProject &project)

the next codes are not parsed.

see the attachment.the CodeCompletion token has no implement file.

[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: Jenna on September 24, 2009, 03:51:00 pm
Another issue in some classes/files I can not see/jump to the implementation.
For example cbEditor.
The toolbar does not show any members of cbEditor if I am in cbeditor.cpp, only cbEditorInternalData.

Jump to declaration and jump to implementation is broken for many functions ( I guess the same that are not listed in toolbar and/or symbols browser).
Title: Re: New code completion remarks/issues
Post by: blueshake on September 24, 2009, 03:54:12 pm
The bug followed the routine .If the golbal function followed some kind of class member function,then the class member function will be not parsed.(e.g. cbeditor.cpp)
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 24, 2009, 04:04:09 pm
The bug followed the routine .If the golbal function followed some kind of class member function,then the class member function will be not parsed.(e.g. cbeditor.cpp)

If I only copy three function implementation from line 517 to line 743 of codecompletion.cpp .

Code
int CodeCompletion::CodeComplete()
{
......
}

bool TestIncludeLine(wxString const &line)
{
......
}

wxArrayString GetIncludeDirs(cbProject &project)
{
......
}



see the screenshot, the parser works quite well.

(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2009-09-24215427.png)

Both two global functions and one class member function will be recognized correctly.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 24, 2009, 04:15:55 pm
I personally believe it's related to the handling of the const keyword in a function signature. For the classes where it works not here *always* the previous method has a const.
I've extarcted all the patches I applied from the beginning of the refactoring meanwhile and try to reproduce by which step this behaviour was introduced. So far I am at step 2...  so this may take a while... (taking Murphy into account it'll be the last step).
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 25, 2009, 06:45:16 am
hello all.

I just enabled the parser_thread_output guard, and try to exam the parser_thread log. It seems when parsing codecompletion.cpp, the parser exits after parsing function: GetIncludeDirs.

Here is the log:

Code
......
DoParse() : Loop:m_Str='int ', token='SortCCList'
HandleFunction() : Adding function 'SortCCList': m_Str='int '
HandleFunction() : name='SortCCList', args='(const wxString& first, const wxString& second)', peek='{'
HandleFunction() : Skipped function SortCCList impl. from 440 to 470
HandleFunction() : Add token name='SortCCList', args='(const wxString& first, const wxString& second)', return type='int '
DoParse() : Loop:m_Str='', token='int'
DoParse() : Loop:m_Str='int ', token='CodeCompletion'
DoParse() : Loop:m_Str='int ', token='CodeComplete'
HandleFunction() : Adding function 'CodeComplete': m_Str='int '
HandleFunction() : name='CodeComplete', args='()', peek='{'
HandleFunction() : Skipped function CodeComplete impl. from 473 to 645
HandleFunction() : Add token name='CodeComplete', args='()', return type='int '
DoParse() : Loop:m_Str='', token='bool'
DoParse() : Loop:m_Str='bool ', token='TestIncludeLine'
HandleFunction() : Adding function 'TestIncludeLine': m_Str='bool '
HandleFunction() : name='TestIncludeLine', args='(wxString const &line)', peek='{'
HandleFunction() : Skipped function TestIncludeLine impl. from 648 to 664
HandleFunction() : Add token name='TestIncludeLine', args='(wxString const &line)', return type='bool '
DoParse() : Loop:m_Str='', token='wxArrayString'
DoParse() : Loop:m_Str='wxArrayString ', token='GetIncludeDirs'
HandleFunction() : Adding function 'GetIncludeDirs': m_Str='wxArrayString '
HandleFunction() : name='GetIncludeDirs', args='(cbProject &project)', peek='{'
HandleFunction() : Skipped function GetIncludeDirs impl. from 667 to 2060
HandleFunction() : Add token name='GetIncludeDirs', args='(cbProject &project)', return type='wxArrayString '

************it seems the parser stops here!!!!!!!********************

Reparsing saved files...
Starting batch parsing



Note:


Skipped function GetIncludeDirs impl. from 667 to 2060

Edit:

So, I still believe it is something related to parsing such statement:

Code
....
fullname.Replace(_T("\\"), _T("/"), true);
...

"\\"   and "/" should carefully be tokenized.


Edit2

If you would like to do such test, you'd better delete all the include statement from the codecompletion.cpp, and copy these code to an empty console project. otherwise, too many logs will be  outputs when parsing these header files. :D
Title: Re: New code completion remarks/issues
Post by: blueshake on September 25, 2009, 07:06:32 am
But what about cbeditor.cpp.
It seems there is no statement fullname.Replace(_T("\\"), _T("/"), true); in the file.
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 25, 2009, 07:23:43 am
But what about cbeditor.cpp.
It seems there is no statement fullname.Replace(_T("\\"), _T("/"), true); in the file.


The same thing happens if you try to parse the cbEditor.cpp, here is the log:

Code
HandleFunction() : Adding function 'IsPreprocessor': m_Str='bool '
HandleFunction() : name='IsPreprocessor', args='(int style)', peek='{'
HandleFunction() : Skipped function IsPreprocessor impl. from 269 to 274
HandleFunction() : Add token name='IsPreprocessor', args='(int style)', return type='bool '
DoParse() : Loop:m_Str='', token='bool'
DoParse() : Loop:m_Str='bool ', token='IsCharacterOrString'
HandleFunction() : Adding function 'IsCharacterOrString': m_Str='bool '
HandleFunction() : name='IsCharacterOrString', args='(int style)', peek='{'
HandleFunction() : Skipped function IsCharacterOrString impl. from 277 to 289
HandleFunction() : Add token name='IsCharacterOrString', args='(int style)', return type='bool '
DoParse() : Loop:m_Str='', token='bool'
DoParse() : Loop:m_Str='bool ', token='IsCharacter'
HandleFunction() : Adding function 'IsCharacter': m_Str='bool '
HandleFunction() : name='IsCharacter', args='(int style)', peek='{'
HandleFunction() : Skipped function IsCharacter impl. from 292 to 304
HandleFunction() : Add token name='IsCharacter', args='(int style)', return type='bool '
DoParse() : Loop:m_Str='', token='void'
DoParse() : Loop:m_Str='void ', token='DoBraceCompletion'
HandleFunction() : Adding function 'DoBraceCompletion': m_Str='void '
HandleFunction() : name='DoBraceCompletion', args='(const wxChar& ch)', peek='{'
HandleFunction() : Skipped function DoBraceCompletion impl. from 307 to 3240
HandleFunction() : Add token name='DoBraceCompletion', args='(const wxChar& ch)', return type='void '
Reparsing saved files...
Starting batch parsing

It seem the parser stops after parsing DoBraceCompletion. note:HandleFunction() : Skipped function DoBraceCompletion impl. from 307 to 3240

It's implementation is list below:

Code
void DoBraceCompletion(const wxChar& ch)
    {
        cbStyledTextCtrl* control = m_pOwner->GetControl();
        int pos = control->GetCurrentPos();
        int style = control->GetStyleAt(pos);
        if ( IsComment(style) || IsPreprocessor(style) )
            return;
        if ( ch == _T('\'') )
        {
            if ( (control->GetCharAt(pos) == ch) && (pos > 1) && (control->GetCharAt(pos-2) != _T('\\')) )
            {
                control->DeleteBack();
                control->GotoPos(pos);
            }
            else
            {
                if ( (control->GetCharAt(pos-2) == _T('\\')) || IsCharacterOrString(style) )
                    return;
                control->AddText(ch);
                control->GotoPos(pos);
            }
            return;
        }
        if ( ch == _T('"') )
        {
            if ( (control->GetCharAt(pos) == ch) && (pos > 1) && (control->GetCharAt(pos-2) != _T('\\')) )
            {
                control->DeleteBack();
                control->GotoPos(pos);
            }
            else
            {
                if ( (control->GetCharAt(pos-2) == _T('\\')) || IsCharacter(style) )
                    return;
                control->AddText(ch);
                control->GotoPos(pos);
            }
            return;
        }
        if ( IsCharacterOrString(style) )
            return;
        const wxString leftBrace(_T("([{"));
        const wxString rightBrace(_T(")]}"));
        int index = leftBrace.find(ch);
        const wxString unWant(_T(");\n\r\t\b "));
        #if wxCHECK_VERSION(2, 9, 0)
        if ((index != wxNOT_FOUND) && (unWant.find(wxUniChar(control->GetCharAt(pos))) != wxNOT_FOUND))
        #else
        if ((index != wxNOT_FOUND) && (unWant.find(control->GetCharAt(pos)) != wxNOT_FOUND))
        #endif
        {
            control->AddText(rightBrace.GetChar(index));
            control->GotoPos(pos);
            if (ch == _T('{'))
            {
                    const wxRegEx reg(_T("^[ \t]*{}[ \t]*"));
                    if (reg.Matches(control->GetCurLine()))
                    {
                control->NewLine();
                control->GotoPos(pos);
                        control->NewLine();
                return;
            }
        }
        }
        else
        {
            index = rightBrace.find(ch);
            if (index != wxNOT_FOUND)
            {
                if (control->GetCharAt(pos) == ch)
                {
                    control->DeleteBack();
                    control->GotoPos(pos);
                    return;
                }
            }
        }
    }

Title: Re: New code completion remarks/issues
Post by: blueshake on September 25, 2009, 07:43:57 am
@ollydbg
I comment wxArrayString GetIncludeDirs(cbProject &project)
and void CodeCompletion::CodeCompleteIncludes() function implementation.Then the cc works in the latest cc.
And I test the codes in codecompletion.cpp without any comment in svn 5731.Everything work perfectly.
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 25, 2009, 08:19:23 am
@ollydbg
I comment wxArrayString GetIncludeDirs(cbProject &project)
and void CodeCompletion::CodeCompleteIncludes() function implementation.Then the cc works in the latest cc.
And I test the codes in codecompletion.cpp without any comment in svn 5731.Everything work perfectly.


I have noticed that there's no difference in the parserthread::SkipBlock() function in the neally two month.

but I found some code changes in the tokenizer.cpp in function SkipToChar from rev 5434->5783.

Code
bool Tokenizer::SkipToChar(const wxChar& ch)
{
    // skip everything until we find ch
    while(true)
    {
        while (CurrentChar() != ch && MoveToNextChar())  // don't check EOF when MoveToNextChar already does
            ;

        if (IsEOF())
            return false;

        if (PreviousChar() != '\\')
            break;
        else
        {
            // check for "\\"
            if (((m_TokenIndex - 2) >= 0) && ((m_TokenIndex - 2) >= m_BufferLen) && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\'))
                break;
        }
        MoveToNextChar();
    }
    return true;
}

Why we need to check this :

Code
            // check for "\\"
            if (((m_TokenIndex - 2) >= 0) && ((m_TokenIndex - 2) >= m_BufferLen) && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\'))


((m_TokenIndex - 2) >= m_BufferLen  ?????


I think it should be:

(m_TokenIndex - 2)<= m_BufferLen
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 25, 2009, 08:33:03 am
Ok, fixed by this patch:

Code
Index: tokenizer.cpp
===================================================================
--- tokenizer.cpp (revision 5818)
+++ tokenizer.cpp (working copy)
@@ -230,7 +230,7 @@
         else
         {
             // check for "\\"
-            if (((m_TokenIndex - 2) >= 0) && ((m_TokenIndex - 2) >= m_BufferLen) && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\'))
+            if (((m_TokenIndex - 2) >= 0) && ((m_TokenIndex - 2) <= m_BufferLen) && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\'))
                 break;
         }
         MoveToNextChar();
@@ -286,7 +286,7 @@
         else
         {
             // check for "\\"
-            if (((m_TokenIndex - 2) >= 0) && ((m_TokenIndex - 2) >= m_BufferLen) && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\'))
+            if (((m_TokenIndex - 2) >= 0) && ((m_TokenIndex - 2) <= m_BufferLen) && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\'))
                 break;
         }
         MoveToNextChar();
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 25, 2009, 08:41:02 am
I think it should be:
(m_TokenIndex - 2)<= m_BufferLen
Definitely. I'll certainly apply this in trunk when tested. But it really looks very ugly to me, too.
Title: Re: New code completion remarks/issues
Post by: blueshake on September 25, 2009, 08:48:12 am
It worked. :D
Cool ,man!!!!!!!!!!!!!!
Title: Re: New code completion remarks/issues
Post by: mmkider on September 25, 2009, 11:54:08 am
New Code Completion plug-in is more powerful.

 :D
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 25, 2009, 09:08:02 pm
Added pointer to here:
http://forums.codeblocks.org/index.php/topic,11240.0.html
(To keep all issues in one place.)
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 26, 2009, 09:54:54 am
Added pointer to here:
http://forums.codeblocks.org/index.php/topic,11240.0.html
(To keep all issues in one place.)

After several email discuss with blueshake, we finally give a patch to solve this problem:

here is the patch:

Code
Index: nativeparser.cpp
===================================================================
--- nativeparser.cpp (revision 5819)
+++ nativeparser.cpp (working copy)
@@ -1412,6 +1420,8 @@
     static wxString cached_search;
     static size_t cached_results_count = 0;
 
+
+
     // early-out opportunity
     // if the user starts typing a token that in our last search had 0 results,
     // and we see that he's continuing typing for that same token,
@@ -1419,7 +1429,8 @@
     if (cached_editor == editor &&
         cached_editor_start_word == m_EditorStartWord &&
         cached_results_count == 0 &&
-        actual.StartsWith(cached_search))
+        actual.StartsWith(cached_search)&&
+        actual.Find(_T('(')) == wxNOT_FOUND)
     {
 #ifdef DEBUG_CC_AI
         if (s_DebugSmartSense)


Here is the explanation of this patch.

for example, you code is like below:

Code
int aaaa;
void f(int result){
if             <---------enter here

}


If we firstly enter
Code
if
then
Code
if(

the AI function may only give 0 result, also, the result was cached as static variables.

if we continue entering like
Code
if(resu
, AI just see that this is already parsed string, and early quite from the function:
NativeParser::AI()

Here is the sample code of such early exit in NativeParser::AI:

Code
static cbEditor* cached_editor = 0;
    static int cached_editor_start_word = 0;
    static wxString cached_search;
    static size_t cached_results_count = 0;



    // early-out opportunity
    // if the user starts typing a token that in our last search had 0 results,
    // and we see that he's continuing typing for that same token,
    // don't even bother to search
    if (cached_editor == editor &&
        cached_editor_start_word == m_EditorStartWord &&
        cached_results_count == 0 &&
        actual.StartsWith(cached_search)&&
        actual.Find(_T('(')) == wxNOT_FOUND) **************************add code here!!!
    {
#ifdef DEBUG_CC_AI
        if (s_DebugSmartSense)
            Manager::Get()->GetLogManager()->DebugLog(_T("Aborting search: last attempt returned 0 results"));
#endif
        // but set m_LastAISearchWasGlobal correctly so C++ keywords can be shown
        std::queue<ParserComponent> components;
        BreakUpComponents(parser, actual, components);
        m_LastAISearchWasGlobal = components.size() <= 1;
        if (!components.empty())
            m_LastAIGlobalSearch = components.front().component;
        return 0;
    }

In fact, at this time( AI entered second time), all the if condition was true.


cached_results_count = 0
cached_search = "if"
actual             = "if(resu"
.....


So, we should add checking actual.Find(_T('(')) == wxNOT_FOUND to avoid the early exit from the function AI.


Thanks to blueshake! :D


Title: Re: New code completion remarks/issues
Post by: ollydbg on September 26, 2009, 03:47:48 pm
hi, there is another patch by blueshake, which is better than mine. :D
Code
Index: src/plugins/codecompletion/nativeparser.cpp
===================================================================
--- src/plugins/codecompletion/nativeparser.cpp (revision 5821)
+++ src/plugins/codecompletion/nativeparser.cpp (working copy)
@@ -1522,6 +1522,7 @@
     }
 
     cached_editor = editor;
+    if (result.size() || (m_EditorEndWord - m_EditorStartWord))
     cached_editor_start_word = m_EditorStartWord;
     cached_search = actual;
     cached_results_count = result.size();
Title: Re: New code completion remarks/issues
Post by: koso on September 27, 2009, 09:18:14 pm
Hello, using latest SVN build (and probably not only latest), I am having problem with code completion for variables declared using "struct" keyword. For example:

Code
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main(void)
{
struct tm *current;
        current->
}

I dont get any suggestions when typing "current->" and also nothing when trying to complete "curr". It works as expected without keyword struct in declaration.
I tried to find symbol current using Code completion debug tool, but i get only empty list with information, that were found Multiple matches and i must choose :P
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 28, 2009, 03:30:40 am
Add pointer to the this bug report:

http://forums.codeblocks.org/index.php/topic,11107.msg76268.html#msg76268

And this problem can be solved by my patch here:

Code
Index: src/plugins/codecompletion/nativeparser.cpp
===================================================================
--- src/plugins/codecompletion/nativeparser.cpp (revision 5824)
+++ src/plugins/codecompletion/nativeparser.cpp (working copy)
@@ -427,71 +427,79 @@
     delete [] Compilers;
 } // end of AddCompilerDirs
 
+
+
 wxArrayString NativeParser::GetGCCCompilerDirs(const wxString &cpp_compiler, const wxString &base)
 {
     wxArrayString gcc_compiler_dirs;
 
     // for starters , only do this for gnu compiler
-//    Manager::Get()->GetLogManager()->DebugLog(_T("CompilerID ") + CompilerID);
-    //    wxString Command("mingw32-g++ -v -E -x c++ - < nul");
-    // specifying "< nul", does not seem to work
-    // workaround : create a dummy file (let's hope it does not exist)
+    //Manager::Get()->GetLogManager()->DebugLog(_T("CompilerID ") + CompilerID);
+    //
+    //   windows: mingw32-g++ -v -E -x c++ nul
+    //   linux  : g++ -v -E -x c++ /dev/null
     // do the trick only for c++, not needed then for C (since this is a subset of C++)
-    wxString DummyFileName = wxFileName::CreateTempFileName(_T("Dummy_z4hsdkl9nf7ba3L9nv41"));
-    if(!DummyFileName.IsEmpty())
+
+
+    // let's construct the command
+    // use a null file handler
+    // both works fine in Windows and linux
+
+#ifdef __WXMSW__
+    wxString Command = cpp_compiler + _T(" -v -E -x c++ nul");
+#else
+    wxString Command = cpp_compiler + _T(" -v -E -x c++ /dev/null");
+#endif
+
+    // action time  (everything shows up on the error stream
+    wxArrayString Output, Errors;
+    wxExecute(Command, Output, Errors, wxEXEC_NODISABLE);
+    int nCount = Errors.GetCount();
+    // the include dir (1 per line) show up between the lines
+    // #include <...> search starts here:
+    // End of search list
+    //   let's hope this does not change too quickly, otherwise we need
+    // to adjust our search code (for several versions ...)
+    bool bStart = false;
+    for(int idxCount = 0; idxCount < nCount; ++idxCount)
     {
-        // let's construct the command
-        wxString Command = cpp_compiler + _T(" -v -E -x c++ ") + DummyFileName;
-        // action time  (everything shows up on the error stream
-        wxArrayString Output, Errors;
-        wxExecute(Command, Output, Errors, wxEXEC_NODISABLE);
-        int nCount = Errors.GetCount();
-        // the include dir (1 per line) show up between the lines
-        // #include <...> search starts here:
-        // End of search list
-        //   let's hope this does not change too quickly, otherwise we need
-        // to adjust our search code (for several versions ...)
-        bool bStart = false;
-        for(int idxCount = 0; idxCount < nCount; ++idxCount)
+        if (!bStart && Errors[idxCount] == _("#include <...> search starts here:"))
         {
-            if (!bStart && Errors[idxCount] == _("#include <...> search starts here:"))
+            bStart = true;
+        }
+        else if (bStart && Errors[idxCount] == _("End of search list."))
+        {
+            bStart = false; // could jump out of for loop if we want
+        }
+        else if (bStart)
+        {
+//                Manager::Get()->GetLogManager()->DebugLog("include dir " + Errors[idxCount]);
+            // get rid of the leading space (more general : any whitespace)in front
+            wxRegEx reg(_T("^[ \t]*(.*)"));
+            if(reg.Matches(Errors[idxCount]))
             {
-                bStart = true;
-            }
-            else if (bStart && Errors[idxCount] == _("End of search list."))
-            {
-                bStart = false; // could jump out of for loop if we want
-            }
-            else if (bStart)
-            {
-//                Manager::Get()->GetLogManager()->DebugLog("include dir " + Errors[idxCount]);
-                // get rid of the leading space (more general : any whitespace)in front
-                wxRegEx reg(_T("^[ \t]*(.*)"));
-                if(reg.Matches(Errors[idxCount]))
+                wxString out = reg.GetMatch(Errors[idxCount], 1);
+                if(!out.IsEmpty())
                 {
-                    wxString out = reg.GetMatch(Errors[idxCount], 1);
-                    if(!out.IsEmpty())
+                    wxFileName dir(out);
+                    if (NormalizePath(dir,base))
                     {
-                        wxFileName dir(out);
-                        if (NormalizePath(dir,base))
-                        {
-                            Manager::Get()->GetLogManager()->DebugLog(_T("Caching GCC dir: ") + dir.GetFullPath());
-                            gcc_compiler_dirs.Add(dir.GetFullPath());
-                        }
-                        else
-                        #if wxCHECK_VERSION(2, 9, 0)
-                            Manager::Get()->GetLogManager()->DebugLog(F(_T("Error normalizing path: '%s' from '%s'"),out.wx_str(),base.wx_str()));
-                        #else
-                            Manager::Get()->GetLogManager()->DebugLog(F(_T("Error normalizing path: '%s' from '%s'"),out.c_str(),base.c_str()));
-                        #endif
+                        Manager::Get()->GetLogManager()->DebugLog(_T("Caching GCC dir: ") + dir.GetFullPath());
+                        gcc_compiler_dirs.Add(dir.GetFullPath());
                     }
+                    else
+                    #if wxCHECK_VERSION(2, 9, 0)
+                        Manager::Get()->GetLogManager()->DebugLog(F(_T("Error normalizing path: '%s' from '%s'"),out.wx_str(),base.wx_str()));
+                    #else
+                        Manager::Get()->GetLogManager()->DebugLog(F(_T("Error normalizing path: '%s' from '%s'"),out.c_str(),base.c_str()));
+                    #endif
                 }
             }
-        } // end for : idx : idxCount
-        // clean up our temp file
-        ::wxRemoveFile(DummyFileName);
-    } // Dummy is open
+        }
+    } // end for : idx : idxCount
 
+
+
     return gcc_compiler_dirs;
 }
 
@@ -1209,11 +1217,16 @@
                     #endif
                     {
                         case ']':
-                        case ')': ++nest; break;
+                        case ')': ++nest; --x;break;
 
                         case '[':
-                        case '(': --nest; break;
+                        case '(': --nest; --x;break;
+
                     }
+                    while ((x >= 0) && (line.GetChar(x) == ' ' || line.GetChar(x) == '\t'))
+                        --x;
+                    if ((x >= 0) && (line.GetChar(x) == ')' || line.GetChar(x) == ']'))
+                        ++nest;
                 }
                 if ((x > 0) && (wxIsalnum(line.GetChar(x - 1)) || line.GetChar(x - 1) == '_'))
                     --x;
@@ -1268,7 +1281,7 @@
         is_function = line.GetChar(startAt) == '(';
 
         ++nest;
-        while (startAt < line.Length() - 1 && nest != 0)
+        while (startAt < line.Length()-1 && nest != 0)
         {
             ++startAt;
             #if wxCHECK_VERSION(2, 9, 0)
@@ -1278,13 +1291,18 @@
             #endif
             {
                 case ']':
-                case ')': --nest; break;
+                case ')': --nest; ++startAt;break;
 
                 case '[':
-                case '(': ++nest; break;
+                case '(': ++nest; ++startAt;break;
             }
+            while (startAt < line.Length()-1&& (line.GetChar(startAt) == ' ' || line.GetChar(startAt) == '\t'))
+                        ++startAt;
+            if (startAt < line.Length()-1 && (line.GetChar(startAt) == '(' || line.GetChar(startAt) == '['))
+                        ++nest;
         }
-        ++startAt;
+        //++startAt;
+
     }
 
     //Manager::Get()->GetLogManager()->DebugLog("Return at %d (%c): res=%s", startAt, line.GetChar(startAt), res.c_str());
@@ -1412,6 +1430,8 @@
     static wxString cached_search;
     static size_t cached_results_count = 0;
 
+
+
     // early-out opportunity
     // if the user starts typing a token that in our last search had 0 results,
     // and we see that he's continuing typing for that same token,
@@ -1522,6 +1542,7 @@
     }
 
     cached_editor = editor;
+    if (result.size() || (m_EditorEndWord - m_EditorStartWord))
     cached_editor_start_word = m_EditorStartWord;
     cached_search = actual;
     cached_results_count = result.size();


Note: this patch, the first part was using a null file to get the GCC include path(this avoid creating a dummy file)

here is the screen shot of codecompletion:

(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2009-09-28091423.png)

I'd like you test it and comments are welcome! thanks.
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 28, 2009, 04:17:21 am
Hello, using latest SVN build (and probably not only latest), I am having problem with code completion for variables declared using "struct" keyword. For example:

Code
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main(void)
{
struct tm *current;
        current->
}

I dont get any suggestions when typing "current->" and also nothing when trying to complete "curr". It works as expected without keyword struct in declaration.
I tried to find symbol current using Code completion debug tool, but i get only empty list with information, that were found Multiple matches and i must choose :P


I can confirm this bug.
The struct tm is correctly by the parser.

But this code works.

Code
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main(void)
{
    struct tm current;
    current.
}


auto completion list can be shown after the dot.


Edit
Quote
I tried to find symbol current using Code completion debug tool, but i get only empty list with information,

This is because the variable "current" is not in the global token tree.
It is just in temporary token tree in the scope of "main".
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 28, 2009, 04:28:21 am
continue the discussion of reply # 55

I just test this code:

Code
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

struct tm * current;

int main(void)
{

    current->
}
But I can't find "current" in the global token tree. So, I believe there is something wrong with parsing the statement:



Code
struct tm * current;

Need time to find the bug...
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 28, 2009, 05:20:57 am
Hello, using latest SVN build (and probably not only latest), I am having problem with code completion for variables declared using "struct" keyword.

This bug is fixed by this patch

see the new patch on this post

http://forums.codeblocks.org/index.php?topic=11187.msg76648#msg76648

here is the test code:

Code
struct tm
{
int tm_sec;
int b;

};

struct tm *current;

int main(void)
{

    current->
}

and this is the screenshot:

(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2009-09-28111002.png)
Title: Re: New code completion remarks/issues
Post by: mmkider on September 28, 2009, 05:25:29 am
I Found a crash, See picture.
At last, I need keyin '\n' but I keyin '\'. it will crash.


[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 28, 2009, 07:40:41 am
I Found a crash, See picture.
At last, I need keyin '\n' but I keyin '\'. it will crash.


I can't reproduce the crash when entering only one statements, can you give a simple testing code?
Title: Re: New code completion remarks/issues
Post by: mmkider on September 28, 2009, 08:39:13 am
I Found a crash, See picture.
At last, I need keyin '\n' but I keyin '\'. it will crash.


I can't reproduce the crash when entering only one statements, can you give a simple testing code?

Code
static wxRegEx rePrint(_T("^(@_@)(.*)"));
void  Test_CodeCompletion()
{
static wxString buffer;
wxArrayString lines = GetArrayFromString(buffer, _T('\n'));
bool bMatch=false;
for (unsigned int i = 0; i < lines.GetCount(); ++i)
{
if (rePrint.Matches(lines[i]))
{

lines[i]=_T("");
bMatch=true;
}
}
if(bMatch)
{
buffer.clear();
for (unsigned int i = 0; i < lines.GetCount(); ++i)
{
if(lines[i]!=_T(""))
{
buffer+=lines[i]+_T("\n"); // this line will be crash.
}
}
}
        return; // come back later
}

Step 1: keyin "buffer"
Step 2: keyin "+"
Step 3: keyin "="
Step 4: keyin "lines"
Step 5: keyin "["
Step 6: keyin "]"
Step 7: keyin "i"
Step 8: keyin "+"
Step 9: keyin "_"
Step 10: keyin "T"
Step 11: keyin "()"
Step 12: keyin "\"\""
Step 13: keyin "\\"  => crash
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 28, 2009, 08:49:50 am
I follow you steps, but still can't reproduce this crash.

By the way, if you enter these statement:

Code
buffer+=lines[i]+_T("\");

You will notice that the open brace and the close brace have the different colors. I guess this is caused by Scintilla gramma highlight.
Title: Re: New code completion remarks/issues
Post by: mmkider on September 28, 2009, 08:56:07 am
I follow you steps, but still can't reproduce this crash.

By the way, if you enter these statement:

Code
buffer+=lines[i]+_T("\");

You will notice that the open brace and the close brace have the different colors. I guess this is caused by Scintilla gramma highlight.
you need create project from  wx project template.
I try this function in the normal C++ project, it will run fine.
But I try this function in the wx project, it still give me  a crash after Popup "(x)".

Title: Re: New code completion remarks/issues
Post by: ollydbg on September 28, 2009, 09:09:14 am
@mmkider
I just create a wx project by wizard, and still can't reproduce crash.
In my test, I have no completion list like "(x)".

Which version of codeblocks did you use? For me, I use latest svn + my own patch. :D
Title: Re: New code completion remarks/issues
Post by: Jenna on September 28, 2009, 09:16:15 am
I can more or less confirm this crash on linux.
It does not crash, but it hangs and eats 100 % of one of my CPU's.

Latest svn-source.
I will look into it.
Title: Re: New code completion remarks/issues
Post by: mmkider on September 28, 2009, 09:19:10 am
I can more or less confirm this crash on linux.
It does not crash, but it hangs and eats 100 % of one of my CPU's.

Latest svn-source.
I will look into it.
Cool.  :D

I get 100 % of one of my CPU's too, and my codeblocks can not respond to any key.

Title: Re: New code completion remarks/issues
Post by: blueshake on September 28, 2009, 10:12:33 am
Quote
This bug is fixed by this patch
does it work for codes like these?
Code
#include <iostream>

using namespace std;
struct qq
{
    int x;
    int y;
};
struct qq tt;
struct qq& xx = tt;
xx.
int main()
{
    cout << "Hello world!" << endl;
    return 0;
}



EDIT:
And the issue in this thread seems not solved yet.http://forums.codeblocks.org/index.php/topic,10966.msg74954.html#msg74954 (http://forums.codeblocks.org/index.php/topic,10966.msg74954.html#msg74954)
Title: Re: New code completion remarks/issues
Post by: blueshake on September 28, 2009, 10:18:25 am
Quote
I get 100 % of one of my CPU's too, and my codeblocks can not respond to any key.
I have this issue too.The programe hang,but 50% of CPU used.

EDIT:
It seems that (x) tip cause this.
Title: Re: New code completion remarks/issues
Post by: Jenna on September 28, 2009, 10:32:24 am
I can more or less confirm this crash on linux.
It does not crash, but it hangs and eats 100 % of one of my CPU's.

Latest svn-source.
I will look into it.
Cool.  :D

I get 100 % of one of my CPU's too, and my codeblocks can not respond to any key.



Should be fixed in svn r5825.

By the way:
we should have a look at all occurrences of MoveToNextChar(), because testing for EOF does not make sense after this function. At EOF it returns false and sets the token-index to bufferlen (this was the cause for the endless-loop in this case).

Title: Re: New code completion remarks/issues
Post by: Jenna on September 28, 2009, 10:37:13 am
Quote
I get 100 % of one of my CPU's too, and my codeblocks can not respond to any key.
I have this issue too.The programe hang,but 50% of CPU used.

EDIT:
It seems that (x) tip cause this.

That's partly correct, the issue does not occur, if I cancel the (x)-tip with escape, but it hangs in in SkipToOneOfChars() because we never reach EOF and so never break out of the while-loop.
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 28, 2009, 10:39:05 am
Quote
This bug is fixed by this patch
does it work for codes like these?
Code
#include <iostream>

using namespace std;
struct qq
{
    int x;
    int y;
};
struct qq tt;
struct qq& xx = tt;
xx.
int main()
{
    cout << "Hello world!" << endl;
    return 0;
}

Not yet, but the statement below are nearly the same. so, it can be easily fixed.

Code
struct qq * xx
struct qq & xx

Here is my modified path, it works ok for both struct reference and pointer .

Code
Index: parserthread.cpp
===================================================================
--- parserthread.cpp (revision 5824)
+++ parserthread.cpp (working copy)
@@ -1350,8 +1350,13 @@
                 HandleFunction(current);
                 break;
             }
-            else if (next.GetChar(0) != '*')
+            else if (next.GetChar(0) == '*' || next.GetChar(0) == '&' )
             {
+                m_Str<<current;
+                break;
+            }
+            else
+            {
                 // might be instantiation, see the following
                 /*
                 struct HiddenStruct {
@@ -1391,8 +1396,7 @@
                     }
                 }
             }
-            else
-                m_Tokenizer.GetToken();
+
         }
         else
             break;



Title: Re: New code completion remarks/issues
Post by: ollydbg on September 29, 2009, 10:37:11 am
EDIT:
And the issue in this thread seems not solved yet.http://forums.codeblocks.org/index.php/topic,10966.msg74954.html#msg74954 (http://forums.codeblocks.org/index.php/topic,10966.msg74954.html#msg74954)

Yes, this issue still exist when enter

Code
while ().

This is because a left search start from the "." will skip the brace pair, and find "while" as the first token.

But it is strange, it seems the right search starting from "w" give the wrong token length. I will look into it.
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 29, 2009, 11:07:14 am
This is the modified version of the "right search", it just solve this problem in my previous reply.

Code
wxString NativeParser::GetNextCCToken(const wxString& line, unsigned int& startAt, bool& is_function)
{
    wxString res;
    int nest = 0;
    if ((startAt < line.Length()) && (line.GetChar(startAt) == '('))
    {
        while (startAt < line.Length() &&
                (line.GetChar(startAt) == '*' ||
                line.GetChar(startAt) == '&' ||
                line.GetChar(startAt) == '('))
        {
            if (line.GetChar(startAt) == '(')
                ++nest;
            ++startAt;
        }
    }
    //Manager::Get()->GetLogManager()->DebugLog(F(_T("at %d (%c): res=%s"), startAt, line.GetChar(startAt), res.c_str()));
    while ((startAt < line.Length()) && (wxIsalnum(line.GetChar(startAt)) || line.GetChar(startAt) == '_'))
    {
        res << line.GetChar(startAt);
        ++startAt;
    }
    while ((nest > 0) && (startAt < line.Length()))
    {
        if (line.GetChar(startAt) == ')')
            --nest;
        ++startAt;
    }
    //Manager::Get()->GetLogManager()->DebugLog(F(_T("Done nest: at %d (%c): res=%s"), startAt, line.GetChar(startAt), res.c_str()));
    while (startAt < line.Length()-1&& (line.GetChar(startAt) == ' ' || line.GetChar(startAt) == '\t'))
        ++startAt;
    if ((startAt < line.Length()) && (line.GetChar(startAt) == '(' || line.GetChar(startAt) == '['))
    {
        is_function = line.GetChar(startAt) == '(';

        ++nest;
        while (startAt < line.Length()-1 && nest != 0)
        {
            ++startAt;
            #if wxCHECK_VERSION(2, 9, 0)
            switch (line.GetChar(startAt).GetValue())
            #else
            switch (line.GetChar(startAt))
            #endif
            {
                case ']':
                case ')': --nest; ++startAt;break;

                case '[':
                case '(': ++nest; ++startAt;break;
            }
            while (startAt < line.Length()-1&& (line.GetChar(startAt) == ' ' || line.GetChar(startAt) == '\t'))
                        ++startAt;
            if (startAt < line.Length()-1 && (line.GetChar(startAt) == '(' || line.GetChar(startAt) == '['))
                        ++nest;
        }
        //++startAt;

    }

    //Manager::Get()->GetLogManager()->DebugLog("Return at %d (%c): res=%s", startAt, line.GetChar(startAt), res.c_str());
    return res;
}


Note, I just add a while statement to skip the spaces between "while" and "()".

Code
    while (startAt < line.Length()-1&& (line.GetChar(startAt) == ' ' || line.GetChar(startAt) == '\t'))
        ++startAt;

Title: Re: New code completion remarks/issues
Post by: MortenMacFly on September 29, 2009, 05:51:50 pm
This is the modified version of the "right search", it just solve this problem in my previous reply.
Committed that to trunk (including some other mods). Hence some cases still do not work, check this snippet:
Code
struct _s
{
  int   x;
  float y;
};
typedef struct _s t_s;

int main()
{
    struct _s  s;
    struct _s& rs = s;
    struct _s* ps = &s;
    t_s        ts;
    t_s&       rts = ts;
    t_s*       pts = &ts;

    s.    // works
    rs.   // works
    ps.   // works
    ps->  // does not work
    ts.   // works
    rts.  // works
    pts.  // works
    pts-> // does not work

    return 0;
}
For ps and pts CC suggests... erm... ps and pts.
Probably I am missing something... it's late... ;-)
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on September 29, 2009, 07:38:16 pm
By the way are there any plans to enable the first combo box in the CC toolbar?
If yes, what needs to be done?
Title: Re: New code completion remarks/issues
Post by: killerbot on September 29, 2009, 08:36:39 pm
the intended purpose of the first combo box, is that you can filter on what you want to see in the right combo box.

For example you filter on a namespace,  class, a class in a namespace.

Basically, my idea was inspired about how it works in M$ Visual Studio.
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on September 29, 2009, 11:23:36 pm
I know the purpose of the combo,
but at the moment it is disabled and useless (wasting lots of gui-estate(place or something)).

By the way one combo with autocompletion/filtering (visual assist style) is better than two (visual c++ style).
Because it requires less actions to find what you want. But I think the combo in wx2.x can't do autocompletion easily :(

My question was, if you plan to make it work, though?
Title: Re: New code completion remarks/issues
Post by: killerbot on September 29, 2009, 11:42:31 pm
Back in the day I could not activate it yet, since the parser was not good enough.
Now it however still has some informative use : it shows the namespace::class, where the right hand side combobox only show the method names (+ arguments + return value) which it should.

But I think the left combobox could be reduced by half in width.
Title: Re: New code completion remarks/issues
Post by: ollydbg on September 30, 2009, 04:11:20 am
This is the modified version of the "right search", it just solve this problem in my previous reply.
Committed that to trunk (including some other mods). Hence some cases still do not work, check this snippet:
Code
struct _s
{
  int   x;
  float y;
};
typedef struct _s t_s;

int main()
{
    struct _s  s;
    struct _s& rs = s;
    struct _s* ps = &s;
    t_s        ts;
    t_s&       rts = ts;
    t_s*       pts = &ts;

    s.    // works
    rs.   // works
    ps.   // works
    ps->  // does not work
    ts.   // works
    rts.  // works
    pts.  // works
    pts-> // does not work

    return 0;
}
For ps and pts CC suggests... erm... ps and pts.
Probably I am missing something... it's late... ;-)


This bug can be fixed by this patch:

Code
Index: nativeparser.cpp
===================================================================
--- nativeparser.cpp (revision 5826)
+++ nativeparser.cpp (working copy)
@@ -1207,7 +1207,7 @@
         --startAt;
     return startAt;
 }
-static bool IsOperator(int startAt, const wxString& line)
+static bool IsOperatorEnd(int startAt, const wxString& line)
 {
     return (   (startAt > 0)
             && (startAt < line.Length())
@@ -1216,6 +1216,17 @@
                 || (   (line.GetChar(startAt) == ':')
                     && (line.GetChar(startAt - 1) == ':') ) ) );
 }
+
+static bool IsOperatorBegin(int startAt, const wxString& line)
+{
+    return (   (startAt > 0)
+            && (startAt + 1< line.Length())
+            && (   (   (line.GetChar(startAt ) == '-') )
+                    && (line.GetChar(startAt + 1) == '>')
+                || (   (line.GetChar(startAt) == ':')
+                    && (line.GetChar(startAt + 1) == ':') ) ) );
+}
+
 static bool IsOperatorDot(int startAt, const wxString& line)
 {
     return (   (startAt >= 0)
@@ -1274,7 +1285,7 @@
         }
         // Check for [Class]-> ('>' pressed)
         // Check for [Class]:: (':' pressed)
-        else if (IsOperator(startAt, line))
+        else if (IsOperatorEnd(startAt, line))
         {
             startAt -= 2;
             repeat = true; // yes -> repeat.
@@ -1394,7 +1405,10 @@
                 ++nest;
         }
     }
+    if (IsOperatorBegin(startAt, line))
+        ++startAt;
 
+
 #if DEBUG_CC_AI
     Manager::Get()->GetLogManager()->DebugLog(F(_T("Return at %d (%c): res=%s"), startAt, line.GetChar(startAt), res.c_str()));
 #endif
@@ -1446,7 +1460,7 @@
         }
         // Check for [Class]-> ('>' pressed)
         // Check for [Class]:: (':' pressed)
-        else if (IsOperator(startAt, line))
+        else if (IsOperatorEnd(startAt, line))
         {
             if (line.GetChar(startAt) == ':')
                 tokenType = pttNamespace;

Note: for checking IsOperator, we need to check whether "it is the beginning of the operator?" or "the end of the operator", because sometimes, we use a "left search" and sometimes we need a "right search".
Title: Re: New code completion remarks/issues
Post by: blueshake on October 02, 2009, 08:51:14 am
May someboby notice that the cc suggestion list window style is different from the notepad++ whose window
stytle is something like pop menu.So if I want to change the cc suggestion list window style to make it similar with notepad++'s,what should I do?(change what codes, I read the codes,found nothing)
any comment? Thanks!
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 03, 2009, 09:03:28 am
I find another issue in the current cc.

If you open the "plugins\debuggergdb\gdb_commands.h"

You will notice that in the CC's toolbar, only one function (wxStrHexTo) is listed. it seems the parser failed parsing this file.  :(

I haven't found the reason....
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 03, 2009, 10:18:08 am
hi, all.

It seems the parser failed parsing this statement in the gdb_commands.h. (comment this statement can let the parser works ok :D)

Code
static wxRegEx rePendingBreakpoint(_T("Breakpoint ([0-9]+)[ \\t]\\(\\\"(.+):([0-9]+)\\)[ \\t]pending\\."));
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 03, 2009, 03:20:29 pm
May someboby notice that the cc suggestion list window style is different from the notepad++ whose window
stytle is something like pop menu.So if I want to change the cc suggestion list window style to make it similar with notepad++'s,what should I do?(change what codes, I read the codes,found nothing)
any comment? Thanks!

I compare the suggestion list window style of notepad++ 5.4 and codeblocks trunk. They are nearly the same. :D

Can you supply a screenshot that their windows are different??
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 03, 2009, 03:53:23 pm
hi, all.

It seems the parser failed parsing this statement in the gdb_commands.h. (comment this statement can let the parser works ok :D)

Code
static wxRegEx rePendingBreakpoint(_T("Breakpoint ([0-9]+)[ \\t]\\(\\\"(.+):([0-9]+)\\)[ \\t]pending\\."));

One catch: it seems the parser failed parsing statement below:

Code
char * aaa = "\\\"";


Title: Re: New code completion remarks/issues
Post by: ollydbg on October 04, 2009, 01:37:05 am
The problem is located in this function in Tokenizer.cpp
Code
bool Tokenizer::SkipToCharBreak()
{
  if (PreviousChar() != '\\')
      return true;
  else
  {
      // check for "\\"
      if (   ((m_TokenIndex - 2) >= 0)
          && ((m_TokenIndex - 2) <= m_BufferLen)
          && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\') )
          return true;
  }
  return false;
}

The main task of this function is try to check the current char is in a "C escape character".

For example, in the statement below:

Code
char * aaa = "\\\"";

There are three quots in the statement above. the first quot and third quot is the start and the end of a C string definition. But the second quot is after a back slash \, is part of an escape character.

The parser starts the search from the first quot, and try to find the end of the C string. When it meets the second quot, This function try to check whether the second quot is the end of the C string.

But this function mistakenly regard the second quot as the end of the C string.

Also, this function checks several characters before the second quot, but that's not enough. In this statement
Code
"\\\""
, the first two back slash
Code
\\
is an escape character, and the third and forth
Code
\"
groups another escape character.

So, how about parsing this statement?
Code
"\\\\\""
or
"\\\\\\\""

I think we should left search from the second quot and count the back slash number until we meet the first non-back-slash char. and check the number is odd or even. :D

Any comments?
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 04, 2009, 02:32:12 am
here is the patch to solve this problem. :D

Code
Index: tokenizer.cpp
===================================================================
--- tokenizer.cpp (revision 5834)
+++ tokenizer.cpp (working copy)
@@ -216,17 +216,22 @@
 
 bool Tokenizer::SkipToCharBreak()
 {
-  if (PreviousChar() != '\\')
-      return true;
-  else
-  {
-      // check for "\\"
-      if (   ((m_TokenIndex - 2) >= 0)
-          && ((m_TokenIndex - 2) <= m_BufferLen)
-          && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\') )
-          return true;
-  }
-  return false;
+    if (PreviousChar() != '\\')
+        return true;
+    else
+    {
+        // check for "\\"
+        unsigned int numBackslash = 2;
+        while(   ((m_TokenIndex - numBackslash) >= 0)
+            && ((m_TokenIndex - numBackslash) <= m_BufferLen)
+            && (m_Buffer.GetChar(m_TokenIndex - numBackslash) == '\\') )
+            ++numBackslash;
+        if(numBackslash%2==1) // number of back slash(include current char ) is even
+            return true;     // eg: "\""
+        else
+            return false;    // eg: "\\""
+    }
+    return false;
 }


@ marton

I think the function name "SkipToCharBreak" is confusion, because it doesn't really do a skip.

So, I suggest using another name like :  IsRealCharBreak or IsCharBreak or IsEscapeChar

Thanks.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 04, 2009, 05:58:58 am
Because my slow Internet,it seems a lot of messages were not caughted. :D
In this threadhttp://forums.codeblocks.org/index.php/topic,10990.msg75079.html#msg75079 (http://forums.codeblocks.org/index.php/topic,10990.msg75079.html#msg75079),the patch I provide is some kind of mistake.So please change it back.I test the codes,it can work under tktypedef,there are so many codes connect with tktypedef,it is risky to do this (apply this patch.)Thanks.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 04, 2009, 06:42:31 am
Because my slow Internet,it seems a lot of messages were not caughted. :D
In this threadhttp://forums.codeblocks.org/index.php/topic,10990.msg75079.html#msg75079 (http://forums.codeblocks.org/index.php/topic,10990.msg75079.html#msg75079),the patch I provide is some kind of mistake.So please change it back.I test the codes,it can work under tktypedef,there are so many codes connect with tktypedef,it is risky to do this (apply this patch.)Thanks.

hi, blueshake. Here is the test code

Code
class sStruct
{
  SomeClass* some_class;
  int some_int;
  void* some_void;
};
typedef class sStruct tStruct;

int main()
{
    tStruct pp;
    pp.
    cout << "Hello world!" << endl;
    return 0;
}

And In my working copy of trunk, code completion works after "pp.".
Why do you want to reverse your patch?

You mean the patch below should be reversed?
Code
Index: parserthread.cpp
===================================================================
--- parserthread.cpp (revision 5730)
+++ parserthread.cpp (working copy)


@@ -1707,15 +1737,60 @@
 #if PARSER_DEBUG_OUTPUT
     Manager::Get()->GetLogManager()->DebugLog(F(_("Adding typedef: name='%s', ancestor='%s'"), components.front().c_str(), ancestor.c_str()));
 #endif
-    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);

+    Token* tdef = DoAddToken(tkClass, components.front(), lineNr, 0, 0, args);


Title: Re: New code completion remarks/issues
Post by: blueshake on October 04, 2009, 06:55:22 am
@ollydbg
The test codes work well under tktypedef now.But it did not work before.Maybe some bugs have been fixeed.The reason I want to change it back to tktypedef is that there are some codes are connected with tktypedef.For example:In the nativeparser.cpp
Code
    if (local_result.size() == 1)
    {
        int id = *local_result.begin();
        Token* token = tree->at(id);

        if (token->m_TokenKind == tkTypedef)
        {
            std::queue<ParserComponent> type_components;
            BreakUpComponents(parser, token->m_ActualType, type_components);

            while(!components.empty())
            {
                ParserComponent comp = components.front();
                components.pop();
                type_components.push(comp);
            }

    #ifdef DEBUG_CC_AI
            if (s_DebugSmartSense)
            #if wxCHECK_VERSION(2, 9, 0)
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.wx_str(), token->m_ActualType.wx_str()));
            #else
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.c_str(), token->m_ActualType.c_str()));
            #endif
    #endif
            return FindAIMatches(parser, type_components, result, parentTokenIdx, noPartialMatch, caseSensitive, use_inheritance, kindMask, search_scope);
        }

    }

I think it is too risky to  change tktypedef to tkclass. :D
Title: Re: New code completion remarks/issues
Post by: blueshake on October 04, 2009, 07:27:19 am
Ok, tiny fixed with ReadClsNames in parserthread.
for broken codes below:
Quote
#include <iostream>

using namespace std;
typedef class qq
{
    int x;
    int y;

}xx
int main()
{
    cout << "Hello world!" << endl;
    return 0;
}

before : the parser will regard xx, int ,main as class.
now: only the main (refer to visual assist ,this is what it do).So I think it will be a better way to do this.
patch:
Code
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5825)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1768,7 +1768,7 @@
     Manager::Get()->GetLogManager()->DebugLog(F(_("HandleTypedef() : Adding typedef: name='%s', ancestor='%s'"), components.front().c_str(), ancestor.c_str()));
 #endif
 //    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);
-    Token* tdef = DoAddToken(tkClass, components.front(), lineNr, 0, 0, args);
+    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);
     if (tdef)
     {
         if (!is_function_pointer)
@@ -1796,7 +1796,7 @@
                 m_Tokenizer.UngetToken();
                 break;
             }
-        else if (wxIsalpha(current.GetChar(0)))
+        else if (wxIsalpha(current.GetChar(0)) && (m_Tokenizer.PeekToken() == ParserConsts::semicolon || m_Tokenizer.PeekToken() == ParserConsts::comma))
         {
 #if PARSERTHREAD_DEBUG_OUTPUT
             Manager::Get()->GetLogManager()->DebugLog(F(_T("ReadClsNames() : Adding variable '%s' as '%s' to '%s'"), current.c_str(), m_Str.c_str(), (m_pLastParent?m_pLastParent->m_Name.c_str():_T("<no-parent>"))));
@@ -1811,7 +1811,10 @@
             }
 
         }
-        else // unexpected
+        else// unexpected
+        {
+            m_Tokenizer.UngetToken();
             break;
+        }
     }
 }
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 04, 2009, 08:42:03 am
Ok, tiny fixed with ReadClsNames in parserthread.
for broken codes below:
Quote
#include <iostream>

using namespace std;
typedef class qq
{
    int x;
    int y;

}xx
int main()
{
    cout << "Hello world!" << endl;
    return 0;
}



Note:This code can't be compiled in G++.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 04, 2009, 09:19:17 am
Exactly,but I refer to visual assist,its parser do what the patch do. :D
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 04, 2009, 09:49:29 am
Exactly,but I refer to visual assist,its parser do what the patch do. :D
I have applied your patch in my local copy, it works. :D. Thanks for your effort!

Title: Re: New code completion remarks/issues
Post by: blueshake on October 04, 2009, 01:27:20 pm
It seems
Code
Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);
can not work for the test codes below.
Code
#include <iostream>
#include <iostream>
using namespace std;
std::vector<string> ssss;
#define  xxxxxxxxx 12e
typedef class qq
{
    int x;
    int y;
}eeee;
typedef unsigned int wwwwwwwww;
www|
int main()
{

    cout << "Hello world!" << endl;
    return 0;
}

In the "|" position , the suggestion list flash and can not show.If I comment the statement
Code
typedef unsigned int wwwwwwwww;
,it works again.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 04, 2009, 01:40:17 pm
@blueshake
That's something related in the nativeparser.cpp, and the code you cited. I will exam that. :D
Title: Re: New code completion remarks/issues
Post by: blueshake on October 04, 2009, 02:10:06 pm
@blueshake
That's something related in the nativeparser.cpp, and the code you cited. I will exam that. :D
Nice to hear that.Thanks. :D
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 04, 2009, 03:01:45 pm
I can only understand a little about the FindAIMatches() function.

For example, your test code:
Code
typedef unsigned int wwwww;

wwww|

You will see in the symbol tree, the "wwwww" is a global "typedef". which is quite different if your code is like below:
Code
typedef class qq
{
    int x;
    int y;
}eeee;
then, "eeee" in the symbol tree is of type "class".


The piece of code in FindAIMatches()
Code
if (local_result.size() == 1)
    {
        int id = *local_result.begin();
        Token* token = tree->at(id);

        if (token->m_TokenKind == tkTypedef)
        {
            std::queue<ParserComponent> type_components;
            BreakUpComponents(parser, token->m_ActualType, type_components);

            while(!components.empty())
            {
                ParserComponent comp = components.front();
                components.pop();
                type_components.push(comp);
            }

    #if DEBUG_CC_AI
            if (s_DebugSmartSense)
            #if wxCHECK_VERSION(2, 9, 0)
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.wx_str(), token->m_ActualType.wx_str()));
            #else
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.c_str(), token->m_ActualType.c_str()));
            #endif
    #endif
            return FindAIMatches(parser, type_components, result, parentTokenIdx, noPartialMatch, caseSensitive, use_inheritance, kindMask, search_scope);
        }

    }

this means, when you only get one match in the current context, then for example, you are entering
Code
www

This time, the only matched string is "wwwww" will be replace with "unsigned int" and the total line will be reparsed. (which means the FindAIMatches will be called iteratively).

 :D

But I think at this test code, this typedef replacement is useless. I can't find a test case that this replacement is meaningful. :(

Can someone explain more about this? Thanks.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 04, 2009, 08:07:42 pm
Guys, I am a bit lost and trying to catch up with all the posts. Am I rioght, that these two patches have to be applied together?

Patch 1:
Code
Index: tokenizer.cpp
===================================================================
--- tokenizer.cpp (revision 5834)
+++ tokenizer.cpp (working copy)
@@ -216,17 +216,22 @@
 
 bool Tokenizer::SkipToCharBreak()
 {
-  if (PreviousChar() != '\\')
-      return true;
-  else
-  {
-      // check for "\\"
-      if (   ((m_TokenIndex - 2) >= 0)
-          && ((m_TokenIndex - 2) <= m_BufferLen)
-          && (m_Buffer.GetChar(m_TokenIndex - 2) == '\\') )
-          return true;
-  }
-  return false;
+    if (PreviousChar() != '\\')
+        return true;
+    else
+    {
+        // check for "\\"
+        unsigned int numBackslash = 2;
+        while(   ((m_TokenIndex - numBackslash) >= 0)
+            && ((m_TokenIndex - numBackslash) <= m_BufferLen)
+            && (m_Buffer.GetChar(m_TokenIndex - numBackslash) == '\\') )
+            ++numBackslash;
+        if(numBackslash%2==1) // number of back slash(include current char ) is even
+            return true;     // eg: "\""
+        else
+            return false;    // eg: "\\""
+    }
+    return false;
 }

Patch 2:
Code
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5825)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1768,7 +1768,7 @@
     Manager::Get()->GetLogManager()->DebugLog(F(_("HandleTypedef() : Adding typedef: name='%s', ancestor='%s'"), components.front().c_str(), ancestor.c_str()));
 #endif
 //    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);
-    Token* tdef = DoAddToken(tkClass, components.front(), lineNr, 0, 0, args);
+    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);
     if (tdef)
     {
         if (!is_function_pointer)
@@ -1796,7 +1796,7 @@
                 m_Tokenizer.UngetToken();
                 break;
             }
-        else if (wxIsalpha(current.GetChar(0)))
+        else if (wxIsalpha(current.GetChar(0)) && (m_Tokenizer.PeekToken() == ParserConsts::semicolon || m_Tokenizer.PeekToken() == ParserConsts::comma))
         {
 #if PARSERTHREAD_DEBUG_OUTPUT
             Manager::Get()->GetLogManager()->DebugLog(F(_T("ReadClsNames() : Adding variable '%s' as '%s' to '%s'"), current.c_str(), m_Str.c_str(), (m_pLastParent?m_pLastParent->m_Name.c_str():_T("<no-parent>"))));
@@ -1811,7 +1811,10 @@
             }
 
         }
-        else // unexpected
+        else// unexpected
+        {
+            m_Tokenizer.UngetToken();
             break;
+        }
     }
 }

???

I think the function name "SkipToCharBreak" is confusion, because it doesn't really do a skip.
So, I suggest using another name like :  IsRealCharBreak or IsCharBreak or IsEscapeChar
That's right and probably was a typo. I'll change that certainly.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 05, 2009, 03:08:00 am
@MortenMacFly
Yes, the two patches can both be applied.

But, blueshake's patch of changing the code :

Code
-    Token* tdef = DoAddToken(tkClass, components.front(), lineNr, 0, 0, args);
+    Token* tdef = DoAddToken(tkTypedef, components.front(), lineNr, 0, 0, args);

should be reconsidered, because a small bug he states in this post:
http://forums.codeblocks.org/index.php/topic,11187.msg76788.html#msg76788
Title: Re: New code completion remarks/issues
Post by: blueshake on October 05, 2009, 05:26:33 am
hello,another bad news.
It seems some bugs hide in the codecompletion.In the file codecompletion.cpp,in this place:
Code
            // Remove duplicate items
            for (size_t i=0; i<items.Count()-1; i++)
                if (items.Item(i)==items.Item(i+1))
                    items.RemoveAt(i);
Manager::ge|
when you type Mana,yes,the suggestion show,then you type ::,yes work again,but when you type ge,the issue come out,the suggestion list flash again,and not show.what is going on here,the suggestion list is try to show,but cancle by something.

Note:
I just use the latest cc,change nothing.
Edit:
Code
Manager::Get()->|
And in the "|" position, the cc not work.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 05, 2009, 05:52:34 am
hello,another bad news.
It seems some bugs hide in the codecompletion.In the file codecompletion.cpp,in this place:
Code
            // Remove duplicate items
            for (size_t i=0; i<items.Count()-1; i++)
                if (items.Item(i)==items.Item(i+1))
                    items.RemoveAt(i);
Manager::ge|
when you type Mana,yes,the suggestion show,then you type ::,yes work again,but when you type ge,the issue come out,the suggestion list flash again,and not show.what is going on here,the suggestion list is try to show,but cancle by something.

Note:
I just use the latest cc,change nothing.
Edit:
Code
Manager::Get()->|
And in the "|" position, the cc not work.

I can confirm both the two bugs.
Is it possible to create a simple test case. So that I can debug it. Currently, Opening the codeblocks's whole source code is not necessary.

Also: I think it's time to create some standard test code for codecompletion now. :D Any comments?

Also 1: @morten, in the current CC code, there are many code using the preprocessor guard.like:

Code
#if PARSERTHREAD_DEBUG_OUTPUT
            Manager::Get()->GetLogManager()->DebugLog(F(_T("HandlePreprocessorBlocks() : Special case \"#if 0\" not skipped.")));
#endif

Is there any way we can just clean up these code, and use something like this:

Code
#if PARSERTHREAD_DEBUG_OUTPUT
#define DebugOutput(a,b,c)  Manager::Get()->GetLogManager()->DebugLog(a,b,c)
#else
#define DebugOutput(a,b,c)  
#endif

So, the the source code, we can only use this:

Code
DebugOutput(F(_T("HandlePreprocessorBlocks() : Special case \"#if 0\" not skipped.")));
This will make the source code more clean.


Also 2:

There are many variable names which are not following the source style guard. like:

Code
    Parser parser(this);
    parser.ParseBufferForFunctions(control->GetTextRange(0, pos));

    wxArrayString funcs;
    TokensTree* tmptree = parser.GetTempTokens();

    // look for implementation functions that enclose our current line number
    for(size_t i = 0; i < tmptree->size();i++)

the tmptree should be tmpTree.

Also, there are many other variables should be renamed. such as:

Code
// find current function's namespace so we can include local scope's tokens
    // we ' ll get the function's token (all matches) and add its parent namespace
    TokenIdxSet scope_result;
    TokenIdxSet proc_result;
    if (FindCurrentFunctionToken(editor, proc_result) != 0)
    {
        for (TokenIdxSet::iterator it = proc_result.begin(); it != proc_result.end(); ++it)
        {
            Token* token = parser->GetTokens()->at(*it);
            if (!token)
                continue;
            scope_result.insert(token->m_ParentIndex);
#if DEBUG_CC_AI
            if (s_DebugSmartSense)
            {
                Token* parent = parser->GetTokens()->at(token->m_ParentIndex);
                Manager::Get()->GetLogManager()->DebugLog(_T("Adding search namespace: ") + (parent ? parent->m_Name : _T("Global namespace")));
            }
#endif
        }
    }

These name should be "scopeResult" and so on...




Title: Re: New code completion remarks/issues
Post by: ollydbg on October 05, 2009, 06:06:39 am
@blueshake:

look at the Get function prototype, it is a static function.
Code
class DLLIMPORT Manager
{
    static Manager* Get();
}
but, in the symbol tree debug dialog, you will see that the
Quote
type is Manager *
actual type is Manager

That's may be the cause of bug...

Title: Re: New code completion remarks/issues
Post by: blueshake on October 05, 2009, 06:36:23 am
@blueshake:

look at the Get function prototype, it is a static function.
Code
class DLLIMPORT Manager
{
    static Manager* Get();
}
but, in the symbol tree debug dialog, you will see that the
Quote
type is Manager *
actual type is Manager

That's may be the cause of bug...


But it worked before.I updated the local copy,and it don't work.
look at these codes.The static are ignored.
Code
        else if (token==ParserConsts::kw_static ||
            token==ParserConsts::kw_virtual ||
            token==ParserConsts::kw_inline)
        {
            // do nothing (skip it)
        }
Title: Re: New code completion remarks/issues
Post by: blueshake on October 05, 2009, 07:08:26 am
anther issue.
Code
std::string ss;
ss. not work
Code
string ss;
ss.work
parser error:
Code
string ss("abc");
ss.
it should be a variable,but ss is parsed as function.

[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 05, 2009, 08:39:38 am
Just a tiny remark: For testing if the current modification (a.k.a patch) does not break something else please make sure that after applying the patch do the following:
1.) Open Code::Blocks having the debug log enabled.
2.) Open CodeBlocks.cbp (project files of core and core plugins)
3.) Wait, until the parser has completed.
3.) Check the debug log for the number of tokens, it should look like the following:
Parsing stage done (1570 total parsed files, 92637 tokens in 1 minute(s), 8.922 seconds).
The tokens should be round about 90000 - 100000 (depending on the patch level of C::B).

Furthermore: I am still happy to collect sample projects for testing CC I already have some...
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 05, 2009, 09:53:43 am
@blueshake:

look at the Get function prototype, it is a static function.
Code
class DLLIMPORT Manager
{
    static Manager* Get();
}
but, in the symbol tree debug dialog, you will see that the
Quote
type is Manager *
actual type is Manager

That's may be the cause of bug...


But it worked before.I updated the local copy,and it don't work.
look at these codes.The static are ignored.
Code
        else if (token==ParserConsts::kw_static ||
            token==ParserConsts::kw_virtual ||
            token==ParserConsts::kw_inline)
        {
            // do nothing (skip it)
        }

Hi, after one and a half hour examing. I find the bug, and the patch to fix it.

Code
Index: nativeparser.cpp
===================================================================
--- nativeparser.cpp (revision 5834)
+++ nativeparser.cpp (working copy)
@@ -1465,7 +1465,7 @@
                 tokenType = pttNamespace;
             else
                 tokenType = pttClass;
-            line.Remove(0, startAt + 2);
+            line.Remove(0, startAt + 1);
         }
         else
             line.Clear();

Here is the explanation:

For the code:
Code
Manager::Get()->
The nativeparser should divide the line to "Manager" and "Get".

The current trunk code just mistakenly divide to "Manager" and "et", so, this patch can solve the bug.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 05, 2009, 10:15:42 am
Cool man.

I just check the codes,found that we used IsOperatorEnd(startAt, line),so this is why patch should be applied.

@ollydbg
watch these codes in nativeparser.cpp
Code
        startAt = BeginOfToken(startAt, line);

        // Check for [Class]. ('.' pressed)
        if (IsOperatorDot(startAt, line))
        {
            --startAt;
            repeat = true; // yes -> repeat.
        }
        // Check for [Class]-> ('>' pressed)
        // Check for [Class]:: (':' pressed)
        else if (IsOperatorEnd(startAt, line))
        {
            startAt -= 2;
            repeat = true; // yes -> repeat.
        }
[/s]

should the startAt -=2;be changed to startAt -=1;.I think the reason is same to the patch. :D
Title: Re: New code completion remarks/issues
Post by: blueshake on October 05, 2009, 10:42:07 am
I can confirm the issues I reported in reply 93,#99 can be fixed by ollydbg's #105 patch,even the tkTypedef can work too.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 05, 2009, 10:46:55 am
I can confirm the issues I reported in reply 93,#99 can be fixed by ollydbg's #105 patch,even the tkTypedef can work too.
I'm so grad to hear that!!!! :lol:
Title: Re: New code completion remarks/issues
Post by: mmkider on October 05, 2009, 12:03:38 pm
Hi
I found a issue.

if stest is a struct;

stest->b. ;   //this is fail. Tip window can't pop up c
stest.b.c  ;  //this  is ok .

Title: Re: New code completion remarks/issues
Post by: blueshake on October 05, 2009, 12:57:41 pm
Hi
I found a issue.

if stest is a struct;

stest->b. ;   //this is fail. Tip window can't pop up c
stest.b.c  ;  //this  is ok .




Full test codes,please.
Title: Re: New code completion remarks/issues
Post by: mmkider on October 05, 2009, 01:30:10 pm
Code

typedef struct U_TKey{
int u32Key;

}U_KEYPAD_TWOKEY;

typedef struct S_KeyPad{
U_KEYPAD_TWOKEY   unTKey;
void *   pParam;
char   u8CallBack_Type;
char  u8DebounceClk;
char  u8IntNum;
char  u8TriggerType;
}S_KEYPAD_CLASS;
It's ok
Code
S_KEYPAD_CLASS  sTest;
sTest.unTKey.u32Key=10;

It's fail.
Code

S_KEYPAD_CLASS  *psTest;
psTest->unTkey.     // can't popup u32Key

Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 05, 2009, 01:56:52 pm
This works fine here:
Code
  S_KEYPAD_CLASS *psTest;
  psTest->unTKey.
("u32Key" will be suggested).

...version, patform etc?
Title: Re: New code completion remarks/issues
Post by: mmkider on October 05, 2009, 02:06:05 pm
This works fine here:
Code
  S_KEYPAD_CLASS *psTest;
  psTest->unTKey.
("u32Key" will be suggested).

...version, patform etc?

I use codeblocks svn 5839 in the WinXp sp2.
In my system, "u32Key" will not be suggested.

Code
S_KEYPAD_CLASS  *psTest;
psTest->unTkey.                      // can't popup u32Key
Title: Re: New code completion remarks/issues
Post by: blueshake on October 05, 2009, 02:14:16 pm
Work here,too.see the attachment.

Sorry to say that,but tktypedef don't work again.ollydbg is right,something is wrong with codes below,if I comment them,worked.!!!!!!!!

Code
        if (token->m_TokenKind == tkTypedef)
        {
            std::queue<ParserComponent> type_components;
            BreakUpComponents(parser, token->m_ActualType, type_components);

            while(!components.empty())
            {
                ParserComponent comp = components.front();
                components.pop();
                type_components.push(comp);
            }

    #if DEBUG_CC_AI
            if (s_DebugSmartSense)
            #if wxCHECK_VERSION(2, 9, 0)
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.wx_str(), token->m_ActualType.wx_str()));
            #else
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.c_str(), token->m_ActualType.c_str()));
            #endif
    #endif
            return FindAIMatches(parser, type_components, result, parentTokenIdx, noPartialMatch, caseSensitive, use_inheritance, kindMask, search_scope);
        }

    }



[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: mmkider on October 05, 2009, 02:33:58 pm
Work here,too.see the attachment.

Sorry to say that,but tktypedef don't work again.ollydbg is right,something is wrong with codes below,if I comment them,worked.!!!!!!!!

Code
        if (token->m_TokenKind == tkTypedef)
        {
            std::queue<ParserComponent> type_components;
            BreakUpComponents(parser, token->m_ActualType, type_components);

            while(!components.empty())
            {
                ParserComponent comp = components.front();
                components.pop();
                type_components.push(comp);
            }

    #if DEBUG_CC_AI
            if (s_DebugSmartSense)
            #if wxCHECK_VERSION(2, 9, 0)
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.wx_str(), token->m_ActualType.wx_str()));
            #else
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Replacing %s to %s"), token->m_Name.c_str(), token->m_ActualType.c_str()));
            #endif
    #endif
            return FindAIMatches(parser, type_components, result, parentTokenIdx, noPartialMatch, caseSensitive, use_inheritance, kindMask, search_scope);
        }

    }



I update src of codeblocks to SVN 5840.
It work well.

Thank all.

Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 05, 2009, 02:54:07 pm
Sorry to say that,but tktypedef don't work again.ollydbg is right,something is wrong with codes below,if I comment them,worked.!!!!!!!!
What does not work exactly? My test cases all work very well...?! Are you using latest SVN code?
Title: Re: New code completion remarks/issues
Post by: mmkider on October 05, 2009, 03:03:13 pm
Sorry to say that,but tktypedef don't work again.ollydbg is right,something is wrong with codes below,if I comment them,worked.!!!!!!!!
What does not work exactly? My test cases all work very well...?! Are you using latest SVN code?

I can't  update all easily.
Because I remain wxflatnotebook in the src of codeblocks.
So I use other folder to update new src of codeblocks  and trandsfer some codes  of  my interest to my old folder.
I thank I do some wrong  thing between the new folder and the old folder :(.

Title: Re: New code completion remarks/issues
Post by: ollydbg on October 05, 2009, 03:17:31 pm
Sorry to say that,but tktypedef don't work again.ollydbg is right,something is wrong with codes below,if I comment them,worked.!!!!!!!!
What does not work exactly? My test cases all work very well...?! Are you using latest SVN code?

@Morten

blueshake's test code in this post:
http://forums.codeblocks.org/index.php/topic,11187.msg76788.html#msg76788

failed when entering "wwww", there is a flash of the suggestion list.
Title: Re: New code completion remarks/issues
Post by: Jenna on October 05, 2009, 03:26:50 pm
failed when entering "wwww", there is a flash of the suggestion list.

CC works for me in this case on linux, but not on windows (immediately close of suggestion window -> flash), but CC believes wwwwwwwww is a class, that's most likely because of the tktypedef-stuff.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 06, 2009, 03:54:58 am
Still can not figure out why this happened.
See the pitcture snap5,the wwwwwwwww value tip is wrong when I use tkTypedef(snap 6).
If I use tkClass,(the pic snap 7) the wwwwwwwww value tip is snap 8.

[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 04:00:17 am
(http://C:\Documents and Settings\Administrator\桌面\Snap5)
See the pitcture,the wwwwwwwww value tip is wrong when I use tkTypedef.
confirmed. that's still something related to the source we mentioned in this thread.

http://forums.codeblocks.org/index.php/topic,11187.msg76830.html#msg76830

I believe comment them will let the suggestion list work ok. :D
Title: Re: New code completion remarks/issues
Post by: blueshake on October 06, 2009, 04:17:04 am
It is strange.I just test the same codes in svn 5731,it worked.



Edit:


I guess maybe the BreakUpComponents function break down something.


[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 04:33:45 am
It is strange.I just test the same codes in svn 5731,it worked.



Edit:


I guess maybe the BreakUpComponents function break down something.

rev 5731 is before the cc_branch is merged. I'm just viewing the log Message of "nativeparser.cpp", and have a comparison with rev5682 of nativeparser.cpp.

Edit:

Both of the rev5844 and rev 5731 give the same token, like your screen shot. so, the main bug was located in the "nativeparser.cpp" :D

(http://forums.codeblocks.org/index.php?action=dlattach;topic=11187.0;attach=3775)
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 05:00:00 am
what I have found is that these code:

Code
  if (local_result.size() == 1)
    {
        int id = *local_result.begin();
        Token* token = tree->at(id);

        if (token->m_TokenKind == tkTypedef)
        {
            std::queue<ParserComponent> type_components;
            BreakUpComponents(parser, token->m_ActualType, type_components);

            while(!components.empty())
            {
                ParserComponent comp = components.front();
                components.pop();
                type_components.push(comp);
            }

#if DEBUG_CC_AI
            if (s_DebugSmartSense)
            #if wxCHECK_VERSION(2, 9, 0)
                Manager::Get()->GetLogManager()->DebugLog(F(_T("FindAIMatches() : Replacing %s to %s"), token->m_Name.wx_str(), token->m_ActualType.wx_str()));
            #else
                Manager::Get()->GetLogManager()->DebugLog(F(_T("FindAIMatches() : Replacing %s to %s"), token->m_Name.c_str(), token->m_ActualType.c_str()));
            #endif
#endif
            return FindAIMatches(parser, type_components, result, parentTokenIdx, noPartialMatch, caseSensitive, use_inheritance, kindMask, search_scope);
        }

    }


Is added from rev 5682 to rev5840 of nativeparser.cpp.

Title: Re: New code completion remarks/issues
Post by: blueshake on October 06, 2009, 05:51:33 am
I just noticed that in the codecompletion.cpp
Code
    if (event.GetEventType() == wxEVT_SCI_CHARADDED &&
            !control->AutoCompActive()) { // not already active autocompletion

They have been changed to
Code
    if (event.GetEventType() == wxEVT_SCI_CHARADDED)

why we do this here,what are the better things?
Title: Re: New code completion remarks/issues
Post by: blueshake on October 06, 2009, 06:03:21 am
what I have found is that these code:

Code
  if (local_result.size() == 1)
    {
        int id = *local_result.begin();
        Token* token = tree->at(id);

        if (token->m_TokenKind == tkTypedef)
        {
            std::queue<ParserComponent> type_components;
            BreakUpComponents(parser, token->m_ActualType, type_components);

            while(!components.empty())
            {
                ParserComponent comp = components.front();
                components.pop();
                type_components.push(comp);
            }

#if DEBUG_CC_AI
            if (s_DebugSmartSense)
            #if wxCHECK_VERSION(2, 9, 0)
                Manager::Get()->GetLogManager()->DebugLog(F(_T("FindAIMatches() : Replacing %s to %s"), token->m_Name.wx_str(), token->m_ActualType.wx_str()));
            #else
                Manager::Get()->GetLogManager()->DebugLog(F(_T("FindAIMatches() : Replacing %s to %s"), token->m_Name.c_str(), token->m_ActualType.c_str()));
            #endif
#endif
            return FindAIMatches(parser, type_components, result, parentTokenIdx, noPartialMatch, caseSensitive, use_inheritance, kindMask, search_scope);
        }

    }


Is added from rev 5682 to rev5840 of nativeparser.cpp.




Can somebody make some explanation about adding these codes here? Thanks.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 09:04:23 am
I would like Morten could give some explanation on the above two reply (reply#125, reply#126).
I also have the same question. :D
Thanks.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 09:27:01 am
Dear Morten,Jens and other devs.
In the post

http://forums.codeblocks.org/index.php/topic,11187.msg76809.html#msg76809
I would like to make the code of DebugLog more clean. Now, I have finally find the way, please see here:

we can define this macro:

Code
#define TOKENIZER_DEBUG_OUTPUT 1

#if TOKENIZER_DEBUG_OUTPUT
    #define TRACE(format, args...)\
    Manager::Get()->GetLogManager()->DebugLog(F( format , ## args))
#else
    #define TRACE(format, args...)
#endif

So, the code can change like below:
Code
#if TOKENIZER_DEBUG_OUTPUT
        Manager::Get()->GetLogManager()->DebugLog(F(_T("Init() : Called without filename.")));
#endif
to
Code
TRACE(_T("Init() : Called without filename."));

Also,

Code
#if TOKENIZER_DEBUG_OUTPUT
        Manager::Get()->GetLogManager()->DebugLog(F(_T("Init() : m_Filename='%s'"), m_Filename.c_str()));
#endif

to

Code
TRACE(_T("Init() : m_Filename='%s'"), m_Filename.c_str());

I have already tested, and it works great! :D

I think all the DebugLog related code can change to this format, which make the code more clean. :)

Any comments? Thanks.



Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 06, 2009, 10:24:50 am
Code
#define TOKENIZER_DEBUG_OUTPUT 1

#if TOKENIZER_DEBUG_OUTPUT
    #define TRACE(format, args...)\
    Manager::Get()->GetLogManager()->DebugLog(F( format , ## args))
#else
    #define TRACE(format, args...)
#endif
Any comments? Thanks.
Sure this looks nice. But keep in mind that this should be in implementation files only, or rename TRACE into a more specific TRACE_NATIVEPARSER or alike. Otherwise you might define the macros also for other files (debug outputs). Patches against trunk are welcome.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 06, 2009, 11:01:58 am
Guys, I've added testing projects (and a workspace) into trunk. If you find functionality to test, feel free to enhance this.

In the CC plugin folder there is now a "testing" sub-folder with a workspace called... erm... TESTING. This will open (so far) 3 projects that test a specific functionality of CC. Ifyou want to enhance this (or provide new tests), please do the following:

- create a new project named like what you are testing (missing is for example "defines", "inheritance" etc.)
- create the files needed *all* having the same prefix like the test
- if you need additional stuff, place it in a sub-folder named like the test.
- provide a main function (named like the test ;-)) and comment lines where a user should do a certain test
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 06, 2009, 11:11:54 am
I would like Morten could give some explanation on the above two reply (reply#125, reply#126).
I can't because I don't recall exactly. This code snippet was part of a patch that enabled parsing of typedefs correctly at all (it was not possible before). Probably it's obsolete now. I am testing CC without this snippet atm (you can do the same, if you like). If I don't see any issues I'll simply remove that portion.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 06, 2009, 01:40:57 pm
In the thread ,http://forums.codeblocks.org/index.php/topic,11187.msg76812.html#msg76812 (http://forums.codeblocks.org/index.php/topic,11187.msg76812.html#msg76812)

for the latest cc,the issue can not be solved.

I believe the std:: int the actualtype cause it.

By the way,how to insert a picture,not as a attachment. :?

[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 01:58:36 pm
By the way,how to insert a picture,not as a attachment. :?
Google: image sharing site
Then choose one, and register an ID, and upload your image, then copy the image link, and add to your post.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 02:05:18 pm
Code
#define TOKENIZER_DEBUG_OUTPUT 1

#if TOKENIZER_DEBUG_OUTPUT
    #define TRACE(format, args...)\
    Manager::Get()->GetLogManager()->DebugLog(F( format , ## args))
#else
    #define TRACE(format, args...)
#endif
Any comments? Thanks.
Sure this looks nice. But keep in mind that this should be in implementation files only, or rename TRACE into a more specific TRACE_NATIVEPARSER or alike. Otherwise you might define the macros also for other files (debug outputs). Patches against trunk are welcome.

Thanks for the hint.
For now, I would prefer still useing the name "TRACE", then, add these macros in every implementation files, such as tokenizer.cpp , parserthread.cpp and nativeparser.cpp.

Patches will be uploaded soon. :D
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 02:07:36 pm
Also, is it possible using the predefined macro name for function name?

Code
__FUNCTION__
or
Code
__PRETTY_FUNCTION__ 

since codeblocks is build under GCC. :D
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 03:31:19 pm
here is the patch of using "TRACE" macro for tokenizer.cpp and parserthread.cpp.




[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 03:50:41 pm
patch for nativeparser.cpp


[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 06, 2009, 04:16:57 pm
I believe the std:: int the actualtype cause it.
Guys, I found the reason for that and it's not easily solvable.

std:: completion worked fine (and still works fine) with GCC3. It is broken with GCC4 and the reason is simple:
Take for example the file vector.tcc (which is the implementation of std::vector in your GCC folder).
In version 3 it was:
Code
#ifndef _VECTOR_TCC
#define _VECTOR_TCC 1

namespace _GLIBCXX_STD
{
  // ...
} // namespace std

#endif /* _VECTOR_TCC */
As we replace "_GLIBCXX_STD" within the parser with "std", std::(vector) was easily resolvable. Now things have changed for GCC4, now the wrapper is as follows:
Code
#ifndef _VECTOR_TCC
#define _VECTOR_TCC 1

_GLIBCXX_BEGIN_NESTED_NAMESPACE(std, _GLIBCXX_STD_D)

  // ...

_GLIBCXX_END_NESTED_NAMESPACE

#endif /* _VECTOR_TCC */
As you see: This is done using a macro. As we don't have macro expansion it won't work with out implementation of CC anymore. So - this is not a bug in CC, "just" a missing feature and it happens now as we all just jumped to GCC4.

Hence I tried replacing:
_GLIBCXX_BEGIN_NESTED_NAMESPACE(std, _GLIBCXX_STD_D)
with
using namespace std;
and:
_GLIBCXX_END_NESTED_NAMESPACE
with "}", but this of course works not, too as the tokenizer has already tokenized the first statement. So the replacement cannot be done anymore. ;-) In addition the macros is not always like that and sometimes appears multiple times / is nested. So replacements have to be done carefully.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 04:27:03 pm
As we replace "_GLIBCXX_STD" within the parser with "std", std::(vector) was easily resolvable. Now things have changed for GCC4, now the wrapper is as follows:
Code
#ifndef _VECTOR_TCC
#define _VECTOR_TCC 1

_GLIBCXX_BEGIN_NESTED_NAMESPACE(std, _GLIBCXX_STD_D)

  // ...

_GLIBCXX_END_NESTED_NAMESPACE

#endif /* _VECTOR_TCC */
As you see: This is done using a macro. As we don't have macro expansion it won't work with out implementation of CC anymore. So - this is not a bug in CC, "just" a missing feature and it happens now as we all just jumped to GCC4.

Hence I tried replacing:
_GLIBCXX_BEGIN_NESTED_NAMESPACE(std, _GLIBCXX_STD_D)
with
using namespace std;
and:
_GLIBCXX_END_NESTED_NAMESPACE
with "empty", but this of course works not, too as the tokenizer has already tokenized the first statement. So the replacement cannot be done anymore. ;-)

For these kind of macros, we can learn from the Ctags.

See this post:

http://forums.codeblocks.org/index.php/topic,10827.msg74127.html#msg74127

We should expand the replacement method in tokenizer.cpp. :D

Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 06, 2009, 04:40:45 pm
We should expand the replacement method in tokenizer.cpp. :D
No. This does not work, because:
Hence I tried replacing:
_GLIBCXX_BEGIN_NESTED_NAMESPACE(std, _GLIBCXX_STD_D)
with
using namespace std;
and:
_GLIBCXX_END_NESTED_NAMESPACE
with "empty", but this of course works not, too as the tokenizer has already tokenized the first statement. So the replacement cannot be done anymore. ;-)
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 05:18:08 pm
We should expand the replacement method in tokenizer.cpp. :D
No. This does not work, because:
Hence I tried replacing:
_GLIBCXX_BEGIN_NESTED_NAMESPACE(std, _GLIBCXX_STD_D)
with
using namespace std;
and:
_GLIBCXX_END_NESTED_NAMESPACE
with "empty", but this of course works not, too as the tokenizer has already tokenized the first statement. So the replacement cannot be done anymore. ;-)

I mean, you can check the token == _GLIBCXX_BEGIN_NESTED_NAMESPACE, if it proves, the tokenizer can eat the following token which is ( "(std, _GLIBCXX_STD_D)"), or do anything you want.

So, what we need to do is "expand the replacement method", which means the tokenizer not only do the replacement, but also do "eat" the next token, or do something else :D
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 06, 2009, 05:24:06 pm
I mean, you can check the token == _GLIBCXX_BEGIN_NESTED_NAMESPACE, if it proves, the tokenizer can eat the following token which is ( "(std, _GLIBCXX_STD_D)"), or do anything you want.
OK - that might work, indeed and would probably be a feasible solution. But: This is very error prone, for sure. If you look deeper into the STL classes the macros contain macros in itself so the order of replacements matters!

However - I did a nasty hack so that std:: completion works again. I am replacing directly within the buffer before the Tokenizer starts any action. This proves that CC would still be able to do the right thing so it's not a bug.
Hence the cost for this is several calls to wxString::Replace which simply takes an awful lot of time. But it was what I could do quickly.

If anybody is interested in a patch, I can so so...?!
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 06, 2009, 05:37:02 pm
I mean, you can check the token == _GLIBCXX_BEGIN_NESTED_NAMESPACE, if it proves, the tokenizer can eat the following token which is ( "(std, _GLIBCXX_STD_D)"), or do anything you want.
OK - that might work, indeed and would probably be a feasible solution. But: This is very error prone, for sure. If you look deeper into the STL classes the macros contain macros in itself so the order of replacements matters!

However - I did a nasty hack so that std:: completion works again. I am replacing directly within the buffer before the Tokenizer starts any action. This proves that CC would still be able to do the right thing so it's not a bug.
Hence the cost for this is several calls to wxString::Replace which simply takes an awful lot of time. But it was what I could do quickly.

If anybody is interested in a patch, I can so so...?!

I really concern the performance of the parsing, if you do this "replacement" in every m_Buffer of Tokenizer.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 06, 2009, 05:41:52 pm
If anybody is interested in a patch, I can so so...?!
Here it is btw...

Edit: Invalid now and not needed anymore, just uncomment the section in trunk.
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on October 06, 2009, 07:47:02 pm
I want to test it, but it failed to apply in the debugger branch... :(
Morten can you sync trunk and debugger branch?
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 06, 2009, 08:05:22 pm
Morten can you sync trunk and debugger branch?
Sure, but not today anymore.
Hence you can surely just checkout the CC plugin from trunk only and do a file comparision/sanc with the debugger branch yourself. The only method you need to take care about is SetSelectionVoid. That should be the only difference to trunk then.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 07, 2009, 03:55:43 am
Hi, here is the patch for tokenizer.cpp and tokenizer.h

I just change some variable name, such as

m_Str in DoGetToken() function changes to str.
Also, I add a function FixArgument to wrap a lot of operations.


[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 07, 2009, 09:16:23 am
Morten can you sync trunk and debugger branch?
Done. Patch is not needed anymore, just uncomment the section in question in the tokenizer.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 07, 2009, 10:04:28 am
For the right valuetip on typedef.
patch:
Code
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5852)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1765,6 +1765,7 @@
         {
             tdef->m_AncestorsString = ancestor;
             tdef->m_ActualType = ancestor;
+            tdef->m_Type = ancestor;
         }
         else
             tdef->m_ActualType = ancestor + args;

[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 07, 2009, 10:05:18 am
I just find a small bug.
When I first only the TESTING workplace, and only the symbol tree. When I double click on the tree item, no jumping was taken.

After I toggle or switch the editor page several times, the jumping works.

So, it seems the "First jump mechanism" was some problem.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 07, 2009, 10:16:52 am
Maybe the parse has not been finished yet. :D
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 07, 2009, 10:17:24 am
For the right valuetip on typedef.
patch:
Code
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5852)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1765,6 +1765,7 @@
         {
             tdef->m_AncestorsString = ancestor;
             tdef->m_ActualType = ancestor;
+            tdef->m_Type = ancestor;
         }
         else
             tdef->m_ActualType = ancestor + args;

Yes, it works on the test code below:

Code
#include <iostream>
#include <iostream>
using namespace std;
std::vector<string> ssss;
#define  xxxxxxxxx 12e
typedef class qq
{
    int x;
    int y;
}eeee;
typedef unsigned int wwwwwwwww;
www!
int main()
{

    cout << "Hello world!" << endl;
    return 0;
}

Title: Re: New code completion remarks/issues
Post by: blueshake on October 07, 2009, 01:35:04 pm
test codes:
Code
#include <iostream>

using namespace std;
struct qq
{
    int x;
    int y;
};
typedef qq (*ptr)(int a, int b);
ptr pp;
pp(3,3). --------not work.
int main()
{
    cout << "Hello world!" << endl;
    return 0;
}

Edit:
relative with these codes:
Code
        if (token.GetChar(0) == '(')
        {
            // function pointer (probably)
            is_function_pointer = true;
            if (peek.GetChar(0) == '(')
            {
                // typedef void (*dMessageFunction)(int errnum, const char *msg, va_list ap);
                // typedef void (MyClass::*Function)(int);

                // remove parentheses and keep everything after the dereferencing symbol
                token.RemoveLast();
                int pos = token.Find('*', true);
                if (pos != wxNOT_FOUND)
                {
                    typ << _T('(') << token.Mid(1, pos) << _T(')');
                    token.Remove(0, pos + 1);
                }
                else
                {
                    typ = _T("(*)");
                    token.Remove(0, 1); // remove opening parenthesis
                }
                args = peek;
                components.push(token);
            }
            else
            {
                // typedef void dMessageFunction (int errnum, const char *msg, va_list ap);

                typ = _T("(*)");
                // last component is already the name and this is the args
                args = token;
            }
            break;
        }
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 07, 2009, 02:07:49 pm
typedef qq (*ptr)(int a, int b);
ptr pp;
pp(3,3). --------not work.

But it seems the parser get the right token type. You can open the CC debug tool dialog.
see here:

(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2009-10-07195917.png)
and
(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2009-10-07195922.png)

So, I think the AI function in nativeparser.cpp has a bug to expand the "ptr".
Title: Re: New code completion remarks/issues
Post by: blueshake on October 08, 2009, 04:17:14 am
Dear ollydbg,
I think the actualtype should be qq,and the type should be qq(*), and the ancestor should be qq too,
for codes:
Code
typedef qq (*ptr)(int a, int b);
can be treat as
Code
typedef qq ptr;
and this maybe can fix the bug.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 08, 2009, 04:25:52 am
Dear ollydbg,
I think the actualtype should be qq,and the type should be qq(*), and the ancestor should be qq too,
for codes:
Code
typedef qq (*ptr)(int a, int b);
can be treat as
Code
typedef qq ptr;
and this maybe can fix the bug.
Good idea, I will try it :D.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 08, 2009, 10:23:15 am
Do some modification on value tip.
patch:
Code
Index: src/plugins/codecompletion/parser/token.cpp
===================================================================
--- src/plugins/codecompletion/parser/token.cpp (revision 5852)
+++ src/plugins/codecompletion/parser/token.cpp (working copy)
@@ -126,15 +126,42 @@
 
 wxString Token::DisplayName() const
 {
-//    wxString result(_T(""));
-//    wxString parentname = GetParentName();
-//    if (!parentname.IsEmpty())
-//        result << parentname << _T("::");
-    wxString result = GetNamespace();
-    result << m_Name << m_Args;
-    if (!m_Type.IsEmpty())
-        result << _T(" : ") << m_Type;
-    return result;
+    wxString result;
+    if (m_TokenKind == tkClass)
+    {
+        result << _T("class ") << m_Name << _T(" {...}");
+        return result;
+    }
+    else if (m_TokenKind == tkNamespace)
+    {
+        result << _T("namespace ") << m_Name << _T(" {...}");
+        return result;
+    }
+    else if (m_TokenKind == tkEnum)
+    {
+        return result << _T("enum ") << m_Name << _T(" {...}");
+    }
+    else if (m_TokenKind == tkTypedef)
+    {
+        result << _T("typedef") ;
+        if (!m_Type.IsEmpty())
+            result << _T(" ") << m_Type;
+        result << _T(" ") << m_Name;
+        return result;
+    }
+    else if (m_TokenKind == tkPreprocessor)
+    {
+        result << _T("#define ") << m_Name << m_Args;
+        if(!m_Type.IsEmpty())
+            result << _T(" ") << m_Type;
+            return result;
+    }
+    else
+    {
+        if (!m_Type.IsEmpty())
+            result << m_Type << _T(" ");
+        return result << GetNamespace() << m_Name << m_Args;
+    }
 }
 
 Token* Token::GetParentToken()

[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: mmkider on October 09, 2009, 03:33:59 am
Hi,

In SVN 5840, code completion can pop up function paramter.
But SVN 5853, it's not.

What happen  ? or do I  some error thing  ? :(

Code

int bbbbb(int cccccc)
{
                  //can't pop up cccccc
}

Title: Re: New code completion remarks/issues
Post by: ollydbg on October 09, 2009, 03:53:22 am
Hi,

In SVN 5840, code completion can pop up function paramter.
But SVN 5853, it's not.

What happen  ? or do I  some error thing  ? :(

Code

int bbbbb(int cccccc)
{
                  //can't pop up cccccc
}


I can confirm this bug.
In the function body, if you enter cccc, then, there is no suggestion list "cccccc".

Need time to find the bug.

By the way, the code in nativeparser.cpp, such as

Code
// Start an Artificial Intelligence (!) sequence to gather all the matching tokens..
// The actual AI is in FindAIMatches() below...
size_t NativeParser::AI(TokenIdxSet& result,
                        cbEditor* editor,
                        Parser* parser,
                        const wxString& lineText,
                        bool noPartialMatch,
                        bool caseSensitive,
                        TokenIdxSet* search_scope,
                        int caretPos)

is really hard to read and understand for me. :( :(
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 09, 2009, 04:30:01 am
I just write a simple test code:

Code
void zfunction(int abcdefg){
ab|         <---
;

};

Then I debug the CC under CB.

In codecompletion.cpp function int CodeCompletion::CodeComplete(), before line 602, the items was add correctly( which means "abcdefg" was correctly added to the suggestion list wxstring array ).

But finally, there is not suggestion list output... :( I don't know why...)

Edit: line 639 and line 640 of codecompletion.cpp, this function run to an early exit.
Code
            if (items.GetCount() == 0)
                return -2;
[/s]

Edit
It seems the match result size is always "0"...

Also, I always meet the "gdb.exe" crash when debugging(TDM-MinGW + GDB 6.8, "evaluate the express under cursor" was checked. )  I think this problem is caused by the "mouse hovers on a complex statement", then the gdb can't evaluate the statement, so ,it crashes. )

Any more comments? Thanks.






Title: Re: New code completion remarks/issues
Post by: blueshake on October 09, 2009, 07:23:11 am
Dear ollydbg:
The function tokens search followed the below steps:
1.serach the function name the line stayed.
2.parse the function arguments and stored tokens in pTempTreepTree ,marked with temp.
3.parse the fucntion body and stored tokens in pTempTreepTree,marked with temp.
I guess we should check whether tokens(e.g. ccccc) is in the pTempTree, and make sure the pTempTree is searched.
should compare with last svn version,I guess it caused by the recent fixed.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 09, 2009, 07:30:16 am
I can confirm it work in svn 5852.
see my screen shot.

[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 09, 2009, 07:33:56 am
Dear ollydbg:
The function tokens search followed the below steps:
1.serach the function name the line stayed.
2.parse the function arguments and stored tokens in pTempTree.
3.parse the fucntion body and stored tokens in pTempTree.
I guess we should check whether tokens(e.g. ccccc) in the pTempTree, and make sure the pTempTree is searched.
should compare with last svn version,I guess it caused by the recent fixed.

Thanks, but in which function takes these steps? I'm examining right now.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 09, 2009, 07:35:48 am
I can confirm it work in svn 5852.
see my screen shot.

Oh, this is a good catch, I was looking at the changes in nativeparser.cpp.
Now, it seems the bug is in other source files, because the newest nativeparser.cpp is rev 5849.

Wait:

It failed in rev 5852 of my working copy.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 09, 2009, 08:02:02 am
I'm sorry, I find the bug, it's here

line 766-787 of nativeparser.cpp.

Code
            if (!token->m_Args.IsEmpty() && !token->m_Args.Matches(_T("()")))
            {
                wxString buffer = token->m_Args;
                buffer.Remove(0, 1); // remove (
                buffer.RemoveLast(); // remove )
                buffer.Replace(_T(","), _T(";")); // replace commas with semi-colons
                buffer << _T(';'); // aid parser ;)
                buffer.Trim();

                if (s_DebugSmartSense)
                {
                #if wxCHECK_VERSION(2, 9, 0)
                    Manager::Get()->GetLogManager()->DebugLog(F(_T("Parsing arguments: \"%s\""), buffer.wx_str()));
                #else
                    Manager::Get()->GetLogManager()->DebugLog(F(_T("Parsing arguments: \"%s\""), buffer.c_str()));
                #endif
                    if (!buffer.IsEmpty() && !parser->ParseBuffer(buffer, false, false, true))
                    {
                        Manager::Get()->GetLogManager()->DebugLog(_T("ERROR parsing arguments"));
                    }
                }
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 09, 2009, 08:05:53 am
I'm sorry, I find the bug, it's here
line 766-787 of nativeparser.cpp.
...so?! What exactly is wrong with that portion?
...besides: it works fine here (trunk)...?!
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 09, 2009, 08:09:33 am
I'm sorry, I find the bug, it's here
line 766-787 of nativeparser.cpp.
...so?! What exactly is wrong with that portion?

The code is wrapped in the if (s_DebugSmartSense )from rev 5840 to 5845 of the nativeparser.cpp.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 09, 2009, 08:14:11 am
The code is wrapped in the if (s_DebugSmartSense )from rev 5840 to 5845 of the nativeparser.cpp.
Argh! Good one. Nevermind... my mistake. :oops: I was always using it *with* DebugSmartSense enabled, so that's why it worked here... Will commit the fix soon...
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 09, 2009, 08:19:09 am
The code is wrapped in the if (s_DebugSmartSense )from rev 5840 to 5845 of the nativeparser.cpp.
Argh! Good one. Nevermind... my mistake. :oops: I was always using it *with* DebugSmartSense enabled, so that's why it worked here... Will commit the fix soon...

haha, I think one reason of the mistake:  #if wxCHECK_VERSION(2, 9, 0)  preprocessor guard is a little annoying. :D
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 09, 2009, 08:19:36 am
Will commit the fix soon...
...done.
Title: Re: New code completion remarks/issues
Post by: mmkider on October 09, 2009, 08:36:18 am
Will commit the fix soon...
...done.
Look so good.
Thank you.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 09, 2009, 08:36:55 am
Will commit the fix soon...
...done.
:lol: :lol: :lol: :lol: :lol: :lol: :lol: :lol:
Title: Re: New code completion remarks/issues
Post by: blueshake on October 09, 2009, 08:38:48 am
Dear ollydbg:
had you tried on reply #155?
Title: Re: New code completion remarks/issues
Post by: daniloz on October 09, 2009, 08:49:56 am
Quick question... in the structs_typedefs.cpp (from the tests - thx Morten):
Code
struct _s
{
  int   x;
  float y;
};
typedef struct _s t_s;

typedef struct _s_inner
{
int z_inner;
} t_s_inner;

C::B shows "t_s" as a typedef, BUT "t_s_inner" as a class. Is this a bug or a feature? :D

Btw, I was expecting to see "t_s_inner" as a typedef... ;)

daniloz

PS: why are structs treated as class by the CC?
Title: Re: New code completion remarks/issues
Post by: blueshake on October 09, 2009, 10:11:58 am
Quick question... in the structs_typedefs.cpp (from the tests - thx Morten):
Code
struct _s
{
  int   x;
  float y;
};
typedef struct _s t_s;

typedef struct _s_inner
{
int z_inner;
} t_s_inner;

C::B shows "t_s" as a typedef, BUT "t_s_inner" as a class. Is this a bug or a feature? :D

Btw, I was expecting to see "t_s_inner" as a typedef... ;)

daniloz

PS: why are structs treated as class by the CC?



not a bug,I do this.read the codes about void ParserThread::ReadClsNames(wxString& ancestor)
as I kown ,there is a bug in typedef handle .so......
Title: Re: New code completion remarks/issues
Post by: blueshake on October 09, 2009, 03:31:57 pm
Quick question... in the structs_typedefs.cpp (from the tests - thx Morten):
Code
struct _s
{
  int   x;
  float y;
};
typedef struct _s t_s;

typedef struct _s_inner
{
int z_inner;
} t_s_inner;

C::B shows "t_s" as a typedef, BUT "t_s_inner" as a class. Is this a bug or a feature? :D

Btw, I was expecting to see "t_s_inner" as a typedef... ;)

daniloz

PS: why are structs treated as class by the CC?



fixed.


and fixed the typedef issue.
test codes:
Code
struct _s
{
  int   x;
  float y;
};
typedef struct _s t_s;
typedef t_s ssss;

ssss.
patch:
Code
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5859)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -1795,13 +1795,15 @@
                   current.c_str(),
                   m_Str.c_str(),
                   (m_pLastParent ? m_pLastParent->m_Name.c_str():_T("<no-parent>")));
-            Token* newToken = DoAddToken(tkClass, current, m_Tokenizer.GetLineNumber());
+            Token* newToken = DoAddToken(tkTypedef, current, m_Tokenizer.GetLineNumber());
             if (!newToken)
                 break;
             else
             {
                 wxString tempAncestor = ancestor;
                 newToken->m_AncestorsString = tempAncestor;
+                newToken->m_ActualType = tempAncestor;
+                newToken->m_Type = tempAncestor;
             }
         }
         else // unexpected
Index: src/plugins/codecompletion/parser/token.cpp
===================================================================
--- src/plugins/codecompletion/parser/token.cpp (revision 5859)
+++ src/plugins/codecompletion/parser/token.cpp (working copy)
@@ -865,7 +865,7 @@
                 {
                     Token* ancestorToken = at(*it);
                     // only classes take part in inheritance
-                    if (ancestorToken && ancestorToken != token && (ancestorToken->m_TokenKind == tkClass || ancestorToken->m_TokenKind == tkEnum))// && !ancestorToken->m_IsTypedef)
+                    if (ancestorToken && ancestorToken != token && (ancestorToken->m_TokenKind == tkClass || ancestorToken->m_TokenKind == tkEnum || ancestorToken->m_TokenKind == tkTypedef))// && !ancestorToken->m_IsTypedef)
                     {
                         token->m_Ancestors.insert(*it);
                         ancestorToken->m_Descendants.insert(i);
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 10, 2009, 07:42:47 am
Here is the patch to add TRACE to the token.cpp.

Also, this patch contains the previous post's patch by blueshake.

Currently, If you test the structs_typedefs.cbp.

Only these statements below failed.

Code
//    t_ptr.
//    t_ptr(3,3).

PS: Is it a typo? the original code is:

Code
//    t_ptr_s(
//    t_ptr_s(3,3).



[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 11, 2009, 01:13:13 pm
This is the testing patch for parsing the Macros for GCC 4.x

See the screenshot.

(http://i683.photobucket.com/albums/vv194/ollydbg_cb/vector.png)

You should also change these section in your default.conf file. Just search by "TOKEN_REPLACEMENTS"



Code
				<TOKEN_REPLACEMENTS>
<ssmap>
<_GLIBCXX_BEGIN_NAMESPACE>
<![CDATA[+namespace]]>
</_GLIBCXX_BEGIN_NAMESPACE>
<_GLIBCXX_BEGIN_NAMESPACE_TR1>
<![CDATA[-namespace tr1 {]]>
</_GLIBCXX_BEGIN_NAMESPACE_TR1>
<_GLIBCXX_BEGIN_NESTED_NAMESPACE>
<![CDATA[+namespace]]>
</_GLIBCXX_BEGIN_NESTED_NAMESPACE>
<_GLIBCXX_END_NAMESPACE>
<![CDATA[}]]>
</_GLIBCXX_END_NAMESPACE>
<_GLIBCXX_END_NAMESPACE_TR1>
<![CDATA[}]]>
</_GLIBCXX_END_NAMESPACE_TR1>
<_GLIBCXX_END_NESTED_NAMESPACE>
<![CDATA[}]]>
</_GLIBCXX_END_NESTED_NAMESPACE>
<_GLIBCXX_STD>
<![CDATA[std]]>
</_GLIBCXX_STD>
</ssmap>
</TOKEN_REPLACEMENTS>


I'm still testing... :D

Edit

it seems works OK!



[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: blueshake on October 11, 2009, 02:40:12 pm
What I want to know is if doing this wil slow down the parse process?
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 11, 2009, 03:07:39 pm
What I want to know is if doing this wil slow down the parse process?

parse C::B source code.
before patch
Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).
Title: Re: New code completion remarks/issues
Post by: koso on October 11, 2009, 04:06:42 pm
When you are talking about performance ...

I was using Symbol browser with view set to "Everything". Problem is, that now when I open project, first few seconds is C::B parsing files, and then it starts to update data in symbol browser. Becase of setting it to "Everything", it takes minutes .. and problem is, that during this operation is C::B dead = marked as "Not responding" = so I can do nothing, only look at white window (even draw events are ignored during this operation) or exit. So is there some way, how to do this more in background and not freeze whole UI?
And btw. this happens also when symbol browser is hidden.
 
Title: Re: New code completion remarks/issues
Post by: blueshake on October 12, 2009, 09:07:19 am
Dear Morten:
Have you started the plan on macro replacement and template parse?
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 18, 2009, 04:40:45 pm
Hi, morten, I find some bugs in CC, but I'm sorry, I can't create a patch, because my local copy was changed a lot. so, here are the modified code:

1.

plugins\codecompletion\classbrowser.cpp

function:

Code
void ClassBrowser::UpdateView()
{
    m_pActiveProject = 0;
    m_ActiveFilename.Clear();
    if (m_pParser && !Manager::IsAppShuttingDown())
    {
        m_pActiveProject = Manager::Get()->GetProjectManager()->GetActiveProject();
        cbEditor* ed = Manager::Get()->GetEditorManager()->GetBuiltinActiveEditor();
        if (ed)
        {
            //m_ActiveFilename = ed->GetFilename().BeforeLast(_T('.'));
            // the above line is a bug (see https://developer.berlios.de/patch/index.php?func=detailpatch&patch_id=1559&group_id=5358)
            m_ActiveFilename = ed->GetFilename().AfterLast(wxFILE_SEP_PATH);

            if(  m_ActiveFilename.Find(_T('.')) != wxNOT_FOUND  )
            {
                m_ActiveFilename = ed->GetFilename().BeforeLast(wxFILE_SEP_PATH) + wxFILE_SEP_PATH + m_ActiveFilename.BeforeLast(_T('.'));
                m_ActiveFilename.Append(_T('.'));
            }
            else
                m_ActiveFilename = ed->GetFilename();

        }

        BuildTree();

        wxSplitterWindow* splitter = XRCCTRL(*this, "splitterWin", wxSplitterWindow);
        if (m_pParser->ClassBrowserOptions().treeMembers)
        {
            splitter->SplitHorizontally(m_Tree, m_TreeBottom);
            m_TreeBottom->Show(true);
        }
        else
        {
            splitter->Unsplit();
            m_TreeBottom->Show(false);
        }

    }
    else
        m_Tree->DeleteAllItems();
}

For example, I have a file:

d:\MinGW\lib\gcc\mingw32\4.4.1\include\c++\vector

So, I would like to see the tokens in this file.


2. Also, the function can give the right tokens in the right file, but it seems the Browser build thread still remove all the nodes.

For example:

d:\MinGW\lib\gcc\mingw32\4.4.1\include\c++\bits\stl_vector.h

Loot at the symbol browser tree window, there is no node shown there. :(


3.
I have change the SkipComments function in tokenizer.cpp

See here

Code
bool Tokenizer::SkipComment(bool skipEndWhite)
{

    bool cstyle;            // C or C++ style comments

    //check the comment prompt
    if(CurrentChar() == '/')
    {
        if(NextChar() == '*')
            cstyle = true;
        else if(NextChar() == '/')
            cstyle = false;
        else
            return true;    //Not the comment prompt, return true;
    }
    else
        return true;        //Not the comment prompt, return true;

    MoveToNextChar(2);      //skip the comment prompt

    while(true)
    {
        if (cstyle)        //c style comment
        {
            SkipToChar('/');
            if (PreviousChar() == '*')//the end of C style comments
            {
                MoveToNextChar();
                break;
            }
            if(!MoveToNextChar())
                break;
        }
        else                //c++ style comment
        {
            SkipToEOL(false, true);//nestBrace = false skipcomment = true
            MoveToNextChar();
            break;
        }
    }


    if (IsEOF())
        return false;

    if (skipEndWhite)
    {
        if(!SkipWhiteSpace())
            return false;
    }
    else
        return true;

    return SkipComment(); // handle chained comments
}

For the old code, if skipEndWhite == false, the function will always return false, this is something wrong.


...

Bed time,,,,  I will add more tomorrow. :D

Title: Re: New code completion remarks/issues
Post by: ollydbg on October 19, 2009, 04:09:49 pm
1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function

Code
    if (    m_LastWasPreprocessor
        && !str.IsSameAs(_T("#"))
        && !m_LastPreprocessor.IsSameAs(_T("#")) )
    {
        if (!m_LastPreprocessor.IsSameAs(TokenizerConsts::include_str))
        {
            // except for #include and #if[[n]def], all other preprocessor directives need only
            // one word exactly after the directive, e.g. #define THIS_WORD
            SkipToEOL();
        }
        m_LastPreprocessor.Clear();
    }
So, these code can be deleted.

2, I tokenizer.h think in the CurrentChar function, we don't need to check index value, because MoveToNextChar already do this.
Code
    bool MoveToNextChar(const unsigned int amount = 1)
    {
        assert(amount);
        if(amount == 1) // compiler will dead-strip this
        {
            ++m_TokenIndex;
            if (IsEOF())
            {
                m_TokenIndex = m_BufferLen;
                return false;
            }

            if (CurrentChar() == _T('\n'))
                ++m_LineNumber;
            return true;
        }
        else
        {
            m_TokenIndex += amount;
            if (IsEOF())
            {
                m_TokenIndex = m_BufferLen;
                return false;
            }

            if (CurrentChar() == _T('\n'))
                ++m_LineNumber;
            return true;
        }
    };

    wxChar CurrentChar() const
    {
        if(m_TokenIndex < m_BufferLen)
            return m_Buffer.GetChar(m_TokenIndex);
        return 0;
    };
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 19, 2009, 08:57:51 pm
Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).
...did you notice? That's weired.

1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function
[...]
So, these code can be deleted.
...which one (which if condition) you mean? For me both still make sense...?!

2, I tokenizer.h think in the CurrentChar function, we don't need to check index value, because MoveToNextChar already do this.
That's true, but I wanted to avoid any issues with API mis-use (meaning calling CurrentChar() before MoveToNextChar() for any obscure reason). I wonder what this would differ in terms of time. Implementing a temporary time measurement would reveal the difference.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 20, 2009, 03:29:50 am
Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).
...did you notice? That's weird.
Yes, it is weird. I don't know why...



1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function
[...]
So, these code can be deleted.
...which one (which if condition) you mean? For me both still make sense...?!

Both the if condition were not be hit if I set breakpoint there.
See my screen shot
(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2009-10-20091723.png)
I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:
Code
bool Tokenizer::SkipUnwanted()
{
    while (CurrentChar() == '#' ||
            (!m_IsOperator && CurrentChar() == '=') ||
            (!m_IsOperator && CurrentChar() == '[') ||
            CurrentChar() == '?' ||
            (CurrentChar() == '/' && (NextChar() == '/' || NextChar() == '*') ))
    {
        bool skipPreprocessor = false; // used for #include
        while (m_Buffer.Mid(m_TokenIndex, 2) == _T("//") ||
                m_Buffer.Mid(m_TokenIndex, 2) == _T("/*"))
        {
            // C/C++ style comments
            SkipComment();
            if (IsEOF())
                return false;
            if (!SkipWhiteSpace())
                return false;
        }

        while (CurrentChar() == '#')
        {
            // preprocessor directives
            // we only care for #include and #define, for now
            unsigned int backupIdx = m_TokenIndex;
            MoveToNextChar();
            SkipWhiteSpace();
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }

Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 20, 2009, 07:52:43 am
I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:
So... now you got me a little confused (or probably it's too early for me). So you propose to remove the "if" conditions because of a bug in SkipUnwanted? Shouldn't we better fix the bug? And then: Where exactly is the bug in the code snippet?
Could you do me a favour and comment the lines / snippets in question in the code you've posted? That'll help me a lot. Please explain a little more... :(
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 20, 2009, 08:15:30 am
I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:
So... now you got me a little confused (or probably it's too early for me). So you propose to remove the "if" conditions because of a bug in SkipUnwanted? Shouldn't we better fix the bug? And then: Where exactly is the bug in the code snippet?
Could you do me a favour and comment the lines / snippets in question in the code you've posted? That'll help me a lot. Please explain a little more... :(

Ok, let me explain:

There are two member variables:

m_LastWasPreprocessor is a bool
m_LastPreprocessor is a wxString

But the DoGetToken() only return the variable "str"

Code
    if (    m_LastWasPreprocessor
        && !str.IsSameAs(_T("#"))
        && !m_LastPreprocessor.IsSameAs(_T("#")) )
    {
        if (!m_LastPreprocessor.IsSameAs(TokenizerConsts::include_str))
        {
            // except for #include and #if[[n]def], all other preprocessor directives need only
            // one word exactly after the directive, e.g. #define THIS_WORD
            SkipToEOL();
        }
        m_LastPreprocessor.Clear();
    }

    if (m_LastWasPreprocessor)
        m_LastPreprocessor << str;

    m_LastWasPreprocessor = false;

    return str;

So, str will never be affected by m_LastWasPreprocessor  or m_LastPreprocessor.

So, these two member variables are useless.


Title: Re: New code completion remarks/issues
Post by: blueshake on October 20, 2009, 10:09:38 am
what more I want to say is the logical error:
The only chance the variable m_LastWasPreprocessor  becomes true is in these codes:
Code
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else

at this time,str contain the string "#",and m_LastPreprocessor is empty.(see codes above)
so the three conditions in if:

m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true

see ,the three conditions can not all be true at the same time.It means the "if " condition can never be true.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 20, 2009, 10:12:06 am
what more I want to say is the logical error:
The only chance the variable m_LastWasPreprocessor  becomes true is in these codes:
Code
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else

at this time,str contain the string "#",and m_LastPreprocessor is empty.(see codes above)
so the three conditions in if:

m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true

see ,the three conditions can not all be true at the same time.It means the "if " condition can never be true.
Hi, blueshake, Thanks for the explanation. :D.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 20, 2009, 03:33:06 pm
m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true
Could it be that simply the "!str..." line should be removed as it is a relict of old times???

From what I read in the code it simplifies preprocessor handling in a way that it just gets rid of everything after the first preprocessor directive as it is unhandled anyways (except for #include and #if[[n]def], as stated in the comment)... which is good.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 20, 2009, 03:59:50 pm
My another question :
what is the effort of m_LastPreprocessor .since when the variable m_LastWasPreprocessor is true,the  m_LastPreprocessor is absoutely empty.see codes below.
Code
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else


EDIT:
In the three conditions,
m_LastWasPreprocessor is true,!m_LastPreprocessor.IsSameAs(_T("#")) is absoutely true too.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 20, 2009, 04:14:58 pm
Hi,
I would like to say:
there is no use of "m_LastPreprocessor" and "m_LastWasPreprocessor".
Let me explain more, In the parserthread, see the code here:

Code
        else if (token==ParserConsts::hash)
        {
            token = m_Tokenizer.GetToken();
            if (token==ParserConsts::kw_include)
                HandleIncludes();
            else if (token==ParserConsts::kw_define)
                HandleDefines();
            else
                HandlePreprocessorBlocks(token);
            m_Str.Clear();
        }

and

Code
void ParserThread::HandlePreprocessorBlocks(const wxString& preproc)
{
    if (preproc.StartsWith(ParserConsts::kw_if)) // #if, #ifdef, #ifndef
    {
        wxString token = preproc;
        ++m_PreprocessorIfCount;

        token = m_Tokenizer.GetToken();
        if (token.IsSameAs(_T("0")))
        {
            // TODO: handle special case "#if 0"
            TRACE(_T("HandlePreprocessorBlocks() : Special case \"#if 0\" not skipped."));
        }
        m_Tokenizer.SkipToEOL();
    }
    else if (preproc==ParserConsts::kw_else || preproc==ParserConsts::kw_elif) // #else, #elif
    {
        TRACE(_T("HandlePreprocessorBlocks() : Saving nesting level: %d"), m_Tokenizer.GetNestingLevel());
        m_Tokenizer.SaveNestingLevel();
        wxString token = preproc;
        while (!token.IsEmpty() && token != ParserConsts::kw_endif)
            token = m_Tokenizer.GetToken();
        --m_PreprocessorIfCount;
#if PARSERTHREAD_DEBUG_OUTPUT
        int l = m_Tokenizer.GetNestingLevel();
#endif
        m_Tokenizer.RestoreNestingLevel();
        TRACE(_T("HandlePreprocessorBlocks() : Restoring nesting level: %d (was %d)"), m_Tokenizer.GetNestingLevel(), l);
    }
    else if (preproc==ParserConsts::kw_endif) // #endif
        --m_PreprocessorIfCount;
}

And In my working copy, I already delete the if statements, and works fine here.
We just want to skip the unnecessory preprocessor directive, and no string information should be kept in m_LastPreprocessor.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 20, 2009, 05:01:24 pm
Code
        else if (token==ParserConsts::hash)
        {
            token = m_Tokenizer.GetToken();
            if (token==ParserConsts::kw_include)
                HandleIncludes();
            else if (token==ParserConsts::kw_define)
                HandleDefines();
            else
                HandlePreprocessorBlocks(token);
            m_Str.Clear();
        }
Notice that there is another call to HandlePreprocessorBlocks() in void ParserThread::SkipBlock() and ParserThread::HandleClass(EClassType ct).
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 20, 2009, 05:33:00 pm
Notice that there is another call to HandlePreprocessorBlocks() in void ParserThread::SkipBlock() and ParserThread::HandleClass(EClassType ct).
I've added a test case for pre-processor handling to trunk. You'll realise that a lot things are not (yet) working (check the symbols browser). A major bug is handling of comments in preprocessors. :-(
Title: Re: New code completion remarks/issues
Post by: blueshake on October 21, 2009, 02:52:52 am
Quote
I've added a test case for pre-processor handling to trunk. You'll realise that a lot things are not (yet) working (check the symbols browser). A major bug is handling of comments in preprocessors. :-(
hello,morten
can you give me more informations about what is wrong with preprocessors,It seems work here.
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 21, 2009, 03:01:55 am
Code
        else if (token==ParserConsts::hash)
        {
            token = m_Tokenizer.GetToken();
            if (token==ParserConsts::kw_include)
                HandleIncludes();
            else if (token==ParserConsts::kw_define)
                HandleDefines();
            else
                HandlePreprocessorBlocks(token);
            m_Str.Clear();
        }
Notice that there is another call to HandlePreprocessorBlocks() in void ParserThread::SkipBlock() and ParserThread::HandleClass(EClassType ct).

Yes, I also noticed that, this function in those places. But this is still nothing related to Tokenizer internals.

The Tokenizer class do quite simple things:

1, it skip the unnecessory preprocessor directives, such as:

Code
#pragma XXXX

2, return a correct token string

So, don't care about that. :D
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 21, 2009, 03:52:44 am
Here is the final patch on Tokenizer, I just do a lot of re-factoring.

1, We don't need to check whether the CurrentCharr is a "C escaped character" if we are not in a C string.
2, I remove the if condition check at the end of the DoGetToken function. see Here and the following discussion (http://forums.codeblocks.org/index.php/topic,11187.msg77414.html#msg77414)
3, I add a Macroreplace function to deal with the new Macros in GCC 4.x. You need to add macro map, see Token replacement map (http://forums.codeblocks.org/index.php/topic,11187.msg77135.html#msg77135)
4, I add some comments of these function in the source file.

I have tested this patch for several days, and works fine. :D

Feed back and test case are all welcome!!! :D


[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 21, 2009, 09:16:58 am
Feed back and test case are all welcome!!! :D
I cannot apply this patch on trunk. Could you please provide a patch against (current) trunk using svn diff?
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 21, 2009, 10:03:55 am
Feed back and test case are all welcome!!! :D
I cannot apply this patch on trunk. Could you please provide a patch against (current) trunk using svn diff?

Sorry, I don't know how to use svn diff. :oops:

So, I upload the source file for you,  :D




[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 21, 2009, 10:13:50 am
Sorry, I don't know how to use svn diff. :oops:
It's very easy:
- Install the SVN command lien client
- Modify the sources as needed
- Within the SVN controlled (sub-)directory, issue this command:
  svn diff > my_patch.patch
Done. :-)

So, I upload the source file for you,  :D
Works, thanks! I'll have a look... I have pending quite some patches now...
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 21, 2009, 10:28:12 am
Sorry, I don't know how to use svn diff. :oops:
It's very easy:
- Install the SVN command lien client
- Modify the sources as needed
- Within the SVN controlled (sub-)directory, issue this command:
  svn diff > my_patch.patch
Done. :-)
Thanks for your help!!
So, here is the svn diff.


[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on October 21, 2009, 10:50:34 am
- Within the SVN controlled (sub-)directory, issue this command:
  svn diff > my_patch.patch
Done. :-)

Within the root of the svn tree is the better place to issue the command (also svn help diff for more info)
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on October 21, 2009, 11:10:47 am
Within the root of the svn tree is the better place to issue the command (also svn help diff for more info)
Not for me, as I don't checkout the whole sources usually. So I need to manually change the patch quite often to remove e.g. the top-level "src" folder. Just do it as you like, I can handle both.
Title: Re: New code completion remarks/issues
Post by: blueshake on October 21, 2009, 03:16:04 pm
patch for this http://forums.codeblocks.org/index.php/topic,11187.msg76954.html#msg76954 (http://forums.codeblocks.org/index.php/topic,11187.msg76954.html#msg76954)

Code
Index: parserthread.cpp
===================================================================
--- parserthread.cpp (revision 5872)
+++ parserthread.cpp (working copy)
@@ -1719,7 +1719,7 @@
             {
                 // typedef void dMessageFunction (int errnum, const char *msg, va_list ap);
 
-                typ = _T("(*)");
+                //typ = _T("(*)");
                 // last component is already the name and this is the args
                 args = token;
             }
@@ -1757,7 +1757,7 @@
             ancestor << _T(' ');
         ancestor << token;
     }
-    ancestor << typ;
+    //ancestor << typ;
 
     // no return type
     m_Str.Clear();
@@ -1773,7 +1773,12 @@
             tdef->m_Type = ancestor;
         }
         else
-            tdef->m_ActualType = ancestor + args;
+        {
+            tdef->m_ActualType = ancestor;
+            tdef->m_Type = ancestor + typ;//+ args;
+            tdef->m_AncestorsString = ancestor;
+        }
+
     }
 }
 
@@ -1800,13 +1805,15 @@
                   current.c_str(),
                   m_Str.c_str(),
                   (m_pLastParent ? m_pLastParent->m_Name.c_str():_T("<no-parent>")));
-            Token* newToken = DoAddToken(tkClass, current, m_Tokenizer.GetLineNumber());
+            Token* newToken = DoAddToken(tkTypedef, current, m_Tokenizer.GetLineNumber());
             if (!newToken)
                 break;
             else
             {
                 wxString tempAncestor = ancestor;
                 newToken->m_AncestorsString = tempAncestor;
+                newToken->m_ActualType = tempAncestor;
+                newToken->m_Type = tempAncestor;
             }
         }
         else // unexpected
Index: token.cpp
===================================================================
--- token.cpp (revision 5872)
+++ token.cpp (working copy)
@@ -138,13 +138,11 @@
     wxString result;
     if (m_TokenKind == tkClass)
     {
-        result << _T("class ") << m_Name << _T(" {...}");
-        return result;
+        return result << _T("class ") << m_Name << _T(" {...}");
     }
     else if (m_TokenKind == tkNamespace)
     {
-        result << _T("namespace ") << m_Name << _T(" {...}");
-        return result;
+        return result << _T("namespace ") << m_Name << _T(" {...}");
     }
     else if (m_TokenKind == tkEnum)
     {
@@ -155,15 +153,18 @@
         result << _T("typedef") ;
         if (!m_Type.IsEmpty())
             result << _T(" ") << m_Type;
-        result << _T(" ") << m_Name;
-        return result;
+        if (result.find('*', true) != wxNOT_FOUND)
+        {
+            result.RemoveLast();
+            return result << m_Name << _T(")") <<  m_Args;
+        }
+        return result << _T(" ") << m_Name;
     }
     else if (m_TokenKind == tkPreprocessor)
     {
         result << _T("#define ") << m_Name << m_Args;
         if(!m_Type.IsEmpty())
-            result << _T(" ") << m_Type;
-        return result;
+        return result << _T(" ") << m_Type;
     }
     else
     {
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 22, 2009, 09:03:26 am
here is the patch to solve parser the vector problem in this post and the following post:

http://forums.codeblocks.org/index.php/topic,11311.msg77512.html#msg77512

For example, the patch can parse this kind of statement:

Code
a  b::c d::e() {;};

Then, c is the return type.
e is the function name.


[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 22, 2009, 10:42:55 am
@blueshake:
Now, the whole test
in  cb_svn\src\plugins\codecompletion\testing\structs_typedefs.cpp
works!!! Thanks very much!!!

Also, a patch to fix a typo.

Code
Index: structs_typedefs.cpp
===================================================================
--- structs_typedefs.cpp (revision 5873)
+++ structs_typedefs.cpp (working copy)
@@ -6,7 +6,7 @@
   float y;
 };
 typedef struct _s t_s;
-typedef _s (*t_ptr)(int a, int b);
+typedef _s (*t_ptr_s)(int a, int b);
 
 typedef struct _s_inner
 {
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 30, 2009, 04:15:54 pm
I have find a bug, that will at least cause the CC to build browser tree twice when we just do a header/implementation file switch after you double click on a tree item. Also, rebuild a browser tree will cause a re-flash on the tree, and your lose the mouse position, this is really annoying!!!

Look at the code:

Code
void ClassBrowser::UpdateView()
{
    m_pActiveProject = 0;
    m_ActiveFilename.Clear();
    if (m_pParser && !Manager::IsAppShuttingDown())
    {
        m_pActiveProject = Manager::Get()->GetProjectManager()->GetActiveProject();
        cbEditor* ed = Manager::Get()->GetEditorManager()->GetBuiltinActiveEditor();
        if (ed)
        {
            //m_ActiveFilename = ed->GetFilename().BeforeLast(_T('.'));
            // the above line is a bug (see https://developer.berlios.de/patch/index.php?func=detailpatch&patch_id=1559&group_id=5358)
            m_ActiveFilename = ed->GetFilename().AfterLast(wxFILE_SEP_PATH);
            if (m_ActiveFilename.Find(_T('.')) != wxNOT_FOUND)
            {
                m_ActiveFilename = ed->GetFilename().BeforeLast(wxFILE_SEP_PATH) + wxFILE_SEP_PATH + m_ActiveFilename.BeforeLast(_T('.'));
                m_ActiveFilename.Append(_T('.'));
            }
            else
                m_ActiveFilename = ed->GetFilename();
        }

        BuildTree();

        wxSplitterWindow* splitter = XRCCTRL(*this, "splitterWin", wxSplitterWindow);
        if (m_pParser->ClassBrowserOptions().treeMembers)
        {
            splitter->SplitHorizontally(m_Tree, m_TreeBottom);
            m_TreeBottom->Show(true);
        }
        else
        {
            splitter->Unsplit();
            m_TreeBottom->Show(false);
        }

    }
    else
        m_Tree->DeleteAllItems();
}

You can see:

Every time, this function is called, the m_ActiveFilename firstly be cleared, then a new m_ActiveFilename value was generated, and the tree was rebuild.

But how do we generate the m_ActiveFilename ?

For example, your active editor was :  "C:\BBB\AAA.cpp", at this time, the m_ActiveFilename was "C:\BBB\AAA.", If you double click on the browser tree item to swap the file, and the caret jump to "C:\BBB\AAA.h", the m_ActiveFilename is still "C:\BBB\AAA.".

At this time, we don't need to rebuild the tree again.

Any comments???
Title: Re: New code completion remarks/issues
Post by: blueshake on October 31, 2009, 05:21:59 am
Quote
At this time, we don't need to rebuild the tree again.
hi,ollydbg:
The contexts in C:\BBB\AAA.cpp are different from C:\BBB\AAA.h,why we don't need to rebuild the tree again?
Title: Re: New code completion remarks/issues
Post by: ollydbg on October 31, 2009, 04:14:48 pm
Quote
At this time, we don't need to rebuild the tree again.
hi,ollydbg:
The contexts in C:\BBB\AAA.cpp are different from C:\BBB\AAA.h,why we don't need to rebuild the tree again?
At this time, we get the same browser tree, if you select "current file's token" in browser view.
Title: Re: New code completion remarks/issues
Post by: blueshake on November 01, 2009, 03:18:43 am
when the parse is done ,the UpdateClassBrowser(); will be called in void NativeParser::OnParserEnd(wxCommandEvent& event)
at this time ,some token maybe has been changed.So the tree should be rebuilt.
and in the situation you mentioned.yes,it don't need be rebuilt.But how do we distinguish these , need or not.


by the way,ollydbg,have you encounter this issue.http://forums.codeblocks.org/index.php/topic,11432.0.html (http://forums.codeblocks.org/index.php/topic,11432.0.html)
Title: Re: New code completion remarks/issues
Post by: ollydbg on November 02, 2009, 10:18:02 am
Here is another bug.

See the screen shot:

(http://i683.photobucket.com/albums/vv194/ollydbg_cb/two_variable.png)

And the test code:

Code
#include <iostream>

using namespace std;

int variableA;

int main()
{
    int variableA;
    variableA = 1;
    cout << "Hello world!" << endl;
    return 0;
}

Both the auto variable in main and global variable will be shown together.

This is because:

1, the global variable "variableA" was already exist in the Token trie.
2, the auto variable "variableA" was added when parsing the function block.

So, my suggestion is:

Search the local tokens first. I mean, if we find a token is already exist in the local context, we don't need to search the global namespace.

This can make code completion more fast.


Title: Re: New code completion remarks/issues
Post by: oBFusCATed on November 02, 2009, 10:24:46 am
Hm, Olly having both in the tooltip can be quite helpful.
If for example the tooltip looks like this:

global int variableA
local int variableA

or
int ::variableA
int variableA
int my_namespace::variableA
Title: Re: New code completion remarks/issues
Post by: blueshake on November 02, 2009, 11:29:36 am
Hm, Olly having both in the tooltip can be quite helpful.
If for example the tooltip looks like this:

global int variableA
local int variableA

or
int ::variableA
int variableA
int my_namespace::variableA

should be easy to do this.
Title: Re: New code completion remarks/issues
Post by: blueshake on November 02, 2009, 11:47:26 am
Hm, Olly having both in the tooltip can be quite helpful.
If for example the tooltip looks like this:

global int variableA
local int variableA

or
int ::variableA
int variableA
int my_namespace::variableA



Patch for doing this.
Code
Index: src/plugins/codecompletion/parser/token.cpp
===================================================================
--- src/plugins/codecompletion/parser/token.cpp (revision 5894)
+++ src/plugins/codecompletion/parser/token.cpp (working copy)
@@ -136,6 +136,10 @@
 wxString Token::DisplayName() const
 {
     wxString result;
+    if (m_IsTemp)
+        result << _T("local ");
+    else
+        result << _T("global ");
     if      (m_TokenKind == tkClass)
         return result << _T("class ")     << m_Name << _T(" {...}");
     else if (m_TokenKind == tkNamespace)
see the screen shot.
(http://www.ikaca.sh.cn/attachment/200911/2/1745_12571579471ZIG.jpg)
Title: Re: New code completion remarks/issues
Post by: jaxon on November 02, 2009, 01:06:04 pm
Sorry, but it seems for me, that we need only variable used by compiler in current context to be shown in the tooltip. And yes, as ollydbg said, it should be faster.
Title: Re: New code completion remarks/issues
Post by: ollydbg on November 02, 2009, 03:54:11 pm
Hi, all, I personally agree with jaxon.

I have debug the code whole day :(, and finally find that this is a bug in the code below:

Code
bool NativeParser::ParseLocalBlock(cbEditor* ed, int caretPos)
{
    if (!ed)
        return false;

    Parser* parser = FindParserFromEditor(ed);
    if (!parser)
        return false;

    if (!parser->Done())
        return false;

    if (s_DebugSmartSense)
        Manager::Get()->GetLogManager()->DebugLog(_T("Parse local block"));

    int blockStart = FindCurrentFunctionStart(ed, 0, 0, caretPos);
    if (blockStart != -1)
    {
        ++blockStart; // skip {
        int blockEnd = caretPos == -1 ? ed->GetControl()->GetCurrentPos() : caretPos;
        if (blockEnd < 0 || blockEnd > ed->GetControl()->GetLength())
            return false;

        if (blockStart >= blockEnd)
            blockStart = blockEnd;

        wxString buffer = ed->GetControl()->GetTextRange(blockStart, blockEnd);
        buffer.Trim();
        if (!buffer.IsEmpty() && !parser->ParseBuffer(buffer, false, false, true))
        {
            if (s_DebugSmartSense)
                Manager::Get()->GetLogManager()->DebugLog(_T("ERROR parsing block:\n") + buffer);
        }
        else
        {
            if (s_DebugSmartSense)
            {
                #if wxCHECK_VERSION(2, 9, 0)
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Block:\n%s"), buffer.wx_str()));
                #else
                Manager::Get()->GetLogManager()->DebugLog(F(_T("Block:\n%s"), buffer.c_str()));
                #endif
                Manager::Get()->GetLogManager()->DebugLog(_T("Local tokens:"));
                for (size_t i = 0; i < parser->GetTokens()->size(); ++i)
                {
                    Token* t = parser->GetTokens()->at(i);
                    if (t && t->m_IsTemp)
                        Manager::Get()->GetLogManager()->DebugLog(_T(" + ") + t->DisplayName());
                }
            }
            return true;
        }
    }
    else
    {
        if (s_DebugSmartSense)
            Manager::Get()->GetLogManager()->DebugLog(_T("Could not determine current block start..."));
    }
    return false;
}

for example, at this time( see my example code), the LocalBlock is the function body of main to the current caret:

Code
int variableA;
    variableA = 1;

After parsing the buffer, the parser mistakenly added the local variableA to the global namespace.. I think the correct way should be :
set the the local variableA's parent as Token "main" (not the global namespace).

Also, there is another issue in these code:

nativeparser.cpp line 1597:

Code
    // always add scope -1 (i.e. global namespace)
    search_scope->insert(-1);

    // find all other matches
    std::queue<ParserComponent> components;
    BreakUpComponents(parser, actual, components);

    m_LastAISearchWasGlobal = components.size() <= 1;
    if (!components.empty())
        m_LastAIGlobalSearch = components.front().component;

    // actually find all matches in selected namespaces
    for (TokenIdxSet::iterator it = search_scope->begin(); it != search_scope->end(); ++it)
    {
        if (s_DebugSmartSense)
        {
            Token* scopeToken = tree->at(*it);
            #if wxCHECK_VERSION(2, 9, 0)
            Manager::Get()->GetLogManager()->DebugLog(F(_T("Parent scope: '%s' (%d)"), scopeToken ? scopeToken->m_Name.wx_str() : _T("Global namespace"), *it));
            #else
            Manager::Get()->GetLogManager()->DebugLog(F(_T("Parent scope: '%s' (%d)"), scopeToken ? scopeToken->m_Name.c_str() : _T("Global namespace"), *it));
            #endif
        }
        FindAIMatches(parser, components, result, *it, noPartialMatch, caseSensitive, true, 0xffff, search_scope);
    }

    cached_editor = editor;
    if (result.size() || (m_EditorEndWord - m_EditorStartWord))
        cached_editor_start_word = m_EditorStartWord;
    cached_search = actual;
    cached_results_count = result.size();

    return result.size();
}


In the code above, the "global namespace" is added by default. I don't think is is necessary, I personally use this method

1, check the result.size() at the end of AI()
2, if the result.size is zero, we should add the global namespace, and do the FindAIMatches() in the global namespace.

In this way, if the token is already found in the local context or matches as a class members, we don't need to use global namespace search.

By the way, the AI() function( and the related functions to do the show the tips) code is really hard to read!!!, that's irradiated me so much!!! :(, I think we should refactor it, especially change the variable names.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on November 02, 2009, 09:45:20 pm
Hm, Olly having both in the tooltip can be quite helpful.
Definitely. Look: Naming a local variable like a global one is bad style. Surely you might forget this sometimes but if the tooltip reminds you accordingly this is just helpful. So please don't strip this (really useful) information. Not even for the sake of speed.
I agree (however) with displayinf this with a "global/local" prefix.
Title: Re: New code completion remarks/issues
Post by: ollydbg on November 10, 2009, 02:11:12 pm
Hi, all, here, I announce this patch of the code completion plugin.

Some time, when you are debugging, both the GDB value tips and CC's tips window will pop up, and overlayed. That's very annoying. The currently method try to close the tip window, when the new tip will be shown, the GDB's tip always shown later, so, GDB's tip always force the CC's tip to close, and win the battle. :D


Edit1
Sorry, When adding the patch, my computer hangs. So, here is the patch:

Code
Index: codecompletion.cpp
===================================================================
--- codecompletion.cpp (revision 5908)
+++ codecompletion.cpp (working copy)
@@ -112,25 +112,26 @@
 // just because we don't know other plugins' used identifiers,
 // we use wxNewId() to generate a guaranteed unique ID ;), instead of enum
 // (don't forget that, especially in a plugin)
-int idMenuCodeComplete = wxNewId();
-int idMenuShowCallTip = wxNewId();
-int idMenuGotoFunction = wxNewId();
-int idMenuGotoPrevFunction = wxNewId();
-int idMenuGotoNextFunction = wxNewId();
-int idMenuGotoDeclaration = wxNewId();
-int idMenuGotoImplementation = wxNewId();
-int idMenuOpenIncludeFile = wxNewId();
-int idViewClassBrowser = wxNewId();
-int idEditorSubMenu = wxNewId();
-int idClassMethod = wxNewId();
+int idMenuCodeComplete          = wxNewId();
+int idMenuShowCallTip           = wxNewId();
+int idMenuGotoFunction          = wxNewId();
+int idMenuGotoPrevFunction      = wxNewId();
+int idMenuGotoNextFunction      = wxNewId();
+int idMenuGotoDeclaration       = wxNewId();
+int idMenuGotoImplementation    = wxNewId();
+int idMenuOpenIncludeFile       = wxNewId();
+int idViewClassBrowser          = wxNewId();
+int idEditorSubMenu             = wxNewId();
+int idClassMethod               = wxNewId();
 int idUnimplementedClassMethods = wxNewId();
-int idGotoDeclaration = wxNewId();
-int idGotoImplementation = wxNewId();
-int idOpenIncludeFile = wxNewId();
-int idStartParsingProjects = wxNewId();
-int idCodeCompleteTimer = wxNewId();
-int idFunctionsParsingTimer = wxNewId();
+int idGotoDeclaration           = wxNewId();
+int idGotoImplementation        = wxNewId();
+int idOpenIncludeFile           = wxNewId();
+int idStartParsingProjects      = wxNewId();
+int idCodeCompleteTimer         = wxNewId();
+int idFunctionsParsingTimer     = wxNewId();
 
+
 // milliseconds
 #define EDITOR_AND_LINE_INTERVAL 150
 
@@ -193,8 +194,14 @@
 
     if (!cfg->Exists(_T("token_replacements")))
     {
-        // first run; add default replacements
+        // first run; add default replacements string
         Tokenizer::SetReplacementString(_T("_GLIBCXX_STD"), _T("std"));
+        Tokenizer::SetReplacementString(_T("_GLIBCXX_BEGIN_NESTED_NAMESPACE"), _T("+namespace"));
+        Tokenizer::SetReplacementString(_T("_GLIBCXX_END_NESTED_NAMESPACE"), _T("}"));
+        Tokenizer::SetReplacementString(_T("_GLIBCXX_BEGIN_NAMESPACE"), _T("+namespace"));
+        Tokenizer::SetReplacementString(_T("_GLIBCXX_END_NAMESPACE_TR1"), _T("}"));
+        Tokenizer::SetReplacementString(_T("_GLIBCXX_BEGIN_NAMESPACE_TR1"), _T("-namespace tr1 {"));
+
     }
     else
         cfg->Read(_T("token_replacements"), &repl);
@@ -522,9 +529,10 @@
     if (!IsAttached() || !m_InitDone)
         return -1;
 
-    ConfigManager* cfg = Manager::Get()->GetConfigManager(_T("code_completion"));
+    ConfigManager* cfg   = Manager::Get()->GetConfigManager(_T("code_completion"));
     EditorManager* edMan = Manager::Get()->GetEditorManager();
-    cbEditor* ed = edMan->GetBuiltinActiveEditor();
+    cbEditor* ed         = edMan->GetBuiltinActiveEditor();
+
     if (!ed)
         return -3;
 
@@ -554,7 +562,8 @@
             ed->GetControl()->ClearRegisteredImages();
 
             bool caseSens = parser ? parser->Options().caseSensitive : false;
-            wxArrayString items; items.Alloc(result.size());
+            wxArrayString items;
+            items.Alloc(result.size());
             int pos   = ed->GetControl()->GetCurrentPos();
             int start = ed->GetControl()->WordStartPosition(pos, true);
             wxArrayInt already_registered;
@@ -748,6 +757,8 @@
     return dirs;
 }
 
+// Do the code completion when we enter:
+// #include "| or #include <|
 void CodeCompletion::CodeCompleteIncludes()
 {
     if (!IsAttached() || !m_InitDone)
@@ -758,20 +769,22 @@
         return;
 
     EditorManager* edMan = Manager::Get()->GetEditorManager();
-    cbEditor* ed = edMan->GetBuiltinActiveEditor();
+    cbEditor* ed         = edMan->GetBuiltinActiveEditor();
+
     if (!ed)
         return;
 
-    Parser* parser = m_NativeParser.FindParserFromActiveEditor();
+    Parser* parser      = m_NativeParser.FindParserFromActiveEditor();
     const bool caseSens = parser ? parser->Options().caseSensitive : false;
 
     FileType ft = FileTypeOf(ed->GetShortName());
     if ( ft != ftHeader && ft != ftSource) // only parse source/header files
         return;
 
-    int pos = ed->GetControl()->GetCurrentPos();
+    int pos          = ed->GetControl()->GetCurrentPos();
     int lineStartPos = ed->GetControl()->PositionFromLine(ed->GetControl()->GetCurrentLine());
-    wxString line = ed->GetControl()->GetLine(ed->GetControl()->GetCurrentLine());
+    wxString line    = ed->GetControl()->GetLine(ed->GetControl()->GetCurrentLine());
+
     //Manager::Get()->GetLogManager()->DebugLog("#include cc for \"%s\"", line.c_str());
     line.Trim();
     if (line.IsEmpty() || !TestIncludeLine(line))
@@ -784,6 +797,7 @@
         return;
     ++quote_pos;
 
+    // now, we are after the quote prompt
     wxString filename = line.substr(quote_pos, pos - lineStartPos - quote_pos);
     filename.Replace(_T("\\"), _T("/"), true);
 
@@ -823,6 +837,7 @@
         }
     }
 
+    // popup the auto completion window
     if (files.GetCount() != 0)
     {
         files.Sort();
@@ -872,11 +887,12 @@
         // else, out of range
     }
 
-    int start = 0;
-    int end = 0;
-    int count = 0;
-    int commas = m_NativeParser.GetCallTipCommas(); // how many commas has the user typed so far?
+    int start           = 0;
+    int end             = 0;
+    int count           = 0;
+    int commas          = m_NativeParser.GetCallTipCommas(); // how many commas has the user typed so far?
     wxArrayString items = m_NativeParser.GetCallTips(maxCalltipLineSizeInChars);
+
     std::set< wxString, std::less<wxString> > unique_tips; // check against this before inserting a new tip in the list
     wxString definition;
     for (unsigned int i = 0; i < items.GetCount(); ++i)
@@ -923,8 +939,11 @@
     }
 
     wxString filename = ed->GetFilename();
+
+    // open the insert class dialog
     InsertClassMethodDlg dlg(Manager::Get()->GetAppWindow(), parser, filename);
     PlaceWindow(&dlg);
+
     if (dlg.ShowModal() == wxID_OK)
     {
         int pos = ed->GetControl()->GetCurrentPos();
@@ -1565,6 +1584,30 @@
         if (!Manager::Get()->GetConfigManager(_T("code_completion"))->ReadBool(_T("eval_tooltip"), true))
             return;
 
+
+        // Check the debugger is running...at this time, don't show cc tips
+        bool debuggerRunning = false;
+        PluginsArray arr = Manager::Get()->GetPluginManager()->GetOffersFor(ptDebugger);
+        if (arr.GetCount())
+        {
+            for(size_t i=0;i<arr.GetCount();i++)
+            {
+                cbDebuggerPlugin* debugger = (cbDebuggerPlugin*)arr[i];
+                if (!debugger)
+                    continue; //kinda scary if this isn't a debugger? perhaps this should be a logged error??
+                if (debugger->IsRunning())
+                {
+                    debuggerRunning=true;
+                    break;
+                }
+            }
+        }
+        if(debuggerRunning)
+        {
+            Manager::Get()->GetLogManager()->DebugLog(F(_T("debugger is running, skip.CC tips..")));
+            return;
+        }
+
         EditorBase* base = event.GetEditor();
         cbEditor* ed = base && base->IsBuiltinEditor() ? static_cast<cbEditor*>(base) : 0;
         if (!ed || ed->IsContextMenuOpened())


I just check if any debugger plugin was running, then, CC's tip should not been shown.

Edit2

I just use the aligner plugin to format some code.


Title: Re: New code completion remarks/issues
Post by: ollydbg on November 10, 2009, 02:50:43 pm
hi, all, here is another patch.

I have always compaint this bug, when the bottom tree was rebuild, the navigate position always go the the bottom of the tree. That means, the vertical scroll bar will always go the bottom of the window, so, the "private member tree items" always been shown, because they were under the "public member tree items".
see this post:
http://forums.codeblocks.org/index.php/topic,10693.0.html

In this patch, I just "remember" the first tree item, and force a navigation when the tree been built.

Code
Index: classbrowserbuilderthread.cpp
===================================================================
--- classbrowserbuilderthread.cpp (revision 5908)
+++ classbrowserbuilderthread.cpp (working copy)
@@ -780,22 +780,29 @@
 #ifdef buildtree_measuring
     wxStopWatch sw;
 #endif
+
         tree->Freeze();
+
+
 #ifdef buildtree_measuring
     Manager::Get()->GetLogManager()->DebugLog(F(_T("tree->Freeze() took : %d ms"),sw.Time()));
     sw.Start();
 #endif
+
         tree->DeleteAllItems();
 #ifdef buildtree_measuring
     Manager::Get()->GetLogManager()->DebugLog(F(_T("tree->DeleteAllItems() took : %d ms"),sw.Time()));
     sw.Start();
 #endif
+
         node = tree->AddRoot(_T("Members")); // not visible, so don't translate
+
 #ifdef buildtree_measuring
     Manager::Get()->GetLogManager()->DebugLog(F(_T("tree->AddRoot() took : %d ms"),sw.Time()));
 #endif
     }
 
+    wxTreeItemId firtItem;
     if (data)
     {
         switch (data->m_SpecialFolder)
@@ -819,6 +826,8 @@
                         AddChildrenOf(tree, rootFuncs, data->m_pToken->GetSelf(), tkFunction);
                         AddChildrenOf(tree, rootVars, data->m_pToken->GetSelf(), tkVariable);
                         AddChildrenOf(tree, rootOthers, data->m_pToken->GetSelf(), ~(tkNamespace | tkClass | tkEnum | tkAnyFunction | tkVariable));
+                        firtItem = rootCtorDtor;
+
                     }
                     else if (m_Options.sortType == bstScope && data->m_pToken->m_TokenKind & tkClass)
                     {
@@ -829,6 +838,9 @@
                         AddChildrenOf(tree, rootPublic, data->m_pToken->GetSelf(), ~(tkNamespace | tkClass | tkEnum), tsPublic);
                         AddChildrenOf(tree, rootProtected, data->m_pToken->GetSelf(), ~(tkNamespace | tkClass | tkEnum), tsProtected);
                         AddChildrenOf(tree, rootPrivate, data->m_pToken->GetSelf(), ~(tkNamespace | tkClass | tkEnum), tsPrivate);
+
+                        firtItem = rootPublic;
+
                     }
                     else
                     {
@@ -866,7 +878,19 @@
     if (bottom)
     {
         tree->ExpandAll();
+////        if(tree->GetRootItem())
+//        tree->EnsureVisible(tree->GetRootItem());
+        //tree->EnsureVisible(node);
+        if(firtItem.IsOk())
+        {
+            tree->ScrollTo(firtItem);
+            tree->EnsureVisible(firtItem);
+        }
         tree->Thaw();
+        //we should scrool to the first item!!
+//        wxTreeItemIdValue cookie;
+//        wxTreeItemId res = tree->GetFirstChild(start, cookie);
+
     }
 }
 


Title: Re: New code completion remarks/issues
Post by: ollydbg on November 10, 2009, 04:23:38 pm
This is the third patch.  I just add one major check,

Code
void NativeParser::OnEditorActivated(EditorBase* editor)
{
    if (!m_pClassBrowser)
        return;
    cbEditor* ed = editor && editor->IsBuiltinEditor() ? static_cast<cbEditor*>(editor) : 0;

//    cbEditor* ed = editor ? static_cast<cbEditor*>(editor) : 0;
//    EditorManager* edMan = Manager::Get()->GetEditorManager();
//    cbEditor* ed         = edMan->GetBuiltinActiveEditor();

    if (ed)
    {
        Parser* parser = FindParserFromEditor(ed);
        if (parser && parser->ClassBrowserOptions().displayFilter == bdfFile)
        {
            // check header and implementation file swap, if yes, don't need to rebuild browser tree
            m_pClassBrowser->UpdateView(true);  ***************here
        }
    }
}

I modify the UpdateView function prototype,  void UpdateView(bool checkHeaderSwap = false);
Which means, if we do the editor change, that is eg: c:/abc.h  to c:/abc.cpp, we don't need to rebuild the Browser tree.

also, a lot of c_str() function and wxWidgets version check macros have been removed, since using wx_str() is enough.



[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on November 10, 2009, 04:32:24 pm
I'm trying to document the parserthread.h. here is the patch to do this, some function prototypes were not described due to my little knowledge and English written ability. :(

I'd prefer some one can supplement this and help me. :D



[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on November 10, 2009, 04:56:18 pm
Hi, all, here, I announce this patch of the code completion plugin.
I am not sure about this one...
I am using CC in fact while debugging and this would not let me.

hi, all, here is another patch.
This is the third patch.  I just add one major check,
I'm trying to document the parserthread.h.
Those three are looking good  to me. I've applied them  in my local copy for testing... (where appropriate).
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 01, 2009, 05:03:30 pm
I'm so grad that several patches of visualfc and blueshake and mine has been tested and finally applied in the trunk. But it seems the documents are not so good, so, I will prepare another patch to fix some "bad or missing documents". :D

Currently, the latest trunk code has broken parsing the OpenCV prototype correctly. because in the OpenCV source code ,there are some statement like this:

Code
CVAPI(void) cvFunction(XXXXX);

The new CC will regard the statement as two tokens, one is "CVAPI(void)", and the other is "cvFunction(XXXXX)", so, I'm not satisfied with this, maybe, I need to implement a macro replacement rule for the "CVAPI" macro. :wink:

By the way, I think this thread should be moved to CodeCompletion redesign (http://forums.codeblocks.org/index.php/board,18.0.html) sub-forum, because people here are all talking about the patches and the new CC features.

Title: Re: New code completion remarks/issues
Post by: blueshake on December 02, 2009, 01:59:36 am
Quote
I'm not satisfied with this, maybe, I need to implement a macro replacement rule for the "CVAPI" macro. Wink


we can make a compare here.
if the position of token (void) and token cvFunction are in the same line.keep the CVAPI(void) in m_Str and continue to parse.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on December 02, 2009, 06:38:40 am
we can make a compare here.
I'd rather go with the token replacement as suggested by OllyDgb. This is configurable and we don't need to implement particular CV SDK hocus-pocus in the plugin. Thus the plugin should not know antyhing about CV. OR make it more generic so that it applies to other similar constructs as well. But this might break something else.
Title: Re: New code completion remarks/issues
Post by: Monolith on December 19, 2009, 11:24:44 am
I have notice a couple issues.

(1) There is a problem with typedefs with multiple definitions. For instance, typedef WNDCLASSEXW WNDCLASSEX,*LPWNDCLASSEX,*PWNDCLASSEX; wont get recognized by code complete, however, typedef WNDCLASSEXW WNDCLASSEX; will work.
(2) The parser should recognize definitions, the difference between UNICODE and not UNICODE. For instance, I have a ANSI app, but I had to edit the UNICODE typedef of WNDCLASSEX to get the Code Complete to work.
(3) #define should be recognized as its counterparts. (#define MessageBox MessageBoxW will not be recognized as MessageBoxW.)

You guys have made great strides lately, keep up the good work.
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on December 19, 2009, 12:54:25 pm
2 and 3 are known problems -> there is no code written to parse the #defines, as far as I know
Title: Re: New code completion remarks/issues
Post by: blueshake on December 20, 2009, 01:35:01 am
Quote
(1) There is a problem with typedefs with multiple definitions. For instance, typedef WNDCLASSEXW WNDCLASSEX,*LPWNDCLASSEX,*PWNDCLASSEX; wont get recognized by code complete, however, typedef WNDCLASSEXW WNDCLASSEX; will work.

um,confirm it.That is because the "," is not handled.thanks for your report.
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 20, 2009, 12:46:38 pm
a bug in rev 5984 (Windows) , if you open the CB project, and try to find the declaration of "set", you will find only three entries.

I just test a nightly build of 20091010, the set class was correctly recognized.

see the screenshot below:


Edit:


To simplify the bug, you can just copy the code in

d:\MinGW\lib\gcc\mingw32\4.4.1\include\c++\bits\stl_set.h

to a console project, delete all the #include directive and #if statement, and the code should like below:

Code


_GLIBCXX_BEGIN_NESTED_NAMESPACE(std, _GLIBCXX_STD_D)

  /**
   *  @brief A standard container made up of unique keys, which can be
   *  retrieved in logarithmic time.
   *
   *  @ingroup associative_containers
   *
   *  Meets the requirements of a <a href="tables.html#65">container</a>, a
   *  <a href="tables.html#66">reversible container</a>, and an
   *  <a href="tables.html#69">associative container</a> (using unique keys).
   *
   *  Sets support bidirectional iterators.
   *
   *  @param  Key  Type of key objects.
   *  @param  Compare  Comparison function object type, defaults to less<Key>.
   *  @param  Alloc  Allocator type, defaults to allocator<Key>.
   *
   *  The private tree data is declared exactly the same way for set and
   *  multiset; the distinction is made entirely in how the tree functions are
   *  called (*_unique versus *_equal, same as the standard).
  */
  template<typename _Key, typename _Compare = std::less<_Key>,
   typename _Alloc = std::allocator<_Key> >
    class set
    {
      // concept requirements
      typedef typename _Alloc::value_type                   _Alloc_value_type;
      __glibcxx_class_requires(_Key, _SGIAssignableConcept)
      __glibcxx_class_requires4(_Compare, bool, _Key, _Key,
_BinaryFunctionConcept)
      __glibcxx_class_requires2(_Key, _Alloc_value_type, _SameTypeConcept)

    public:
      // typedefs:
      //@{
      /// Public typedefs.
      typedef _Key     key_type;
      typedef _Key     value_type;
      typedef _Compare key_compare;
      typedef _Compare value_compare;
      typedef _Alloc   allocator_type;
      //@}

    private:
      typedef typename _Alloc::template rebind<_Key>::other _Key_alloc_type;

      typedef _Rb_tree<key_type, value_type, _Identity<value_type>,
       key_compare, _Key_alloc_type> _Rep_type;
      _Rep_type _M_t;  // Red-black tree representing set.

    public:
      //@{
      ///  Iterator-related typedefs.
      typedef typename _Key_alloc_type::pointer             pointer;
      typedef typename _Key_alloc_type::const_pointer       const_pointer;
      typedef typename _Key_alloc_type::reference           reference;
      typedef typename _Key_alloc_type::const_reference     const_reference;
      // _GLIBCXX_RESOLVE_LIB_DEFECTS
      // DR 103. set::iterator is required to be modifiable,
      // but this allows modification of keys.
      typedef typename _Rep_type::const_iterator            iterator;
      typedef typename _Rep_type::const_iterator            const_iterator;
      typedef typename _Rep_type::const_reverse_iterator    reverse_iterator;
      typedef typename _Rep_type::const_reverse_iterator    const_reverse_iterator;
      typedef typename _Rep_type::size_type                 size_type;
      typedef typename _Rep_type::difference_type           difference_type;
      //@}

      // allocation/deallocation
      /**
       *  @brief  Default constructor creates no elements.
       */
      set()
      : _M_t() { }

      /**
       *  @brief  Creates a %set with no elements.
       *  @param  comp  Comparator to use.
       *  @param  a  An allocator object.
       */
      explicit
      set(const _Compare& __comp,
  const allocator_type& __a = allocator_type())
      : _M_t(__comp, __a) { }


......
......


and see the parser works ok or not.



[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 20, 2009, 01:01:07 pm
A further test give the bad result, see the screen shot below:
(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2009-12-20194347.png)
The parser result is totally wrong.

I personally suspect this bug is introduced by the patch of reading the "template" argument of "visualfc", I'm still examining. Enable PARSERTHREAD_DEBUG_OUTPUT and see the output.
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 20, 2009, 02:20:02 pm
For example, the code in parserthread.cpp

Code
        if (token == _T("template"))
        {
            wxString args = m_Tokenizer.GetToken(false,true);
            token = m_Tokenizer.GetToken();
            if (args == _T("<>"))
            {
                m_Str.Clear();
                wxString name = m_Tokenizer.GetToken();
                args = m_Tokenizer.PeekToken(false,true);
                m_Tokenizer.UngetToken();
            }
            if (token==ParserConsts::kw_class)
            {
                m_Str.Clear();
                if (m_Options.handleClasses)
                    HandleClass(ctClass,args);
                else
                    SkipToOneOfChars(ParserConsts::semicolonclbrace, true);
                continue;
            }
            else if (token==ParserConsts::kw_struct)
            {
                m_Str.Clear();
                if (m_Options.handleClasses)
                    HandleClass(ctStructure,args);
                else
                    SkipToOneOfChars(ParserConsts::semicolonclbrace, true);
                continue;
            }
            else
            {
                m_Tokenizer.UngetToken();
            }
        }


It is really hard to understand the function call like "m_Tokenizer.GetToken(false,true);" , I think the function should still keep the consistent prototype like( simple and easy to understand) :

GetToken()
PeekToken()

I still can't understand what does these prototype means:
Code
    wxString GetToken(bool bGetValue = false, bool bTemplate = false);
    wxString PeekToken(bool bGetValue = false, bool bTemplate = false);
:(
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on December 20, 2009, 02:50:47 pm
I still can't understand what does these prototype means:
Code
    wxString GetToken(bool bGetValue = false, bool bTemplate = false);
    wxString PeekToken(bool bGetValue = false, bool bTemplate = false);
:(
Agreed, the interface could be more clear. Please PM visualfc so he can have a look into...
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 20, 2009, 03:00:17 pm
I still can't understand what does these prototype means:
Code
    wxString GetToken(bool bGetValue = false, bool bTemplate = false);
    wxString PeekToken(bool bGetValue = false, bool bTemplate = false);
:(
Agreed, the interface could be more clear. Please PM visualfc so he can have a look into...
Thanks, that's great that we have the same consideration  :D.

Yes, I have notified him.

Currently, a workaround is to remove the "handling template related code". :D


Title: Re: New code completion remarks/issues
Post by: MortenMacFly on December 20, 2009, 03:26:24 pm
Currently, a workaround is to remove the "handling template related code". :D
True, but in fact I added this to trunk because in fact it helps a lot if it works. In my own projects I use templates on a "normal" level and it works very well there and thus is really helpful (for me at least).
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 21, 2009, 05:47:31 am
A reminder: This bug was introduced in the rev 5945.

In DoGetToken function, these code:

Code

    else if (CurrentChar() == '<' && bTemplate)
    {
        MoveToNextChar();
        if (!SkipToOneOfChars(_T(">\r\n")), false)
            return wxEmptyString;

        MoveToNextChar();
        wxString tmp = m_Buffer.Mid(start+1,m_TokenIndex-start-2);
        tmp.Trim();
        str = _T("<");
        str += tmp;
        str += _T(">"); // m_Buffer.Mid(start, m_TokenIndex - start);
    }

So, what does the SkipToOneOfChars(_T(">\r\n")), false) means?

It seems the ">" or "\r" or "\n" was the end of a template argument???
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on December 21, 2009, 06:40:27 am
So, what does the SkipToOneOfChars(_T(">\r\n")), false) means?
So it seems VisualFC assumed that template arguments are no longer than a single line which isn't true for the STL set as you've seen... :lol:
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 21, 2009, 06:49:51 am
Ok, I'm planning to add a "TokenizerState" variable in the Tokenizer class.

The TokenizerState variable is normally a enum variable, and could have the following types:

Code
UnWanted
Preprocessor
Template

Then, the pseudo code could be something like from the old style:

Code
void ParserThread::HandleXXXXXX()
{
    // need to force the tokenizer _not_ skip anything
    // or else default values for template params would cause us to miss everything (because of the '=' symbol)
    bool oldState = m_Tokenizer.IsSkippingUnwantedTokens();
    m_Tokenizer.SetSkipUnwantedTokens(false);
   
    DoSomething();

    m_Tokenizer.SetSkipUnwantedTokens(oldState);
}

The new style:

Code
void ParserThread::HandleXXXXXX()
{

    m_Tokenizer.PushState();
   
    m_Tokenizer.SetState(YYYYY);

    DoSomething();
   
    m_Tokenizer.PopState();
}

I'm not sure the "POP" and "PUSH" mechanism can work here.

Title: Re: New code completion remarks/issues
Post by: ollydbg on December 21, 2009, 07:00:23 am
So, what does the SkipToOneOfChars(_T(">\r\n")), false) means?
So it seems VisualFC assumed that template arguments are no longer than a single line which isn't true for the STL set as you've seen... :lol:

Also, what about use :

SkipToOneOfChars(_T(">")), true)

because we should let the template argument support nesting of like <   XXXX<YYYYY>>.
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 21, 2009, 09:00:40 am
Wait, there is a big syntax error in the if statement.

Code
 if (   !SkipToOneOfChars(_T(">\r\n")), false      )

look at the pair of parentheses!!!????

it is
Code
if ( XXXX, false)
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 21, 2009, 09:22:30 am
Patch to fix the "template reading bug" , this patch contains some patch in CC comments improvement (http://forums.codeblocks.org/index.php/topic,11655.0.html)



[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on December 21, 2009, 11:25:33 am
Patch to fix the "template reading bug" ,
Wow, that was fast (I didn't even manage to run C::B today). IF this works that would be nice!

BTW: Please don't include other patches next time, if possible. This makes merging for me a little easier. ;-)
Title: Re: New code completion remarks/issues
Post by: ollydbg on December 22, 2009, 07:27:34 am
hi, morten, I have an idea, I want to hear your comments.

Currently, the Tokenizer is only return a wxString as a Token.
But my idea is, Tokenizer should return a Token structure.

For example

Code
void f(XXX);

In the current trunk code, the Tokenizer will return four wxString.

Code
void
f
(XXX)
;

But my suggestion is to create a Token structure like
Code
struct Token{
wxString name;
Tokentype type;
}

So, the Tokenizer::GetToken will return four strictures, because When Tokenizer do the "DoGetToken" function, it is already know the Token's type, so, we don't need the check the wxString in ParserThread.(mostly, we need to check the first character of the wxstring to determine this is a (XXXX)).
Code
void its type is normalText
f     its type is normalText
(XXX) its type is functionArgument
;       its type is endofStatement

This will make parserThread more happier and easy to determine the syntax.


Edit

following my idea, the DoGetToken can be changed to like below:
Code
    else if (CurrentChar() == '<' && bTemplate)
    {
        MoveToNextChar();
        if (!SkipToOneOfChars(_T(">")), true)
            return wxEmptyString;

        MoveToNextChar();
        wxString tmp = m_Buffer.Mid(start+1,m_TokenIndex-start-2);
        tmp.Trim();
        str = _T("<");
        str += tmp;
        str += _T(">"); // m_Buffer.Mid(start, m_TokenIndex - start);
        type = TemplateArgument;
    }
    else if (c == '(')
    {
        m_IsOperator = false;

        // skip blocks () []
        if (!SkipBlock(CurrentChar()))
            return wxEmptyString;

        str = FixArgument(m_Buffer.Mid(start, m_TokenIndex - start));
        CompactSpaces(str);
        type = functionArgument;
    }
    else
    {
        if      (c == '{')
            ++m_NestLevel;
            type = OpenBrace
        else if (c == '}')
            --m_NestLevel;
            type = CloseBrace

        str = c;
        MoveToNextChar();
    }

Take care of the "type" variable, it is a enum variable, can have the type like:

Code
enum Tokentype{

normalText;
OpenBrace;
CloseBrace;
EndOfStatement;
FunctionArgument;
TemplateArgument;
....

}

So, in the parserThread void ParserThread::DoParse, we can use a switch statement


Code
switch(type){

case normaltext: XXXX
case:functionArgument: XXXX


}

This will runs faster then the many "if - else if ....."
Title: Re: New code completion remarks/issues
Post by: blueshake on December 24, 2009, 10:10:15 am
Quote
Patch to fix the "template reading bug" , this patch contains some patch in CC comments improvement

forget to do this in patch.(will make the parser in a long work time.) :lol:
Code
#define PARSERTHREAD_DEBUG_OUTPUT 0
Title: Re: New code completion remarks/issues
Post by: blueshake on December 25, 2009, 06:20:57 am
hi,mortern:
can you comment these codes,because they make no sense.

Code
Index: src/plugins/codecompletion/parser/parserthread.cpp
===================================================================
--- src/plugins/codecompletion/parser/parserthread.cpp (revision 5986)
+++ src/plugins/codecompletion/parser/parserthread.cpp (working copy)
@@ -661,17 +661,17 @@
                     && (!m_pLastParent || m_pLastParent->m_Name != token) ) // if func has same name as current scope (class)
                 {
                     // test Macro
-                    if (token.Upper() == token)
+                    /*if (token.Upper() == token)
                     {
                         HandleMacro(token, peek);
                         m_Tokenizer.GetToken();
                         m_Str.Clear();
                     }
                     else
-                    {
+                    {*/
                         wxString arg = m_Tokenizer.GetToken(); // eat args ()
                         m_Str = token+arg;
-                    }
+                    //}
                 }
                 else if (peek.GetChar(0) == '(' && m_Options.handleFunctions)
                 {
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on December 25, 2009, 12:03:48 pm
can you comment these codes,because they make no sense.
Why exactly does that make no sense? Because if commented macros are not being handeled at all.
Title: Re: New code completion remarks/issues
Post by: blueshake on December 25, 2009, 12:40:47 pm
check this
Code
if (token.Upper() == token)


it make a hypothesis here that all macro should be capital,but if the macros are not capital letters,this will not work.

for example:

LINUX(ARG)  ---------------work.


linUX(ARG)    ----------------not work


and in this situation,should in using macro but not defining a macro.so it is no reason to add macro here.
Title: Re: New code completion remarks/issues
Post by: MortenMacFly on December 25, 2009, 08:07:24 pm
LINUX(ARG)  ---------------work.
linUX(ARG)    ----------------not work
This is true, however, macros are usually uppercase it's like a rule everybody should follow. In addition it worked nicely for me for quite some macros that were CC'ed very nicely afterwards. However, I agree that the detection algorithm should be more precise. So I'll comment it for a while and add a ToDo there...

Pending changes will be committed a little later... I am still testing some other CC related stuff (including your other patch...).
Title: Re: New code completion remarks/issues
Post by: Techy on January 01, 2010, 05:34:58 pm
Hi,

I started experiencing quite serious performance issue after rev. 5945 for big projects. To be more precise - the code that 5945 introduced doesn't cause the problem - it just made the problem more visible. What happened after this commit was that the number of enums found in the project increased to about 18000 (this looks like a correct number - I was grepping through the whole project and there are about 60000 occurrences of enum there - some possibly comments so this seems to be more or less OK). Parsing is quite fast (about 10s), but then the whole GUI freezes for about 40 seconds (CPU 100%). This happens even when re-saving a file (parsing 0.1s), but again the whole 40s freeze of the GUI. I experienced this before as well, but the freeze took only 2 seconds so I thought it was the old Linux I was using.

OK, what happens - parsing is really fast and because it runs in a separate thread, id doesn't block the GUI. However, when parsing is finished, NativeParser::OnParserEnd is invoked and there parser->LinkInheritance(false) is called, which is the thing that takes the terrible amount of time. So my question is:

1. Could this function be called only when "Display inheritance info" is selected in the options page or "Show inherited members" is selected from the right-click menu? Is the info used by calling this function used by something else as well? (I'm running codeblocks with the problematic command commented-out now without any major issue so I guess this should be possible). It would be good to be able to at least disable it.

2. Could this thing be performed in the parser thread? This at least wouldn't freeze the GUI. (Again, I think this might be possible as well.)

3. Could this function be done in a smarter way? For instance building the complete tree only once at the beginning and adding the classes from the saved file to the hierarchy. I haven't studied the algorithm of this thing in detail yet but I think it has quadratic complexity (at least when thinking how to implement it, the most trivial way has quadratic complexity). This maybe could be improved as well. Finally, I've seen that TokensTree::RecalcData() goes through all classes, typedefs and enums - only typedefs for classes and structs are interesting for this function so the rest could be filtered out. Why are the enums there? (Again, I haven't studied this in detail so maybe the above is done already - just putting some ideas I have here).

Your opinion?


Title: Re: New code completion remarks/issues
Post by: MortenMacFly on January 01, 2010, 07:56:29 pm
1. Could this function be called only when "Display inheritance info" is selected in the options page or "Show inherited members" is selected from the right-click menu?
2. Could this thing be performed in the parser thread?
3. Could this function be done in a smarter way?

Your opinion?
To make it simple: 1.) Yes, 2.) Yes, 3.) Yes.

However, I didn't realise such a bottleneck with enums. What project are you talking about? Is this a public one so I can try?
Title: Re: New code completion remarks/issues
Post by: Techy on January 02, 2010, 03:27:32 pm
First please replace the word "enum" with "typedef" in my previous post - the typedefs and classes are those problematic things...

(But I'm still wondering why the enums are there in RecalcData() [and typedefs as well] - it's a bit array so [without looking at the parser itself] I guess that for "typedef struct" you set both tkClass and tkTypedef so you can test for tkClass only here...)

Well, the project I'm working on isn't free so you cannot try it. However, the good news is that the problematic part of the project is the boost library it uses (the source tree contains a partial copy of boost). Download the full sources from here:

http://www.boost.org/

and add all the source files recursively to a project. With full copy of boost I wasn't even patient enough to wait until the tree is generated and killed codeblocks... (let me know if you are able to reproduce it)

I've also analysed this behaviour a bit: the part marked as     

// first loop to convert ancestors string to token indices for each token

takes about 7 seconds - even though this not so much compared to the total time, I think that this can be done completely during the parsing stage and not being reparsed every time (anyway, this would have to be done in a thread not to block the GUI).

But the most problematic part is

// second loop to calculate full inheritance for each token

I think that the recursive algorithm you use is suboptimal - for every class you compute its ancestors again and again, but you could reuse the work you have already done for the subclasses. With the copy of boost my project uses, the maximal depth of the inheritance hierarchy was 74. After thinking about it a bit (and getting some graph algorithms from the back of my brain), this should be much more optimal algorithm:

1. Set X equal to the set of all classes and Y equal to empty set
2. Set Z equal to the set of all classes that do not inherit anything
3. Set the set of all ancestors and descendants of all members of Z equal to empty set

do while X is getting smaller:
    4. Set X = X - Z, Y = Y union Z, and Z = emptyset
    5. Set Z equal to the set of all classes that inherit from some class from Y (in the case of multiple inheritance all the ancestors have to be from Y - otherwise don't put the class to Z)
    6. For every member M of Z set the set of all ancestors equal to the set of ancestors of the classes it inherits from (those from Y) plus the classes themselves. At the same time, for each of the ancestors of M add M as its descendant.

(In correctly written C++ program X should be empty set but before compiling the user can make an error so there is a loop in the inheritance. This is the reason why there is the condition "do while X is getting smaller")

The advantage is clear - you go up from the classes that don't inherit anything and reuse what you did in the previous step (and in addition, you avoid recursion which can speed up things a bit as well)

So a good start would be to implement this algorithm and see how much it helps (right now I don't have time to do this myself, sorry). If it's not enough (even if the time reduces to 1s, the freeze is still quite unpleasant), then the function should be put into the parser thread.
Title: Re: New code completion remarks/issues
Post by: Techy on January 02, 2010, 05:16:05 pm
Just one more note about data types when implementing it - X can be a list of pointers to tokens, Y should be a map indexed by token index with a pointer to the token as the contents (we'll be mainly searching in it so it should be a map) and Z will not exist at all (you'll take members of X one by one and either move them to Y or leave them in X).
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on January 02, 2010, 06:55:59 pm
Simple example that breaks the completion. Place the code below in the main.cpp of new console project:
Code

class AClass
{
    AClass(int a, int b)
    {
    }
    int method(int b) { return 0; }
};

int main()
{
    AClass a(5, 6);
    a.method(1);

    return 0;
}

Steps
0. Save the file
1. Place the currsor at the end of line "a.method(1);"
2. Press enter
3. Type "a."
4. Press ctrl+space
5. Nothing shows

Also the calltip for the constructor call doesn't work :(
Using latest version from trunk (6023).
Title: Re: New code completion remarks/issues
Post by: Jenna on January 02, 2010, 07:39:19 pm
Simple example that breaks the completion. Place the code below in the main.cpp of new console project:
Code

class AClass
{
    AClass(int a, int b)
    {
    }
    int method(int b) { return 0; }
};

int main()
{
    AClass a(5, 6);
    a.method(1);

    return 0;
}

Steps
0. Save the file
1. Place the currsor at the end of line "a.method(1);"
2. Press enter
3. Type "a."
4. Press ctrl+space
5. Nothing shows

Also the calltip for the constructor call doesn't work :(
Using latest version from trunk (6023).

Works fine here (debian 64-bit).
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on January 02, 2010, 07:45:25 pm
Works fine here (debian 64-bit).
Both work (the completion and the calltip problems)?
Title: Re: New code completion remarks/issues
Post by: Jenna on January 02, 2010, 08:26:39 pm
Works fine here (debian 64-bit).
Both work (the completion and the calltip problems)?

No only the completion and the tooltip for method.
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on January 29, 2010, 11:28:41 pm
Another bug, this time 100% reproducible:

(http://smrt.is-a-geek.org/codeblocks/cb_cc_bug.png)

Steps:
1. Create console project
2. Remove the auto generated code
3. Paste the code below:
Code

struct TestStruct
{
    int a;

    int Test()
    {
        int test_1, test_2;
        tes
    }
};

4. Hit ctrl + space at "tes" line
5. Type t_ and repeat step 4. ... works as expected.

Tested on gentoo amd64 + cb 6112 debugger branch
Title: Re: New code completion remarks/issues
Post by: ollydbg on January 31, 2010, 02:23:10 am
Another bug, this time 100% reproducible:

Ok, I just do more test, for example, if you paste these code to your main.cpp.
Test Code one:
Code
struct TestStruct
{
    int test_3;

    int Test()
    {
        int test_1, test_2;
        te
    }
};
Then you get the auto-completion list is:(see the screen shot as attachment 1)
Code
template
Test
test_3

These Code two:
Code
struct TestStruct
{
    int test_3;

    int Test()
    {
        int test_1, test_2;
        test_
    }
};
Then you will get only "test_3" in auto-completion list.(see attachment2)

These code three:
Code
struct TestStruct
{
    int aaaaa;

    int Test()
    {
        int test_1, test_2;
        test_
    }
};

Now, it can autocomplete both "test_1" and "test_3". see the screen shot of attachment 3.




[attachment deleted by admin]
Title: Re: New code completion remarks/issues
Post by: ollydbg on January 31, 2010, 02:54:04 am
Let me explain why this bug happened.
It is related to the "initial search scope" in the CC's source code of AI() function.

Code
// Start an Artificial Intelligence (!) sequence to gather all the matching tokens..
// The actual AI is in FindAIMatches() below...
size_t NativeParser::AI(TokenIdxSet& result,
                        cbEditor* editor,
                        const wxString& lineText,
                        bool noPartialMatch,
                        bool caseSensitive,
                        TokenIdxSet* search_scope,
                        int caretPos)
{
......
......



    if (result.size()<1) // found nothing in the search_scope, add global namespace
    {
        if (s_DebugSmartSense)
            Manager::Get()->GetLogManager()->DebugLog(F(_T("AI() result is zero. Adding global namespace.")));

        search_scope->insert(-1);
        FindAIMatches(components, result, -1, noPartialMatch, caseSensitive, true, 0xffff, search_scope);
    }

......



In the AI function, we firstly collect the "initial search scope", then do the recursive matching stage. I have already written some steps in the wiki, you can see here: 5 Automatic Code Completion (http://wiki.codeblocks.org/index.php?title=Code_Completion_Design#Automatic_Code_Completion), so, the "global namespace scope" is not added by default.( if I can remember, in a earlier revision, when we were collecting initial search scopes", global namespace is always added)

You can see the "if" condition.

For example, when we do the AI function in the code below:
Code
struct TestStruct
{
    int test_3;

    int Test()
    {
        int test_1, test_2;
        te
    }
};

We firstly add the initial scope is "struct TestStruct", then do the AIMatch routing in this scope. Too bad that when the "local parser" parsing the local statement, both "test_1" and "test_2" will added to the "global scope namespace":
Code
{
        int test_1, test_2;
        te
    }

So, when we do a AIMatch in the "strct TestStruct", we find we get the two matches: "Test()" and the member variable "test_3". At this time, we have already get the matching result size >1, so, due to the if condition, global namespace is not searched any more.

Solution:

We can have two solutions:
One: we can force to add the "global search scope" to the initial search, so that we can get the right matching result. the Pros is it is quite simple, the cons is that you will loose the performance and cause other issue.

For example:

Code
int abcd;

int main()
{
    int abcd;
   ab
}

Here, both global variable and auto variable name will be prompted. In CB forum, some people just suggest the "auto variable" in the function body.

Two:
We don't need to add the global namespace in the initial search scope.
When the "local parser" parse the function body, the "auto variable" added as the children of the function body.

For example:
Code
struct TestStruct
{
    int test_3;

    int Test()
    {
        int test_1, test_2;
        te
    }
};
This time, the initial search scope should be: "struct TestStruct" and the "Test()".
So, when we do an AIMatch under "TestStruct" scope, we get the "test_3" and "Test()" matches.
Also, when we do an AIMatch under "Test()", we get the "test_1" and "test_2" matches.
So, we get the expected prompted list.

I have tried to using this method, but I didn't get success, because, when a parserthread attached with a "local body" is initialed, the defualt m_Parent Token is always the "global namespace", so the "test_2" and "test_1" will always be added to the "global namespace".

If we can supply a way that we can "set" m_Parent pointer to "Test() Token", we can totally solved this problem.





Title: Re: New code completion remarks/issues
Post by: oBFusCATed on February 07, 2010, 05:58:32 pm
We firstly add the initial scope is "struct TestStruct", then do the AIMatch routing in this scope. Too bad that when the "local parser" parsing the local statement, both "test_1" and "test_2" will added to the "global scope namespace":
Code
{
        int test_1, test_2;
        te
    }

So, when we do a AIMatch in the "strct TestStruct", we find we get the two matches: "Test()" and the member variable "test_3". At this time, we have already get the matching result size >1, so, due to the if condition, global namespace is not searched any more.

So, this is a parser bug? Can it be fixed easily?
Title: Re: New code completion remarks/issues
Post by: blueshake on February 09, 2010, 01:41:45 pm
It is not a bug.it can be fixed easily. :D
Title: Re: New code completion remarks/issues
Post by: damur on February 18, 2010, 02:11:47 am
Hi!

I tried to use wxWidgets in Codeblocks, but Code Completion does not work correctly. And I think it's because of an #elif directive. Try this:
Code
#define CONDITION

#if defined(SOMETHING)
    #include "something.h"
#elif defined(CONDITION)
    #include "class1.h"
#endif


int main()
{
    return 0;
}
class1.h is a file which is not part of your project (because else it would be parsed either way). So the file class1.h doesn't get parsed.
Interesting: replace "something.h" with "class1.h" and class1.h gets parsed, although SOMETHING is not defined.

Directives like this are used in most header files of wx base classes, for example "wx/button.h"

I'm using ubuntu9.10 and codeblocks SVN 6088. I tried also the 8.02 stable release, but it had the same bug.
Title: Re: New code completion remarks/issues
Post by: blueshake on February 18, 2010, 02:24:51 am
it is not a bug.it is just not implemented yet.Be patient! Ollydbg had done a little work.it will be solved soon.
Title: Re: New code completion remarks/issues
Post by: ollydbg on March 02, 2010, 07:13:00 am
It is not a bug.it can be fixed easily. :D
@oBFusCATed
This is fixed in blueshakes this post: new cc search implemention (http://forums.codeblocks.org/index.php/topic,12096.0.html).
Title: Re: New code completion remarks/issues
Post by: oBFusCATed on March 02, 2010, 09:09:48 am
OK, Might test it...
Title: Re: New code completion remarks/issues
Post by: ollydbg on March 02, 2010, 09:13:58 am
OK, Might test it...
In fact, only one statement do the hack:

Code
search_scope->insert(-1);//alwayser search the global scope.