Author Topic: New code completion remarks/issues  (Read 211960 times)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #180 on: October 11, 2009, 03:07:39 pm »
What I want to know is if doing this wil slow down the parse process?

parse C::B source code.
before patch
Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline koso

  • Multiple posting newcomer
  • *
  • Posts: 58
Re: New code completion remarks/issues
« Reply #181 on: October 11, 2009, 04:06:42 pm »
When you are talking about performance ...

I was using Symbol browser with view set to "Everything". Problem is, that now when I open project, first few seconds is C::B parsing files, and then it starts to update data in symbol browser. Becase of setting it to "Everything", it takes minutes .. and problem is, that during this operation is C::B dead = marked as "Not responding" = so I can do nothing, only look at white window (even draw events are ignored during this operation) or exit. So is there some way, how to do this more in background and not freeze whole UI?
And btw. this happens also when symbol browser is hidden.
 

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #182 on: October 12, 2009, 09:07:19 am »
Dear Morten:
Have you started the plan on macro replacement and template parse?
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #183 on: October 18, 2009, 04:40:45 pm »
Hi, morten, I find some bugs in CC, but I'm sorry, I can't create a patch, because my local copy was changed a lot. so, here are the modified code:

1.

plugins\codecompletion\classbrowser.cpp

function:

Code
void ClassBrowser::UpdateView()
{
    m_pActiveProject = 0;
    m_ActiveFilename.Clear();
    if (m_pParser && !Manager::IsAppShuttingDown())
    {
        m_pActiveProject = Manager::Get()->GetProjectManager()->GetActiveProject();
        cbEditor* ed = Manager::Get()->GetEditorManager()->GetBuiltinActiveEditor();
        if (ed)
        {
            //m_ActiveFilename = ed->GetFilename().BeforeLast(_T('.'));
            // the above line is a bug (see https://developer.berlios.de/patch/index.php?func=detailpatch&patch_id=1559&group_id=5358)
            m_ActiveFilename = ed->GetFilename().AfterLast(wxFILE_SEP_PATH);

            if(  m_ActiveFilename.Find(_T('.')) != wxNOT_FOUND  )
            {
                m_ActiveFilename = ed->GetFilename().BeforeLast(wxFILE_SEP_PATH) + wxFILE_SEP_PATH + m_ActiveFilename.BeforeLast(_T('.'));
                m_ActiveFilename.Append(_T('.'));
            }
            else
                m_ActiveFilename = ed->GetFilename();

        }

        BuildTree();

        wxSplitterWindow* splitter = XRCCTRL(*this, "splitterWin", wxSplitterWindow);
        if (m_pParser->ClassBrowserOptions().treeMembers)
        {
            splitter->SplitHorizontally(m_Tree, m_TreeBottom);
            m_TreeBottom->Show(true);
        }
        else
        {
            splitter->Unsplit();
            m_TreeBottom->Show(false);
        }

    }
    else
        m_Tree->DeleteAllItems();
}

For example, I have a file:

d:\MinGW\lib\gcc\mingw32\4.4.1\include\c++\vector

So, I would like to see the tokens in this file.


2. Also, the function can give the right tokens in the right file, but it seems the Browser build thread still remove all the nodes.

For example:

d:\MinGW\lib\gcc\mingw32\4.4.1\include\c++\bits\stl_vector.h

Loot at the symbol browser tree window, there is no node shown there. :(


3.
I have change the SkipComments function in tokenizer.cpp

See here

Code
bool Tokenizer::SkipComment(bool skipEndWhite)
{

    bool cstyle;            // C or C++ style comments

    //check the comment prompt
    if(CurrentChar() == '/')
    {
        if(NextChar() == '*')
            cstyle = true;
        else if(NextChar() == '/')
            cstyle = false;
        else
            return true;    //Not the comment prompt, return true;
    }
    else
        return true;        //Not the comment prompt, return true;

    MoveToNextChar(2);      //skip the comment prompt

    while(true)
    {
        if (cstyle)        //c style comment
        {
            SkipToChar('/');
            if (PreviousChar() == '*')//the end of C style comments
            {
                MoveToNextChar();
                break;
            }
            if(!MoveToNextChar())
                break;
        }
        else                //c++ style comment
        {
            SkipToEOL(false, true);//nestBrace = false skipcomment = true
            MoveToNextChar();
            break;
        }
    }


    if (IsEOF())
        return false;

    if (skipEndWhite)
    {
        if(!SkipWhiteSpace())
            return false;
    }
    else
        return true;

    return SkipComment(); // handle chained comments
}

For the old code, if skipEndWhite == false, the function will always return false, this is something wrong.


...

Bed time,,,,  I will add more tomorrow. :D

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #184 on: October 19, 2009, 04:09:49 pm »
1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function

Code
    if (    m_LastWasPreprocessor
        && !str.IsSameAs(_T("#"))
        && !m_LastPreprocessor.IsSameAs(_T("#")) )
    {
        if (!m_LastPreprocessor.IsSameAs(TokenizerConsts::include_str))
        {
            // except for #include and #if[[n]def], all other preprocessor directives need only
            // one word exactly after the directive, e.g. #define THIS_WORD
            SkipToEOL();
        }
        m_LastPreprocessor.Clear();
    }
So, these code can be deleted.

2, I tokenizer.h think in the CurrentChar function, we don't need to check index value, because MoveToNextChar already do this.
Code
    bool MoveToNextChar(const unsigned int amount = 1)
    {
        assert(amount);
        if(amount == 1) // compiler will dead-strip this
        {
            ++m_TokenIndex;
            if (IsEOF())
            {
                m_TokenIndex = m_BufferLen;
                return false;
            }

            if (CurrentChar() == _T('\n'))
                ++m_LineNumber;
            return true;
        }
        else
        {
            m_TokenIndex += amount;
            if (IsEOF())
            {
                m_TokenIndex = m_BufferLen;
                return false;
            }

            if (CurrentChar() == _T('\n'))
                ++m_LineNumber;
            return true;
        }
    };

    wxChar CurrentChar() const
    {
        if(m_TokenIndex < m_BufferLen)
            return m_Buffer.GetChar(m_TokenIndex);
        return 0;
    };
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: New code completion remarks/issues
« Reply #185 on: October 19, 2009, 08:57:51 pm »
Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).
...did you notice? That's weired.

1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function
[...]
So, these code can be deleted.
...which one (which if condition) you mean? For me both still make sense...?!

2, I tokenizer.h think in the CurrentChar function, we don't need to check index value, because MoveToNextChar already do this.
That's true, but I wanted to avoid any issues with API mis-use (meaning calling CurrentChar() before MoveToNextChar() for any obscure reason). I wonder what this would differ in terms of time. Implementing a temporary time measurement would reveal the difference.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #186 on: October 20, 2009, 03:29:50 am »
Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).
...did you notice? That's weird.
Yes, it is weird. I don't know why...



1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function
[...]
So, these code can be deleted.
...which one (which if condition) you mean? For me both still make sense...?!

Both the if condition were not be hit if I set breakpoint there.
See my screen shot

I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:
Code
bool Tokenizer::SkipUnwanted()
{
    while (CurrentChar() == '#' ||
            (!m_IsOperator && CurrentChar() == '=') ||
            (!m_IsOperator && CurrentChar() == '[') ||
            CurrentChar() == '?' ||
            (CurrentChar() == '/' && (NextChar() == '/' || NextChar() == '*') ))
    {
        bool skipPreprocessor = false; // used for #include
        while (m_Buffer.Mid(m_TokenIndex, 2) == _T("//") ||
                m_Buffer.Mid(m_TokenIndex, 2) == _T("/*"))
        {
            // C/C++ style comments
            SkipComment();
            if (IsEOF())
                return false;
            if (!SkipWhiteSpace())
                return false;
        }

        while (CurrentChar() == '#')
        {
            // preprocessor directives
            // we only care for #include and #define, for now
            unsigned int backupIdx = m_TokenIndex;
            MoveToNextChar();
            SkipWhiteSpace();
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: New code completion remarks/issues
« Reply #187 on: October 20, 2009, 07:52:43 am »
I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:
So... now you got me a little confused (or probably it's too early for me). So you propose to remove the "if" conditions because of a bug in SkipUnwanted? Shouldn't we better fix the bug? And then: Where exactly is the bug in the code snippet?
Could you do me a favour and comment the lines / snippets in question in the code you've posted? That'll help me a lot. Please explain a little more... :(
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #188 on: October 20, 2009, 08:15:30 am »
I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:
So... now you got me a little confused (or probably it's too early for me). So you propose to remove the "if" conditions because of a bug in SkipUnwanted? Shouldn't we better fix the bug? And then: Where exactly is the bug in the code snippet?
Could you do me a favour and comment the lines / snippets in question in the code you've posted? That'll help me a lot. Please explain a little more... :(

Ok, let me explain:

There are two member variables:

m_LastWasPreprocessor is a bool
m_LastPreprocessor is a wxString

But the DoGetToken() only return the variable "str"

Code
    if (    m_LastWasPreprocessor
        && !str.IsSameAs(_T("#"))
        && !m_LastPreprocessor.IsSameAs(_T("#")) )
    {
        if (!m_LastPreprocessor.IsSameAs(TokenizerConsts::include_str))
        {
            // except for #include and #if[[n]def], all other preprocessor directives need only
            // one word exactly after the directive, e.g. #define THIS_WORD
            SkipToEOL();
        }
        m_LastPreprocessor.Clear();
    }

    if (m_LastWasPreprocessor)
        m_LastPreprocessor << str;

    m_LastWasPreprocessor = false;

    return str;

So, str will never be affected by m_LastWasPreprocessor  or m_LastPreprocessor.

So, these two member variables are useless.


If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #189 on: October 20, 2009, 10:09:38 am »
what more I want to say is the logical error:
The only chance the variable m_LastWasPreprocessor  becomes true is in these codes:
Code
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else

at this time,str contain the string "#",and m_LastPreprocessor is empty.(see codes above)
so the three conditions in if:

m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true

see ,the three conditions can not all be true at the same time.It means the "if " condition can never be true.
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #190 on: October 20, 2009, 10:12:06 am »
what more I want to say is the logical error:
The only chance the variable m_LastWasPreprocessor  becomes true is in these codes:
Code
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else

at this time,str contain the string "#",and m_LastPreprocessor is empty.(see codes above)
so the three conditions in if:

m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true

see ,the three conditions can not all be true at the same time.It means the "if " condition can never be true.
Hi, blueshake, Thanks for the explanation. :D.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: New code completion remarks/issues
« Reply #191 on: October 20, 2009, 03:33:06 pm »
m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true
Could it be that simply the "!str..." line should be removed as it is a relict of old times???

From what I read in the code it simplifies preprocessor handling in a way that it just gets rid of everything after the first preprocessor directive as it is unhandled anyways (except for #include and #if[[n]def], as stated in the comment)... which is good.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: New code completion remarks/issues
« Reply #192 on: October 20, 2009, 03:59:50 pm »
My another question :
what is the effort of m_LastPreprocessor .since when the variable m_LastWasPreprocessor is true,the  m_LastPreprocessor is absoutely empty.see codes below.
Code
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else


EDIT:
In the three conditions,
m_LastWasPreprocessor is true,!m_LastPreprocessor.IsSameAs(_T("#")) is absoutely true too.
« Last Edit: October 20, 2009, 04:02:09 pm by blueshake »
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: New code completion remarks/issues
« Reply #193 on: October 20, 2009, 04:14:58 pm »
Hi,
I would like to say:
there is no use of "m_LastPreprocessor" and "m_LastWasPreprocessor".
Let me explain more, In the parserthread, see the code here:

Code
        else if (token==ParserConsts::hash)
        {
            token = m_Tokenizer.GetToken();
            if (token==ParserConsts::kw_include)
                HandleIncludes();
            else if (token==ParserConsts::kw_define)
                HandleDefines();
            else
                HandlePreprocessorBlocks(token);
            m_Str.Clear();
        }

and

Code
void ParserThread::HandlePreprocessorBlocks(const wxString& preproc)
{
    if (preproc.StartsWith(ParserConsts::kw_if)) // #if, #ifdef, #ifndef
    {
        wxString token = preproc;
        ++m_PreprocessorIfCount;

        token = m_Tokenizer.GetToken();
        if (token.IsSameAs(_T("0")))
        {
            // TODO: handle special case "#if 0"
            TRACE(_T("HandlePreprocessorBlocks() : Special case \"#if 0\" not skipped."));
        }
        m_Tokenizer.SkipToEOL();
    }
    else if (preproc==ParserConsts::kw_else || preproc==ParserConsts::kw_elif) // #else, #elif
    {
        TRACE(_T("HandlePreprocessorBlocks() : Saving nesting level: %d"), m_Tokenizer.GetNestingLevel());
        m_Tokenizer.SaveNestingLevel();
        wxString token = preproc;
        while (!token.IsEmpty() && token != ParserConsts::kw_endif)
            token = m_Tokenizer.GetToken();
        --m_PreprocessorIfCount;
#if PARSERTHREAD_DEBUG_OUTPUT
        int l = m_Tokenizer.GetNestingLevel();
#endif
        m_Tokenizer.RestoreNestingLevel();
        TRACE(_T("HandlePreprocessorBlocks() : Restoring nesting level: %d (was %d)"), m_Tokenizer.GetNestingLevel(), l);
    }
    else if (preproc==ParserConsts::kw_endif) // #endif
        --m_PreprocessorIfCount;
}

And In my working copy, I already delete the if statements, and works fine here.
We just want to skip the unnecessory preprocessor directive, and no string information should be kept in m_LastPreprocessor.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: New code completion remarks/issues
« Reply #194 on: October 20, 2009, 05:01:24 pm »
Code
        else if (token==ParserConsts::hash)
        {
            token = m_Tokenizer.GetToken();
            if (token==ParserConsts::kw_include)
                HandleIncludes();
            else if (token==ParserConsts::kw_define)
                HandleDefines();
            else
                HandlePreprocessorBlocks(token);
            m_Str.Clear();
        }
Notice that there is another call to HandlePreprocessorBlocks() in void ParserThread::SkipBlock() and ParserThread::HandleClass(EClassType ct).
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ