Author Topic: New code completion remarks/issues (Read 361808 times)

ollydbg · « **Reply #180 on:** October 11, 2009, 03:07:39 pm »

Quote from: blueshake on October 11, 2009, 02:40:12 pm

What I want to know is if doing this wil slow down the parse process?

parse C::B source code.
before patch
Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).

koso · « **Reply #181 on:** October 11, 2009, 04:06:42 pm »

When you are talking about performance ...

I was using Symbol browser with view set to "Everything". Problem is, that now when I open project, first few seconds is C::B parsing files, and then it starts to update data in symbol browser. Becase of setting it to "Everything", it takes minutes .. and problem is, that during this operation is C::B dead = marked as "Not responding" = so I can do nothing, only look at white window (even draw events are ignored during this operation) or exit. So is there some way, how to do this more in background and not freeze whole UI?
And btw. this happens also when symbol browser is hidden.

blueshake · « **Reply #182 on:** October 12, 2009, 09:07:19 am »

Dear Morten:
Have you started the plan on macro replacement and template parse?

ollydbg · « **Reply #183 on:** October 18, 2009, 04:40:45 pm »

Hi, morten, I find some bugs in CC, but I'm sorry, I can't create a patch, because my local copy was changed a lot. so, here are the modified code:

1.

plugins\codecompletion\classbrowser.cpp

function:

Code

void ClassBrowser::UpdateView()
{
    m_pActiveProject = 0;
    m_ActiveFilename.Clear();
    if (m_pParser && !Manager::IsAppShuttingDown())
    {
        m_pActiveProject = Manager::Get()->GetProjectManager()->GetActiveProject();
        cbEditor* ed = Manager::Get()->GetEditorManager()->GetBuiltinActiveEditor();
        if (ed)
        {
            //m_ActiveFilename = ed->GetFilename().BeforeLast(_T('.'));
            // the above line is a bug (see https://developer.berlios.de/patch/index.php?func=detailpatch&patch_id=1559&group_id=5358)
            m_ActiveFilename = ed->GetFilename().AfterLast(wxFILE_SEP_PATH);

            if(  m_ActiveFilename.Find(_T('.')) != wxNOT_FOUND  )
            {
                m_ActiveFilename = ed->GetFilename().BeforeLast(wxFILE_SEP_PATH) + wxFILE_SEP_PATH + m_ActiveFilename.BeforeLast(_T('.'));
                m_ActiveFilename.Append(_T('.'));
            }
            else
                m_ActiveFilename = ed->GetFilename();

        }

        BuildTree();

        wxSplitterWindow* splitter = XRCCTRL(*this, "splitterWin", wxSplitterWindow);
        if (m_pParser->ClassBrowserOptions().treeMembers)
        {
            splitter->SplitHorizontally(m_Tree, m_TreeBottom);
            m_TreeBottom->Show(true);
        }
        else
        {
            splitter->Unsplit();
            m_TreeBottom->Show(false);
        }

    }
    else
        m_Tree->DeleteAllItems();
}

For example, I have a file:

d:\MinGW\lib\gcc\mingw32\4.4.1\include\c++\vector

So, I would like to see the tokens in this file.

2. Also, the function can give the right tokens in the right file, but it seems the Browser build thread still remove all the nodes.

For example:

d:\MinGW\lib\gcc\mingw32\4.4.1\include\c++\bits\stl_vector.h

Loot at the symbol browser tree window, there is no node shown there.

3.
I have change the SkipComments function in tokenizer.cpp

See here

Code

bool Tokenizer::SkipComment(bool skipEndWhite)
{

    bool cstyle;            // C or C++ style comments

    //check the comment prompt
    if(CurrentChar() == '/')
    {
        if(NextChar() == '*')
            cstyle = true;
        else if(NextChar() == '/')
            cstyle = false;
        else
            return true;    //Not the comment prompt, return true;
    }
    else
        return true;        //Not the comment prompt, return true;

    MoveToNextChar(2);      //skip the comment prompt

    while(true)
    {
        if (cstyle)        //c style comment
        {
            SkipToChar('/');
            if (PreviousChar() == '*')//the end of C style comments
            {
                MoveToNextChar();
                break;
            }
            if(!MoveToNextChar())
                break;
        }
        else                //c++ style comment
        {
            SkipToEOL(false, true);//nestBrace = false skipcomment = true
            MoveToNextChar();
            break;
        }
    }


    if (IsEOF())
        return false;

    if (skipEndWhite)
    {
        if(!SkipWhiteSpace())
            return false;
    }
    else
        return true;

    return SkipComment(); // handle chained comments
}

For the old code, if skipEndWhite == false, the function will always return false, this is something wrong.

...

Bed time,,,, I will add more tomorrow.

ollydbg · « **Reply #184 on:** October 19, 2009, 04:09:49 pm »

1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function

Code

    if (    m_LastWasPreprocessor
        && !str.IsSameAs(_T("#"))
        && !m_LastPreprocessor.IsSameAs(_T("#")) )
    {
        if (!m_LastPreprocessor.IsSameAs(TokenizerConsts::include_str))
        {
            // except for #include and #if[[n]def], all other preprocessor directives need only
            // one word exactly after the directive, e.g. #define THIS_WORD
            SkipToEOL();
        }
        m_LastPreprocessor.Clear();
    }

So, these code can be deleted.

2, I tokenizer.h think in the CurrentChar function, we don't need to check index value, because MoveToNextChar already do this.

Code

    bool MoveToNextChar(const unsigned int amount = 1)
    {
        assert(amount);
        if(amount == 1) // compiler will dead-strip this
        {
            ++m_TokenIndex;
            if (IsEOF())
            {
                m_TokenIndex = m_BufferLen;
                return false;
            }

            if (CurrentChar() == _T('\n'))
                ++m_LineNumber;
            return true;
        }
        else
        {
            m_TokenIndex += amount;
            if (IsEOF())
            {
                m_TokenIndex = m_BufferLen;
                return false;
            }

            if (CurrentChar() == _T('\n'))
                ++m_LineNumber;
            return true;
        }
    };

    wxChar CurrentChar() const
    {
        if(m_TokenIndex < m_BufferLen)
            return m_Buffer.GetChar(m_TokenIndex);
        return 0;
    };

MortenMacFly · « **Reply #185 on:** October 19, 2009, 08:57:51 pm »

Quote from: ollydbg on October 11, 2009, 03:07:39 pm

Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).

...did you notice? That's weired.

Quote from: ollydbg on October 19, 2009, 04:09:49 pm

1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function
[...]
So, these code can be deleted.

...which one (which if condition) you mean? For me both still make sense...?!

Quote from: ollydbg on October 19, 2009, 04:09:49 pm

2, I tokenizer.h think in the CurrentChar function, we don't need to check index value, because MoveToNextChar already do this.

That's true, but I wanted to avoid any issues with API mis-use (meaning calling CurrentChar() before MoveToNextChar() for any obscure reason). I wonder what this would differ in terms of time. Implementing a temporary time measurement would reveal the difference.

ollydbg · « **Reply #186 on:** October 20, 2009, 03:29:50 am »

Quote from: MortenMacFly on October 19, 2009, 08:57:51 pm

Quote from: ollydbg on October 11, 2009, 03:07:39 pm
Parsing stage done (1511 total parsed files, 84442 tokens in 0 minute(s), 12.468 seconds).
after patch
Parsing stage done (1510 total parsed files, 84506 tokens in 0 minute(s), 12.578 seconds).
...did you notice? That's weird.

Yes, it is weird. I don't know why...

Quote from: MortenMacFly on October 19, 2009, 08:57:51 pm

Quote from: ollydbg on October 19, 2009, 04:09:49 pm
1. In Tokenizer.cpp. the if condition never be true in the DoGetToken function
[...]
So, these code can be deleted.
...which one (which if condition) you mean? For me both still make sense...?!

Both the if condition were not be hit if I set breakpoint there.
See my screen shot

I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:

Code

bool Tokenizer::SkipUnwanted()
{
    while (CurrentChar() == '#' ||
            (!m_IsOperator && CurrentChar() == '=') ||
            (!m_IsOperator && CurrentChar() == '[') ||
            CurrentChar() == '?' ||
            (CurrentChar() == '/' && (NextChar() == '/' || NextChar() == '*') ))
    {
        bool skipPreprocessor = false; // used for #include
        while (m_Buffer.Mid(m_TokenIndex, 2) == _T("//") ||
                m_Buffer.Mid(m_TokenIndex, 2) == _T("/*"))
        {
            // C/C++ style comments
            SkipComment();
            if (IsEOF())
                return false;
            if (!SkipWhiteSpace())
                return false;
        }

        while (CurrentChar() == '#')
        {
            // preprocessor directives
            // we only care for #include and #define, for now
            unsigned int backupIdx = m_TokenIndex;
            MoveToNextChar();
            SkipWhiteSpace();
            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }

MortenMacFly · « **Reply #187 on:** October 20, 2009, 07:52:43 am »

Quote from: ollydbg on October 20, 2009, 03:29:50 am

I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:

So... now you got me a little confused (or probably it's too early for me). So you propose to remove the "if" conditions because of a bug in SkipUnwanted? Shouldn't we better fix the bug? And then: Where exactly is the bug in the code snippet?
Could you do me a favour and comment the lines / snippets in question in the code you've posted? That'll help me a lot. Please explain a little more...

ollydbg · « **Reply #188 on:** October 20, 2009, 08:15:30 am »

Quote from: MortenMacFly on October 20, 2009, 07:52:43 am

Quote from: ollydbg on October 20, 2009, 03:29:50 am
I has discussed this with blueshake, he can confirm this. That's because in the logic bug in SkipUnwanted.
see the code:
So... now you got me a little confused (or probably it's too early for me). So you propose to remove the "if" conditions because of a bug in SkipUnwanted? Shouldn't we better fix the bug? And then: Where exactly is the bug in the code snippet?
Could you do me a favour and comment the lines / snippets in question in the code you've posted? That'll help me a lot. Please explain a little more...

Ok, let me explain:

There are two member variables:

m_LastWasPreprocessor is a bool
m_LastPreprocessor is a wxString

But the DoGetToken() only return the variable "str"

Code

    if (    m_LastWasPreprocessor
        && !str.IsSameAs(_T("#"))
        && !m_LastPreprocessor.IsSameAs(_T("#")) )
    {
        if (!m_LastPreprocessor.IsSameAs(TokenizerConsts::include_str))
        {
            // except for #include and #if[[n]def], all other preprocessor directives need only
            // one word exactly after the directive, e.g. #define THIS_WORD
            SkipToEOL();
        }
        m_LastPreprocessor.Clear();
    }

    if (m_LastWasPreprocessor)
        m_LastPreprocessor << str;

    m_LastWasPreprocessor = false;

    return str;

So, str will never be affected by m_LastWasPreprocessor or m_LastPreprocessor.

So, these two member variables are useless.

blueshake · « **Reply #189 on:** October 20, 2009, 10:09:38 am »

what more I want to say is the logical error:
The only chance the variable m_LastWasPreprocessor becomes true is in these codes:

Code

            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else

at this time,str contain the string "#",and m_LastPreprocessor is empty.(see codes above)
so the three conditions in if:

m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true

see ,the three conditions can not all be true at the same time.It means the "if " condition can never be true.

ollydbg · « **Reply #190 on:** October 20, 2009, 10:12:06 am »

Quote from: blueshake on October 20, 2009, 10:09:38 am

what more I want to say is the logical error:
The only chance the variable m_LastWasPreprocessor becomes true is in these codes:

Code

            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else

at this time,str contain the string "#",and m_LastPreprocessor is empty.(see codes above)
so the three conditions in if:

m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true

see ,the three conditions can not all be true at the same time.It means the "if " condition can never be true.

Hi, blueshake, Thanks for the explanation.

.

MortenMacFly · « **Reply #191 on:** October 20, 2009, 03:33:06 pm »

Quote from: blueshake on October 20, 2009, 10:09:38 am

m_LastWasPreprocessor------------------------true
!str.IsSameAs(_T("#"))-------------------------false
!m_LastPreprocessor.IsSameAs(_T("#"))-------true

Could it be that simply the "!str..." line should be removed as it is a relict of old times???

From what I read in the code it simplifies preprocessor handling in a way that it just gets rid of everything after the first preprocessor directive as it is unhandled anyways (except for #include and #if[[n]def], as stated in the comment)... which is good.

blueshake · « **Reply #192 on:** October 20, 2009, 03:59:50 pm »

My another question :
what is the effort of m_LastPreprocessor .since when the variable m_LastWasPreprocessor is true,the m_LastPreprocessor is absoutely empty.see codes below.

Code

            if ((CurrentChar() == 'i' && NextChar() == 'n') || // in(clude)
                (CurrentChar() == 'i' && NextChar() == 'f') || // if(|def|ndef)
                (CurrentChar() == 'e' && NextChar() == 'l') || // el(se|if)
                (CurrentChar() == 'e' && NextChar() == 'n') || // en(dif)
                (m_TokenizerOptions.wantPreprocessor && CurrentChar() == 'd' && NextChar() == 'e')) // de(fine)
            {
                // ok, we have something like #in(clude)
                m_LastWasPreprocessor = true;
                m_LastPreprocessor.Clear();
                m_TokenIndex = backupIdx; // keep #
                skipPreprocessor = true;
                break;
            }
            else

EDIT:
In the three conditions,
m_LastWasPreprocessor is true,!m_LastPreprocessor.IsSameAs(_T("#")) is absoutely true too.

ollydbg · « **Reply #193 on:** October 20, 2009, 04:14:58 pm »

Hi,
I would like to say:
there is no use of "m_LastPreprocessor" and "m_LastWasPreprocessor".
Let me explain more, In the parserthread, see the code here:

Code

        else if (token==ParserConsts::hash)
        {
            token = m_Tokenizer.GetToken();
            if (token==ParserConsts::kw_include)
                HandleIncludes();
            else if (token==ParserConsts::kw_define)
                HandleDefines();
            else
                HandlePreprocessorBlocks(token);
            m_Str.Clear();
        }

and

Code

void ParserThread::HandlePreprocessorBlocks(const wxString& preproc)
{
    if (preproc.StartsWith(ParserConsts::kw_if)) // #if, #ifdef, #ifndef
    {
        wxString token = preproc;
        ++m_PreprocessorIfCount;

        token = m_Tokenizer.GetToken();
        if (token.IsSameAs(_T("0")))
        {
            // TODO: handle special case "#if 0"
            TRACE(_T("HandlePreprocessorBlocks() : Special case \"#if 0\" not skipped."));
        }
        m_Tokenizer.SkipToEOL();
    }
    else if (preproc==ParserConsts::kw_else || preproc==ParserConsts::kw_elif) // #else, #elif
    {
        TRACE(_T("HandlePreprocessorBlocks() : Saving nesting level: %d"), m_Tokenizer.GetNestingLevel());
        m_Tokenizer.SaveNestingLevel();
        wxString token = preproc;
        while (!token.IsEmpty() && token != ParserConsts::kw_endif)
            token = m_Tokenizer.GetToken();
        --m_PreprocessorIfCount;
#if PARSERTHREAD_DEBUG_OUTPUT
        int l = m_Tokenizer.GetNestingLevel();
#endif
        m_Tokenizer.RestoreNestingLevel();
        TRACE(_T("HandlePreprocessorBlocks() : Restoring nesting level: %d (was %d)"), m_Tokenizer.GetNestingLevel(), l);
    }
    else if (preproc==ParserConsts::kw_endif) // #endif
        --m_PreprocessorIfCount;
}

And In my working copy, I already delete the if statements, and works fine here.
We just want to skip the unnecessory preprocessor directive, and no string information should be kept in m_LastPreprocessor.

MortenMacFly · « **Reply #194 on:** October 20, 2009, 05:01:24 pm »

Quote from: ollydbg on October 20, 2009, 04:14:58 pm

Code

        else if (token==ParserConsts::hash)
        {
            token = m_Tokenizer.GetToken();
            if (token==ParserConsts::kw_include)
                HandleIncludes();
            else if (token==ParserConsts::kw_define)
                HandleDefines();
            else
                HandlePreprocessorBlocks(token);
            m_Str.Clear();
        }

Notice that there is another call to HandlePreprocessorBlocks() in void ParserThread::SkipBlock() and ParserThread::HandleClass(EClassType ct).

Code::Blocks Forums

News:

Author Topic: New code completion remarks/issues (Read 361808 times)

ollydbg

Re: New code completion remarks/issues

koso

Re: New code completion remarks/issues

blueshake

Re: New code completion remarks/issues

ollydbg

Re: New code completion remarks/issues

ollydbg

Re: New code completion remarks/issues

MortenMacFly

Re: New code completion remarks/issues

ollydbg

Re: New code completion remarks/issues

MortenMacFly

Re: New code completion remarks/issues

ollydbg

Re: New code completion remarks/issues

blueshake

Re: New code completion remarks/issues

ollydbg

Re: New code completion remarks/issues

MortenMacFly

Re: New code completion remarks/issues

blueshake

Re: New code completion remarks/issues

ollydbg

Re: New code completion remarks/issues

MortenMacFly

Re: New code completion remarks/issues