User forums > Nightly builds

The 08 February 2014 build (9639) is out.

<< < (5/8) > >>

blauzahn:

--- Quote from: MortenMacFly on February 11, 2014, 09:14:03 pm ---For me its different: It looks like in bool ParserThread::Parse() in the else if (!switchHandled) clause it always end in the m_Str << token << ParserConsts::space_chr; line which will crash sooner or later if the wxString cannot append more characters. at that time, token is something like (wxMACRO(...)) which cannot be resolved / handled in this loop.

--- End quote ---
You mean void ParserThread::DoParse(), I assume.

dmoore:

--- Quote from: kingfox on February 11, 2014, 01:19:07 pm ---"new command line option --user-data-dir=<path>" is very good !  :D

--- End quote ---

Good! Let us know if there are issues.

ollydbg:

--- Quote from: MortenMacFly on February 11, 2014, 09:14:03 pm ---
--- Quote from: ollydbg on February 11, 2014, 01:33:36 pm ---Here is the BT that I see m_TokenIndex always stay in the same value (infinite loop)

--- End quote ---
For me its different: It looks like in bool ParserThread::Parse() in the else if (!switchHandled) clause it always end in the m_Str << token << ParserConsts::space_chr; line which will crash sooner or later if the wxString cannot append more characters. at that time, token is something like (wxMACRO(...)) which cannot be resolved / handled in this loop.

--- End quote ---

I debugged a little, I think it was a regression after a patch by Huki, that is:


--- Code: ---Revision: f7cb07c8dcf613f6786eaf0ee34e908ec218d2c8
Author: ollydbg <ollydbg@2a5c6006-c6dd-42ca-98ab-0921f2732cef>
Date: 2013-9-29 16:25:10
Message:
* CC: reliable working of UngetToken() when macro expansion is involved, it set the undo index value correctly(thanks Huki)

* CC: avoid take backward step (UngetToken) twice in the Tokenizer class
- CC: comments added for Tokenizer class

git-svn-id: https://svn.code.sf.net/p/codeblocks/code/trunk@9369 2a5c6006-c6dd-42ca-98ab-0921f2732cef
----
Modified: src/plugins/codecompletion/parser/tokenizer.cpp
Modified: src/plugins/codecompletion/parser/tokenizer.h


--- End code ---

which have such diff:

--- Code: ---@@ -1148,6 +1148,8 @@ wxString Tokenizer::PeekToken()
         unsigned int savedLineNumber = m_LineNumber;
         unsigned int savedNestLevel  = m_NestLevel;
 
+        int savedReplaceCount = m_IsReplaceParsing ? m_RepeatReplaceCount : -1;
+
         if (SkipUnwanted())
             m_PeekToken = DoGetToken();
         else
@@ -1156,17 +1158,33 @@ wxString Tokenizer::PeekToken()
         m_PeekTokenIndex             = m_TokenIndex;
         m_PeekLineNumber             = m_LineNumber;
         m_PeekNestLevel              = m_NestLevel;
-
-        m_TokenIndex                 = savedTokenIndex;
-        m_LineNumber                 = savedLineNumber;
-        m_NestLevel                  = savedNestLevel;
+        // Check whether a ReplaceBufferForReparse() was done in DoGetToken().
+        // We assume m_Undo... have already been reset in ReplaceBufferForReparse().
+        if (m_IsReplaceParsing && savedReplaceCount != (int)m_RepeatReplaceCount)
+        {
+            m_TokenIndex             = m_UndoTokenIndex;
+            m_LineNumber             = m_UndoLineNumber;
+            m_NestLevel              = m_UndoNestLevel;
+        }
+        else
+        {
+            m_TokenIndex             = savedTokenIndex;
+            m_LineNumber             = savedLineNumber;
+            m_NestLevel              = savedNestLevel;
+        }
     }
 
     return m_PeekToken;

--- End code ---

So, bug is here:

--- Code: ---wxString Tokenizer::PeekToken()
{
    if (!m_PeekAvailable)
    {
        m_PeekAvailable = true;

        unsigned int savedTokenIndex = m_TokenIndex;
        unsigned int savedLineNumber = m_LineNumber;
        unsigned int savedNestLevel  = m_NestLevel;

        int savedReplaceCount = m_IsReplaceParsing ? m_RepeatReplaceCount : -1;

        if (SkipUnwanted())
            m_PeekToken = DoGetToken();
        else
            m_PeekToken.Clear();

        m_PeekTokenIndex             = m_TokenIndex;
        m_PeekLineNumber             = m_LineNumber;
        m_PeekNestLevel              = m_NestLevel;
        // Check whether a ReplaceBufferText() was done in DoGetToken().
        // We assume m_Undo... have already been reset in ReplaceBufferText().
        if (m_IsReplaceParsing && savedReplaceCount != (int)m_RepeatReplaceCount)
        {
            m_TokenIndex             = m_UndoTokenIndex;                       //*************bug here******************
            m_LineNumber             = m_UndoLineNumber;
            m_NestLevel              = m_UndoNestLevel;
        }
        else
        {
            m_TokenIndex             = savedTokenIndex;
            m_LineNumber             = savedLineNumber;
            m_NestLevel              = savedNestLevel;
        }
    }

    return m_PeekToken;
}

--- End code ---

m_TokenIndex is always remain/reset to the same value, so Tokenizer won't go forward any more.

blauzahn:
Has it been considered to group

TokenIndex
LineNumber
NestLevel

into a tiny class/struct?

ollydbg:

--- Quote from: blauzahn on February 12, 2014, 07:00:42 am ---Has it been considered to group

TokenIndex
LineNumber
NestLevel

into a tiny class/struct?

--- End quote ---
Hi, blauzahn, thanks. Sure this will make the code more clean and readable(I even thought the lexeme string should be bundled with those information to a single struct), but the change is not simple, because two many places use those variables. :)
I'm currently don't have the time to do that, so this can be a TO-DO.
Let's firstly fix the hang issue. ;)

Navigation

[0] Message Index

[#] Next page

[*] Previous page