Code::Blocks

User forums => Nightly builds => Topic started by: killerbot on February 08, 2014, 03:39:39 pm

Title: The 08 February 2014 build (9639) is out.
Post by: killerbot on February 08, 2014, 03:39:39 pm
Get quick announcements through the RSS feed http://www.codeblocks.org/nightly/CodeBlock_RSS.xml

Before you use a nightly make sure you understand how it works (http://forums.codeblocks.org/index.php/topic,3232.0.html).

A link to the unicode windows wxWidget dll for Code::Blocks : http://prdownload.berlios.de/codeblocks/wxmsw28u_gcc_cb_wx2812_gcc481-TDM.7z

For those who might need this one (when no MingW installed on your system) : the mingw10m.dll : http://prdownload.berlios.de/codeblocks/mingwm10_gcc481-TDM.7z

The 08 Feb 2014 build is out.
  - Windows :
   http://prdownload.berlios.de/codeblocks/CB_20140208_rev9639_win32.7z
  - Linux :
   none

Resolved Fixed:


Regressions/Confirmed/Annoying/Common bugs:


Title: Re: The 08 February 2014 build (9639) is out.
Post by: maryjeck on February 09, 2014, 11:43:16 am
There must be something wrong with it。

When I open an Project.
There are lost a word like like this "NativeParser::OnParserEnd(): Project '******' parsing stage done!".

and I can do nothing with it,it FC.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: jens on February 09, 2014, 11:56:06 am
There must be something wrong with it。

When I open an Project.
There are lost a word like like this "NativeParser::OnParserEnd(): Project '******' parsing stage done!".

and I can do nothing with it,it FC.
Where do you see this messages ?
Can you provide a sample (minimal) workspace/project where this occurs ?
Title: Re: The 08 February 2014 build (9639) is out.
Post by: jens on February 09, 2014, 11:57:16 am
As usua:

Debian packages (binaries and sources) for 32-bit and 64-bit systems can be found in my debian-repo (http://apt.jenslody.de/).
Fedora packages (binaries and sources) for 32-bit and 64-bit systems (fc18, fc19, fc20 and rawhide) and RedHat/CentOS 5 and 6 packages (also 32-bit and 64-bit) can be found in my rpm-repo (http://rpm.jenslody.de) .
Title: Re: The 08 February 2014 build (9639) is out.
Post by: shurick on February 09, 2014, 02:03:15 pm
Packages for openSUSE 12.2 - 13.1 (http://download.opensuse.org/repositories/home:/NarkoZ:/Nightlies/) (binaries and sources) for 32-bit and 64-bit.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: dmoore on February 09, 2014, 03:18:42 pm
Ubuntu packages building here (https://code.launchpad.net/~damien-moore/+archive/codeblocks-nightly)
Title: Re: The 08 February 2014 build (9639) is out.
Post by: damorin on February 09, 2014, 05:11:00 pm
Hi,

something new in this release. Each time I try "Find functions calling <x>", CSCOPE crash.

I noticed the cscope database "cscope.out" have been renamed "ncscope.out".

Renaming it to the original name is fixing the problem.

I'm using cscope v15.7a



Title: Re: The 08 February 2014 build (9639) is out.
Post by: maryjeck on February 10, 2014, 03:46:16 am
There must be something wrong with it。

When I open an Project.
There are lost a word like like this "NativeParser::OnParserEnd(): Project '******' parsing stage done!".

and I can do nothing with it,it FC.
Where do you see this messages ?
Can you provide a sample (minimal) workspace/project where this occurs ?

build(9619):
The message is like this:
NativeParser::CreateParser(): Finish creating a new parser for project '***********'
NativeParser::OnParserEnd(): Project '********' parsing stage done!

build(9639):
like this:
NativeParser::CreateParser(): Finish creating a new parser for project '***********'

Only one word.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 10, 2014, 04:35:08 am
There must be something wrong with it。

When I open an Project.
There are lost a word like like this "NativeParser::OnParserEnd(): Project '******' parsing stage done!".

and I can do nothing with it,it FC.
Where do you see this messages ?
Can you provide a sample (minimal) workspace/project where this occurs ?

build(9619):
The message is like this:
NativeParser::CreateParser(): Finish creating a new parser for project '***********'
NativeParser::OnParserEnd(): Project '********' parsing stage done!

build(9639):
like this:
NativeParser::CreateParser(): Finish creating a new parser for project '***********'

Only one word.
Ok, so it looks like the CC's parser hangs or go to some infinite loop.
If you see the change between rev 9619 and 9639, there are few CC related changed.

Code: [Select]
Revision: 9639
Author: ollydbg
Date: 2014-2-8 17:33:14
Message:
- CC: use wx_str() instead of c_str().
-------------------------------
M(T ) : /trunk/src/plugins/codecompletion/nativeparser.cpp

Revision: 9638
Author: ollydbg
Date: 2014-2-8 17:32:33
Message:
* CC: fix a bug (logic error) introduced in rev 9601.
-------------------------------
M(T ) : /trunk/src/plugins/codecompletion/parser/tokenizer.cpp


Revision: 9636
Author: fuscated
Date: 2014-2-4 7:15:50
Message:
- CC: Make the code a bit more readable
-------------------------------
M(T ) : /trunk/src/plugins/codecompletion/parser/parserthread.cpp

Revision: 9635
Author: fuscated
Date: 2014-2-4 7:15:45
Message:
- wx30: Fix a format specifier mismatch assert
-------------------------------
M(T ) : /trunk/src/plugins/codecompletion/parser/parserthread.cpp


I guess the change of rev9638 (* CC: fix a bug (logic error) introduced in rev 9601.) cause this bug. But in-fact rev 9638 is going fix a logic error, so another guess is that rev 9601 does not truly fix the bug.

I would welcome a test project/workspace. Meanwhile, I will re-test the sample project in this discussion:  NativeParser lockup when parsing Visual Studio 2013 and Boost 1.55.0 headers (http://forums.codeblocks.org/index.php/topic,18801.msg128775.html#msg128775) later today.

EDIT:
Code blocks using too much cpu (http://forums.codeblocks.org/index.php/topic,18824.msg128966.html#msg128966), this contains a sample project which will lead to hang issue.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 10, 2014, 04:53:51 am
...
I would welcome a test project/workspace. Meanwhile, I will re-test the sample project in this discussion:  NativeParser lockup when parsing Visual Studio 2013 and Boost 1.55.0 headers (http://forums.codeblocks.org/index.php/topic,18801.msg128775.html#msg128775) later today.

EDIT:
Code blocks using too much cpu (http://forums.codeblocks.org/index.php/topic,18824.msg128966.html#msg128966), this contains a sample project which will lead to hang issue.
FYI: I test the sample project in above link with C::B rev 9643, I can't find see the hang issue here. So, I definitely need a reproducer, thanks.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: grf on February 10, 2014, 12:40:24 pm
After starting CB and opening some projects, the CPU load increases up to 100% and the GUI doesn't react anymore. I tried three different projects: Two projects for programs with GUI (wxWidgets based) and one simple C/C++ project. Only these both projects with wxWidgets lead to this behaviour.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: stahta01 on February 10, 2014, 04:34:24 pm
After starting CB and opening some projects, the CPU load increases up to 100% and the GUI doesn't react anymore. I tried three different projects: Two projects for programs with GUI (wxWidgets based) and one simple C/C++ project. Only these both projects with wxWidgets lead to this behaviour.

I am guessing this info might be needed; wxWidgets version, OS name and version, Compiler name and version?

Are you using Precompiled Headers?

Tim S.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: grf on February 10, 2014, 06:45:56 pm
wxWidgets 3.0.0, no precompiled headers, tdm-gcc 4.7.1-2, Windows XP

The problem starts directly after loading the project, without any compilation. CB opens the project and then the CPU load goes up to 100% and it doesn't react anymore. After some time I killed it via task manager.
The nightly svn9619 works without this problem.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 11, 2014, 02:24:53 am
wxWidgets 3.0.0, no precompiled headers, tdm-gcc 4.7.1-2, Windows XP

The problem starts directly after loading the project, without any compilation. CB opens the project and then the CPU load goes up to 100% and it doesn't react anymore. After some time I killed it via task manager.
The nightly svn9619 works without this problem.
Can you show a sample/minimal C::B project which cause the hang issue? I'm interest to catch those bugs.  :)
Title: Re: The 08 February 2014 build (9639) is out.
Post by: grf on February 11, 2014, 10:14:34 am
What I tried is to create a new wxWidgets project, without PCH, a debug configuration (which uses the wxWidgets 3.0.0 release libs) and a release configuration. The project contains no code changes, only the generated files. After opening the project, the CPU load increases up to 100% and CB can only be killed via Task Manager.

Here is the content of the project file (I don't want to provide any zip archives):

Code: [Select]
?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<CodeBlocks_project_file>
<FileVersion major="1" minor="6" />
<Project>
<Option title="minimal_test" />
<Option pch_mode="2" />
<Option compiler="gcc" />
<Build>
<Target title="Debug">
<Option output="bin/minimal_test" prefix_auto="1" extension_auto="1" />
<Option object_output="bin/" />
<Option type="0" />
<Option compiler="gcc" />
<Option projectLinkerOptionsRelation="2" />
<Compiler>
<Add option="-g" />
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib/mswu" />
</Compiler>
<ResourceCompiler>
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib/mswu" />
</ResourceCompiler>
<Linker>
<Add library="libwxmsw30u_core.a" />
<Add library="libwxbase30u.a" />
<Add library="libwxpng.a" />
<Add library="libwxzlib.a" />
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib" />
</Linker>
</Target>
<Target title="Release">
<Option output="bin/minimal_test" prefix_auto="1" extension_auto="1" />
<Option object_output="bin/" />
<Option type="0" />
<Option compiler="gcc" />
<Option projectLinkerOptionsRelation="2" />
<Compiler>
<Add option="-O2" />
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib/mswu" />
</Compiler>
<ResourceCompiler>
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib/mswu" />
</ResourceCompiler>
<Linker>
<Add option="-s" />
<Add library="libwxmsw30u_core.a" />
<Add library="libwxbase30u.a" />
<Add library="libwxpng.a" />
<Add library="libwxzlib.a" />
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib" />
</Linker>
</Target>
</Build>
<Compiler>
<Add option="-pipe" />
<Add option="-mthreads" />
<Add option="-D__GNUWIN32__" />
<Add option="-D__WXMSW__" />
<Add option="-DwxUSE_UNICODE" />
<Add option="-Wall" />
<Add directory="C:/Libs/wxWidgets_gcc/include" />
</Compiler>
<ResourceCompiler>
<Add directory="C:/Libs/wxWidgets_gcc/include" />
</ResourceCompiler>
<Linker>
<Add option="-mthreads" />
<Add library="libkernel32.a" />
<Add library="libuser32.a" />
<Add library="libgdi32.a" />
<Add library="libwinspool.a" />
<Add library="libcomdlg32.a" />
<Add library="libadvapi32.a" />
<Add library="libshell32.a" />
<Add library="libole32.a" />
<Add library="liboleaut32.a" />
<Add library="libuuid.a" />
<Add library="libcomctl32.a" />
<Add library="libwsock32.a" />
<Add library="libodbc32.a" />
</Linker>
<Unit filename="minimal_testApp.cpp" />
<Unit filename="minimal_testApp.h" />
<Unit filename="minimal_testMain.cpp" />
<Unit filename="minimal_testMain.h" />
<Unit filename="resource.rc">
<Option compilerVar="WINDRES" />
</Unit>
<Extensions>
<code_completion />
<envvars />
<debugger />
<lib_finder disable_auto="1" />
</Extensions>
</Project>
</CodeBlocks_project_file>

Title: Re: The 08 February 2014 build (9639) is out.
Post by: kingfox on February 11, 2014, 01:19:07 pm
"new command line option --user-data-dir=<path>" is very good !  :D
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 11, 2014, 01:33:36 pm
What I tried is to create a new wxWidgets project, without PCH, a debug configuration (which uses the wxWidgets 3.0.0 release libs) and a release configuration. The project contains no code changes, only the generated files. After opening the project, the CPU load increases up to 100% and CB can only be killed via Task Manager.

Here is the content of the project file (I don't want to provide any zip archives):

Code: [Select]
?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<CodeBlocks_project_file>
<FileVersion major="1" minor="6" />
<Project>
<Option title="minimal_test" />
<Option pch_mode="2" />
<Option compiler="gcc" />
<Build>
<Target title="Debug">
<Option output="bin/minimal_test" prefix_auto="1" extension_auto="1" />
<Option object_output="bin/" />
<Option type="0" />
<Option compiler="gcc" />
<Option projectLinkerOptionsRelation="2" />
<Compiler>
<Add option="-g" />
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib/mswu" />
</Compiler>
<ResourceCompiler>
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib/mswu" />
</ResourceCompiler>
<Linker>
<Add library="libwxmsw30u_core.a" />
<Add library="libwxbase30u.a" />
<Add library="libwxpng.a" />
<Add library="libwxzlib.a" />
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib" />
</Linker>
</Target>
<Target title="Release">
<Option output="bin/minimal_test" prefix_auto="1" extension_auto="1" />
<Option object_output="bin/" />
<Option type="0" />
<Option compiler="gcc" />
<Option projectLinkerOptionsRelation="2" />
<Compiler>
<Add option="-O2" />
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib/mswu" />
</Compiler>
<ResourceCompiler>
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib/mswu" />
</ResourceCompiler>
<Linker>
<Add option="-s" />
<Add library="libwxmsw30u_core.a" />
<Add library="libwxbase30u.a" />
<Add library="libwxpng.a" />
<Add library="libwxzlib.a" />
<Add directory="C:/Libs/wxWidgets_gcc/lib/gcc_lib" />
</Linker>
</Target>
</Build>
<Compiler>
<Add option="-pipe" />
<Add option="-mthreads" />
<Add option="-D__GNUWIN32__" />
<Add option="-D__WXMSW__" />
<Add option="-DwxUSE_UNICODE" />
<Add option="-Wall" />
<Add directory="C:/Libs/wxWidgets_gcc/include" />
</Compiler>
<ResourceCompiler>
<Add directory="C:/Libs/wxWidgets_gcc/include" />
</ResourceCompiler>
<Linker>
<Add option="-mthreads" />
<Add library="libkernel32.a" />
<Add library="libuser32.a" />
<Add library="libgdi32.a" />
<Add library="libwinspool.a" />
<Add library="libcomdlg32.a" />
<Add library="libadvapi32.a" />
<Add library="libshell32.a" />
<Add library="libole32.a" />
<Add library="liboleaut32.a" />
<Add library="libuuid.a" />
<Add library="libcomctl32.a" />
<Add library="libwsock32.a" />
<Add library="libodbc32.a" />
</Linker>
<Unit filename="minimal_testApp.cpp" />
<Unit filename="minimal_testApp.h" />
<Unit filename="minimal_testMain.cpp" />
<Unit filename="minimal_testMain.h" />
<Unit filename="resource.rc">
<Option compilerVar="WINDRES" />
</Unit>
<Extensions>
<code_completion />
<envvars />
<debugger />
<lib_finder disable_auto="1" />
</Extensions>
</Project>
</CodeBlocks_project_file>


OK, many thanks. I can reproduce the hang issue with the latest C::B SVN head, so I will try to see what cause this hang issue.

Here is the BT that I see m_TokenIndex always stay in the same value (infinite loop)
Code: [Select]
[debug]#0  Tokenizer::ReplaceMacro (this=0x940fc98, str="wxDEFINE_UNICHARREF_OPERATOR") at F:\cb_sf_git\trunk\src\plugins\codecompletion\parser\tokenizer.cpp:1259
[debug]#1  0x65ef64b1 in Tokenizer::DoGetToken (this=0x940fc98) at F:\cb_sf_git\trunk\src\plugins\codecompletion\parser\tokenizer.cpp:1230
[debug]#2  0x65ef5e19 in Tokenizer::PeekToken (this=0x940fc98) at F:\cb_sf_git\trunk\src\plugins\codecompletion\parser\tokenizer.cpp:1064
[debug]#3  0x65ee3ba9 in ParserThread::DoParse (this=0x940fc90) at F:\cb_sf_git\trunk\src\plugins\codecompletion\parser\parserthread.cpp:1002
[debug]#4  0x65ee7107 in ParserThread::HandleClass (this=0x940fc90, ct=ParserThread::ctClass) at F:\cb_sf_git\trunk\src\plugins\codecompletion\parser\parserthread.cpp:1971
[debug]#5  0x65ee3262 in ParserThread::DoParse (this=0x940fc90) at F:\cb_sf_git\trunk\src\plugins\codecompletion\parser\parserthread.cpp:811
[debug]#6  0x65ee23df in ParserThread::Parse (this=0x940fc90) at F:\cb_sf_git\trunk\src\plugins\codecompletion\parser\parserthread.cpp:513
[debug]#7  0x65f0607e in ParserThread::Execute (this=0x940fc90) at F:\cb_sf_git\trunk\src\plugins\codecompletion\parser\parserthread.h:155
[debug]#8  0x010ab44b in cbThreadPool::cbWorkerThread::Entry (this=0x6bab438) at F:\cb_sf_git\trunk\src\sdk\cbthreadpool.cpp:216
[debug]#9  0x62764ba9 in wxThreadInternal::DoThreadStart(wxThread*) () from E:\code\wx-mingw-build-481-dw2\wxWidgets-2.8.12\lib\gcc_dll\wxmsw28u_gcc_custom.dll
[debug]#10 0x62764c85 in wxThreadInternal::WinThreadStart(void*)@4 () from E:\code\wx-mingw-build-481-dw2\wxWidgets-2.8.12\lib\gcc_dll\wxmsw28u_gcc_custom.dll
[debug]#11 0x77c3a3b0 in msvcrt!_endthreadex () from C:\WINDOWS\system32\msvcrt.dll
[debug]#12 0x7c80b729 in KERNEL32!GetModuleFileNameA () from C:\WINDOWS\system32\kernel32.dll
[debug]#13 0x00000000 in ?? ()
Title: Re: The 08 February 2014 build (9639) is out.
Post by: MortenMacFly on February 11, 2014, 04:24:40 pm
OK, many thanks. I can reproduce the hang issue with the latest C::B SVN head
I can do as well, meanwhile with all my wx30 based projects... dammed.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: MortenMacFly on February 11, 2014, 09:14:03 pm
Here is the BT that I see m_TokenIndex always stay in the same value (infinite loop)
For me its different: It looks like in bool ParserThread::Parse() in the else if (!switchHandled) clause it always end in the m_Str << token << ParserConsts::space_chr; line which will crash sooner or later if the wxString cannot append more characters. at that time, token is something like (wxMACRO(...)) which cannot be resolved / handled in this loop.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: blauzahn on February 12, 2014, 12:15:33 am
a superficial glance for typical suspects shows:

class Token's ctor should initialize its members to avoid UB:

m_IsConst(false),
m_IsNoExcept(false),

I see no reason why its data members m_TokenTree and m_ticket have to be protected instead of private.
Is this class still in use?

class Tokenizer's ctor looks like it better should be explicit. Its 2nd argument has a default value and an
accidential implicit construction out of a TokenTree* feels quite unsound to me.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: blauzahn on February 12, 2014, 12:27:29 am
For me its different: It looks like in bool ParserThread::Parse() in the else if (!switchHandled) clause it always end in the m_Str << token << ParserConsts::space_chr; line which will crash sooner or later if the wxString cannot append more characters. at that time, token is something like (wxMACRO(...)) which cannot be resolved / handled in this loop.
You mean void ParserThread::DoParse(), I assume.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: dmoore on February 12, 2014, 02:16:36 am
"new command line option --user-data-dir=<path>" is very good !  :D

Good! Let us know if there are issues.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 12, 2014, 03:51:52 am
Here is the BT that I see m_TokenIndex always stay in the same value (infinite loop)
For me its different: It looks like in bool ParserThread::Parse() in the else if (!switchHandled) clause it always end in the m_Str << token << ParserConsts::space_chr; line which will crash sooner or later if the wxString cannot append more characters. at that time, token is something like (wxMACRO(...)) which cannot be resolved / handled in this loop.

I debugged a little, I think it was a regression after a patch by Huki, that is:

Code: [Select]
Revision: f7cb07c8dcf613f6786eaf0ee34e908ec218d2c8
Author: ollydbg <[email protected]>
Date: 2013-9-29 16:25:10
Message:
* CC: reliable working of UngetToken() when macro expansion is involved, it set the undo index value correctly(thanks Huki)

* CC: avoid take backward step (UngetToken) twice in the Tokenizer class
- CC: comments added for Tokenizer class

git-svn-id: https://svn.code.sf.net/p/codeblocks/code/[email protected] 2a5c6006-c6dd-42ca-98ab-0921f2732cef
----
Modified: src/plugins/codecompletion/parser/tokenizer.cpp
Modified: src/plugins/codecompletion/parser/tokenizer.h


which have such diff:
Code: [Select]
@@ -1148,6 +1148,8 @@ wxString Tokenizer::PeekToken()
         unsigned int savedLineNumber = m_LineNumber;
         unsigned int savedNestLevel  = m_NestLevel;
 
+        int savedReplaceCount = m_IsReplaceParsing ? m_RepeatReplaceCount : -1;
+
         if (SkipUnwanted())
             m_PeekToken = DoGetToken();
         else
@@ -1156,17 +1158,33 @@ wxString Tokenizer::PeekToken()
         m_PeekTokenIndex             = m_TokenIndex;
         m_PeekLineNumber             = m_LineNumber;
         m_PeekNestLevel              = m_NestLevel;
-
-        m_TokenIndex                 = savedTokenIndex;
-        m_LineNumber                 = savedLineNumber;
-        m_NestLevel                  = savedNestLevel;
+        // Check whether a ReplaceBufferForReparse() was done in DoGetToken().
+        // We assume m_Undo... have already been reset in ReplaceBufferForReparse().
+        if (m_IsReplaceParsing && savedReplaceCount != (int)m_RepeatReplaceCount)
+        {
+            m_TokenIndex             = m_UndoTokenIndex;
+            m_LineNumber             = m_UndoLineNumber;
+            m_NestLevel              = m_UndoNestLevel;
+        }
+        else
+        {
+            m_TokenIndex             = savedTokenIndex;
+            m_LineNumber             = savedLineNumber;
+            m_NestLevel              = savedNestLevel;
+        }
     }
 
     return m_PeekToken;

So, bug is here:
Code: [Select]
wxString Tokenizer::PeekToken()
{
    if (!m_PeekAvailable)
    {
        m_PeekAvailable = true;

        unsigned int savedTokenIndex = m_TokenIndex;
        unsigned int savedLineNumber = m_LineNumber;
        unsigned int savedNestLevel  = m_NestLevel;

        int savedReplaceCount = m_IsReplaceParsing ? m_RepeatReplaceCount : -1;

        if (SkipUnwanted())
            m_PeekToken = DoGetToken();
        else
            m_PeekToken.Clear();

        m_PeekTokenIndex             = m_TokenIndex;
        m_PeekLineNumber             = m_LineNumber;
        m_PeekNestLevel              = m_NestLevel;
        // Check whether a ReplaceBufferText() was done in DoGetToken().
        // We assume m_Undo... have already been reset in ReplaceBufferText().
        if (m_IsReplaceParsing && savedReplaceCount != (int)m_RepeatReplaceCount)
        {
            m_TokenIndex             = m_UndoTokenIndex;                       //*************bug here******************
            m_LineNumber             = m_UndoLineNumber;
            m_NestLevel              = m_UndoNestLevel;
        }
        else
        {
            m_TokenIndex             = savedTokenIndex;
            m_LineNumber             = savedLineNumber;
            m_NestLevel              = savedNestLevel;
        }
    }

    return m_PeekToken;
}

m_TokenIndex is always remain/reset to the same value, so Tokenizer won't go forward any more.


Title: Re: The 08 February 2014 build (9639) is out.
Post by: blauzahn on February 12, 2014, 07:00:42 am
Has it been considered to group

TokenIndex
LineNumber
NestLevel

into a tiny class/struct?
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 12, 2014, 07:14:20 am
Has it been considered to group

TokenIndex
LineNumber
NestLevel

into a tiny class/struct?
Hi, blauzahn, thanks. Sure this will make the code more clean and readable(I even thought the lexeme string should be bundled with those information to a single struct), but the change is not simple, because two many places use those variables. :)
I'm currently don't have the time to do that, so this can be a TO-DO.
Let's firstly fix the hang issue. ;)
Title: Re: The 08 February 2014 build (9639) is out.
Post by: MortenMacFly on February 12, 2014, 08:16:13 am
m_TokenIndex is always remain/reset to the same value, so Tokenizer won't go forward any more.
That's too bad. Can you fix it? Otherwise I would strongly recommend to revert this change completely - it makes C::B unusable with wx30.

Edit: What SVN revision this belongs to, btw?! ???
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 12, 2014, 08:27:38 am
m_TokenIndex is always remain/reset to the same value, so Tokenizer won't go forward any more.
That's too bad. Can you fix it? Otherwise I would strongly recommend to revert this change completely - it makes C::B unusable with wx30.

Edit: What SVN revision this belongs to, btw?! ???

Follow this post:
Re: Several improvements to Code Completion plugin (http://forums.codeblocks.org/index.php/topic,18315.msg125473.html#msg125473)
and
Re: Several improvements to Code Completion plugin (http://forums.codeblocks.org/index.php/topic,18315.msg125579.html#msg125579)

The change is rev 9369, you can see it from the git log message in my previous posts.
Quote
Revision: f7cb07c8dcf613f6786eaf0ee34e908ec218d2c8
Author: ollydbg <[email protected]>
Date: 2013-9-29 16:25:10
Message:
* CC: reliable working of UngetToken() when macro expansion is involved, it set the undo index value correctly(thanks Huki)

* CC: avoid take backward step (UngetToken) twice in the Tokenizer class
- CC: comments added for Tokenizer class

git-svn-id: https://svn.code.sf.net/p/codeblocks/code/[email protected]9369 2a5c6006-c6dd-42ca-98ab-0921f2732cef

Currently I can't fix it, but I'm debugging now for hours......
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 12, 2014, 09:27:41 am
The code do the macro replacement is a mass, the big error looks like it is not the way a C Processor do the replacement.

For example, if we have such code

Code: [Select]
/****************************************/

#define A(x,y)  (x+y)
#define B 1
#define C 2


#if A(B,C)

void f1234();

#endif

void f2345();


// f123      //f1234
// f234      //f2345
To expand the A(B,C), we should first apply the macro definition 1, so it becomes (B+C), then we should "rescan" the result token list, and finally get the (2+1), so the final result is 3.

But from my point of view, our Tokenizer don't have a "rescan" stage, the trick thing is that it did some text substitute on formal parameter to actual parameter translation, some kind of recursive call.

Note, with the latest cc-test frame, I get such result
Code: [Select]
-PASS:  f234  f2345
*FAIL:  f123  f1234
--------------------------------------------------------
Total 2 tests, 1 PASS, 1 FAIL
--------------------------------------------------------
So, the #if branch is abandoned.

EDIT:

If I change the #if line to "#if A(2,3)", then I get two PASSes.
Code: [Select]
-PASS:  f234  f2345
-PASS:  f123  f1234
--------------------------------------------------------
Total 2 tests, 2 PASS, 0 FAIL
--------------------------------------------------------

Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 12, 2014, 02:48:24 pm
@morten, revert rev9369 can avoid the hang issue, so I'm going to commit the change now. (This is a workaround, not a true fix).
Title: Re: The 08 February 2014 build (9639) is out.
Post by: osdt on February 12, 2014, 03:01:21 pm
@morten, revert rev9369 can avoid the hang issue, so I'm going to commit the change now. (This is a workaround, not a true fix).

@ollydbg: reverting r9638 fixes this issue too!

I can confirm that r9639 has this issue while r9619 works flawlessly. Between this revisions, CC-related changes are r9636, r9638 and r9639.

- osdt
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 12, 2014, 03:08:36 pm
@morten, revert rev9369 can avoid the hang issue, so I'm going to commit the change now. (This is a workaround, not a true fix).

@ollydbg: reverting r9638 fixes this issue too!

I can confirm that r9639 has this issue while r9619 works flawlessly. Between this revisions, CC-related changes are r9636 and r9638.

- osdt
Dear osdt, thanks for the reply. Rev9638 is a solid fix for rev 9601, because when in rev 9601, I leave a logic error.

See what rev9638 does:
Code: [Select]
src/plugins/codecompletion/parser/tokenizer.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/plugins/codecompletion/parser/tokenizer.cpp b/src/plugins/codecompletion/parser/tokenizer.cpp
index 6761574..25b0a6d 100644
--- a/src/plugins/codecompletion/parser/tokenizer.cpp
+++ b/src/plugins/codecompletion/parser/tokenizer.cpp
@@ -1985,7 +1985,7 @@ bool Tokenizer::GetMacroExpendedText(const Token* tk, wxString& expandedText)
 
     // sanity check if we have such macro definition that #define AAA(x,y) x+y+AAA
     // which means a macro name is exists in its definition, which will cause a infinite expansion loop
-    if (tk->m_FullType.Find(tk->m_Name) ==wxNOT_FOUND)
+    if (tk->m_FullType.Find(tk->m_Name) != wxNOT_FOUND)
         return false;
 
     // Now, tk is a function like macro definition we are going to expand, it's m_Args contains the

You see, if you leave the  if (tk->m_FullType.Find(tk->m_Name) ==wxNOT_FOUND) here, than you just disable the macro expansion feature now, so it should definitely be  != for this sanity check.

Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 12, 2014, 03:36:46 pm
@morten, revert rev9369 can avoid the hang issue, so I'm going to commit the change now. (This is a workaround, not a true fix).
Done in r9647, sorry to all who have the hang issue.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: osdt on February 12, 2014, 03:37:46 pm
Dear osdt, thanks for the reply. Rev9638 is a solid fix for rev 9601, because when in rev 9601, I leave a logic error.

Ok. I was just bitsecting this issue and came across r9638.

-osdt
Title: Re: The 08 February 2014 build (9639) is out.
Post by: Huki on February 13, 2014, 05:36:24 pm
I debugged a little, I think it was a regression after a patch by Huki,
So, bug is here:
Code: [Select]
       // Check whether a ReplaceBufferText() was done in DoGetToken().
        // We assume m_Undo... have already been reset in ReplaceBufferText().
        if (m_IsReplaceParsing && savedReplaceCount != (int)m_RepeatReplaceCount)
        {
            m_TokenIndex             = m_UndoTokenIndex;                       //*************bug here******************
            m_LineNumber             = m_UndoLineNumber;
            m_NestLevel              = m_UndoNestLevel;
        }
m_TokenIndex is always remain/reset to the same value, so Tokenizer won't go forward any more.

Are we dealing with really complex nested macros in wx3.0? I'm assuming that is the problem...
A quick explanation of my patch for PeekToken(): the goal is to reset m_TokenIndex every time a macro replacement occurs inside ReplaceBufferText(). So we use the test "if (m_IsReplaceParsing && savedReplaceCount != (int)m_RepeatReplaceCount)", to see if m_RepeatReplaceCount has increased after calling DoGetToken(). This should be safe because there is a limit on the number of nested macro replacements allowed (s_MaxRepeatReplaceCount), so infinite loop is impossible.

But seeing the concerned code, I think the problem is obvious: (in tokenizer.cpp)
Code: [Select]
bool Tokenizer::ReplaceBufferText(const wxString& target, bool updatePeekToken)
{
    if (target.IsEmpty())
        return false;

    if (m_IsReplaceParsing && ++m_RepeatReplaceCount > s_MaxRepeatReplaceCount)
    {
        m_TokenIndex = m_BufferLen - m_FirstRemainingLength;
        m_PeekAvailable = false;
        SkipToEOL(false);
        return false;
    }

    ...


We can see that the m_RepeatReplaceCount will keep increasing even after s_MaxRepeatReplaceCount is reached, even though no more replacement occurs. We should change it to something like this:
Code: [Select]
bool Tokenizer::ReplaceBufferText(const wxString& target, bool updatePeekToken)
{
    if (target.IsEmpty())
        return false;

    if (m_IsReplaceParsing && m_RepeatReplaceCount >= s_MaxRepeatReplaceCount)
        return false;
 
    ++m_RepeatReplaceCount;
    
    ...


@ollydbg: Does my patch still cause the hang with the above fix?
Title: Re: The 08 February 2014 build (9639) is out.
Post by: ollydbg on February 14, 2014, 02:49:32 am
...
@ollydbg: Does my patch still cause the hang with the above fix?
@Huki, many thanks, fairly clear explanation, I just test the code and no hang now.
This is the code I use:
Code: [Select]
bool Tokenizer::ReplaceBufferText(const wxString& target, bool updatePeekToken)
{
    if (target.IsEmpty())
        return false;

    if (m_IsReplaceParsing)
    {
        if (m_RepeatReplaceCount >= s_MaxRepeatReplaceCount)
        {
            m_TokenIndex = m_BufferLen - m_FirstRemainingLength;
            m_PeekAvailable = false;
            SkipToEOL(false);
            return false;
        }
        else
            ++m_RepeatReplaceCount;
    }

.....

BTW: I originally thought the test condition in PeekToken() can be changed to

Quote
       if (m_IsReplaceParsing && savedReplaceCount < (int)m_RepeatReplaceCount)

Because in some cases, m_RepeatReplaceCount will reset to zero, but finally I found that is not necessary, because both m_IsReplaceParsing and m_RepeatReplaceCount will be reset to zero in the same time.

Code: [Select]
   if (m_FirstRemainingLength != 0 && m_BufferLen - m_FirstRemainingLength < m_TokenIndex)
    {
        m_FirstRemainingLength = 0;
        m_IsReplaceParsing = false;
        m_RepeatReplaceCount = 0;
    }

BTW2: Why we need two variables? I think m_IsReplaceParsing is a redundant variable of m_RepeatReplaceCount. Checking all the source code, I realize that
Code: [Select]
RepeatReplaceCount > 0 means m_IsReplaceParsing == true
RepeatReplaceCount == 0 means m_IsReplaceParsing == false
So, if we remove the m_IsReplaceParsing, we can change the test to
Quote
       if (savedReplaceCount < m_RepeatReplaceCount)
But the code to initialize savedReplaceCount can be simply
Code: [Select]
size_t savedReplaceCount = m_RepeatReplaceCount;
Then
Code: [Select]
if (m_IsReplaceParsing)can change to
Code: [Select]
if (m_RepeatReplaceCount > 0)
Also need to adjust the code below
Code: [Select]
   // Set replace parsing state, and save first replace token index
    if (!m_IsReplaceParsing)
    {
        m_FirstRemainingLength = m_BufferLen - m_TokenIndex;
        m_IsReplaceParsing = true;
    }

I will prepare two commits:
1, fix the hang issue as you suggest.
2, code refactoring by remove the m_IsReplaceParsing variable.

Finally, thanks for your help!


Title: Re: The 08 February 2014 build (9639) is out.
Post by: Huki on February 15, 2014, 03:32:48 am
Glad to know the hang was fixed. :)

I will prepare two commits:
1, fix the hang issue as you suggest.
2, code refactoring by remove the m_IsReplaceParsing variable.
I checked both commits, it's ok for me.. and yes, removing the redundant m_IsReplaceParsing var is a nice improvement.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: stahta01 on February 18, 2014, 05:05:35 pm
Can someone apply this short fix?

Patch that get rid of the Install Plugin error on Windows; after 2 hours of trying semi-random stuff I found a fix.
Error message was "One or more plugins were not installed successfully".

Tim S.

Code: [Select]
Index: src/sdk/pluginmanager.cpp
===================================================================
--- src/sdk/pluginmanager.cpp (revision 9653)
+++ src/sdk/pluginmanager.cpp (working copy)
@@ -282,7 +282,7 @@
         settingsOnName.Remove(0, 3);
     if (!platform::windows && settingsOffName.StartsWith(_T("lib")))
         settingsOffName.Remove(0, 3);
-    wxString pluginFilename = pluginDir + _T('/') + localName;
+    wxString pluginFilename = UnixFilename(pluginDir + _T('/') + localName);
 //    Manager::Get()->GetLogManager()->DebugLog(F(_T("Plugin filename: ") + pluginFilename));
 //    Manager::Get()->GetLogManager()->DebugLog(F(_T("Plugin resources: ") + ConfigManager::GetDataFolder() + _T('/') + resourceName));
 

Thanks.

Tim S.
Title: Re: The 08 February 2014 build (9639) is out.
Post by: MortenMacFly on February 18, 2014, 05:17:44 pm
Thanks.
I would, but currently I've no access to SVN. I've noted it down... stay tuned...