Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => Topic started by: Pecan on July 12, 2013, 06:48:24 pm

Title: Debugging code completion plugin
Post by: Pecan on July 12, 2013, 06:48:24 pm
I'm getting an intermittent loop in code completion. It goes away when I turn off symbol browsing.

I'd like to turn on tracing to get closer to the problem.

Would someone give me a hint about the proper way to do that.

thanks
Title: Re: Debugging code completion plugin
Post by: ollydbg on July 14, 2013, 05:58:40 am
The loop is mainly in function "void ParserThread::DoParse()", if you would like to enable the TRACE macro, it is setting:
Code
#define CC_PARSERTHREAD_DEBUG_OUTPUT 1

But it will log MANY messages.

I think you can try to disable all the four options in, Editor->Code Completion->C/C++ Parser, so that only the files in your cbp file will be parsed.

If you have narrowed down the endless loop in one file, then you can try the cc_test project, and put the code snippet in "test.h", this will find the bug quickly.
Title: Re: Debugging code completion plugin
Post by: MortenMacFly on July 16, 2013, 06:24:11 pm
But it will log MANY messages.
Not if you have a small project to reproduce and probably do not scan system headers.
Title: Re: Debugging code completion plugin
Post by: ollydbg on July 17, 2013, 02:18:06 am
Also, the log and debug log message should be redirected to files (I think C::B has such start up options), if C::B hangs, you are hard to see them in the main GUI.
Title: Re: Debugging code completion plugin
Post by: ollydbg on July 18, 2013, 10:58:58 am
Also, the log and debug log message should be redirected to files (I think C::B has such start up options)
FYI: they are options below:
Code
--log-to-file --debug-log-to-file
Title: Re: Debugging code completion plugin
Post by: Pecan on August 22, 2013, 08:24:38 pm
ImageCraft Creations has found a quirk in cbProject::CalculateCommonTopLevelPath() that allows root to be set as the common top level path. This in turn causes CodeCompletion to always search the whole of the HD from root when "View: Current file's symbols" is set.
For each open file, this stalls CB for about 7 seconds.

An erroneously specified file, such as "..\..\..\..\iccv8cortex\include\CMSIS\core_cm3.h" where iccv8cortex is actually located at c:\iccv8cortex will cause the top level path to be set to C:\  

ImageCraft proposes the following fix:

Code
Index: src/sdk/cbproject.cpp
===================================================================
--- src/sdk/cbproject.cpp (revision 9271)
+++ src/sdk/cbproject.cpp (working copy)
@@ -364,6 +364,21 @@
             tmpbase << sep << tmp.Left(pos) << sep;
 
         wxFileName tmpbaseF(tmpbase); tmpbaseF.Normalize(wxPATH_NORM_DOTS);
+
+        // Avoid missing files causing mis-setting of common top level path
+        if (not wxFileExists(f->file.GetFullPath()))
+        {
+            Manager::Get()->GetLogManager()->Log(_T("Non-existent file: ") + f->relativeFilename);
+            continue;
+        }
+        // Don't allow root to be set as common top level search path
+        if ((tmpbaseF.GetFullPath().Mid(1).Lower() == _T(":\\"))
+            or (tmpbaseF.GetFullPath().Lower() == _T("/")))
+        {
+            Manager::Get()->GetLogManager()->Log(_T("Skipped setting root as CommonTopLevelPath from: ") + f->relativeFilename);
+            continue;
+        }
+
         if (   (tmpbaseF.GetDirCount() < base.GetDirCount())
             && (base.GetPath().StartsWith(tmpbaseF.GetPath())) )
         {



Title: Re: Debugging code completion plugin
Post by: ollydbg on August 23, 2013, 09:00:33 am
Hi, pecan, I have two questions:
1, if CC cause the hang/slowness issue, is it possible to fix in CC?
2, I don't find a way that in CC's symbol browser, "View: Current file's symbols" is set" will cause the hang issue, can you give me a hint (maybe call stack?)
Thanks.
Title: Re: Debugging code completion plugin
Post by: Jenna on August 23, 2013, 10:22:01 am
If CC really causes the slowdown, than CC should be fixed.

I'm against silently dropping the root-folder as common toplevel path, if it is calculated correctly.

Why should the IDE fix a poor project layout, like e.g. cmake-projects can have, if they add cmake files from below /usr to the project, even if the user is not allowed to do anything with them.
Title: Re: Debugging code completion plugin
Post by: Pecan on August 23, 2013, 04:33:54 pm
Well, yes, it's a user layout problem that caused CC to scan all files from root.

But the user has no way to find or fix the problem without a message or warning. If we don't want to prevent CC scanning for symbols from root, can we put out the warning messages instead?

Ollydbg:
CC is doing just what it's supposed to do. It's just that when CC calls CalculateCommonTopLevelPath() when dbfFile is set AND the user has specified a project file like "..\..\..\..\iccv8cortex\include\CMSIS\core_cm3.h" where iccv8cortex is actually located at c:\iccv8cortex AND because that specification resolves to a file that actually exists, CC is handed back a top level path of "C:\".

CC wil then scan for all open file's symbols starting at root. It will scan c:\windows and all it subdirectories taking many seconds of time for each open file.

Thats why I called it a quirk, not a bug. But the user suffers....
because with many files, such as in FreeRTOS source, the problem is almost impossible to find.

I'll upload (somewhere) the source that causes this and get back to you.
 
Title: Re: Debugging code completion plugin
Post by: Pecan on August 27, 2013, 11:55:50 pm
To recreate the CC root scan quirk:

Download:
https://dl.dropboxusercontent.com/u/46870262/iccv8cortexCCRootScan.7z

Unpack and place the folder iccv8cortex at the root of drive c:\

Load the examples project:
c:\iccv8cortex\examples.cortex\examples.workspace

Alt-G to load tasks.c
Alt-G to load croutine.c

close workspace, then reload it.
CC will scan all of the hard disk starting at root C:\
 
Title: Re: Debugging code completion plugin
Post by: ollydbg on August 28, 2013, 09:09:22 am
To recreate the CC root scan quirk:

Download:
https://dl.dropboxusercontent.com/u/46870262/iccv8cortexCCRootScan.7z

Unpack and place the folder iccv8cortex at the root of drive c:\

Load the examples project:
c:\iccv8cortex\examples.cortex\examples.workspace

Alt-G to load tasks.c
Alt-G to load croutine.c

close workspace, then reload it.
CC will scan all of the hard disk starting at root C:\
 

Thanks for the example projects. But I can't reproduce this bug.
What is your CC setting? One parser per project(in this case the maximal allowed parser number is?) or one parser for the whole workspace?

Besides that, how do you know "CC scan the root of C:\"?  Can you put a breakpoint in the line below (in plugins\codecompletion\systemheadersthread.cpp):

Code
        HeaderDirTraverser traverser(this, m_SystemHeadersThreadCS, m_SystemHeadersMap, dirs[i]);
        dir.Traverse(traverser, wxEmptyString, wxDIR_FILES | wxDIR_DIRS);

Then you can see what is the actual dir the Traverser's working one. If it contains a C:\, then the traverser will scan all the files/sub-folders of the C:\.

Title: Re: Debugging code completion plugin
Post by: Jenna on August 28, 2013, 09:46:14 am
I just tested the workspace (but I do not have icc installed, just gcc).

After opening the workspace everything is okay, symbolsbrowser shows "Everything", but switching to "Curent file" starts hdd-activity for a long time (C::B is not responsible).
Parser-settings are "One parser per project) with a maximum of 5 parsers.

The funny thing is, that I am not able to reproduce this behaviour, neither by clicking "Reparse now", by closing and reopneing the workspace or by closing and reopening C::B.

I can not debug the sources here (at work) and I use the 12.11 release.
I can try to do tests with trunk on win7, xp and win8 this evening.
Title: Re: Debugging code completion plugin
Post by: Jenna on August 28, 2013, 09:55:47 am
What I find if I start C::B with debug-log is:
Code
Traversing 'C:\iccv8cortex\examples.cortex\FreeRTOS\Source' for: tasks.*
Traversing ' - C:\' for: tasks.*
Found 1 files:
- C:\iccv8cortex\examples.cortex\FreeRTOS\Source\tasks.c

But even renaming the folder, so that it starts with "Z"  the file is found nearly immediately (without much hdd-activity).
Title: Re: Debugging code completion plugin
Post by: Pecan on August 28, 2013, 07:40:28 pm
Be sure that View: is set to "Current File's symbols" and that task.c and croutine.c is loaded in the editor.

"Use one parser per project" is set.

Yes, I've set the trap and GetTopLevelPath() hands back "C:\" to CC.


svn build  rev 9271 (2013-08-18 05:51:11)   gcc 4.6.1 Windows/unicode - 32 bit

Note below that C:\ is being traversed.
NativeParser::GetAllPathsByFilename(): Traversing ' - C:\' for: croutine.*

Code
NativeParser::DoFullParsing(): Adding three kind of files to batch-parser
NativeParser::DoFullParsing(): Added 127 header&source file(s) for project 'FreeRTOS_NXP_demo' to batch-parser...
NativeParser::CreateParser(): Finish creating a new parser for project 'FreeRTOS_NXP_demo'
NativeParser::OnParsingOneByOneTimer(): Start switch from OnParsingOneByOneTimer
NativeParser::GetAllPathsByFilename(): Traversing 'C:\iccv8cortex\examples.cortex\FreeRTOS\Source' for: croutine.*
NativeParser::GetAllPathsByFilename(): Traversing ' - C:\' for: croutine.*
NativeParser::GetAllPathsByFilename(): Found 1 files:
- C:\iccv8cortex\examples.cortex\FreeRTOS\Source\croutine.c
Switch parser to project 'FreeRTOS_NXP_demo'
ClassBrowser::OnThreadEvent(): Updating class browser...
NativeParser::OnParserStart(): Starting batch parsing for project 'FreeRTOS_NXP_demo'...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::OnParserEnd(): Project 'FreeRTOS_NXP_demo' parsing stage done!
Project 'FreeRTOS_NXP_demo' parsing stage done (126 total parsed files, 3262 tokens in 0 minute(s), 1.014 seconds).
NativeParser::GetAllPathsByFilename(): Traversing 'C:\iccv8cortex\examples.cortex\FreeRTOS\Source' for: croutine.*
NativeParser::GetAllPathsByFilename(): Traversing ' - C:\' for: croutine.*
NativeParser::GetAllPathsByFilename(): Found 1 files:
- C:\iccv8cortex\examples.cortex\FreeRTOS\Source\croutine.c
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
Title: Re: Debugging code completion plugin
Post by: Jenna on August 28, 2013, 07:57:26 pm
That's similar to my output, but as written it was slow the first time and then it found the file immediately.
Maybe WinXP has kept the index of the hdd in memory (3.3 GB Ram on the laptop I used for testing).
I will try it on virtual WinXP, Win7 and Win8 this evening.
Title: Re: Debugging code completion plugin
Post by: Jenna on August 28, 2013, 10:02:30 pm
In this case the issue is not a C::B or CC issue, it's a broken projectfile.
You have he same file twice in it (with different relative paths):
Code
		<Unit filename="..\..\..\..\iccv8cortex\include\CMSIS\core_cm3.h" />
<Unit filename="..\..\..\include\CMSIS\core_cm3.h" />

The issue can simply be fixed by explicitely saving the projectfile, closing and reopening it.

Nevertheless the calculating of the common toplevel path should work correctly in this case also in my opinion.
Title: Re: Debugging code completion plugin
Post by: Jenna on August 28, 2013, 10:43:28 pm
Nevertheless the calculating of the common toplevel path should work correctly in this case also in my opinion.

The problem is that CalculateCommonTopLevelPath() relies on a correct relativeFilename. This will be set correctly when the project gets saved, so it must have been added in a wrong way (manually or by a plugin or whatever ? ).

To avoid this we either have to recalculate the relativeFilename when a project is opened or everytime when we use it (e.g. CalculateCommonTopLevelPath() ).
The first attempt would most likely slow down opening large projects and the second one would make storing it useless and would surely lead to more serious performance issues with large projects.