Debugging code completion plugin

[Pecan]

I'm getting an intermittent loop in code completion. It goes away when I turn off symbol browsing.

I'd like to turn on tracing to get closer to the problem.

Would someone give me a hint about the proper way to do that.

thanks

[ollydbg]

The loop is mainly in function "void ParserThread::DoParse()", if you would like to enable the TRACE macro, it is setting:

Code

#define CC_PARSERTHREAD_DEBUG_OUTPUT 1

But it will log MANY messages.

I think you can try to disable all the four options in, Editor->Code Completion->C/C++ Parser, so that only the files in your cbp file will be parsed.

If you have narrowed down the endless loop in one file, then you can try the cc_test project, and put the code snippet in "test.h", this will find the bug quickly.

[MortenMacFly]

But it will log MANY messages.

Not if you have a small project to reproduce and probably do not scan system headers.

[ollydbg]

Also, the log and debug log message should be redirected to files (I think C::B has such start up options), if C::B hangs, you are hard to see them in the main GUI.

[ollydbg]

Also, the log and debug log message should be redirected to files (I think C::B has such start up options)

FYI: they are options below:

Code

--log-to-file --debug-log-to-file

[Pecan]

ImageCraft Creations has found a quirk in cbProject::CalculateCommonTopLevelPath() that allows root to be set as the common top level path. This in turn causes CodeCompletion to always search the whole of the HD from root when "View: Current file's symbols" is set.
For each open file, this stalls CB for about 7 seconds.

An erroneously specified file, such as "..\..\..\..\iccv8cortex\include\CMSIS\core_cm3.h" where iccv8cortex is actually located at c:\iccv8cortex will cause the top level path to be set to C:\  

ImageCraft proposes the following fix:

Code

Index: src/sdk/cbproject.cpp
===================================================================
--- src/sdk/cbproject.cpp	(revision 9271)
+++ src/sdk/cbproject.cpp	(working copy)
@@ -364,6 +364,21 @@
             tmpbase << sep << tmp.Left(pos) << sep;
 
         wxFileName tmpbaseF(tmpbase); tmpbaseF.Normalize(wxPATH_NORM_DOTS);
+
+        // Avoid missing files causing mis-setting of common top level path
+        if (not wxFileExists(f->file.GetFullPath()))
+        {
+            Manager::Get()->GetLogManager()->Log(_T("Non-existent file: ") + f->relativeFilename);
+            continue;
+        }
+        // Don't allow root to be set as common top level search path
+        if ((tmpbaseF.GetFullPath().Mid(1).Lower() == _T(":\\"))
+            or (tmpbaseF.GetFullPath().Lower() == _T("/")))
+        {
+            Manager::Get()->GetLogManager()->Log(_T("Skipped setting root as CommonTopLevelPath from: ") + f->relativeFilename);
+            continue;
+        }
+
         if (   (tmpbaseF.GetDirCount() < base.GetDirCount())
             && (base.GetPath().StartsWith(tmpbaseF.GetPath())) )
         {

[ollydbg]

Hi, pecan, I have two questions:
1, if CC cause the hang/slowness issue, is it possible to fix in CC?
2, I don't find a way that in CC's symbol browser, "View: Current file's symbols" is set" will cause the hang issue, can you give me a hint (maybe call stack?)
Thanks.

[Jenna]

If CC really causes the slowdown, than CC should be fixed.

I'm against silently dropping the root-folder as common toplevel path, if it is calculated correctly.

Why should the IDE fix a poor project layout, like e.g. cmake-projects can have, if they add cmake files from below /usr to the project, even if the user is not allowed to do anything with them.

[Pecan]

Well, yes, it's a user layout problem that caused CC to scan all files from root.

But the user has no way to find or fix the problem without a message or warning. If we don't want to prevent CC scanning for symbols from root, can we put out the warning messages instead?

Ollydbg:
CC is doing just what it's supposed to do. It's just that when CC calls CalculateCommonTopLevelPath() when dbfFile is set AND the user has specified a project file like "..\..\..\..\iccv8cortex\include\CMSIS\core_cm3.h" where iccv8cortex is actually located at c:\iccv8cortex AND because that specification resolves to a file that actually exists, CC is handed back a top level path of "C:\".

CC wil then scan for all open file's symbols starting at root. It will scan c:\windows and all it subdirectories taking many seconds of time for each open file.

Thats why I called it a quirk, not a bug. But the user suffers....
because with many files, such as in FreeRTOS source, the problem is almost impossible to find.

I'll upload (somewhere) the source that causes this and get back to you.
 

[Pecan]

To recreate the CC root scan quirk:

Download:
https://dl.dropboxusercontent.com/u/46870262/iccv8cortexCCRootScan.7z

Unpack and place the folder iccv8cortex at the root of drive c:\

Load the examples project:
c:\iccv8cortex\examples.cortex\examples.workspace

Alt-G to load tasks.c
Alt-G to load croutine.c

close workspace, then reload it.
CC will scan all of the hard disk starting at root C:\
 

[ollydbg]

To recreate the CC root scan quirk:

Download:
https://dl.dropboxusercontent.com/u/46870262/iccv8cortexCCRootScan.7z

Unpack and place the folder iccv8cortex at the root of drive c:\

Load the examples project:
c:\iccv8cortex\examples.cortex\examples.workspace

Alt-G to load tasks.c
Alt-G to load croutine.c

close workspace, then reload it.
CC will scan all of the hard disk starting at root C:\
 

Thanks for the example projects. But I can't reproduce this bug.
What is your CC setting? One parser per project(in this case the maximal allowed parser number is?) or one parser for the whole workspace?

Besides that, how do you know "CC scan the root of C:\"?  Can you put a breakpoint in the line below (in plugins\codecompletion\systemheadersthread.cpp):

Code

        HeaderDirTraverser traverser(this, m_SystemHeadersThreadCS, m_SystemHeadersMap, dirs[i]);
        dir.Traverse(traverser, wxEmptyString, wxDIR_FILES | wxDIR_DIRS);

Then you can see what is the actual dir the Traverser's working one. If it contains a C:\, then the traverser will scan all the files/sub-folders of the C:\.

[Jenna]

I just tested the workspace (but I do not have icc installed, just gcc).

After opening the workspace everything is okay, symbolsbrowser shows "Everything", but switching to "Curent file" starts hdd-activity for a long time (C::B is not responsible).
Parser-settings are "One parser per project) with a maximum of 5 parsers.

The funny thing is, that I am not able to reproduce this behaviour, neither by clicking "Reparse now", by closing and reopneing the workspace or by closing and reopening C::B.

I can not debug the sources here (at work) and I use the 12.11 release.
I can try to do tests with trunk on win7, xp and win8 this evening.

[Jenna]

What I find if I start C::B with debug-log is:

Code

Traversing 'C:\iccv8cortex\examples.cortex\FreeRTOS\Source' for: tasks.*
Traversing ' - C:\' for: tasks.*
Found 1 files:
- C:\iccv8cortex\examples.cortex\FreeRTOS\Source\tasks.c

But even renaming the folder, so that it starts with "Z"  the file is found nearly immediately (without much hdd-activity).

[Pecan]

Be sure that View: is set to "Current File's symbols" and that task.c and croutine.c is loaded in the editor.

"Use one parser per project" is set.

Yes, I've set the trap and GetTopLevelPath() hands back "C:\" to CC.


svn build  rev 9271 (2013-08-18 05:51:11)   gcc 4.6.1 Windows/unicode - 32 bit

Note below that C:\ is being traversed.
NativeParser::GetAllPathsByFilename(): Traversing ' - C:\' for: croutine.*

Code

NativeParser::DoFullParsing(): Adding three kind of files to batch-parser
NativeParser::DoFullParsing(): Added 127 header&source file(s) for project 'FreeRTOS_NXP_demo' to batch-parser...
NativeParser::CreateParser(): Finish creating a new parser for project 'FreeRTOS_NXP_demo'
NativeParser::OnParsingOneByOneTimer(): Start switch from OnParsingOneByOneTimer
NativeParser::GetAllPathsByFilename(): Traversing 'C:\iccv8cortex\examples.cortex\FreeRTOS\Source' for: croutine.*
NativeParser::GetAllPathsByFilename(): Traversing ' - C:\' for: croutine.*
NativeParser::GetAllPathsByFilename(): Found 1 files:
- C:\iccv8cortex\examples.cortex\FreeRTOS\Source\croutine.c
Switch parser to project 'FreeRTOS_NXP_demo'
ClassBrowser::OnThreadEvent(): Updating class browser...
NativeParser::OnParserStart(): Starting batch parsing for project 'FreeRTOS_NXP_demo'...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::OnParserEnd(): Project 'FreeRTOS_NXP_demo' parsing stage done!
Project 'FreeRTOS_NXP_demo' parsing stage done (126 total parsed files, 3262 tokens in 0 minute(s), 1.014 seconds).
NativeParser::GetAllPathsByFilename(): Traversing 'C:\iccv8cortex\examples.cortex\FreeRTOS\Source' for: croutine.*
NativeParser::GetAllPathsByFilename(): Traversing ' - C:\' for: croutine.*
NativeParser::GetAllPathsByFilename(): Found 1 files:
- C:\iccv8cortex\examples.cortex\FreeRTOS\Source\croutine.c
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.

[Jenna]

That's similar to my output, but as written it was slow the first time and then it found the file immediately.
Maybe WinXP has kept the index of the hdd in memory (3.3 GB Ram on the laptop I used for testing).
I will try it on virtual WinXP, Win7 and Win8 this evening.