Author Topic: CC upper limits ? BUT for sure BUGs  (Read 21323 times)

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5267
CC upper limits ? BUT for sure BUGs
« on: January 22, 2014, 06:38:25 pm »
Are there any upper limits for CC (parsing ?)  related to :
- number of projects in workspace
- number of files
- number of tokens identified

I have noticed this for a very long time (1 year  + ?), when I load a workspace, which contains +- 100 projects (maybe even more), the CC works very bad. It can't find declarations, implementations, even when the stuff is within the same (active) project, when looking at the log of the parsing, I notice it for sure does not parse all projects.

So lot of frustration with the users :-(

« Last Edit: January 25, 2014, 11:11:52 pm by killerbot »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5295
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ?
« Reply #1 on: January 24, 2014, 07:31:55 am »
Are there any upper limits for CC (parsing ?)  related to :
- number of projects in workspace

There is a limit for project numbers if you are in "One Parser per Project" case. I don't think there is a limit for the "One Parser for the whole Workspace" case.

Code: [Select]
- number of files
- number of tokens identified
No.

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5267
Re: CC upper limits ? BUT for sure BUGs
« Reply #2 on: January 25, 2014, 11:20:14 pm »
did some experiment, I noticed already in the past the parsing (in the CB log tab) stopped at the same project in the workspace, it is not due to the fact it is a given Nth project, sicne other projects have been added to the workspace in front of the project where the parsing stopped upon.

First description of the workspace.
- one parser per workspace
- it contains 477 projects
- the log shows the pairs of AddProjectToParser/OnParserEnd, it show 76 pairs and then it ends with AddProjectToParser on the problematic project "FOO"

One thing to note about project FOO : it is a library project with only headers, so when building this project there's nothing to do.

Made this project FOO the active project in the workspace, that means it would become the first project to be parsed, the one which triggers the CreaterParser.
This does make a difference !!!

It is parsed and more projects (starting from the first one in the workspace are being added/parsed).
The results are better, but still extremely bad. Now 228 projects made it through the parsing step. Again it halts on project "BAR".
Special thing about this project "BAR" : it is a library project with only headers.

For further tests I made a workspace with 4 projects, one of them being "FOO" (it was the second one, and the first project being the active), in this case all did get parsed ...
EDIT : an 11 projects workspace containing FOO and BAR, all parsed ...
« Last Edit: January 25, 2014, 11:23:05 pm by killerbot »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5295
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #3 on: January 26, 2014, 02:49:50 am »
Is it happens when there are some editor opened when you load the workspace?
What happens if you have *NO* editors opened when you load the workspace?
Please test and report back, I suspect that the opened editor cause this issue.


Also, when in one parser for whole workspace mode, the project files will be parsed one another. But the next project will only be triggered if the previous project parsing send an "OnParserEnd" event.

Another guess is that if one Foo project contains some files, but those files were already be parsed before, then in-fact, there is no actual Parserthread tasks assigned to the threadedtask-pool, then, there will be no parser-end event,  because the parser-end event were initially sent from the threaded task pool that all tasks were done.

EDIT: If possible, can you post the debug log messages? Thanks.
« Last Edit: January 26, 2014, 03:07:55 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5295
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #4 on: January 26, 2014, 03:16:35 am »
@Morten

I see a comment by you in the source code:

Code: [Select]
void NativeParser::OnParsingOneByOneTimer(cb_unused wxTimerEvent& event)
{
    TRACE(_T("NativeParser::OnParsingOneByOneTimer(): Enter"));

    std::pair<cbProject*, ParserBase*> info = GetParserInfoByCurrentEditor();
    if (m_ParserPerWorkspace)
    {
        // If there is no parser and an active editor file can be obtained, parse the file according the active project
        if (!info.second && Manager::Get()->GetEditorManager()->GetBuiltinActiveEditor())
        {
            // NOTE (Morten#1#): Shouldn't this actually be a temp parser??? I think this screws things with re-opening files on load of a projects...
            AddProjectToParser(info.first);
            CCLogger::Get()->DebugLog(_T("NativeParser::OnParsingOneByOneTimer(): Add foreign active editor to current active project's parser."));
        }
        // Otherwise, there is a parser already present
        else
        {
            // First: try to parse the active project (if any)
            cbProject* activeProject = Manager::Get()->GetProjectManager()->GetActiveProject();
            if (m_ParsedProjects.find(activeProject) == m_ParsedProjects.end())
            {
                AddProjectToParser(activeProject);
                CCLogger::Get()->DebugLog(_T("NativeParser::OnParsingOneByOneTimer(): Add new (un-parsed) active project to parser."));
            }
            // Else: add remaining projects one-by-one (if any)
            else
            {
                ProjectsArray* projs = Manager::Get()->GetProjectManager()->GetProjects();
                for (size_t i = 0; i < projs->GetCount(); ++i)
                {
                    // Only add, if the project is not already parsed
                    if (m_ParsedProjects.find(projs->Item(i)) == m_ParsedProjects.end())
                    {
                        AddProjectToParser(projs->Item(i));
                        CCLogger::Get()->DebugLog(_T("NativeParser::OnParsingOneByOneTimer(): Add additional (next) project to parser."));
                        break;
                    }
                }
            }
        }
    }
    else if (info.first && !info.second)
    {
        info.second = CreateParser(info.first);
        if (info.second && info.second != m_Parser)
        {
            CCLogger::Get()->DebugLog(_T("NativeParser::OnParsingOneByOneTimer(): Start switch from OnParsingOneByOneTimer"));
            SwitchParser(info.first, info.second); // Calls SetParser() which also calls UpdateClassBrowserView()
        }
    }
    TRACE(_T("NativeParser::OnParsingOneByOneTimer(): Leave"));
}


Here
Code: [Select]
// NOTE (Morten#1#): Shouldn't this actually be a temp parser??? I think this screws things with re-opening files on load of a projects...
I agree on this, why do we need to check the editor on the OnParsingOneByOneTimer event? This is basically to parse the project sequentially in one parser for whole workspace mode.

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5267
Re: CC upper limits ? BUT for sure BUGs
« Reply #5 on: January 26, 2014, 09:48:36 am »
during the entire tests, no editor was open.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5267
Re: CC upper limits ? BUT for sure BUGs
« Reply #6 on: January 26, 2014, 10:02:59 am »
new test : I create a "DUMMY" project, which cpp file was including the headers that the "FOO" library has, meaning every file of project "FOO" (remember header only) CAN/HAS been seen by the parser.

When DUMMY is after FOO : all ok.
When DUMMY is before FOO, the above mentioned problem pops up, and all parsing stops, the OnParserEnd for "FOO" never happens.

Code: [Select]
NativeParser::AddProjectToParser(): Add project (Dummy) to parser
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (FOO) to parser

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5295
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #7 on: January 26, 2014, 10:46:58 am »
new test : I create a "DUMMY" project, which cpp file was including the headers that the "FOO" library has, meaning every file of project "FOO" (remember header only) CAN/HAS been seen by the parser.

When DUMMY is after FOO : all ok.
When DUMMY is before FOO, the above mentioned problem pops up, and all parsing stops, the OnParserEnd for "FOO" never happens.

Code: [Select]
NativeParser::AddProjectToParser(): Add project (Dummy) to parser
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (FOO) to parser
Ok, so it looks like my guess was correct.
Quote
Another guess is that if one Foo project contains some files, but those files were already be parsed before, then in-fact, there is no actual Parserthread tasks assigned to the threadedtask-pool, then, there will be no parser-end event,  because the parser-end event were initially sent from the threaded task pool that all tasks were done.

I will see I can take some time to fix this issue.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5295
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #8 on: January 26, 2014, 04:20:59 pm »
But I can't reproduce this bug here, see my testing workspace with three projects (in attachment).
Can you give me some sample workspace?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: CC upper limits ? BUT for sure BUGs
« Reply #9 on: January 26, 2014, 05:00:43 pm »
The errors appears on my system.
Here are the logs if "a" is the active project:
"Code::Blocks"-tab
Code: [Select]
Opening /home/jens/Downloads/test/a.cbp
Fertig.
Opening /home/jens/Downloads/test/b.cbp
Fertig.
Opening /home/jens/Downloads/test/c.cbp
Fertig.
NativeParser::CreateParser(): Finish creating a new parser for project 'a'
NativeParser::OnParserEnd(): Project 'a' parsing stage done!
NativeParser::AddProjectToParser(): Add project (b) to parser

"Code::Blocks Debug"-tab
Code: [Select]
Loading workspace "/home/jens/Downloads/test/abc.workspace"
Loading project file...
Parsing project file...
Loading target Debug
Loading target Release
Loading project files...
4 files loaded
Done loading project in 1ms
Project's base path: /home/jens/Downloads/test/
Project's common toplevel path: /home/jens/Downloads/test/
Loading project file...
Parsing project file...
Loading target Debug
Loading target Release
Loading project files...
2 files loaded
Done loading project in 1ms
Project's base path: /home/jens/Downloads/test/
Project's common toplevel path: /home/jens/Downloads/test/
Loading project file...
Parsing project file...
Loading target Debug
Loading target Release
Loading project files...
2 files loaded
Done loading project in 1ms
Project's base path: /home/jens/Downloads/test/
Project's common toplevel path: /home/jens/Downloads/test/
Workspace layout file doesn't exist "/home/jens/Downloads/test/abc.workspace.layout"
NativeParser::GetGCCCompilerDirs(): Caching GCC default include dir: /usr/include/c++/4.8.2
NativeParser::GetGCCCompilerDirs(): Caching GCC default include dir: /usr/include/c++/4.8.2/x86_64-redhat-linux
NativeParser::GetGCCCompilerDirs(): Caching GCC default include dir: /usr/include/c++/4.8.2/backward
NativeParser::GetGCCCompilerDirs(): Caching GCC default include dir: /usr/lib/gcc/x86_64-redhat-linux/4.8.2/include
NativeParser::GetGCCCompilerDirs(): Caching GCC default include dir: /usr/local/include
NativeParser::GetGCCCompilerDirs(): Caching GCC default include dir: /usr/include
NativeParser::DoFullParsing(): AddCompilerPredefinedMacros failed!
NativeParser::DoFullParsing(): Adding three kind of files to batch-parser
NativeParser::DoFullParsing(): Add priority header file: '/usr/include/c++/4.8.2/cstddef'
NativeParser::DoFullParsing(): Add priority header file: '/usr/include/boost/config.hpp'
NativeParser::DoFullParsing(): Add priority header file: '/usr/include/boost/filesystem/config.hpp'
NativeParser::DoFullParsing(): Add 3 priority file(s) for project 'a'...
NativeParser::DoFullParsing(): Added 4 header&source file(s) for project 'a' to batch-parser...
NativeParser::CreateParser(): Finish creating a new parser for project 'a'
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::OnParserStart(): Starting batch parsing for project 'a'...
NativeParser::OnParserEnd(): Project 'a' parsing stage done!
Project 'a' parsing stage done (19 total parsed files, 478 tokens in 0 minute(s), 0.047 seconds).
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
SystemHeadersThread: /usr/include/c++/4.8.2/ , 602
SystemHeadersThread: /usr/include/c++/4.8.2/x86_64-redhat-linux/ , 46
SystemHeadersThread: /usr/include/c++/4.8.2/backward/ , 8
SystemHeadersThread: /usr/lib/gcc/x86_64-redhat-linux/4.8.2/include/ , 52
SystemHeadersThread: /usr/local/include/ , 0
SystemHeadersThread: /usr/include/ , 16270
SystemHeadersThread: Total number of paths: 6
NativeParser::AddProjectToParser(): Add project (b) to parser
NativeParser::AddProjectToParser(): AddCompilerPredefinedMacros failed!
NativeParser::AddProjectToParser(): Done adding 0 files of project (b) to parser.
NativeParser::OnParsingOneByOneTimer(): Add additional (next) project to parser.
Saving workspace layout "/home/jens/Downloads/test/abc.workspace.layout"


and with "b" as active project:

"Code::Blocks"-tab
Code: [Select]
Opening /home/jens/Downloads/test/a.cbp
Fertig.
Opening /home/jens/Downloads/test/b.cbp
Fertig.
Opening /home/jens/Downloads/test/c.cbp
Fertig.
NativeParser::CreateParser(): Finish creating a new parser for project 'b'
NativeParser::OnParserEnd(): Project 'b' parsing stage done!
NativeParser::AddProjectToParser(): Add project (a) to parser
NativeParser::OnParserEnd(): Project 'a' parsing stage done!
NativeParser::AddProjectToParser(): Add project (c) to parser
NativeParser::OnParserEnd(): Project 'c' parsing stage done!


"Code::Blocks Debug"-tab
Code: [Select]
Loading workspace "/home/jens/Downloads/test/abc.workspace"
Loading project file...
Parsing project file...
Loading target Debug
Loading target Release
Loading project files...
4 files loaded
Done loading project in 1ms
Project's base path: /home/jens/Downloads/test/
Project's common toplevel path: /home/jens/Downloads/test/
Loading project file...
Parsing project file...
Loading target Debug
Loading target Release
Loading project files...
2 files loaded
Done loading project in 1ms
Project's base path: /home/jens/Downloads/test/
Project's common toplevel path: /home/jens/Downloads/test/
Loading project file...
Parsing project file...
Loading target Debug
Loading target Release
Loading project files...
2 files loaded
Done loading project in 1ms
Project's base path: /home/jens/Downloads/test/
Project's common toplevel path: /home/jens/Downloads/test/
Loading workspace layout "/home/jens/Downloads/test/abc.workspace.layout"
Project /home/jens/Downloads/test/b.cbp has been activated.
NativeParser::DoFullParsing(): AddCompilerPredefinedMacros failed!
NativeParser::DoFullParsing(): Adding three kind of files to batch-parser
NativeParser::DoFullParsing(): Add priority header file: '/usr/include/c++/4.8.2/cstddef'
NativeParser::DoFullParsing(): Add priority header file: '/usr/include/boost/config.hpp'
NativeParser::DoFullParsing(): Add priority header file: '/usr/include/boost/filesystem/config.hpp'
NativeParser::DoFullParsing(): Add 3 priority file(s) for project 'b'...
NativeParser::DoFullParsing(): Added 2 header&source file(s) for project 'b' to batch-parser...
NativeParser::CreateParser(): Finish creating a new parser for project 'b'
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::OnParserStart(): Starting batch parsing for project 'b'...
NativeParser::OnParserEnd(): Project 'b' parsing stage done!
Project 'b' parsing stage done (17 total parsed files, 476 tokens in 0 minute(s), 0.046 seconds).
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::AddProjectToParser(): Add project (a) to parser
NativeParser::AddProjectToParser(): AddCompilerPredefinedMacros failed!
NativeParser::AddProjectToParser(): Done adding 2 files of project (a) to parser.
NativeParser::OnParsingOneByOneTimer(): Add additional (next) project to parser.
NativeParser::OnParserStart(): Starting batch parsing for project 'a'...
NativeParser::OnParserEnd(): Project 'a' parsing stage done!
Project 'a' parsing stage done (19 total parsed files, 478 tokens in 0 minute(s), 0.011 seconds).
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::AddProjectToParser(): Add project (c) to parser
NativeParser::AddProjectToParser(): AddCompilerPredefinedMacros failed!
NativeParser::AddProjectToParser(): Done adding 2 files of project (c) to parser.
NativeParser::OnParsingOneByOneTimer(): Add additional (next) project to parser.
NativeParser::OnParserStart(): Starting batch parsing for project 'c'...
NativeParser::OnParserEnd(): Project 'c' parsing stage done!
Project 'c' parsing stage done (21 total parsed files, 480 tokens in 0 minute(s), 0.012 seconds).
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5267
Re: CC upper limits ? BUT for sure BUGs
« Reply #10 on: January 26, 2014, 09:05:11 pm »
Attached archive, contains a workspace containing 3 projects :

- Dummy
- NamingService  (the header only, nearly no more code in it, and where parsing stops upon)
- Dummy2

Here's the log :

Code: [Select]
Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy/Project/Dummy.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/NamingService/Project/NamingService.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy2/Project/Dummy2.cbp
Done.
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser


Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5295
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #11 on: January 27, 2014, 02:43:47 am »
Hi, killerbot and Jens, thanks.

I just tried the sample workspace supplied by killerbot, but I still couldn't reproduce this bug. I'm using the nightly build 9605 on WinXP.

See my log when open this workspace.
C::B log:
Quote
Opening E:\cc-bug\libraries\common\Dummy\Project\Dummy.cbp
Done.
Opening E:\cc-bug\libraries\common\NamingService\Project\NamingService.cbp
Done.
Opening E:\cc-bug\libraries\common\Dummy2\Project\Dummy2.cbp
Done.
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParserEnd(): Project 'NamingService' parsing stage done!
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!


C::B debug log:
Quote
Loading workspace "E:\cc-bug\test.workspace"
Loading project file...
Parsing project file...
Loading target GnuDebug
Loading target GnuRelease
Loading target powerpc-linux-gnu-ltibDebug
Loading target powerpc-linux-gnu-ltibRelease
Loading target ClangDebug
Loading target ClangRelease
Loading project files...
2 files loaded
Done loading project in 0ms
Project's base path: E:\cc-bug\libraries\common\Dummy\Project\
Project's common toplevel path: E:\cc-bug\libraries\common\Dummy\
Loading project file...
Parsing project file...
Loading target GnuDebug
Loading target GnuRelease
Loading target powerpc-linux-gnu-ltibDebug
Loading target powerpc-linux-gnu-ltibRelease
Loading project files...
1 files loaded
Done loading project in 0ms
Project's base path: E:\cc-bug\libraries\common\NamingService\Project\
Project's common toplevel path: E:\cc-bug\libraries\common\NamingService\
Loading project file...
Parsing project file...
Loading target GnuDebug
Loading target GnuRelease
Loading target powerpc-linux-gnu-ltibDebug
Loading target powerpc-linux-gnu-ltibRelease
Loading target ClangDebug
Loading target ClangRelease
Loading project files...
2 files loaded
Done loading project in 16ms
Project's base path: E:\cc-bug\libraries\common\Dummy2\Project\
Project's common toplevel path: E:\cc-bug\libraries\common\Dummy2\
Workspace layout file doesn't exist "E:\cc-bug\test.workspace.layout"
EnvVars: Obtained 'default' as active envvar set from config.
EnvVars: Set 'default' will not be applied (already active).
NativeParser::DoFullParsing(): Adding three kind of files to batch-parser
NativeParser::DoFullParsing(): Add priority header file: 'D:\mingw-builds\473\mingw32\lib\gcc\i686-w64-mingw32\4.7.3\include\c++\cstddef'
NativeParser::DoFullParsing(): Add priority header file: 'D:\mingw-builds\473\mingw32\i686-w64-mingw32\include\w32api.h'
NativeParser::DoFullParsing(): Add 2 priority file(s) for project 'Dummy'...
NativeParser::DoFullParsing(): Added 2 header&source file(s) for project 'Dummy' to batch-parser...
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::OnParserStart(): Starting batch parsing for project 'Dummy'...
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
Project 'Dummy' parsing stage done (75 total parsed files, 3462 tokens in 0 minute(s), 0.578 seconds).
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::AddProjectToParser(): Done adding 0 files of project (NamingService) to parser.
NativeParser::OnParsingOneByOneTimer(): Add additional (next) project to parser.
NativeParser::OnParserStart(): Starting batch parsing for project 'NamingService'...
NativeParser::OnParserEnd(): Project 'NamingService' parsing stage done!
Project 'NamingService' parsing stage done (75 total parsed files, 3462 tokens in 0 minute(s), 0.000 seconds).
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::AddProjectToParser(): Done adding 2 files of project (Dummy2) to parser.
NativeParser::OnParsingOneByOneTimer(): Add additional (next) project to parser.
NativeParser::OnParserStart(): Starting batch parsing for project 'Dummy2'...
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!
Project 'Dummy2' parsing stage done (77 total parsed files, 3465 tokens in 0 minute(s), 0.062 seconds).
ClassBrowser::OnThreadEvent(): Updating class browser...
ClassBrowser::OnThreadEvent(): Class browser updated.


So, I guess this is the bug happens under Linux?

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5267
Re: CC upper limits ? BUT for sure BUGs
« Reply #12 on: January 27, 2014, 08:46:20 am »
yes I am on linux.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5267
Re: CC upper limits ? BUT for sure BUGs
« Reply #13 on: January 27, 2014, 08:56:03 am »
tried in on my win virtual box (winxp) :


Code: [Select]
Opening C:\Projects\cc-bug\libraries\common\Dummy\Project\Dummy.cbp
Done.
Opening C:\Projects\cc-bug\libraries\common\NamingService\Project\NamingService.cbp
Done.
Opening C:\Projects\cc-bug\libraries\common\Dummy2\Project\Dummy2.cbp
Done.
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParserEnd(): Project 'NamingService' parsing stage done!
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

So seems it is a linux only issue.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5295
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #14 on: January 27, 2014, 09:57:07 am »
tried in on my win virtual box (winxp) :


Code: [Select]
Opening C:\Projects\cc-bug\libraries\common\Dummy\Project\Dummy.cbp
Done.
Opening C:\Projects\cc-bug\libraries\common\NamingService\Project\NamingService.cbp
Done.
Opening C:\Projects\cc-bug\libraries\common\Dummy2\Project\Dummy2.cbp
Done.
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParserEnd(): Project 'NamingService' parsing stage done!
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

So seems it is a linux only issue.
This is quite strange. Can you try the CC patch below to see whether it works OK under Linux? Thanks.

Code: [Select]
c423a5611f77ecc95925e3fb7d75042cbcbfb83b
 src/plugins/codecompletion/nativeparser.cpp | 23 +++++++++++++++++------
 src/plugins/codecompletion/nativeparser.h   |  4 ++--
 2 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/src/plugins/codecompletion/nativeparser.cpp b/src/plugins/codecompletion/nativeparser.cpp
index c3d659e..cded451 100644
--- a/src/plugins/codecompletion/nativeparser.cpp
+++ b/src/plugins/codecompletion/nativeparser.cpp
@@ -2643,12 +2643,18 @@ void NativeParser::OnParsingOneByOneTimer(cb_unused wxTimerEvent& event)
             else
             {
                 ProjectsArray* projs = Manager::Get()->GetProjectManager()->GetProjects();
+                // loop on the whole workspace, and only add a new project to the parser
+                // here the "new" means a project haven't been parsed. Once it was parsed, it is
+                // added to the m_ParsedProjects
                 for (size_t i = 0; i < projs->GetCount(); ++i)
                 {
                     // Only add, if the project is not already parsed
                     if (m_ParsedProjects.find(projs->Item(i)) == m_ParsedProjects.end())
                     {
-                        AddProjectToParser(projs->Item(i));
+                        // if total files of current project were already parsed, we should try the next project
+                        int actualAddFileNumber = AddProjectToParser(projs->Item(i));
+                        if (!actualAddFileNumber)
+                            continue;
                         CCLogger::Get()->DebugLog(_T("NativeParser::OnParsingOneByOneTimer(): Add additional (next) project to parser."));
                         break;
                     }
@@ -2763,24 +2769,24 @@ void NativeParser::InitCCSearchVariables()
     Reset();
 }
 
-void NativeParser::AddProjectToParser(cbProject* project)
+int NativeParser::AddProjectToParser(cbProject* project)
 {
     wxString prj = (project ? project->GetTitle() : _T("*NONE*"));
     ParserBase* parser = GetParserByProject(project);
     if (parser)
-        return;
+        return 0;
 
     if (m_ParsedProjects.empty())
-        return;
+        return 0;
 
     m_ParsedProjects.insert(project);
     parser = GetParserByProject(project);
     if (!parser)
-        return;
+        return 0;
     else if (!parser->UpdateParsingProject(project))
     {
         m_ParsedProjects.erase(project);
-        return;
+        return 0;
     }
 
     // TODO (ollydbg#1#) did exactly the same thing as the function NativeParser::DoFullParsing()?
@@ -2820,6 +2826,9 @@ void NativeParser::AddProjectToParser(cbProject* project)
         }
 
         CCLogger::Get()->DebugLog(F(_("NativeParser::AddProjectToParser(): Done adding %lu files of project (%s) to parser."), static_cast<unsigned long>(fileCount), prj.wx_str()));
+
+        // in some cases, all the files were already be parsed, so fileCount is still 0
+        return fileCount;
     }
     else
     {
@@ -2831,8 +2840,10 @@ void NativeParser::AddProjectToParser(cbProject* project)
             m_StandaloneFiles.Add(editor->GetFilename());
 
             CCLogger::Get()->DebugLog(F(_("NativeParser::AddProjectToParser(): Done adding stand-alone file (%s) of editor to parser."), editor->GetFilename().wx_str()));
+            return 1;
         }
     }
+    return 0;
 }
 
 bool NativeParser::RemoveProjectFromParser(cbProject* project)
diff --git a/src/plugins/codecompletion/nativeparser.h b/src/plugins/codecompletion/nativeparser.h
index 333166e..48b65a4 100644
--- a/src/plugins/codecompletion/nativeparser.h
+++ b/src/plugins/codecompletion/nativeparser.h
@@ -416,8 +416,8 @@ private:
     /** Init cc search member variables */
     void InitCCSearchVariables();
 
-    /** Add all project files to parser */
-    void AddProjectToParser(cbProject* project);
+    /** Add all project's files to parser */
+    int AddProjectToParser(cbProject* project);
 
     /** Remove cbp from the common parser, this only happens in one parser for whole workspace mode
      * when a parser is removed from the workspace, we should remove the project from the parser


The description of the patch is:
Quote
* CC: Fix a bug in the one parser for the whole workspace mode, if a project's file was already parsed by previous projects, then this project don't actually run a parsing tasks (for parsing source files), thus there is not ParserEnd event happened.

The general logic are below:

thread pool done-> send event to Parser object
Parser::OnAllThreadsDone(CodeBlocksEvent& event) -> send a Parser End event
NativeParser::OnParserEnd -> m_TimerParsingOneByOne.Start(500, wxTIMER_ONE_SHOT);
NativeParser::OnParsingOneByOneTimer -> AddProjectToParser(projs->Item(i)); (adding the next project)

But if AddProjectToParser() add nothing to the Task pool, there will be no "thread pool done" again, so all the remaining projects has no chance to be done.

So, I simply let the AddProjectToParser() function return actually added file numbers, if it is zero, we try to parse the next project.

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.