CC upper limits ? BUT for sure BUGs

[killerbot]

This seems to work.

Code

Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy/Project/Dummy.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/NamingService/Project/NamingService.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy2/Project/Dummy2.cbp
Done.
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

also the big workspace I had completed till the end.

Note : OnParserEnd() is printed for these header only projects, and the same for a project which is based upon a makefile. But stuff continued till the end, the projects I had which are based on  makefile are at the end, but all of them passed by, so I guess these are also ok.

[killerbot]

question :" can CC already deal with the C++11 syntax :

using Foo = int;


aka the new typedef style.

[ollydbg]

This seems to work.

Code

Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy/Project/Dummy.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/NamingService/Project/NamingService.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy2/Project/Dummy2.cbp
Done.
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

also the big workspace I had completed till the end.

Good result!

Note : OnParserEnd() is printed for these header only projects, and the same for a project which is based upon a makefile.

Do you mean "is not printed"?

But stuff continued till the end, the projects I had which are based on  makefile are at the end, but all of them passed by, so I guess these are also ok.

I will commit the patch if no objections.

question :" can CC already deal with the C++11 syntax :

using Foo = int;


aka the new typedef style.

No, not sure how easy we can add such feature.
C++11 has more and more features that our CC parser doesn't support, currently, I think only Clang can help us.

BTW: I have read some posts in GCC maillists and Emacs maillist recently, sounds like RMS does not like Clang, since it fosters many proprietary tools. I personally hope GCC can have such feature, but it's not.

[killerbot]

indeed : "NOT" printed, sorry for the confusion.

Commit welcome.

[ollydbg]

indeed : "NOT" printed, sorry for the confusion.

Commit welcome.

I try to find the reason why this is a Linux only bug, with the new patch, I would like to see what's the log of your test workspace.
This is mine on Windows.

Code

Opening E:\cc-bug\libraries\common\Dummy\Project\Dummy.cbp
Done.
Opening E:\cc-bug\libraries\common\NamingService\Project\NamingService.cbp
Done.
Opening E:\cc-bug\libraries\common\Dummy2\Project\Dummy2.cbp
Done.

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
NativeParser::OnParserEnd(): Project 'NamingService' parsing stage done!
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

EDIT: patch deleted, because later posts contain new patches.

[killerbot]

will try to test this evening,  wrt previous path, just one change , right, log before the continue.

[ollydbg]

wrt previous path, just one change , right, log before the continue.

yes.

[killerbot]

Code

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

[ollydbg]

Code

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

Thanks, now, I need to understand why in Windows, there is two message:

Code

NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.

But it only has one on Linux.

I debugged the code a little, found that my patch was not quite logically OK. By testing your sample workspace, I noticed that the second cbp add no source files, but I see it did add some macro definitions (from the GCC -E ... command), so adding the third project in that loop(see my patch) is not a good idea, because it will cause the third cbp added, and removed again, then added again....

[ollydbg]

Code

    if (   !m_PriorityHeaders.empty()
        || !m_BatchParseFiles.empty()
        || !m_PredefinedMacros.IsEmpty() )
    {
        CC_LOCKER_TRACK_P_MTX_LOCK(ParserCommon::s_ParserMutex)

        ParserThreadedTask* thread = new ParserThreadedTask(this, ParserCommon::s_ParserMutex);
        TRACE(_T("Parser::OnBatchTimer(): Adding a ParserThreadedTask thread to m_Pool."));
        m_Pool.AddTask(thread, true); //once this function is called, the thread will be executed from the pool.

If m_PredefinedMacros is not empty, there is still one ParserThreadedTask put in the m_Pool,  that's the reason the bug doesn't happened under Windows(without my patch). So, there comes a guess, if under Linux, C::B svn code (without my patch), I guess that the cbp is not a standard project(such as a makefile project) Or in some wired reason , then m_PredefinedMacros is empty, so no thread task will be put in the pool, so the bug happens because the m_Pool have not tasks to run.

[ollydbg]

...if under Linux, C::B svn code (without my patch), I guess that the cbp is not a standard project(such as a makefile project) Or in some wired reason , then m_PredefinedMacros is empty, so no thread task will be put in the pool, so the bug happens because the m_Pool have not tasks to run.

Ok, my guess is right, refer to  Jen's reply
In the first test case, I did see in the debug log:

Code

...
NativeParser::DoFullParsing(): AddCompilerPredefinedMacros failed!
...
NativeParser::AddProjectToParser(): AddCompilerPredefinedMacros failed!
NativeParser::AddProjectToParser(): Done adding 0 files of project (b) to parser.

Look at the log, when parsing project(a), the variable m_PredefinedMacros is empty, but m_BatchParseFiles is not empty, m_PriorityHeaders is not empty.
But when parsing project(b), I see that m_PredefinedMacros and m_BatchParseFiles are both empty. I don't know m_PriorityHeaders, I think it is also empty. So, that's the bug, every thing is empty, so the m_Pool is free now.

Compare to my test under Windows, debug log, I think the m_PredefinedMacros is not empty (I don't receive "NativeParser::AddProjectToParser()/DoFullParsing(): AddCompilerPredefinedMacros failed!" error message) when parsing the second project, so at least on thread task is put in the m_Pool.

Now, one question: why fetch compilerPredefinedMacros get failed in Linux?

[ollydbg]

This is the new patch, I think the bug is correctly fixed.
Please apply the patch, and give the logs.
Under Windows, I can correctly get the compiler predefined macros, so I get the following log (after applying the patch)

Code

Opening E:\cc-bug\libraries\common\Dummy\Project\Dummy.cbp
Done.
Opening E:\cc-bug\libraries\common\NamingService\Project\NamingService.cbp
Done.
Opening E:\cc-bug\libraries\common\Dummy2\Project\Dummy2.cbp
Done.

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParserEnd(): Project 'NamingService' parsing stage done!
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

I guess under Linux (you failed to fetch the compiler predefined macros), so you should get something like:

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParsingOneByOneTimer(): nothing need to parse in this project, try next project.
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

Thanks.

EDIT: patch deleted, because later posts contain new patches.

[killerbot]

result of testing the new patch :

Code

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser

So it seems it doesn't work anymore

[ollydbg]

result of testing the new patch :

Code

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser

So it seems it doesn't work anymore

I have no idea how to fix it now, because I don't have a Linux system to test/debug this patch. I'd welcome any one to help, thanks.

[dmoore]

Maybe i can help. Is there a test project? What patch/branch should I look at.