Author Topic: CC upper limits ? BUT for sure BUGs  (Read 37976 times)

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: CC upper limits ? BUT for sure BUGs
« Reply #15 on: January 27, 2014, 08:45:45 pm »
This seems to work.

Code
Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy/Project/Dummy.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/NamingService/Project/NamingService.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy2/Project/Dummy2.cbp
Done.
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

also the big workspace I had completed till the end.

Note : OnParserEnd() is printed for these header only projects, and the same for a project which is based upon a makefile. But stuff continued till the end, the projects I had which are based on  makefile are at the end, but all of them passed by, so I guess these are also ok.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: CC upper limits ? BUT for sure BUGs
« Reply #16 on: January 27, 2014, 08:47:08 pm »
question :" can CC already deal with the C++11 syntax :

using Foo = int;


aka the new typedef style.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #17 on: January 28, 2014, 03:21:42 am »
This seems to work.

Code
Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy/Project/Dummy.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/NamingService/Project/NamingService.cbp
Done.
Opening /home/killerbot/Projects/CC-bug/libraries/common/Dummy2/Project/Dummy2.cbp
Done.
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

also the big workspace I had completed till the end.
Good result!

Quote
Note : OnParserEnd() is printed for these header only projects, and the same for a project which is based upon a makefile.
Do you mean "is not printed"?

Quote
But stuff continued till the end, the projects I had which are based on  makefile are at the end, but all of them passed by, so I guess these are also ok.
I will commit the patch if no objections.

question :" can CC already deal with the C++11 syntax :

using Foo = int;


aka the new typedef style.
No, not sure how easy we can add such feature.
C++11 has more and more features that our CC parser doesn't support, currently, I think only Clang can help us.

BTW: I have read some posts in GCC maillists and Emacs maillist recently, sounds like RMS does not like Clang, since it fosters many proprietary tools. I personally hope GCC can have such feature, but it's not. :(

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: CC upper limits ? BUT for sure BUGs
« Reply #18 on: January 28, 2014, 07:43:52 am »
indeed : "NOT" printed, sorry for the confusion.

Commit welcome.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #19 on: January 28, 2014, 10:51:34 am »
indeed : "NOT" printed, sorry for the confusion.

Commit welcome.
I try to find the reason why this is a Linux only bug, with the new patch, I would like to see what's the log of your test workspace.
This is mine on Windows.
Code
Opening E:\cc-bug\libraries\common\Dummy\Project\Dummy.cbp
Done.
Opening E:\cc-bug\libraries\common\NamingService\Project\NamingService.cbp
Done.
Opening E:\cc-bug\libraries\common\Dummy2\Project\Dummy2.cbp
Done.

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
NativeParser::OnParserEnd(): Project 'NamingService' parsing stage done!
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

EDIT: patch deleted, because later posts contain new patches.
« Last Edit: January 31, 2014, 04:14:37 pm by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: CC upper limits ? BUT for sure BUGs
« Reply #20 on: January 28, 2014, 01:33:01 pm »
will try to test this evening,  wrt previous path, just one change , right, log before the continue.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #21 on: January 28, 2014, 02:15:06 pm »
wrt previous path, just one change , right, log before the continue.
yes.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: CC upper limits ? BUT for sure BUGs
« Reply #22 on: January 28, 2014, 10:58:24 pm »
Code
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #23 on: January 29, 2014, 03:48:02 pm »
Code
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!
Thanks, now, I need to understand why in Windows, there is two message:
Code
NativeParser::OnParsingOneByOneTimer(): all files in this project were already parsed, try next project.
But it only has one on Linux.

I debugged the code a little, found that my patch was not quite logically OK. By testing your sample workspace, I noticed that the second cbp add no source files, but I see it did add some macro definitions (from the GCC -E ... command), so adding the third project in that loop(see my patch) is not a good idea, because it will cause the third cbp added, and removed again, then added again....

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #24 on: January 29, 2014, 04:04:50 pm »
Code
    if (   !m_PriorityHeaders.empty()
        || !m_BatchParseFiles.empty()
        || !m_PredefinedMacros.IsEmpty() )
    {
        CC_LOCKER_TRACK_P_MTX_LOCK(ParserCommon::s_ParserMutex)

        ParserThreadedTask* thread = new ParserThreadedTask(this, ParserCommon::s_ParserMutex);
        TRACE(_T("Parser::OnBatchTimer(): Adding a ParserThreadedTask thread to m_Pool."));
        m_Pool.AddTask(thread, true); //once this function is called, the thread will be executed from the pool.
If m_PredefinedMacros is not empty, there is still one ParserThreadedTask put in the m_Pool,  that's the reason the bug doesn't happened under Windows(without my patch). So, there comes a guess, if under Linux, C::B svn code (without my patch), I guess that the cbp is not a standard project(such as a makefile project) Or in some wired reason ???, then m_PredefinedMacros is empty, so no thread task will be put in the pool, so the bug happens because the m_Pool have not tasks to run.
« Last Edit: January 29, 2014, 04:17:31 pm by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #25 on: January 29, 2014, 04:33:33 pm »
...if under Linux, C::B svn code (without my patch), I guess that the cbp is not a standard project(such as a makefile project) Or in some wired reason ???, then m_PredefinedMacros is empty, so no thread task will be put in the pool, so the bug happens because the m_Pool have not tasks to run.

Ok, my guess is right, refer to  Jen's reply
In the first test case, I did see in the debug log:

Code
...
NativeParser::DoFullParsing(): AddCompilerPredefinedMacros failed!
...
NativeParser::AddProjectToParser(): AddCompilerPredefinedMacros failed!
NativeParser::AddProjectToParser(): Done adding 0 files of project (b) to parser.

Look at the log, when parsing project(a), the variable m_PredefinedMacros is empty, but m_BatchParseFiles is not empty, m_PriorityHeaders is not empty.
But when parsing project(b), I see that m_PredefinedMacros and m_BatchParseFiles are both empty. I don't know m_PriorityHeaders, I think it is also empty. So, that's the bug, every thing is empty, so the m_Pool is free now.

Compare to my test under Windows, debug log, I think the m_PredefinedMacros is not empty (I don't receive "NativeParser::AddProjectToParser()/DoFullParsing(): AddCompilerPredefinedMacros failed!" error message) when parsing the second project, so at least on thread task is put in the m_Pool.

Now, one question: why fetch compilerPredefinedMacros get failed in Linux?





If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #26 on: January 30, 2014, 03:42:30 am »
This is the new patch, I think the bug is correctly fixed.
Please apply the patch, and give the logs.
Under Windows, I can correctly get the compiler predefined macros, so I get the following log (after applying the patch)
Code
Opening E:\cc-bug\libraries\common\Dummy\Project\Dummy.cbp
Done.
Opening E:\cc-bug\libraries\common\NamingService\Project\NamingService.cbp
Done.
Opening E:\cc-bug\libraries\common\Dummy2\Project\Dummy2.cbp
Done.

NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParserEnd(): Project 'NamingService' parsing stage done!
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

I guess under Linux (you failed to fetch the compiler predefined macros), so you should get something like:
Quote
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser
NativeParser::OnParsingOneByOneTimer(): nothing need to parse in this project, try next project.
NativeParser::AddProjectToParser(): Add project (Dummy2) to parser
NativeParser::OnParserEnd(): Project 'Dummy2' parsing stage done!

Thanks.

EDIT: patch deleted, because later posts contain new patches.
« Last Edit: January 31, 2014, 04:15:16 pm by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: CC upper limits ? BUT for sure BUGs
« Reply #27 on: January 30, 2014, 10:50:18 pm »
result of testing the new patch :


Code
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser

So it seems it doesn't work anymore

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: CC upper limits ? BUT for sure BUGs
« Reply #28 on: January 31, 2014, 03:09:15 pm »
result of testing the new patch :


Code
NativeParser::CreateParser(): Finish creating a new parser for project 'Dummy'
NativeParser::OnParserEnd(): Project 'Dummy' parsing stage done!
NativeParser::AddProjectToParser(): Add project (NamingService) to parser

So it seems it doesn't work anymore

I have no idea how to fix it now, because I don't have a Linux system to test/debug this patch. I'd welcome any one to help, thanks.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: CC upper limits ? BUT for sure BUGs
« Reply #29 on: January 31, 2014, 03:18:06 pm »
Maybe i can help. Is there a test project? What patch/branch should I look at.