Author Topic: Solve the header file parsing problem  (Read 12190 times)

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Solve the header file parsing problem
« on: May 11, 2010, 10:41:55 am »
Hi, all. After several days hard work, I have finally release the patch to solve the header file parsing problem.

Problem description:

At present, for example, the parser try to parse a main.cpp, and there are two "include directive".

main.cpp
Code
#include <windows.h>
#include <exception>
......

When the parsing stage analyze these statement, it will create two Parserthread objects( One attach to the "windows.h" and one attached to "exception.h".) Then put the two objects into the cbThreadPool. Finally, It is the Thread Pool's duty do schedule the running of each Parserthread object. As we know, there are many included header inside the "windows.h" file, such as "WinBash.h", so,  more Parserthread objects will be created and added to the ThreadPool.

Basically, There's no priority about the sequence about which header file should be parsed first, and which header file should parsed secondly.

So, you can imagine:
The Parserthread obj attached to the header file "exception" maybe runs earlier than the one attach with "WinBash.h". (As a general guess, the ThreadPool is a FIFO structure, so the one first added to ThreadPool should run firstly).

This may cause some problems when I have already implement the "handling of conditional preprocessor". (If we don't care about the conditional proprocessor handling and always choose the #if branch, this is not a big problem ).

For example, in the exception header file, there are some code snippet:

Code
#if _HAS_EXCEPTIONS

 _STD_BEGIN

class _CRTIMP_PURE exception
    {    // base of all library exceptions
...

_STD_END

 #else /* _HAS_EXCEPTIONS */

_STDEXT_BEGIN
class exception
    {    // base of all library exceptions
public:

...

_STDEXT_END

_STD_BEGIN

 #endif /* _HAS_EXCEPTIONS */


These code analysis should mainly depend on whether the macro "_HAS_EXCEPTIONS" is defined. Suppose this Macro is defined in "WinBash.h", but as I said before, the parsing of "exception" is earlier than paring of "WinBash.h".

The result is:
Parsing "exception" file is not correctly as we expected.
As a conclusion, the currently CC parsing is all in parallel mode.

My solution:

I add a Edit control in the CC's configure dialog, named "Files to Parse up-front", which means all these files in this should be parsed firstly when a batch parsing started. Also, the files will be parsed in a sequence mode (opposite to the parallel mode), so that the maximal file dependence were reserved.

Test result: parsing codeblocks.cbp

1, Applying the patches of "conditional preprocessor handling" and "strip comment in #defines"
http://forums.codeblocks.org/index.php/topic,12388.0.html
http://forums.codeblocks.org/index.php/topic,12499.0.html
result:
Parsing stage done (1264 total parsed files, 54064 tokens in 0 minute(s), 8.313 seconds).
Parsing stage done (1264 total parsed files, 54064 tokens in 0 minute(s), 8.312 seconds).

2, additionally add this patch to solve header file dependency:
Parsing stage done (1277 total parsed files, 63181 tokens in 0 minute(s), 8.719 seconds).
Parsing stage done (1277 total parsed files, 63181 tokens in 0 minute(s), 8.672 seconds).

It seems we have 10000 tokens increase when applied this patch, but it is without performance loss.

Welcome to test, and I would like to hear your comment!!!

Thanks ollydbg.

[attachment deleted by admin]
« Last Edit: May 12, 2010, 04:04:12 pm by Loaden »