Author Topic: ParserTester for codecompletion plugin  (Read 26281 times)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5737
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ParserTester for codecompletion plugin
« Reply #15 on: February 23, 2010, 09:56:59 am »
@morten:
Loaden means to parse the VC header files correctly, We need to add two rules in the wxstring-wxstring replacement map.

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline blueshake

  • Regular
  • ***
  • Posts: 459
Re: ParserTester for codecompletion plugin
« Reply #16 on: February 23, 2010, 12:18:18 pm »
@Loaden

if you want to show pictures,you can use outside repository.then cite the picture addresses here. :D
Keep low and hear the sadness of little dog.
I fall in love with a girl,but I don't dare to tell her.What should I do?

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5737
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ParserTester for codecompletion plugin
« Reply #17 on: February 25, 2010, 06:15:29 am »
In the ParserTest.cpp, if you would like to add the replacement tokens, use these statement:

Code
void Start()
    {
        Parser client(NULL);
        wxString fileName = _T("test.cpp");
        FileLoader* loader = new FileLoader(fileName);
        (*loader)();

        TokensTree* tree = new TokensTree();

        Tokenizer::SetReplacementString(_T("_GLIBCXX_STD"),                    _T("std"));

        Tokenizer::SetReplacementString(_T("_GLIBCXX_BEGIN_NESTED_NAMESPACE"), _T("+namespace"));
        Tokenizer::SetReplacementString(_T("_GLIBCXX_END_NESTED_NAMESPACE"),   _T("}"));

        Tokenizer::SetReplacementString(_T("_GLIBCXX_BEGIN_NAMESPACE"),        _T("+namespace"));
        Tokenizer::SetReplacementString(_T("_GLIBCXX_END_NAMESPACE"),          _T("}"));

        Tokenizer::SetReplacementString(_T("_GLIBCXX_END_NAMESPACE_TR1"),      _T("}"));
        Tokenizer::SetReplacementString(_T("_GLIBCXX_BEGIN_NAMESPACE_TR1"),    _T("-namespace tr1 {"));

        // for VC2005/2008
        Tokenizer::SetReplacementString(_T("_STD_BEGIN"),    _T("-namespace std {"));
        Tokenizer::SetReplacementString(_T("_STD_END"),      _T("}"));


        ParserThreadOptions opts;
        opts.wantPreprocessor = false;
        opts.useBuffer = false;
        opts.bufferSkipBlocks = false;
        opts.bufferSkipOuterBlocks = false;
        opts.followLocalIncludes = false;
        opts.followGlobalIncludes = false;
        opts.loader = loader;

        ParserThread* ph = new ParserThread(&client, fileName, true, opts, tree);
        bool b = ph->Parse();
        delete ph;

        ShowLog();
    }
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9662
Re: ParserTester for codecompletion plugin
« Reply #18 on: February 25, 2010, 07:28:25 am »
In the ParserTest.cpp, if you would like to add the replacement tokens, use these statement:
Why not the others, too? (The ones from the actual plugin.)

Edit: Argh, forget about it: I missed the scrollbar in your post. :lol:
« Last Edit: February 25, 2010, 07:30:15 am by MortenMacFly »
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5737
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ParserTester for codecompletion plugin
« Reply #19 on: February 25, 2010, 07:45:02 am »
Edit: Argh, forget about it: I missed the scrollbar in your post. :lol:
:D, these two replacement rules should be added to the standard CC plug-in too.

By the way, Loaden suggests that we could have different replacement string set for different compiler. :D
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9662
Re: ParserTester for codecompletion plugin
« Reply #20 on: February 26, 2010, 06:07:49 pm »
By the way, Loaden suggests that we could have different replacement string set for different compiler. :D
I wouldn't do that by the compiler. The reasons are:

1.) CC needs to "talk" to the compiler which may not be present. CC should also work without the compiler (the compiler is another plugin in the end).
2.) You can use e.g. Visual C++ SDK's with GCC just fine (like Direct/X for example). So the replacements do in fact not depend on the compiler as you see.

Better proposal: Try to "identify" commonly used SDK's (like base SDK's) upon typical identifiers in the header files For example: If you encounter _GLIBCXX_STD switch to the right set of replacements tokens and re-parse. If you encounter _STD_BEGIN switch (or better append) this set.

Notice that you can also use GCC / VC libraries / header files mixed. So a combination os replacements tokens may make sense.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5737
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ParserTester for codecompletion plugin
« Reply #21 on: February 27, 2010, 04:37:04 am »
Notice that you can also use GCC / VC libraries / header files mixed. So a combination os replacements tokens may make sense.
Ok, in this situation, both replacement string for GCC and VC should be used.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5737
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ParserTester for codecompletion plugin
« Reply #22 on: February 27, 2010, 04:50:16 am »
Here is a improved parserTester version 2.
A lot of GUI improved by Loaden. Added a "find" non-model dialog.
Also, we have moved the project sources to a separate folder instead of the "/plugins/codecompletion/parser"

So, you need to download the 7z package, and unzip it. it should have a "ptest" folder, you need to paste this folder (include all the files) to "codecompletion/parser", so, now, all the source files were in "codecompletion/parser/ptest".

Also, a you need to apply these patches to let the "parserTester" project compile successfully. ( This patch never hurt the building of standard CC)

In this patch, I have fix this problem: Insert all class method without implementation question

Comments are welcome!!!

By the way, I don't change these code snippet in tokenizer.cpp ,but the TortoiseSVN always thought I was changing that...
Code
#ifdef __WXMSW__ // This is a Windows only bug!
    else if (c == 178 || c == 179 || c == 185) // fetch ² and ³
    {
        str = c;
        MoveToNextChar();
    }
#endif


I'm not sure why.






[attachment deleted by admin]
« Last Edit: February 27, 2010, 04:52:03 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: ParserTester for codecompletion plugin
« Reply #23 on: February 27, 2010, 06:43:22 am »
Sorry, my fault, in "ParserTesterV2.7z" compressed package, the "mainfdsfef.cpp" is unnecessary, and could be deleted.

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: ParserTester for codecompletion plugin
« Reply #24 on: February 27, 2010, 11:45:43 am »
Refactoring code, and add two methods: PrintTree and PrintList.
It should work better.

[attachment deleted by admin]
« Last Edit: February 27, 2010, 11:54:31 am by Loaden »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5737
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ParserTester for codecompletion plugin
« Reply #25 on: February 27, 2010, 01:31:38 pm »
Refactoring code, and add two methods: PrintTree and PrintList.
It should work better.
Great! It works better now! Thanks for your work!
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: ParserTester for codecompletion plugin
« Reply #26 on: March 02, 2010, 03:43:11 pm »
Hi,all, I created a unix project for ptest, and after testing, it works well.
Another change: I changed test.cpp TO test.h, so that it can be added to the project, for easy testing.

[attachment deleted by admin]

Offline code robot

  • Multiple posting newcomer
  • *
  • Posts: 14
Re: ParserTester for codecompletion plugin
« Reply #27 on: March 13, 2010, 09:55:19 am »
THANK YOU!!!

I can now code again using the VC++ compiler :)

Please add this to the main repo a.s.a.p.
(In case anyone else needs this fix right away, I manually copy/pasted the .cpp/.h files from 'parserTesterV1.zip' , into my local SVN checkout folder, and recompiled Code::Blocks)
« Last Edit: March 13, 2010, 09:58:30 am by code robot »

Offline Loaden

  • Lives here!
  • ****
  • Posts: 1014
Re: ParserTester for codecompletion plugin
« Reply #28 on: March 21, 2010, 06:27:08 am »
By the way, I don't change these code snippet in tokenizer.cpp ,but the TortoiseSVN always thought I was changing that...
Code
#ifdef __WXMSW__ // This is a Windows only bug!
    else if (c == 178 || c == 179 || c == 185) // fetch ² and ³
    {
        str = c;
        MoveToNextChar();
    }
#endif


I find the reason: don't use TortoiseSVN , you can use patch in MSYS.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5737
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ParserTester for codecompletion plugin
« Reply #29 on: May 12, 2010, 04:24:50 pm »
This is a pure patch to let the parserthread.cpp and tokenizer.cpp supporting ParserTester project.

Code
Index: parserthread.cpp
===================================================================
--- parserthread.cpp (revision 6218)
+++ parserthread.cpp (working copy)
@@ -21,12 +21,18 @@
 
 #define PARSERTHREAD_DEBUG_OUTPUT 0
 
+#ifdef PARSER_TEST
+  extern void ParserTrace(const wxChar* format, ...);
+  #define TRACE(format, args...)\
+  ParserTrace(format , ## args)
+#else
 #if PARSERTHREAD_DEBUG_OUTPUT
-    #define TRACE(format, args...)\
-    Manager::Get()->GetLogManager()->DebugLog(F( format , ## args))
+   #define TRACE(format, args...)\
+   Manager::Get()->GetLogManager()->DebugLog(F( format , ## args))
 #else
-    #define TRACE(format, args...)
+   #define TRACE(format, args...)
 #endif
+#endif
 
 int THREAD_START       = wxNewId();
 int THREAD_END         = wxNewId();
@@ -1184,11 +1190,13 @@
         while (!token.IsEmpty() && token != ParserConsts::kw_endif)
             token = m_Tokenizer.GetToken();
         --m_PreprocessorIfCount;
-#if PARSERTHREAD_DEBUG_OUTPUT
+#if PARSERTHREAD_DEBUG_OUTPUT || defined PARSER_TEST
         int l = m_Tokenizer.GetNestingLevel();
 #endif
         m_Tokenizer.RestoreNestingLevel();
+#if PARSERTHREAD_DEBUG_OUTPUT || defined PARSER_TEST
         TRACE(_T("HandlePreprocessorBlocks() : Restoring nesting level: %d (was %d)"), m_Tokenizer.GetNestingLevel(), l);
+#endif
     }
     else if (preproc==ParserConsts::kw_endif) // #endif
         --m_PreprocessorIfCount;
Index: tokenizer.cpp
===================================================================
--- tokenizer.cpp (revision 6218)
+++ tokenizer.cpp (working copy)
@@ -15,15 +15,22 @@
 #include "manager.h"
 #include <cctype>
 #include <globals.h>
+#include "logmanager.h"
 
 #define TOKENIZER_DEBUG_OUTPUT 0
 
+#ifdef PARSER_TEST
+    extern void ParserTrace(const wxChar* format, ...);
+    #define TRACE(format, args...)\
+    ParserTrace(format , ## args)
+#else
 #if TOKENIZER_DEBUG_OUTPUT
     #define TRACE(format, args...)\
     Manager::Get()->GetLogManager()->DebugLog(F( format , ## args))
 #else
     #define TRACE(format, args...)
 #endif
+#endif
 
 namespace TokenizerConsts
 {
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.