Hi, with some further work of the post:
Intergrate the nativeparser_base to our parsertest project (http://forums.codeblocks.org/index.php/topic,16559.msg112342.html#msg112342)
Here is the initial patch. The main change is:
1, NativeParserTest contains a ParserBase instance
2, a ParserBase class contains a TokenTree for Parserthread to fill
3, the test case was in the source file (in the last section of the file, all the test cases begin with C++ style comment), see cctest_function_decls.cpp in the patch.
4, I remove the cc_test class, I think their function should be moved to NativeParserTest class
5, The limitation is currently the test case is all in global namespace
Result from the test log message:
********************************************************
Testing in file: cctest\cctest_function_decls.cpp
********************************************************
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 2 "g_S."
GetNextCCToken() : at 2 (g): res=
GetNextCCToken() : Done nest: at 5 (.): res=g_S
GetNextCCToken() : Return at 5 (.): res=g_S
GetCCToken() : FindCCTokenStart returned 5 " g_S."
GetCCToken() : GetNextCCToken returned 5 "g_S"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 0 "S"
GetNextCCToken() : at 0 (S): res=
GetNextCCToken() : Done nest: at 1 (
GetNextCCToken() : Return at 1 (
GetCCToken() : FindCCTokenStart returned 1 "S"
GetCCToken() : GetNextCCToken returned 1 "S"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
NativeParserBase::GenerateResultSet_2()
****************************
PASS: g_S. i
****************************
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 2 "g_"
GetNextCCToken() : at 2 (g): res=
GetNextCCToken() : Done nest: at 4 (
GetNextCCToken() : Return at 4 (
GetCCToken() : FindCCTokenStart returned 4 " g_"
GetCCToken() : GetNextCCToken returned 4 "g_"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
****************************
PASS: g_ g_S
****************************
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 2 "tc.Vo"
GetNextCCToken() : at 2 (t): res=
GetNextCCToken() : Done nest: at 4 (.): res=tc
GetNextCCToken() : Return at 4 (.): res=tc
GetCCToken() : FindCCTokenStart returned 4 " tc.Vo"
GetCCToken() : GetNextCCToken returned 4 "tc"
GetCCToken() : Left "Vo"
FindCCTokenStart() : Starting at 0 "Vo"
GetNextCCToken() : at 0 (V): res=
GetNextCCToken() : Done nest: at 2 (
GetNextCCToken() : Return at 2 (
GetCCToken() : FindCCTokenStart returned 2 "Vo"
GetCCToken() : GetNextCCToken returned 2 "Vo"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 0 "TheClass"
GetNextCCToken() : at 0 (T): res=
GetNextCCToken() : Done nest: at 8 (
GetNextCCToken() : Return at 8 (
GetCCToken() : FindCCTokenStart returned 8 "TheClass"
GetCCToken() : GetNextCCToken returned 8 "TheClass"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 2 valid text matched tokens from the tree.
NativeParserBase::GenerateResultSet_2()
Find 2 valid text matched tokens from the tree.
****************************
PASS: tc.Vo Void
****************************
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 2 "tc.St"
GetNextCCToken() : at 2 (t): res=
GetNextCCToken() : Done nest: at 4 (.): res=tc
GetNextCCToken() : Return at 4 (.): res=tc
GetCCToken() : FindCCTokenStart returned 4 " tc.St"
GetCCToken() : GetNextCCToken returned 4 "tc"
GetCCToken() : Left "St"
FindCCTokenStart() : Starting at 0 "St"
GetNextCCToken() : at 0 (S): res=
GetNextCCToken() : Done nest: at 2 (
GetNextCCToken() : Return at 2 (
GetCCToken() : FindCCTokenStart returned 2 "St"
GetCCToken() : GetNextCCToken returned 2 "St"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 0 "TheClass"
GetNextCCToken() : at 0 (T): res=
GetNextCCToken() : Done nest: at 8 (
GetNextCCToken() : Return at 8 (
GetCCToken() : FindCCTokenStart returned 8 "TheClass"
GetCCToken() : GetNextCCToken returned 8 "TheClass"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 2 valid text matched tokens from the tree.
NativeParserBase::GenerateResultSet_2()
Find 2 valid text matched tokens from the tree.
****************************
PASS: tc.St StaticVoid
****************************
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 2 "tc.GetC"
GetNextCCToken() : at 2 (t): res=
GetNextCCToken() : Done nest: at 4 (.): res=tc
GetNextCCToken() : Return at 4 (.): res=tc
GetCCToken() : FindCCTokenStart returned 4 " tc.GetC"
GetCCToken() : GetNextCCToken returned 4 "tc"
GetCCToken() : Left "GetC"
FindCCTokenStart() : Starting at 0 "GetC"
GetNextCCToken() : at 0 (G): res=
GetNextCCToken() : Done nest: at 4 (
GetNextCCToken() : Return at 4 (
GetCCToken() : FindCCTokenStart returned 4 "GetC"
GetCCToken() : GetNextCCToken returned 4 "GetC"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 0 "TheClass"
GetNextCCToken() : at 0 (T): res=
GetNextCCToken() : Done nest: at 8 (
GetNextCCToken() : Return at 8 (
GetCCToken() : FindCCTokenStart returned 8 "TheClass"
GetCCToken() : GetNextCCToken returned 8 "TheClass"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 2 valid text matched tokens from the tree.
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
****************************
PASS: tc.GetC GetClass
****************************
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 2 "tc.GetI"
GetNextCCToken() : at 2 (t): res=
GetNextCCToken() : Done nest: at 4 (.): res=tc
GetNextCCToken() : Return at 4 (.): res=tc
GetCCToken() : FindCCTokenStart returned 4 " tc.GetI"
GetCCToken() : GetNextCCToken returned 4 "tc"
GetCCToken() : Left "GetI"
FindCCTokenStart() : Starting at 0 "GetI"
GetNextCCToken() : at 0 (G): res=
GetNextCCToken() : Done nest: at 4 (
GetNextCCToken() : Return at 4 (
GetCCToken() : FindCCTokenStart returned 4 "GetI"
GetCCToken() : GetNextCCToken returned 4 "GetI"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
NativeParserBase::BreakUpComponents()
FindCCTokenStart() : Starting at 0 "TheClass"
GetNextCCToken() : at 0 (T): res=
GetNextCCToken() : Done nest: at 8 (
GetNextCCToken() : Return at 8 (
GetCCToken() : FindCCTokenStart returned 8 "TheClass"
GetCCToken() : GetNextCCToken returned 8 "TheClass"
GetCCToken() : Left ""
NativeParserBase::GenerateResultSet_2()
Find 2 valid text matched tokens from the tree.
NativeParserBase::GenerateResultSet_2()
Find 1 valid text matched tokens from the tree.
****************************
PASS: tc.GetI GetInt
****************************
Patch is in the attachment.
Comments are welcome! Thanks.
An modified patch, new tests were added. the result is below, the main change is: adjust the GCC/wx path for pcx's mingw 4.6.3.
********************************************************
Testing in file: cctest\cctest_function_decls.cpp
********************************************************
****************************
PASS: g_S. i
****************************
****************************
PASS: g_ g_S
****************************
****************************
PASS: tc.Vo Void
****************************
****************************
PASS: tc.St StaticVoid
****************************
****************************
PASS: tc.GetC GetClass
****************************
****************************
PASS: tc.GetI GetInt
****************************
****************************
PASS: str. size
****************************
****************************
PASS: str. length
****************************
----------------------------
FAIL: vt[1]. GetInt
----------------------------
Look, it seems it fails on some test of std::vector.
EDIT: remove a debug code, and update this patch.
Update to the batch_test_for_codecompletion_v2.patch
Currently, the default file: cc_function_decls.cpp is used for testing.
The contents of cc_function_decls.cpp is below:
#include <string>
#include <vector>
extern "C"
{
double ExternallyC(double* my_double, std::vector<int> int_vec)
{
// nothing
return 1.0;
}
}
class TheClass
{
public:
TheClass () { ; }
virtual ~TheClass () { ; }
int GetInt (size_t idx) const { int i=0; return i; }
const TheClass* GetClass (const TheClass& tc) { return this; }
static void StaticVoid() { ; }
void Void (void* pVoid) { ; }
};
extern int ExternInt(TheClass* the_class, int my_int);
int InternInt(TheClass* the_class, int my_int)
{
// nothing
return 42;
}
void VoidFunction(void* my_void_pointer, int my_int)
{
// nothing
}
std::string GetString(TheClass* the_class, const std::string& in_str)
{
std::string str(in_str);
return str;
}
const float& ConstFloatRef(void)
{
// nothing
static float f = 0.0;
return f;
}
static int StaticInt(const int& in_int)
{
// nothing
int i = 1;
return i;
}
struct S
{
int i;
};
S g_S;
TheClass tc;
std::string str;
std::vector<TheClass> vt;
// str. //size,length
// tc.GetI //GetInt
// tc.GetC //GetClass
// tc.St //StaticVoid
// tc.Vo //Void
// g_ //g_S
// g_S. //i
// vt[1]. //GetInt
Please look at the last lines of the file:
// str. //size,length
// tc.GetI //GetInt
// tc.GetC //GetClass
// tc.St //StaticVoid
// tc.Vo //Void
// g_ //g_S
// g_S. //i
// vt[1]. //GetInt
These are the test cases, the grammar is quite simple, E.g.
The string between the double backslash is the statement "str.", it is expected that the user put the caret after the statement, and press the CTRL+SPACE key to let the CodeCompletion plugin to list the auto suggestion list.
The string after the second double backslash is the match cases. In the above example, the auto suggestion list should/must contains "size" and "length". If yes, then this test case passes, otherwise, this test case failed.
You can change the test case in the cc_test's wxScintilla control, and press "parse" button again to run the new tests.
Currently, all the parsing messages are logged out, so if you are parsing a cpp file which contains an include file, then all the include files will be parsed recursively(this is the correct way of parsing a cpp file as a normal compiler does), in such cases, you get thousands of log messages. :)
Comments are welcome.
BTW: the attachment is in git format(generated from Tortoisegit/msysgit), because I'm currently maintain a local git branch to implement this feature.