Scripting Test framework

[BlueHazzard]

As obfuscated asked in the other topic (http://forums.codeblocks.org/index.php/topic,18933.msg132717.html#msg132717) i ported the test-framework i use in my sqrat port to the old squirrel implementation, so it can be used to test the current implementation in c::b.
It isn't this useful, because (at least i hope) the current implemented functions should work, and also should never get changed.
But nevertheless here it is.

The framework consists of 3 plugins that can be loaded over the scriptconsole:

• script-test-plugin.script: Test the general scripting framework (mostly the wx stuff like strings etc. )
• user-dialog-test-plugin.script: Test the user dialogs
• menu-test-plugin.script: Test the creation of menus (this isn't working probably, i think the current implementation is faulty, i have fixed this in the sqrat port)

There are additional files:

• test_base.script: Base functionality for the test framework
• wx-test.script: Test wx- test code

To test/install it simply open the script console and open the one of the *-plugin.script files. They should create a menu entry in "Plugins" and output the results in the console window...

greetings

[Edit:] it isn't a patch, because there is no dependency to the current HEAD of c::b or its source...

[oBFusCATed]

Great, it will be best if we can integrate it in the svn source tree, so it can be run regularly and easily.
Is this doable?

[MortenMacFly]

Who Do you ask?

[oBFusCATed]

By courtesy, probably.

This is the result on my linux cb build:

Code

Global executed tests: 73
Global passed tests:   68
Global failed tests:   5

Is this expected?

[BlueHazzard]

I get on win7

Code

Global executed tests: 73
Global passed tests:   67
Global failed tests:   6

Code

======= Test wxSize BEGINN ======= 
[ERROR] Test: _tostring           FAILED (got "(instance : 0x0D8A16B0)" needed "[10,10] ")

not implemented in this c::b version, so no error (but wrong behavior)

Code

======= Test wxPoint BEGINN ======= 
[ERROR] Test: compare doesn't seem to work                      FAILED error: "comparsion between 'instance' and 'instance'" 
[ERROR] Test: _tostring           FAILED (got "(instance : 0x0D98B470)" needed "[10,10] ")

not implemented in this c::b version, so no error (but wrong behavior)

Code

[ERROR] Test: HasVolume           FAILED (got false needed true)

i have absolute no idea what the expected value here is, i always thought windows has volumes and linux don't but it seems that i'm missing something here...

Code

======= Test wx String unicode BEGINN ======= 
test string: !5Aa ЉЩщӃ ウインカリ
[ERROR] Test: This implementation doesn't support UNICODE       FAILED not implemented, as you can see on the wrong test string (it has to show Cyrillic  and ?Japanese? symbols

Executed tests: 0
passed tests:   0
failed tests:   1
======= Test wx String unicode END ======= 

not implemented in this c::b version, so expected, but wrong behavior

Code

======= Test Test environment BEGINN ======= 
 this is intended to fail: 
[ERROR] Test: test_string         FAILED (got "hallo" needed "nope ")

Executed tests: 1
passed tests:   0
failed tests:   1
======= Test Test environment END ======= 

well this is intended

Great, it will be best if we can integrate it in the svn source tree, so it can be run regularly and easily.
Is this doable?

i see no problem, but also no great benefit... first there are missing some tests of the wx system, and second, the "important" tests for c::b like the creation and manipulation of projects and targets is not implemented, and i have no idea how to implement it... Any ideas are welcome!

greetings

[oBFusCATed]

My failures are similar.

Partial test suite is better than no test suite. Also this might encourage other people to contribute.

About the project bindings:
1. tests query functionality - save some projects on disk then use the apis to load then and get the various bits of information in them
2. tests for creation - create projects on the fly then, write some tests to verify that the written information is correct.

Writing tests for the wizard scripts looks a lot more challenging

[oBFusCATed]

The initial version of the scripts is in trunk now.