Doing unit tests (cppunit)

[nofaber]

Hi all,

I have just discovered code::blocks and I find it very powerful. Nevertheless, I want to do unit test (under windows). So I wonder if a cppunit plugin is available ? Or if not, how can I do unit tests with code::blocks under windows

Thanks

nofaber

[thomas]

There is no such plugin available at this time.

Neither have I ever done unit testing or intend to myself, so my knowledge is somewhat limited in that matter, too. However, I found a project at Sourceforge which may be helpful: http://sourceforge.net/projects/cppunit

If you get this to work (I haven't tried, but it looks promising), then you should not have too much difficulties to make a plugin for Code::Blocks from it.

[killerbot]

I haven't used those thing either, but I am looking around to start using them, I already read some comparisons articles, and cpp unit seems not to be that good, there was more positive acceptance on : cpptest : http://cpptest.sourceforge.net/

[Game_Ender]

Neither have I ever done unit testing or intend to myself, so my knowledge is somewhat limited in that matter, too. However, I found a project at Sourceforge which may be helpful: http://sourceforge.net/projects/cppunit

*Says silent prayer for future of CB stability*

Unit tests are really helpful.  At first there is some overhead for writing the test but they help you in numerous ways.  If you write a piece of code don't you want to know that it works?  Wouldn't it be amazing if you felt complete freedom to completely rewrite the code from class and be able to tell in seconds in whether it performs just as it did?  Once you have something written and tested you get a sense of freedom.  Freedom from the fear that ever touching that piece of code again is going to break it into pieces.  It also means you are no longer responsible for that code, the unit tests are.  As long as whoever maintains the code in future keeps the code passing the unit tests everything is fine.  They don't have to try and figure what a class does, because the unit tests define this.

Numerous plugins and parts of codeblocks would benefit from unit testing.  Things the like the new code completetion should have a test suite written for it.  Here is one of my favorite examples of agile development and unit testing.  See the obligatory wikipedia article on unit testing and extreme programming explanation of unit tests.  As for the XP link, I know we aren't developing like that but it does do a good explanation of the benefits of testing.

Besides, how can you argue against validating your code without looking lazy .

About frameworks, I currently use the xUnit testing framework (CPPUnit, JUnit, NUnit...).  Here is a good comparison of the different unit testing frameworks, but it doesn't have cpptest.  So I will have a look at that right now.

EDIT- From a popularity standpoint CppUnit is about 10 times as used as cpptest, and Cppunit is more active.

[killerbot]

Check out C/C++ Users Journal : Edition December 2005
"Unit Testing & CxxTest"

So it's CxxTest and not cppTest, my mistake.

Comparison article :
http://www.gamesfromwithin.com/articles/0412/000061.html

[thomas]

In my opinion, unit testing is a way of moronizing a process. Yes, I know, moronizing processes is very much "en vogue" these days, but I prefer to do it the good old way.

The industry pays millions and millions for a very similar business process concept and loses billions because of it every year. All those ISO 900[123] certification processes do nothing different than unit testing. They set up a protocol and that protocol has to be followed. Also, the process is documented at discrete points defined by the protocol. If you pass those milestones, everything is good, otherwise it is not.
The problem is that it does not work! Different from what the smooth-tongued advocates tell you, this does not detect key problems. What it detects is violation of the protocol, nothing else.

I think that a system that every monkey can use (passed test: yes or no?) is good for some things. However, personally I still prefer to validate manually.

[Michael]

The industry pays millions and millions for a very similar business process concept and loses billions because of it every year. All those ISO 900[123] certification processes do nothing different than unit testing. They set up a protocol and that protocol has to be followed. Also, the process is documented at discrete points defined by the protocol. If you pass those milestones, everything is good, otherwise it is not.
The problem is that it does not work! Different from what the smooth-tongued advocates tell you, this does not detect key problems. What it detects is violation of the protocol, nothing else.

Years ago during my studies, I have had an introduction of all the ISO 900[123] & ISO 1400[...] bla bla . I am not sure with computer, but in the domain of the ecotoxicology, these protocols and the tests they described seemed to be a relatively good way to go (their costs were one of the main negative points anyway ). If really effective or not, I am not sure (the postgraduated course related specifically to the ISO topic was really too much expensive for most of the students ). IMHO, they should not take as unique reference as they do not are perfect and their costs are not to underestimated.

Best wishes,
Michael

[thomas]

My experience with the indroduction of ISO 9000 in a large general hospital some years ago was a 50% increase in administrative work for everybody (and a respective overall decrease in patient care).
Half a dozen new administrative positions filled with entirely unqualified people doing assessment and quality control (read: prevent more qualified people from doing their work, to justify their own position), and no improvements whatsoever. 

[Game_Ender]

I am still failing to see how designing a class to X, then writing test code that test to make sure it does X, and running those test every time to tweak the class is a bad thing.  Either you, or the computer is going to test that class, and I think we can all agree this is something a computer can do faster (we are using computer right now aren't we).

To bring this thread a little more back to topic.  There is a wxWidgets test runner written for cppunit.  So that interface could be used and then you just have to integrate it with CB.

[Michael]

My experience with the indroduction of ISO 9000 in a large general hospital some years ago was a 50% increase in administrative work for everybody (and a respective overall decrease in patient care).

Yes, this is a ISO negative point.

Half a dozen new administrative positions filled with entirely unqualified people doing assessment and quality control (read: prevent more qualified people from doing their work, to justify their own position), and no improvements whatsoever. 

I think the problem come from unqualified people doing assessment and quality control. This will hurts more than be useful.

The course mentioned above was to give students who decided to specialize in ISO (and have the money for doing that a relatively good preparation. One time I was speaking with a person working in a pharmaceutical laboratory who was searching someone for an opened position of responsible for quality control of the whole production chain (from the beginning until the end). The learning part of this position was of more than 1 years, because all what she/he had to know/follow was written in several huge books (encyclopedia format . Moreover, it was requested that the person "accepted" to work for around 10 years (the laboratory did not want to invest a lot of money and time during more than one year for someone who will leave in 2-3 years .

Personally, I think that ISO could be useful if done correctly. If not it is just a waste of money and nothing else.

Best wishes,
Michael

[MortenMacFly]

My experience with the indroduction of ISO 9000 in a large general hospital some years ago was a 50% increase in administrative work for everybody

You're so right! At my work they started to introduce quality assurancy and certification of several working tasks based on an ISO somewhat (I don't know exactly). I don't see the promised pros yet only the increase of administration as you did...
Maybe I change my mind at someday when I see any trade-off... but I'm already waiting for about a year or so...
Morten.

Ps.: Sorry for being completely off-topic meanwhile... :oops:

[kfmfe04]

This is a very old thread, but in case other developers are trying to use cxxtest, I managed to get it working using this setup:

1. Create two projects - a static library project called mylib, and a console project called mylib_test
2. Make mylib_test depend on mylib via Project > Properties > Project dependencies...
3. Put my tests into the mylib_test project's include folder.  eg Test_smart_ptr.hpp
4. In mylib_test, go to Project > Build options... > Pre/post build steps and add a line to Pre-build steps:
cxxtestgen.pl --gui=QtGui -o $(PROJECT_DIR)src/runner.cpp $(PROJECT_DIR)include/*.hpp

Of course, you can configure this last cxxtestgen.pl line to output just text, if you prefer.

Under this setup, the build will fail (with red in the GUI) upon failure of any unit tests.
I usually have mylib_test selected on Release and hit <F9>, change/add code, hit <F9>, etc...

In other IDEs/unit-tests, I tend to create a tests folder within the same project, but for this cxxtest/C::B combo, a separate project seems to be easier to set up.

Hope someone finds this useful.

Check out C/C++ Users Journal : Edition December 2005
"Unit Testing & CxxTest"

So it's CxxTest and not cppTest, my mistake.

Comparison article :
http://www.gamesfromwithin.com/articles/0412/000061.html

[kfmfe04]

Just a quick update on building with cxxtest inside C::B.

cxxtestgen.pl --gui=QtGui -o $(PROJECT_DIR)src/runner.cpp $(PROJECT_DIR)include/*.hpp

was taking too long when I got up to about 20 test files, so I cranked out a little ruby script which effectively acts as a no-maintenance Makefile.  I run a for-loop over the files in my include directory, check the timestamps between my include and src directory, and generate with (inside a ruby script)

`cxxtestgen.pl --part -o #{full_cpp} #{full_hpp}`

if a rebuild is necessary, where full_cpp is a full-path to my cpp file and full_hpp to my hpp file.  I don't really know ruby, but it's such an intuitive language that I was able to crank out my script in about 30 minutes (about 25 lines) - took me longest to find out that #{var} is used to escape a variable inside a shell `cmd` invocation.

For the build dependencies to work properly inside C::B, add all the *.cpp files generated.

There are a few extra lines wrapping:

`cxxtestgen.pl --gui=QtGui --root -o #{full_cpp}`

Now, when I do my pre-build from inside C::B, the ruby script only calls cxxtestgen.pl on the headers I touched so the build/link/test cycle is much, much faster than before.

============================================================

#!/usr/bin/ruby

indir    = "/foobar/include"

hppnames = Dir.glob( indir + "/*.hpp" )
hppnames.each {
   |hppname|
   cppname  = hppname.sub(".hpp",".cpp")
   cppname  = cppname.sub("/include","/src")
   htime    = File::mtime( hppname )
   rebuild  = true
   if ( File::exists?( cppname ))
       ctime   = File::mtime( cppname );
       rebuild = ( htime >= ctime )
   end
   if ( rebuild )
       puts "Regenerating " + cppname
       `cxxtestgen.pl --part -o #{cppname} #{hppname}`
   end
}

cppname = indir.sub("/include","/src") + "/runner.cpp"
if ( !File::exists?( cppname ))
   `cxxtestgen.pl --gui=QtGui --root -o #{cppname}`
end

[oBFusCATed]

Hm, this cxxtest looks ugly by the way kfmfe04 is using it. Lots of unnecessary complexity.

After seeing this thread, I've decided to give a try of the TDD religion. So I've reread the http://gamesfromwithin.com/?p=50 and then about their UnitTest++ lib.
Downloaded it and setup a test project for a testbed CB plugin I'm doing at the moment.
I've added it as a separate project and added the files that I want to test from the original project to the test project, written the some tests and added a post-build step to run the test executable.
Everything works fine, except that if I modify the real project the test won't recompile and rerun.
There are two options that I have:
1. Convert the test project to target in the real project (will try it tonight)
2. Somehow make C::B to recompile the test project every time I build the real project. I've tried the project dependencies dialog, but it does not work The thing I need is to setup the post-build step of the real project to build my test project, but I suppose that is not possible at the moment (and never will be).

[killerbot]

unittest++ works very well in CB. I have nearly finished my wiki article preparation. Will be completely finished in 3 weeks time when I have some time off.


The only issue is project dependencies in CB, but that's a general CB issue.

[kfmfe04]

Unfortunately, the "ugliness" is a byproduct of the fact that cxxtest generates .cpp files from .hpp files and then expects you to compile and link them.  Luckily, this results in a very lightweight and easy-to-use system.

I'm not sure what is happening on your system configuration, but dependencies between projects seem to work okay for me - ie if I edit my real library code and target my test-driver application, C::B knows to compile the library before linking.

What I'd really like to see is a JUnit or NUnit like system with a GUI, but none seems to be available C++ - it's convenient to be able to run a sub-set of all the tests through a GUI when I am concentrating on something specific.

Anyhow, best not to waste too much time thinking about this - as long as the framework does its job, better to just get some framework up, and concentrate on the coding itself...

[Jenna]

Everything works fine, except that if I modify the real project the test won't recompile and rerun.
There are two options that I have:
1. Convert the test project to target in the real project (will try it tonight)
2. Somehow make C::B to recompile the test project every time I build the real project. I've tried the project dependencies dialog, but it does not work The thing I need is to setup the post-build step of the real project to build my test project, but I suppose that is not possible at the moment (and never will be).

What about using the C::B-project as dependency for the test-project and recompile the test-project ?

[oBFusCATed]

Dependencies don't work at all, no matter if test_app depends on the plugin or the vice-versa.
Could it be because I don't link the test_app to the plugin?

[kfmfe04]

From my limited experience with cxxtest and C::B (about 2 weeks), I was able to get dependencies working correctly, but it was time-consuming figuring out how to set-up a test framework to work well within C::B.  When I say correctly, I mean if I edit any of my main code, a minimal-but-necessary number of compiles/links occur in my system.

As far as I can tell, C::B dependencies work fine, but settings in 3 or 4 or 5 different places can definitely have an impact on whether dependencies work the way you expect.

If you describe >very specifically<, what you mean by "Dependencies don't work at all", maybe some of us can help. 

What is your setup for your code and testing framework?  In the same project?  In different projects?  Are you using any pre-build or post-build steps for your testing framework?  Is the test failing to compile or link or both?

Dependencies don't work at all, no matter if test_app depends on the plugin or the vice-versa.
Could it be because I don't link the test_app to the plugin?

[oBFusCATed]

I've explained my setup in my first post (not very clear though, excuse me for that )

Here it is simplified:

1. my real project - in the current case a C::B plugin
2. my test project - do not like with the real project, because I don't export anything useful for testing. Instead I have added the files I want to test from the real project in the test project.

I've put the two projects in a workspace, then I've tried to make the test project depend on the real project (and vice versa) through "project -> properties -> project dependencies". I've not set any external dependencies, because the two output files don't depend on each other.

[kfmfe04]

That's better.  I've never done plug-in development, but I assume that your test project should link with the plug-in library (is it a .a or .dll or .so)?  Correct me if I'm wrong.

1. Is it a compile failure, a link failure, or a dependency failure which is preventing some automatic compile or link?
2. Specifically, what did you try that causes you to think a dependency failed (compile or link)?
3. Has the dependency succeeded under any condition?  In other words, is it failing under a certain condition or does it always fail?

My setup is very similar to yours, but I have found that I DO NOT need to add any files from my real project to my test project - only the directory for the real project headers under the test project's build directory to search for.

I've explained my setup in my first post (not very clear though, excuse me for that )

Here it is simplified:

1. my real project - in the current case a C::B plugin
2. my test project - do not like with the real project, because I don't export anything useful for testing. Instead I have added the files I want to test from the real project in the test project.

I've put the two projects in a workspace, then I've tried to make the test project depend on the real project (and vice versa) through "project -> properties -> project dependencies". I've not set any external dependencies, because the two output files don't depend on each other.

[oBFusCATed]

I've found my problem ... the two projects need to have the same Targets, otherwise the dependencies doesn't work.
In my case my plugin project has only the default target, but my test project have the Debug/Release combo.
I've renamed default to be Debug and now it works....