Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Plugins development => Topic started by: Ptomaine on September 11, 2006, 03:11:23 pm

Title: Refactoring tool?
Post by: Ptomaine on September 11, 2006, 03:11:23 pm
Maybe it is time to implement the Refactoring plug-in?
Maybe it has to be an additional functionality of the "code completion" plug-in?
Maybe it has to be a separate plug-in based on the code of the "code completion" plug-in?

What do you think?
Title: Re: Refactoring tool?
Post by: tiwag on September 11, 2006, 03:20:53 pm
Be patient!
You'll get it it when it is implemented ...
Title: Re: Refactoring tool?
Post by: Game_Ender on September 11, 2006, 11:13:07 pm
Its been time for that in the C++ world forever, no one will disagree.  C++ is just not a langauge that likes to be parsed so its difficult.
Title: Re: Refactoring tool?
Post by: byo on September 11, 2006, 11:42:41 pm
Making refactoring tools for c++ could take years in company where everybody works 8 hours (or more) a day. And I guess we don't have such possibilities ;)
C++ from parser-point-of-view is really a nightmare (I've been working on such parser some time ago but it took to much time).  And such tool can not be done to work just for some easy cases. You have to take care of scopes / types / templates (it's even hard to say whether statement is declaration or not,even standard says so, and #if... #else blocks - I can't even imagine what cases should be taken into consideration here).
Without precise parsing, you can forget about stl and other nice thigs. Refactoring tool would simply mess up your code.
Because of such problems, other languages (like D, java or C#) were created. They are really easy to parse and modify.
Maybe refactoring tool for c++ is just a matter of time (after reading svn logs I'm more optimistic about that ;)). But if we could make refactoring tool we could also be able to create our own c++ compiler (wow, that would be something :D)
Title: Re: Refactoring tool?
Post by: takeshimiya on September 12, 2006, 12:13:51 pm
Making refactoring tools for c++ could take years in company where everybody works 8 hours (or more) a day. And I guess we don't have such possibilities ;)
C++ from parser-point-of-view is really a nightmare (I've been working on such parser some time ago but it took to much time).  And such tool can not be done to work just for some easy cases. You have to take care of scopes / types / templates (it's even hard to say whether statement is declaration or not,even standard says so, and #if... #else blocks - I can't even imagine what cases should be taken into consideration here).
Without precise parsing, you can forget about stl and other nice thigs. Refactoring tool would simply mess up your code.
Because of such problems, other languages (like D, java or C#) were created. They are really easy to parse and modify.
Maybe refactoring tool for c++ is just a matter of time (after reading svn logs I'm more optimistic about that ;)). But if we could make refactoring tool we could also be able to create our own c++ compiler (wow, that would be something :D)
What makes you think that such precise parser doesn't exists yet? :)

The ANTLR (http://ANTLR) parser generator (C++ binary and C++ grammar) is the best - most correct I could find.
There are lot's of programs using it for C++ refactoring (http://www.refpp.com/), code coverage, code analysis, etc.

This parser generator is so powerful that is being used for language translations (ie. from Java to C and viceversa).
If you wonder what parser the IntelliJ IDEA guys are using for one of the best existing refactoring tools (http://www.jetbrains.com/idea/training/demos/overview.html), you guessed right, it's ANTLR too.

So yes, refactoring C++ is not only possible but doable without "taking years in company where everybody works 8 hours (or more) a day"  :wink: That is, if you choose the right parser instead of creating your own (ANTLR took 15 years :D).
Title: Re: Refactoring tool?
Post by: byo on September 12, 2006, 07:37:09 pm
What makes you think that such precise parser doesn't exists yet? :)

The ANTLR (http://ANTLR) parser generator (C++ binary and C++ grammar) is the best - most correct I could find.
There are lot's of programs using it for C++ refactoring (http://www.refpp.com/), code coverage, code analysis, etc.

This parser generator is so powerful that is being used for language translations (ie. from Java to C and viceversa).
If you wonder what parser the IntelliJ IDEA guys are using for one of the best existing refactoring tools (http://www.jetbrains.com/idea/training/demos/overview.html), you guessed right, it's ANTLR too.

So yes, refactoring C++ is not only possible but doable without "taking years in company where everybody works 8 hours (or more) a day"  :wink: That is, if you choose the right parser instead of creating your own (ANTLR took 15 years :D).

:shock: Seems quite optimistic (it looks like my knowledge is outdated in c++ parsing). But why we don't use ANTLR for CodeCompletion parser? There's whole C++ grammar available for free, few small modifications could make it really usefl in codecompletion case. Ive looked into forums but haven't found real explanation why it is not used yet in C::B
Title: Re: Refactoring tool?
Post by: takeshimiya on September 12, 2006, 08:38:37 pm
What makes you think that such precise parser doesn't exists yet? :)

The ANTLR (http://ANTLR) parser generator (C++ binary and C++ grammar) is the best - most correct I could find.
There are lot's of programs using it for C++ refactoring (http://www.refpp.com/), code coverage, code analysis, etc.

This parser generator is so powerful that is being used for language translations (ie. from Java to C and viceversa).
If you wonder what parser the IntelliJ IDEA guys are using for one of the best existing refactoring tools (http://www.jetbrains.com/idea/training/demos/overview.html), you guessed right, it's ANTLR too.

So yes, refactoring C++ is not only possible but doable without "taking years in company where everybody works 8 hours (or more) a day"  :wink: That is, if you choose the right parser instead of creating your own (ANTLR took 15 years :D).

:shock: Seems quite optimistic (it looks like my knowledge is outdated in c++ parsing). But why we don't use ANTLR for CodeCompletion parser? There's whole C++ grammar available for free, few small modifications could make it really usefl in codecompletion case. Ive looked into forums but haven't found real explanation why it is not used yet in C::B
ddiego has modified ANTLR a little for easier usage in CodeCompletion, along with ucpp, ctags and sqlite, see here: http://vcfbuilder.org/?q=node/139
and he's making it a standard library for using in any IDE. The repository is here: https://svn.sourceforge.net/svnroot/classdom

On the other hand, eranif is doing something very similar with CodeLite: ctags and sqlite, with a little usage of flex.
See here: https://opensvn.csie.org/traccgi/CodeLite/wiki

Ctags is very fast, but not precise, so it's best usage is for codecompletion. A list of supported languages is here (http://ctags.sourceforge.net/languages.html).

ANTLR is not fast, but it's very precise, so it's best usage is things that require this precision: refactoring tools, code translators, compilers, etc. A list of available grammars is here (http://antlr.org/grammar/list).

Of course a hybrid solution is possible (combining ctags & antlr), like what ddiego is doing with "ClassDOM".

The problem with ClassDOM is that its a pure iso c/c++ library, so no wxWidgets facilities.

CodeLite  (https://opensvn.csie.org/traccgi/CodeLite/wiki)on the other hand is being made as a wxWidgets library. We plan to make it integrated as a C::B plugin (any hand is welcomed!) when this component' functionality is finished.

A list of current supported features is here: https://opensvn.csie.org/traccgi/CodeLite/wiki/Features
Title: Re: Refactoring tool?
Post by: byo on September 12, 2006, 11:43:30 pm
Sounds great. But I hope it won't break in expressions like:

Code
a::b<c,d>::e<f,g> (h<i,j>::k) = reinterpret_cast<o>(l(--m++).n ... 
;) (Yop, I'm really waiting for some unbreakable tool)

Is there any estimated time when we could find out how these libraries work ?
Title: Re: Refactoring tool?
Post by: eranif on September 13, 2006, 12:28:52 am
Quote
Sounds great. But I hope it won't break in expressions like:

Code:

a::b<c,d>::e<f,g> (h<i,j>::k) = reinterpret_cast<o>(l(--m++).n ...

Wink (Yop, I'm really waiting for some unbreakable tool)

Is there any estimated time when we could find out how these libraries work ?
You are more then welcome to download CodeLite and test it using the supplied demo.

I suggest using the sources from SVN rather the zip or the installation provided since the sources are far more updated.

The svn respository is located at:
https://opensvn.csie.org/CodeLite/ (https://opensvn.csie.org/CodeLite/)

There are several ways of building it:
using makfile - for MinGW
using VC7.1 solution
makefile.linux - linux ...
and codeblock project supplied by Takeshi & tiwag

For the timeline:

I believe that CodeLite will be ready by the end of next week.

Eran
Title: Re: Refactoring tool?
Post by: kai on September 13, 2006, 01:37:43 am
after reading these posts i did a little research on refactoring c++ code. i'm surprised that there are very few projects about it. anyway, let me share what i found.

cerp is a refactoring tool for eclipse/cdt. i don't know the development status, but it had a few releases and the latest is called 1.0.1
features:

CERP offers some refactorings for C++. these should'nt be too difficult to implement but they are already very useful.

 * Declare Method -- method from .cpp to .h
 * Extract Baseclass -- create abstract base class from existing class in new file
 * Hide Method -- makes a method private
 * Implement Method -- method from .h to .cpp
 * Separate Class -- moves a class to an extra file
 * Synchronize Method -- sync e.g. parameters of a function in header and source file

(german) homepage: http://ifs.hsr.ch/374

elsa seems to be a c++ parser that is worth a look:
http://www.cs.berkeley.edu/~smcpeak/elkhound/

and the harmonia project is very interesting. it provides lots of tools to analyse source code and applications that use this analysis. one application is programming with speech input for handicapped people. harmonia-mode can e.g. provide syntax highlighting of errors before compile time.

a few quotes from the page http://harmonia.cs.berkeley.edu/harmonia/projects/shorthand/index.html :

shorthand project: The key idea behind this work is to use a combination of language awareness and contextual analysis to interpret the commands. The interface can then be made intuitive and natural. For example, inside a Java class, the input “p s v main ( s [ argv” would be taken to designate “public static void main ( String [] argv ) {}”.

codelink project: Studies have repeatedly shown that the majority of computer programs contain a lot duplicated source code. Existing research into this issue has assumed that the problem is in poor programming -- that programmers should abstract their code more extensively with functions and macros. However, we claim that the tools of abstraction (functions and macros) are the problem. They inadequately support the ergonomic and cognitive needs of programmers in real-life situations. We have developed a new editor-assisted alternative to functions and macros. This tool maintains a persistent editor-level "link" between similar regions of code in order to support high-level visualization, editing, and navigation. Our hypothesis is that this tool enables programmers to work faster and with fewer errors, and that in turn they are likely to use the tool in situations where they wouldn't use a traditional abstraction.



Title: Re: Refactoring tool?
Post by: phlox81 on September 14, 2006, 08:28:06 am

ddiego has modified ANTLR a little for easier usage in CodeCompletion, along with ucpp, ctags and sqlite, see here: http://vcfbuilder.org/?q=node/139
and he's making it a standard library for using in any IDE. The repository is here: https://svn.sourceforge.net/svnroot/classdom

Of course a hybrid solution is possible (combining ctags & antlr), like what ddiego is doing with "ClassDOM".

The problem with ClassDOM is that its a pure iso c/c++ library, so no wxWidgets facilities.

I personally think that you don't need wxWidgets at all for parsing. So for me, ClassDOM is interesting because
it does it all in iso c/c++, so you have no dependencies to wxWidgets.
Maybe some should write a wrapper to make it easier to access from wxWidgets.

I personally write currently some sort of my own parser for C++ with boost::spirit, but only for testing,
and currently I only need to validate methodheads. Still a challenge :D

phlox
Title: Re: Refactoring tool?
Post by: takeshimiya on September 14, 2006, 10:17:49 am

ddiego has modified ANTLR a little for easier usage in CodeCompletion, along with ucpp, ctags and sqlite, see here: http://vcfbuilder.org/?q=node/139
and he's making it a standard library for using in any IDE. The repository is here: https://svn.sourceforge.net/svnroot/classdom

Of course a hybrid solution is possible (combining ctags & antlr), like what ddiego is doing with "ClassDOM".

The problem with ClassDOM is that its a pure iso c/c++ library, so no wxWidgets facilities.

I personally think that you don't need wxWidgets at all for parsing.
Not for parsing, but for the GUI and crossplatform bits...
The parsers itself (ctags and flex) doesn't even requieres a c++ compilers, since it's plain c.
Neither for sqlite3, it's plain c too. I hope you don't wonder why wxSQlite3 exists. :wink:

Maybe some should write a wrapper to make it easier to access from wxWidgets.
And that' is what is CodeLite for, it wraps everything in one library and gives you the parsed tree and GUI chores, for example as a derived wxTreeCtrl with all the symbols, or call tips strings for feeding your wxScintilla (or whatever) component.
Title: Re: Refactoring tool?
Post by: phlox81 on September 14, 2006, 11:07:32 am
I personally think that you don't need wxWidgets at all for parsing.
Not for parsing, but for the GUI and crossplatform bits...
The parsers itself (ctags and flex) doesn't even requieres a c++ compilers, since it's plain c.
Neither for sqlite3, it's plain c too. I hope you don't wonder why wxSQlite3 exists. :wink:
crossplattform isn't only wxWidgets. Actually, for everything else then GUI you can use other libraries, with
a more modern C++ Design, boost is here a nice candidate, since its gonna be part of the next standard. (threads, sockets, filesystem f.e.)

Maybe some should write a wrapper to make it easier to access from wxWidgets.
And that' is what is CodeLite for, it wraps everything in one library and gives you the parsed tree and GUI chores, for example as a derived wxTreeCtrl with all the symbols, or call tips strings for feeding your wxScintilla (or whatever) component.
I've taken a look at it. Looks pretty nice :)
But why isn't there a namespace for the lib?
And what license does it have?
Title: Re: Refactoring tool?
Post by: eranif on September 14, 2006, 02:05:44 pm
Quote
But why isn't there a namespace for the lib?

I think I will add namespace, since I am using some very common names such as "Tree" doh :D

Quote
And what license does it have?

The license will be wxWindows, once it is completed, I will add license to the header files.

Quote
I personally think that you don't need wxWidgets at all for parsing.
CodeLite is more than pure C++ parser library, it provides many other facilities, which aimed for wxWidgets applications, such as:

- Unicode supported
- It provides you with the GUI tree, including updating it in run time, anyone who wrote such tree, knows that it is a tedious task to write one.
- Parser thread that runs in the background and updates the tree
- It uses wxWdigets event system to notify about changes
- Keeping the database up-to-date

Today, I am going to start on adding new features, that extends it beyond the standard CodeCompletion:
- Comment tips
- Basic refactoring:
    + extract method
    + move implementation to cpp file
    + create declration from implementation
 The ideas for this I took from VisualAssist (thanks to Takeshi for the tip)

I will also provide my own custom AutoComplete popup box (it will be configurable, I am not happy with scintilla's one, it is very basic)

Eran
Title: Re: Refactoring tool?
Post by: blockn102 on October 24, 2008, 10:03:32 am
Netbeans C/C++ has a good feature
http://img112.imageshack.us/my.php?image=netbeanscod3.png
(http://img112.imageshack.us/img112/7706/netbeanscod3.png)
Title: Re: Refactoring tool?
Post by: killerbot on October 24, 2008, 09:44:07 pm
what feature do you mean. Fact that all occurrences are highlighted, CB does that.
But in CB you can not set the minimum number of characters (I think it is 4, would be better if this is a setting). Like in the screenshot example above, 3.

By the way, what does the red underlining stand for ?
Title: Re: Refactoring tool?
Post by: blockn102 on October 25, 2008, 03:52:14 am
what feature do you mean. Fact that all occurrences are highlighted, CB does that.
But in CB you can not set the minimum number of characters (I think it is 4, would be better if this is a setting). Like in the screenshot example above, 3.

By the way, what does the red underlining stand for ?

Try download: http://bits.netbeans.org/download/trunk/nightly/latest/zip/
netbeans-trunk-nightly-200810241401-cpp.zip

Create a sample project. Then, left click on std or pid and press Ctrl+R to rename class or variable name.