Author Topic: New code completion plugin w/ refactoring  (Read 63006 times)

Offline artoj

  • Almost regular
  • **
  • Posts: 206
  • Location: Supporting my team
    • http://ajonsson.kapsi.fi/
New code completion plugin w/ refactoring
« on: February 08, 2006, 11:55:19 am »
Yannis and I have talked about creating a new code completion plugin with support for refactoring, it will be a full rewrite. If I'm lucky I will get a summer job in this project, that's about 3 months working full-time with C::B. Hopefully after summer we have something done (I'm not promising anything though).

I'm currently working with the most important part of the plugin, the planning. I know that there are very talented programmers/designers in these forums so I would like to hear your opinions about the plugin when we have something to show you. I want the planning phase to be as open as possible, like rickg22 did with the current plugin.

The initial plan is to create the plugin using eranif's ideas: CTags and SQLite. I've made very, very, very simple layout of the plugin which I had in mind:

Code
                                           "The plugin"
                                                 |
                                             Languages
                                          /      |       \
                                       C/C++  Python    Java
                                        |        |        |
                                      P&R E     P&R E   P&R E
                                --------------------------------
                               Common Parser | Common Refactor
                                --------------------------------

So what these all mean? At the top we have the plugin which will implement all our features. The plugin defines a set of languages. These can be anything. I'm not saying C::B should handle Python or Java code, it's just an example (we'll focus the C/C++ part). Each language defines its own Parser & Refactor Engine (P&R E) which will called when needed. These "engines" are subclasses of Common Parser and Common Refactor. Common Parser and Common Refactor are just abstract classes which will define what features parser and refactor have.

For example, in Common Refactor we could say:

Code
bool ExtractMethod(wxString[] codeblock);

or something like that. Now, each language defines this ExtractMethod refactoring scheme anyway it wants. This was designed (well, though, because the design phase is just beginning) this way because we don't know what languages we will support in the future.

Because there is about 70 different refactoring schemes (some of them are very complicated), we won't be creating them all at once. I'm thinking that ExtractMethod and RenameMethod are the two that we want to implement first.

We want to be able to extend the Common Parser too. I'm thinking that the Common Parser could just be a wrapper to CTags, if we want to change CTags to a different parsing system, so be it, the methods in the Common Parser remain the same.

I haven't thought the SQL system much yet. I don't know should we create another wrapper to the cache and use SQLite under it. This could be a good decision because, again, we don't know what happens in future.

Another thing I thought this morning was how to efficiently use the the cache between restarts. We want to save the database to disk, but what about files that have changed since we last used the database. I was thinking some extremely simple (and fast) checksum system that we could use to check if the files are changed.

So what happens now?

You can just sit back and relax. :D I don't think I will be presenting anything major in this month. When the design phase is over I let you guys know. After we got the full documentation this plugin is 50 % complete.

And please don't ask when we are going to release this plugin, honestly, we don't know. :)

rickg22, thank you for your hard work with the current plugin.

- Arto

Offline Michael

  • Lives here!
  • ****
  • Posts: 1608
Re: New code completion plugin w/ refactoring
« Reply #1 on: February 08, 2006, 12:06:12 pm »
Hello,

I have just a small question. I would like to know why you want to do a full-rewrite, instead of improving the already existing plugin :?. Not that I have anything against a full-rewrite, I just would like to know. AFAIK, the code completion of rickg22 is not that bad :).

Thank you very much.

Best wishes,
Michael

PS.: Anyway, your design looks promising :).

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: New code completion plugin w/ refactoring
« Reply #2 on: February 08, 2006, 12:13:55 pm »
I would like to know why you want to do a full-rewrite, instead of improving the already existing plugin :?.

Because there comes a point where the weaknesses of the design start to show through ;).
Honestly, although I initially created the codecompletion plugin, I don't feel like it has much potential in it yet...
Yes, Rick did a hell of a job with it during the last months. It will not be dropped, at least not any time soon ;)
Even if Arto comes up with a working plugin during the summer, we will have a choice between two plugins. Choice is A Good Thing (tm). Only when (and if) Arto's plugin proves its worthiness it will become our standard :) (I hope it will, though).
Be patient!
This bug will be fixed soon...

Offline Michael

  • Lives here!
  • ****
  • Posts: 1608
Re: New code completion plugin w/ refactoring
« Reply #3 on: February 08, 2006, 12:30:31 pm »
I would like to know why you want to do a full-rewrite, instead of improving the already existing plugin :?.

Because there comes a point where the weaknesses of the design start to show through ;).
Honestly, although I initially created the codecompletion plugin, I don't feel like it has much potential in it yet...
Yes, Rick did a hell of a job with it during the last months. It will not be dropped, at least not any time soon ;)
Even if Arto comes up with a working plugin during the summer, we will have a choice between two plugins. Choice is A Good Thing (tm). Only when (and if) Arto's plugin proves its worthiness it will become our standard :) (I hope it will, though).

Thank you very much for the explanation :D. Yes, I agree with you. It is good to have the choice and a bit of competition can stimulate people to provide better software. It seems too me that after its design, the Arto's plugin looks promising and something good will come out. Anyway, there will be probably people in this forum that would like to help or at least to give useful advices/comments/critics.

Michael

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: New code completion plugin w/ refactoring
« Reply #4 on: February 08, 2006, 12:56:31 pm »
Anyway, there will be probably people in this forum that would like to help or at least to give useful advices/comments/critics.

I 'm sure there are. That's why Arto made this post ;).
Be patient!
This bug will be fixed soon...

Offline TDragon

  • Lives here!
  • ****
  • Posts: 943
    • TDM-GCC
Re: New code completion plugin w/ refactoring
« Reply #5 on: February 08, 2006, 03:07:30 pm »
1. Replaceable and subclassable interfaces are a Good Thing™. That way it'll require minimal redesign on my end and no redesign on the main plugin when I unveil my hyper-advanced C/C++ parser. ;)

2. Make sure you provide functionality to let the parsers do structure introspection too. C++ in particular is a highly context-relative language, so it needs to be able to look back at what it's already parsed in order to understand what it's currently parsing.

3. The saying "Make it work. Make it fast. Make it right. Pick any two" doesn't apply to code completion / visual assist software. It needs to work quickly and correctly, or it'll be a turn-off (both literally and figuratively).

More as I think of them,
Cheers,
Good luck,
JohnE / TDM
« Last Edit: February 08, 2006, 03:13:43 pm by TDragon »
https://jmeubank.github.io/tdm-gcc/ - TDM-GCC compiler suite for Windows (GCC 9.2.0 2020-03-08, 32/64-bit, no extra DLLs)

Offline Game_Ender

  • Lives here!
  • ****
  • Posts: 551
Re: New code completion plugin w/ refactoring
« Reply #6 on: February 08, 2006, 04:15:48 pm »
I love the plugable parser model, I am sure everyone has a pet parser they would love to see placed in the codecompeletion pluggin so we can finally get some template and typedef recognition.  I have my hat tossed in with the Elsa parser, especially if you want to do refactoring.

One thing your model does not address is threading.  This needs to thought of from the beginning, because it has a very flaky feel in the current plugin.  There is no offense meant to the authors, it just that the threading code is not very robust. If we work out the specific thread design we want to use (worker thread(s), a new thread per parsing job etc) now it will be easy to design the interactions of the components to place nice in that system. 

I also don't know if using something like boost.threads would help or not, but I would consider heavier use of helper libraries like boost.  IMHO the wxWidgets "support" classes should probably be avoided if there they are already in boost or the stl, but that is another issue all together.

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: New code completion plugin w/ refactoring
« Reply #7 on: February 08, 2006, 06:09:34 pm »
Suggestion, make the parsing per-workspace and not per-project, otherwise we'll still suffer the memory issues...

Pete

  • Guest
Re: New code completion plugin w/ refactoring
« Reply #8 on: February 13, 2006, 11:17:19 am »
I'd like to propose the more separted model
Code: dos
------------------------------------
internal source code model
-----------------------------------------------------------------------------------
an interface for manipulations with internal model
-----------------------------------------------------------------------------------
/\               |                         |    /\            /\   |
|                |                        \/    |             |   \/
------      ------------------     -------------------     ----------
parser      source code styler     UML representation      refactoring
------     -------------------     -------------------      plugin
|                 |                                        ----------- 
|                \/
-------------------------------
source code listing,
edit window or something similar
-------------------------------
This way parsing and refactoring will be separated from internal source code representation, common refactor will be obsolete.
Another idea is a script wrapper for manipulations with source code model. I think it's more easy to write for code analizing and refactoring, than writing a c++ plugin or external application with another source parsing.

PS sorry for my poor english :(

takeshimiya

  • Guest
Re: New code completion plugin w/ refactoring
« Reply #9 on: February 13, 2006, 01:45:04 pm »
Agreed, there should be one C++ parser for all plugins, no more.

Offline artoj

  • Almost regular
  • **
  • Posts: 206
  • Location: Supporting my team
    • http://ajonsson.kapsi.fi/
Re: New code completion plugin w/ refactoring
« Reply #10 on: February 13, 2006, 03:03:18 pm »
So we could have something like this:

In core (build in Code::Blocks itself) it's the Language manager. This language manager will load/unload language libraries. Language manager will also handle file extensions and syntax lighting (= all language properties in one place). The language manager will define set of properties that every language must handle, e.g. these could be functions and members (we need these to build the cache).

The language library would handle all implementation, the language parser, refactoring, styles and parser. In the most optimal case extending C::B to use new language would only require dropping one library to the plugins path or separate languages path.

---

The work that this would require would be massive, period. I'm not saying this cannot be done and I agree that currently the language/parser/syntax lighting/file extensions system is a mess. I think that some vital functions should be in core, the cache would be good example because practically every language needs it. One could argue that there's no need to build anything to the core if we make plugin which uses plugins, great. :lol:

It's great to see new ideas, keep them coming 8)

Pete

  • Guest
Re: New code completion plugin w/ refactoring
« Reply #11 on: February 14, 2006, 01:24:19 pm »
May be caсhe can be implemented as a service. If cache implementation for c++ will provide all functionality needed for other languages, i can use cache service whitout integrating its source code in my language library.

sethjackson

  • Guest
Re: New code completion plugin w/ refactoring
« Reply #12 on: February 14, 2006, 04:44:09 pm »
I think that is good. One thing though. Keep the styling done through XML.  :)

Offline artoj

  • Almost regular
  • **
  • Posts: 206
  • Location: Supporting my team
    • http://ajonsson.kapsi.fi/
Re: New code completion plugin w/ refactoring
« Reply #13 on: February 14, 2006, 08:04:22 pm »
May be caсhe can be implemented as a service. If cache implementation for c++ will provide all functionality needed for other languages, i can use cache service whitout integrating its source code in my language library.

Yup, that could be possible. This could lead to "plugin hell" though, separate plugins relay on different versions of each other. The interface that the cache would be used would have to be so good that it wouldn't require changes often. The PluginManager included currently in Code::Blocks allows us to call different plugins. I don't know about PluginManager's thread safety (pretty sure it isn't), the cache system must be thread safe because it will be called all over the place.

Another good idea for the separate cache would be the class treeview system / function & class listboxes in separate plugins BUT because we must assume that we don't know anything about the language we are using can causes several problems: how are we going to layout the tree/whatever if don't know if the language has namespaces or classes or is the language only procedural. We could however use namespaces and classes in every language but put something like "(global)" or something like that to the name of the class/namespace if the language doesn't support classes/namespaces.

In the original idea I was going to add some sort of meta block to the cache and every language could add some important data to it for further usage. If language "x" supports some new function type "y" it could mark those functions with "y" in the meta block and then later use that info for example showing user in the UI/class treeview that these functions are "y" type. I'm not sure if I'm going to trash this idea or not. If this idea will be implemented it would mean that the UI usage and language specification must be very close to each other. We could again however somehow query the language plugin how we should layout this data we have.

It could work. 8)

Offline Game_Ender

  • Lives here!
  • ****
  • Posts: 551
Re: New code completion plugin w/ refactoring
« Reply #14 on: February 14, 2006, 10:31:21 pm »
The only thing you can do with the current level of information the code-completion provides is code complete.  You need a full abstract syntax tree of a language to create good UML diagrams and do reliable refactoring.  That AST will be at the very least langauge specific and even parser specific.  The current code completion plugin is general enough for any language.  Non OO languages can just have everything at global scope for now.

I don't think you can create a general AST that a UML/Refactoring program can rely on.  The UML/Refactoring program is going to have to be dependent on the langauge module (parser and AST).  Like you said it will probably be possible to create a general thread safe cache system that can store any kind of data you want.

Maybe if I have the time I will whip a UML diagram, in the end there should definetly be one made before the codeing starts.