Requirements / Guidelines for re-writing the Code Completion

[rickg22]

Requirements notation:

   This document occasionally uses terms that appear in capital letters.
   When the terms "MUST", "SHOULD", "MUST NOT", "SHOULD NOT", and "MAY"
   appear capitalized, they are being used to indicate particular
   requirements of this specification. A discussion of the meanings of
   these terms appears in RFC 2119.

For those who want to rewrite CodeCompletion using an existing C++ parser, here's what we should do:

(These guidelines are subject to change without notice)

• It SHOULD be designed around the class browser: Classes, Functions, Variables, Enums, Preprocessor.
• The code completion plugin MUST hold an in-memory structure (maps, arrays, whatever is necessary) to contain all the tokens. For example, a class would include member variables, methods, and the methods would include the local variables.
• This structure MUST be independent from the GUI. It MUST be a container. All code MUST be done by another class, but it MAY include saving, reading / writing from cache, and necessary functions.
• There SHOULD be a structure per development language in the project. It's no use mixing C++ and Perl tokens, it would only create confusion.
• The codecompletion plugin SHOULD have modules for different languages, these modules would consist of thread-safe functions to add the tokens into the main structure.
• The ParserThread (this part we'll keep, but with the necessary adaptations) will call the modules to parse the different files.
• For classes, use of STL classes (like vector and map) is RECOMMENDED, but you MAY NOT use std::string. You MUST use wxString instead. Use of only the std::string compatibility functions is RECOMMENDED.

(Contributions to these ideas are welcome)

[David Perfors]

Should it use wxWidgets functions? like wxString and wxArray, etc.?

[thomas]

I would use string and abstain from using wx containers to maximum extend possible.

There are three reasons for this:
1. wxString has std::string compatibility functions. If you use string, then you can do:

Code

#if USE_WXWIDGETS
  #include <wx/wx.h>
  #define string wxString
#else
  #include <string>
#endif

This will compile anywhere, with wx or without.

2. wx containers suck really bad. You cannot typedef them, you cannot declare them inside a class, they use obscure macros... Standard containers work really well.

Wait... did I say three?  :lol:

[Michael]

2. wx containers suck really bad. You cannot typedef them, you cannot declare them inside a class, they use obscure macros... Standard containers work really well.

That's interesting . Just a question. There are not problems by using standard containers with wxWidgets? Otherside, is not the use of std::string not adviced to be used with wxWidgets?

Michael

[takeshimiya]

The picture is:

wxWidgets containers sucks.
STL containers are great.

wxString is great.
STL string sucks (written by english people, for english people).

So I agree with thomas: using STL containers, and using wxString, but trying to use the STL string compatible functions.

[Michael]

The picture is:

wxWidgets containers sucks.
STL containers are great.

wxString is great.
STL string sucks (written by english people, for english people).

So I agree with thomas: using STL containers, and using wxString, but trying to use the STL string compatible functions.

STL Containers are good, that's right , but wxWidgets advices not to use STL. I have had some problems by using STL, wxWidgets and std:string . Therefore, I am not sure it would be good by mixing containers. Anyway, I would be happy if I could use STL with wxWidgets without any problems .

Michael

[David Perfors]

I thought the devs of wxWidgets are planning to fully support STL in some version of wxWidgets (wasn't it for 3.0...)
But I agree that we should not mix STL and wxWidgets as long as wxWidgets doesn't fully support this...

[takeshimiya]

Mm, what kind of problems you had? I mean, STL containers + wxString, instead of wxWidgets+std::string.

And anyways, wxWidgets 3.0 is planned to be release the next year, which will be very STL friendly.

wxWidgets currently advices not to use STL mainly because at the time of 2.0 release, the STL implementations were really bad, especially the MSVC6 one. But now the situation is better (less people using MSVC6 .

EDIT: mispunt won me

[Michael]

Mm, what kind of problems you had? I mean, STL containers + wxString, instead of wxWidgets+std::string.

I have had cast problems, problems with STL vectors and linking problems. With some work I solved the cast and the vector problems, but I still could not linking my application in relaese, because of wxWidgets link errors.

And anyways, wxWidgets 3.0 is planned to be release the next year, which will be very STL friendly.

That's would be really good .

wxWidgets currently advices not to use STL mainly because at the time of 2.0 release, the STL implementations were really bad, especially the MSVC6 one. But now the situation is better (less people using MSVC6 .

Yes, MSVC6 has a real bad STL implementation. Better is to use STLport . A lot of users solve their STL problems with MSVC6 by using it.


Concerning the use of templates or string class I would like to add that wxWidgets FAQ proposes a workaround if you want to use templates or the string class. But it does not help me by solving my problems .

There is nothing to stop an application using templates or the string class for its own purposes. With wxWidgets debugging options on, you may find you get errors when including STL headers. You can work around it either by switching off memory checking, or by adding this to a header before you include any STL files:

Code

#ifdef new
#undef new
#endif

Michael

[takeshimiya]

I think they redefine (by default) the new operator, for debugging purposes, which I think can be safely turned off.

[Michael]

I think they redefine (by default) the new operator, for debugging purposes, which I think can be safely turned off.

Yes, it seems so. But still have some doubts in using STL with wxWidgets.

Michael

[thomas]

I use STL a lot, because I hate wx containers. The blockallocator in Code::Blocks uses STL, the SVN plugin does, the Uservariable manager does... we compile tinyXML in STL mode, too (although we could do otherwise).  Rick uses STL in his new tree class, the AStyle plugin uses STL if I remember correctly...  just try and do a "find in files" on the project.

All you have to remember is to #undef new to undo the evil of wxWidgets   *shudders*

[Michael]

I use STL a lot, because I hate wx containers. The blockallocator in Code::Blocks uses STL, the SVN plugin does, the Uservariable manager does... we compile tinyXML in STL mode, too (although we could do otherwise).  Rick uses STL in his new tree class, the AStyle plugin uses STL if I remember correctly...  just try and do a "find in files" on the project.

That's is interesting. My doubts are vanishing . Probably, the problems I have had by using STL and wxWidgets have another source(s).

Michael

[thomas]

Here are another few:

Parsing MUST be done in a separate thread.
Several threads MAY be used.
The parser thread(s) MUST have lower priority than the main thread.
The parser MUST NOT do insane things from the main thread. Events MUST be handed to the worker thread and MUST be handled there.

These requirements address the fact that save all files currently freezes the application for half a minute if you have a dozen modified files open. This is because the parser walks through the tokens array one by one and removes one token at a time for every file saved (causing ~250.000 memcpy() calls on a memory block half the total size of the tokens array). Furthermore, due to thread synchronisation, execution resumes only after all files have been parsed.
It does not matter at all that the actual parsing runs in a separate thread, the GUI still freezes for a time which is clearly longer than necessary and longer than acceptable. This will not change, even if we  make parsing 500 times faster.

[rickg22]

<i>These requirements address the fact that save all files currently freezes the application for half a minute if you have a dozen modified files open. This is because the parser walks through the tokens array one by one and removes one token at a time for every file saved</i>

THAT was what caused the freezes? Oh man! I knew it was bad, but not THAT bad! :?

I'll try to figure out how to fix it.
Edit: I'll post about this on another thread.

[pass86]

Like VisualAssistXv10.

[rickg22]

I'm adding this link:

http://www.codeproject.com/showcase/VisualAssistX.asp

for reference, to see where we are and where we want to get.

[byo]

Out of STL vs wxWidgets debate:
Maybe it would be good idea to split current CC plugin into two plugins - one for actual code assisting and one for strict code parsing. If it will be done in this way, one can simply write parsing-only plugin to support extra language an thus allowing it inside assist window. And what's also nice about this - other plugins may use tree generated by parser plugin to do some magic - like CodeBrowser plugin (presented somewhere in this forum), becuase there's no use writing separate parser for each plugin that require some code analysis (and just imagine how much it could ease writing refactoring plugin).
Assist plugin could be developed separately giving better code assisting no matter what language is used.

What do you think ?

Regards
   BYO

[Der Meister]

Sounds great but I wonder if it would be easier and better to start from scratch than to split up the existing CC plugin. Because the CC plugin is already quite complex and I doubt that splitting it can easily be done without a lot of code-rewriting. Probably this will get worse if we try to give both separated plugins a good interface, but this would be needed. But this is just a guess, maybe someone who is more familier with the CC plugin than I am can say more about it

[pass86]

It needs to be running more quickly.

[patlecat]

If you want to improve CB, toward what's the best CodeCompletion systems today, then orient yourself instead on Visual Slick Edit (go to Cool Demos). Not on an outdated VC plugin!

I began a project this summer for my employer to reimplement a server written in Perl on a new machine in C++. Target machine is Solaris10-Sparc. So I thought: Great I can develop on Windows with CodeBlocks and just move the files over and compile them there. But apart from the lack of Makefile support the biggest downer for me was the very incomplete support for CodeCompletion that helps you with new Code and Libraries!
So, after 3 weeks of struggling with CB I gave up and bought the excellent Visual Slick Edit.

I have used VSE for some years and have not come across a more convenient Editor since, including VisualC 6-8 ! So I really think that you should let yourself be influenced by this editor as the top notch on the market. Because I would love to use CB for everything as soon as possible!

One key of success is the excellent Tagging-System in VSE which allows you to add new interfaces anytime!

[Raindog]

If you want to improve CB, toward what's the best CodeCompletion systems today, then orient yourself instead on Visual Slick Edit (go to Cool Demos). Not on an outdated VC plugin!

I began a project this summer for my employer to reimplement a server written in Perl on a new machine in C++. Target machine is Solaris10-Sparc. So I thought: Great I can develop on Windows with CodeBlocks and just move the files over and compile them there. But apart from the lack of Makefile support the biggest downer for me was the very incomplete support for CodeCompletion that helps you with new Code and Libraries!
So, after 3 weeks of struggling with CB I gave up and bought the excellent Visual Slick Edit.

I have used VSE for some years and have not come across a more convenient Editor since, including VisualC 6-8 ! So I really think that you should let yourself be influenced by this editor as the top notch on the market. Because I would love to use CB for everything as soon as possible!

One key of success is the excellent Tagging-System in VSE which allows you to add new interfaces anytime!

Well, if you define "superior" as not properly handing the most basic refactorings on code with templates (including renames), requiring "tag" files for code completion and not actual intelligent knowledge of the source, poor syntax highlighting and extremely outdated editor, then yeah, I would wholeheartedly suggest VSE over VAX.

VAX provides the following features:
* improved syntax highlighting.
* improved code completion. (as good as java or c# intellisense features).
* c/c++ refactoring.
* outline view of code that enables navigation to and re-arrangement of code from within the view.
* spell checking.
* quick templates.
* ability to find references to variables, functions, classes.
* supremely awesome code navigation features/shortcuts.

If anyone develops w/ Visual Studio and does not use VAX, you might as well code in notepad.

[patlecat]

You've got to be kidding me...

[AlekseyT]

May be better use sqlite (www.sqlite.or) for store current project information (parsed function, variable and so on). Back-end in background get all information and write it to database. Only back-end may write and delete information from sqlite database. Front-end(s) use database for reading. Front-end(s) can get information in one transaction (for get consistent information).

With this scheme developers can integrate other back-end (other parse engine) into CB. And plugin-developer can also easy get needed info from parser.

[stevenkaras]

May be better use sqlite (www.sqlite.or) for store current project information (parsed function, variable and so on). Back-end in background get all information and write it to database. Only back-end may write and delete information from sqlite database. Front-end(s) use database for reading. Front-end(s) can get information in one transaction (for get consistent information).

With this scheme developers can integrate other back-end (other parse engine) into CB. And plugin-developer can also easy get needed info from parser.

True. You should take a look at the wiki page, where you can find some more info on the C::C rewrite:
http://wiki.codeblocks.org/index.php?title=Code::Completion_Rewrite

[ollydbg]

Hi, everyone, I found another good IDE named Codelite which has functionality of code completion. I think we could learn from it. By the way, If would help to develop this plugin ( add something to support namespaces), how can I get involved?

It seems that some document? some guideline? and some plug in source code is need.

Thanks

[Ceniza]

Integrating CodeLite's parser into Code::Blocks has been discussed before, but no one has dared to do it. eranif (CodeLite's developer) was helping us to some extent, and still no one decided to step forward, take the challenge and do it. Just search the forums, and you will see.

[ollydbg]

Thanks.
Now, I have download the whole source code of codeblocks from svn. And I'm reading the code completion part. But it is difficult to read, because no document is supported ....

[cdavalillo]

You guys could use std::u32string which is based on char32_t and can represent any UTF-32 char including obviously any UTF-8 char. With this you could make the IDE support even all Chinese ideograms. You could convert this type of string to the normal std::string which is based on char. And the interaction with wxWidgets is not a problem because you can convert from and to std::string and from there to wxString, std::string is considered a binary buffer of wider characters representation when converted from std::u32string. Furthermore, as std::u32string is based on char32_t you could make arrays of char32_t and used them equally as std::u32string or even a std::vector<std::u32string>.