Suggestion: Using ctags & sqlite for code completion

[takeshimiya]

if you want to run it from the LiteEditor\bin directory, you have to copy the files
liteeditor.xml, sqlite3.dll and ctags.exe there too , it runs fine for me using wx263.dll

For the wx-config LiteEditor project I've set post-build steps:

Code

cmd /c copy /y ..\sdk\sqlite3\lib\msw_gcc\sqlite3.dll bin\msw_gcc\
cmd /c copy /y ..\sdk\ctags\bin\msw_gcc\ctags.exe bin\msw_gcc\
cmd /c copy /y liteeditor.xml bin\msw_gcc\

i would be interested if it builds and runs fine for you too

Everything works fine now in all configurations in both projects.

[tiwag]

Everything works fine now in all configurations in both projects.

[Blue-Tiger]

Hi there!
As C++ supports multiple inheritance it is possible that a class has more than 1 parent. Currently your implementation has a single "parent" column in your database. So if a class has more than superclasses, you'd have to store something like "ParentA, ParentB" in that column, or write 2 entries that only differ by their parent-value. This is usually considered bad database design (If you're interested might want to check out http://en.wikipedia.org/wiki/First_normal_form ), as it makes queries like "find me all the parents of class Foo" a bit akward. I don't know if this is an issue for you or not. Probably the whole thing would work just fine anyways. I just thought I let you know, because it might need a restructuration of your database to fix it. So it should better not be left until the end of the coding


P. S.I didn't like LiteEditor being distributed with an Installer, personally I like simple zip-archives more, because you can usually just delete & forget them once you're done with them ( + installation is much faster). This is just my personal opinion, though.

[eranif]

Just a quick update on this subject:
the library + a demo for how to use it, is ready with the following features:

With minimum work on the Editor side, the CodeLite library can provide the following:

- Built-in Symbol browser  (automatic update etc)
- Built-in parsing thread
- *very* fast parsing of sources (took me less then 30 secs to parse and build database with the whole gnu include directory, STL included)
- CodeCompletion works with STL, wxWidgets ect (ofcourse bug are there, just waiting to be discovered)
- WordCompletion
- Static database in addition to the workspace database, the static database contains information from unlikely to change headers files (for example, gcc include directory /usr/include, wxWidgets etc)
- Hover tips (assiting tooltips)
- Goto definition/declaration
- Comment parsing and can be displayed in the Hover toolip

Other features can be added on demand, with minimum work.

If any developer would like to create a plugin for C::B, drop me a message

the most updated source file can be retrieved here via SVN:
https://opensvn.csie.org/CodeLite
Latest revision (at the time of typing this message): 140

With regards,
Eran


Eran

[tiwag]

ctags is obviously not good enough to parse wxwidgets
and gets confused with the heavy usage of macros

when parsing .\CodeLiteEditor\LiteEditor\frame.cpp

it shows WXUNUSED(event) instead of
OnUseExternalDatabase(wxCommandEvent& WXUNUSED(event))

and all eventhandler functions are missing

screenshot

[takeshimiya]

ctags is obviously not good enough to parse wxwidgets
and gets confused with the heavy usage of macros

Yes I've noticed that about WXUNUSED.
Fortunately ctags haves a switch to specially handle identifiers like that.
See the -I identifier-list option in the manual: http://ctags.sourceforge.net/ctags.html

So we'll only need to have a user-configurable list of troublesome identifiers.
Ideal for libraries which haves heavy macro usage. Modern libraries with little-to-no-macro-usage will not need that.

Little example from the manual:

Code: cpp

CLASS Example
{
    // your content here
}; 

The example above uses "CLASS" as a preprocessor macro which expands to something different for each platform. For instance CLASS may be defined as "class __declspec(dllexport)" on Win32 platforms and simply "class" on UNIX. Normally, the absence of the C++ keyword "class" would cause the source file to be incorrectly parsed. Correct behavior can be restored by specifying -I CLASS=class.

[eranif]

ctags is obviously not good enough to parse wxwidgets
and gets confused with the heavy usage of macros

when parsing .\CodeLiteEditor\LiteEditor\frame.cpp

it shows WXUNUSED(event) instead of
OnUseExternalDatabase(wxCommandEvent& WXUNUSED(event))

and all eventhandler functions are missing

they are missing in the GUI tree, but they should appear in the CodeCompletion part, if not then it is a bug inside CodeLite and not within ctags.

I chose not to display the parent(s) functions members only because it adds more complication to the GUI tree, which if a plugin will be implemented, and a requirment will arise, I will add it.

Like Takeshi mentioned, a dialog with a list of pre-processor arguments can be added and be passed to ctags (ctags command line are stored inside a map, and can be changed) to resolve such issues.

Another problem, one can think of, is that in the GNU header files (specially the STL implementation), the vector class is located inside namespace _GNUCXX_STD (or something similar) while map is inside std, so typing std:: (obvioulsly one is a define of the other) will yield a completion box with alot of members but vector is not one of them ...) Takeshis' solutuion will solve this one too.

There are probabaly many other bugs outhere, but I think that it is in a good position that people can work with.

Eran

[Pecan]

I cannot compile "Debug wxdll unicode" project under XP CB SVN 3087.

Even though readtags.cpp and readtags.h are checked to be compiled under the ctags target, they are never loaded and compiled. Note that compilation jumps from read.c to rexx.c skipping readtags.cpp

Code

-------------- Build: ctags in Debug wxdll unicode ---------------
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\args.c -o .objs\msw_wxdll_ud\sdk\ctags\args.o
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\asm.c -o .objs\msw_wxdll_ud\sdk\ctags\asm.o
...<snip>...
-IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\python.c -o .objs\msw_wxdll_ud\sdk\ctags\python.o
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\read.c -o .objs\msw_wxdll_ud\sdk\ctags\read.o
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\rexx.c -o .objs\msw_wxdll_ud\sdk\ctags\rexx.o
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\routines.c -o .objs\msw_wxdll_ud\sdk\ctags\routines.o
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\ruby.c -o .objs\msw_wxdll_ud\sdk\ctags\ruby.o
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\scheme.c -o .objs\msw_wxdll_ud\sdk\ctags\scheme.o
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\sh.c -o .objs\msw_wxdll_ud\sdk\ctags\sh.o
mingw32-gcc.exe -Wall -g -pipe -fmessage-length=0 -mthreads -Winvalid-pch -DHAVE_W32API_H -D__WXMSW__ -DWXUSINGDLL -DwxUSE_UNICODE  -I.objs\msw_wxdll_ud\CodeLite -I- -I.objs\msw_wxdll_ud\CodeLite -I. -IC:\Usr\Proj\wxWidgets263\include -IC:\Usr\Proj\wxWidgets263\lib\gcc_dll\mswu -Isdk\ctags -IC:\MinGW\include -Isdk\ctags -IC:\Usr\Proj\cbCodeLite\ -c sdk\ctags\slang.c -o .objs\msw_wxdll_ud\sdk\ctags\slang.o
...<snip>...
.objs\msw_wxdll_ud\sdk\ctags\sql.o .objs\msw_wxdll_ud\sdk\ctags\strlist.o .objs\msw_wxdll_ud\sdk\ctags\tcl.o .objs\msw_wxdll_ud\sdk\ctags\verilog.o .objs\msw_wxdll_ud\sdk\ctags\vim.o .objs\msw_wxdll_ud\sdk\ctags\vstring.o .objs\msw_wxdll_ud\sdk\ctags\yacc.o    -lwxmsw26u 
mingw32-g++.exe: .objs\msw_wxdll_ud\sdk\ctags\readtags.o: No such file or directory
Process terminated with status 1 (0 minutes, 3 seconds)
0 errors, 0 warnings 

[eranif]

MM, maybe I am not udnerstanding something here, but:

Why does readtags.cpp/h are under ctags target? they should be under CodeLite.

They are the interface the tags files (although I developed my own API for it, they still there just incase)

Eran

[Pecan]

MM, maybe I am not udnerstanding something here, but:

Why does readtags.cpp/h are under ctags target? they should be under CodeLite.

They are the interface the tags files (although I developed my own API for it, they still there just incase)

Eran

You are correct. I unchecked readtags.cpp/h from the ctags target and it compiled and is running just fine.

[Pecan]

 The tree label is missing the "wxke"

[Pecan]

For my education:

How is this program any better than the symbol tree and function lister box already included in CB?

Who would use it (as opposed to those already using the current plugin) ?

Does it provide a means for code completion?

(Yes. I've read the whole thread twice, but I'm still wondering if it's worth the time to convert this to a plugin).

Is anyone else working to turn this into a plugin?

Even if it was turned into a plugin, would it ever replace the current tree,function box, code completion?

[eranif]

I dont think it will replace to current tree.
The tree is not a must, u can still use the current tree, (unless the current implementation cant be removed from the current CodeCompletion).

How is it better?

Stability - I dont think this will rarely cause C::B to freeze/hangs/crash since it uses well matured product (ctags) to perform its parsing tasks.
Speed - compare parsing of huge files with this implementation and the current one (I havnt got to do it myself, but I got feedback from people)
Comment parsing - I dont think that the current implementation supports comments parsing, this implementation does (and acheived with minimum effort)
Local variable parsing

this implementation is a suggestion, as stated in the topic and not a must, I cant point out who will use it, maybe it will swift the croud and maybe it will not have even a single user, BUT I still think an alternative to the current implementation should exist.

I myself, didnt invest too much time in invistigating the reasons why people are not happy with the current implementation, (half of the solution to C::B hangs/freezs/crushes are caused because of the current CodeCompletion "disable codecompletion").

Eran

[tiwag]

MM, maybe I am not udnerstanding something here, but:
Why does readtags.cpp/h are under ctags target? they should be under CodeLite.
They are the interface the tags files (although I developed my own API for it, they still there just incase)
Eran

You are correct. I unchecked readtags.cpp/h from the ctags target and it compiled and is running just fine.

the problem is, that the source file, which belongs to ctags target, is named readtags.c  (instead of cpp)
i don't understand why it built without errors in the past  :shock: (can't find anything related to this in the svn log)
after changing the filename in the projectfile it builds fine again.

sorry for the inconvenience, fixed in svn rev142

brgds, tiwag

[joubertdj]

Are you guys busy with a plugin to get ctags & sqlite up and running in code::blocks?
If so do you perhaps have a Nightly which I can test?