Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: Alpha on January 11, 2014, 01:45:53 am

Title: Clang based CC, new CC interface
Post by: Alpha on January 11, 2014, 01:45:53 am
I made an attempt to just get clang CC running with the working state of the CC plugin interface redesign.  Code attached.  (This is purely for demonstration, as it currently runs synchronously and does not cache results, so usability is low.)
Only tested under linux, some search directories are hard coded (and may need to be changed), the project directory expects to be extracted into src/plugins/, it may be necessary to disable the main CC plugin to see any results (CCManager will only query one of them because both claim to support the same files).

(For some reason, it appears clang_codeCompleteAt() is simply dumping all tokens that it knows, instead of using any sort of heuristic search/namespace resolution.  I do not know if that is error on my part, or if that is the expected output...)
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 12, 2014, 02:22:36 am
(For some reason, it appears clang_codeCompleteAt() is simply dumping all tokens that it knows, instead of using any sort of heuristic search/namespace resolution.  I do not know if that is error on my part, or if that is the expected output...)
Oops.  User error on my part.  The location I passed for line/column was 0-index based, however, clang was expecting 1-index based.

(I will start a git repository for this plugin soon.)
Title: Re: Clang based CC, new CC interface
Post by: dmoore on January 12, 2014, 02:39:54 am
Cool! I will give it a try when it makes it to git. Never understood how clang's CC works in terms of finding symbols outside of local file scope. Presumably it already knows how to find std lib stuff, but what about something like wxwidgets?

Btw, when do we get the merge of your cc branch into trunk?
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 12, 2014, 11:12:20 pm
Repository is up.
Code
git clone https://github.com/alpha0010/ClangLib.git
https://github.com/alpha0010/ClangLib (https://github.com/alpha0010/ClangLib)
Please copy the attached image into src/devel/share/codeblocks/images/codecompletion/ before testing.

Btw, when do we get the merge of your cc branch into trunk?
From what I can tell, it runs as stable as the trunk (I am using the cc_interface branch as my main code editor).  Part of my purpose with testing this Clang plugin is to see if any obvious changes must be made to the interface.
My thoughts are, merge the branch once the interface works without losing any features from the main CC plugin (done), tests from this Clang plugin (in progress...), and the FortranProject (not yet looked at).  If we merge before these are complete, the interface would be likely to change soon after (defeating the purpose of working in a branch).
That said, my ideal target is one/two months (which depends a lot on how easy porting the FortranProject turns out to be).

Never understood how clang's CC works in terms of finding symbols outside of local file scope.
The way it is set up, Clang essentially compiles the file (into memory), so all of the #includes are resolved and expanded.  Searches are done per translation unit, not per file (which does mean I had to do a bit of file trickery in this plugin).
Title: Re: Clang based CC, new CC interface
Post by: dmoore on January 15, 2014, 11:03:14 pm
Repository is up.
Code
git clone https://github.com/alpha0010/ClangLib.git
https://github.com/alpha0010/ClangLib (https://github.com/alpha0010/ClangLib)

Ubuntu 13.10's clang libs seems to be missing clang-c/Index.h, so I can't build it. Anyone know of a workaround? (Installing clang from source not an option in the near term)

Quote
Please copy the attached image into src/devel/share/codeblocks/images/codecompletion/ before testing.

Why don't you just deploy this file with a post build step in the project settings?

That said, my ideal target is one/two months (which depends a lot on how easy porting the FortranProject turns out to be).

If my python experience is a guide, this should be relatively straightforward. Your code eliminates a lot of the fiddling with cbEditor/wxScintilla.

Has anyone contacted the fortran maintainer? (I think he has the username darmar)
Title: Re: Clang based CC, new CC interface
Post by: ollydbg on January 15, 2014, 11:13:06 pm
If I remember correct, index.h and other Clang shared libraries can be copied from CodeLite IDE's code repo.
Title: Re: Clang based CC, new CC interface
Post by: oBFusCATed on January 16, 2014, 01:22:39 am
dmoore: Have you tried to install clang-dev or -devel?
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 16, 2014, 01:38:35 am
Ubuntu 13.10's clang libs seems to be missing clang-c/Index.h, so I can't build it. Anyone know of a workaround? (Installing clang from source not an option in the near term)
The package libclang-3.4-dev has it (libclang-dev also works, but why not use the newer version since it is available?).  (Note that in the project file, the search path for it is currently hard coded, so you may have to adapt it to your computer.)

Why don't you just deploy this file with a post build step in the project settings?
I was planing to have CCManager provide a default set of icons, so I would end up deleting this from the Clang project later... Not really a valid excuse :) .  Just lazy.

Has anyone contacted the fortran maintainer? (I think he has the username darmar)
I have not yet; I will send a PM now.
Title: Re: Clang based CC, new CC interface
Post by: dmoore on January 16, 2014, 02:05:37 am
dmoore: Have you tried to install clang-dev or -devel?

Yes. No sign of the needed headers. I will try installing 3.4...
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 16, 2014, 02:16:41 am
Yes. No sign of the needed headers. I will try installing 3.4...
Hmm... I found them under /usr/lib/llvm-3.2/include and /usr/lib/llvm-3.4/include respectively.
Title: Re: Clang based CC, new CC interface
Post by: dmoore on January 16, 2014, 03:37:09 am
Yes. No sign of the needed headers. I will try installing 3.4...
Hmm... I found them under /usr/lib/llvm-3.2/include and /usr/lib/llvm-3.4/include respectively.

Ohh... completely obvious that they would put it there  ::) and who needs pkg-config files these days.  :-\
Title: Re: Clang based CC, new CC interface
Post by: dmoore on January 16, 2014, 04:33:25 am
Anyway, I should have looked more closely at your project file and I would have figured this out. doh!

This works really well and as far as I can tell is every bit as quick as our CC plugin. I didn't really do much testing but it seemed to be able to complete most things I tried. (Some things seemingly took a bit of time to populate in whatever DB clang uses so didn't come up first time)
Title: Re: Clang based CC, new CC interface
Post by: MortenMacFly on January 16, 2014, 09:28:43 am
...did you try that on Windows already?
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 16, 2014, 06:28:04 pm
I have it running on Windows now as well.  I will commit the project file once I am done cleaning it up.
Title: Re: Clang based CC, new CC interface
Post by: thomas on January 16, 2014, 07:51:59 pm
You got a working Clang on Windows? How so?
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 16, 2014, 10:26:17 pm
You got a working Clang on Windows? How so?
I downloaded Clang for Windows (http://llvm.org/releases/3.4/LLVM-3.4-win32.exe) form here (http://llvm.org/releases/download.html#3.4), installed it, added the proper paths to the project file, and it just worked (which did actually surprise me).  Windows .cbp is now committed.
Title: Re: Clang based CC, new CC interface
Post by: thomas on January 17, 2014, 11:33:48 am
Oh wow... last time I checked, that page had something "Windows support is troublesome, but we have a 3.2 experimental build for MSVC only". I had tried their "build from source" instructions (using MinGW, but that failed miserably).

Great, I'll have a look, thank you :)
Title: Re: Clang based CC, new CC interface
Post by: thomas on January 17, 2014, 11:42:15 am
That doesn't seem to have anything like WinAPI headers (or libs) or even C++SL, need to copy those over from MinGW?
Title: Re: Clang based CC, new CC interface
Post by: killerbot on January 17, 2014, 01:10:33 pm
as for linux, distros have clang, but for a modern C++ developer  :P , you want to use C++11, and then the problem starts, you need their new libc++ library implementation.
As far as i know still hard or impossible to have that on linux, unless someones can tell me otherwise (an the how ) ...
Title: Re: Clang based CC, new CC interface
Post by: MortenMacFly on January 17, 2014, 01:50:37 pm
That doesn't seem to have anything like WinAPI headers (or libs) or even C++SL, need to copy those over from MinGW?
I've seen that, too - in fact I am monitoring this site. The thing is, you can and have to adjust the compiler config to make use of another compiler that provides those API. If I am not mistaken, this works well with that release for VC, but I doubt it will work reliable for MinGW. What I did was compiling clang itself using MinGW (based on Makefiles created for Code::Blocks using cmake) which ensure ABI compatibility with that compiler. Then you can adjust the compiler settings so they point to exactly this MinGW distribution - and have to add very low-level include folders so it actually FINDS the references. This depends on the standard used, so its really fiddling with settings.

Well - that's how far I came. Compiling a very basic app works, but I am far from compiling anything else than that. At linking time you'll see that (i.e.) wxWidgets needs better to be compiled with clang before, too. And so on...

Still a never-ending story. I'm afraid its still no prime time on Windows, yet. :-(
Title: Re: Clang based CC, new CC interface
Post by: thomas on January 17, 2014, 03:56:28 pm
The way this download page looks, there should actually be C++ standard library somewhere. It starts with "sources", and among these is LibC++, which I assume is CLang's equivalent of glibc++. But I can't find it anywhere in the installed compiler. Be nice if they had something like "install instructions", hehehe...

So basically CLang seems to work, as long as you don't try anything like #include <algorithm> or #include <vector>. It compiles and links int main() { return 0; } lightning fast, too  :P. If only it worked for anything else as well...

Well, maybe in 3-4 months. They do seem to be working on getting something usable together.

Quote
What I did was compiling clang itself using MinGW
Sadly, wasting days and days on compiling compilers to no avail is something I can't afford any more :(
For me, it really has to work by unpacking a .tar.gz or a .zip file, or by double-clicking an installer.

(Or, in the most extreme case, a script that I need to run unattended (and leave the computer on over night) to do a fully automatic build. But that one had better work without guessing and trying afterwards.)
Title: Re: Clang based CC, new CC interface
Post by: MortenMacFly on January 17, 2014, 04:52:44 pm
So basically CLang seems to work, as long as you don't try anything like #include <algorithm> or #include <vector>. It compiles and links int main() { return 0; } lightning fast, too  :P. If only it worked for anything else as well...
That basically summarises my experience except hat I provided a C library through MinGW but the linker complained anyways.

Well, maybe in 3-4 months. They do seem to be working on getting something usable together.
I am waiting ~1,5 years now.

Sadly, wasting days and days on compiling compilers to no avail is something I can't afford any more :(
I never compiled compilers before at all, but this one is rather easy, as configuring, adjusting and start compiling takes ~10 minutes and uses Code::Blocks which works on Windows. Otherwise I wouldn't have wasted my time.
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 17, 2014, 05:15:45 pm
I downloaded Clang for Windows (http://llvm.org/releases/3.4/LLVM-3.4-win32.exe) form here (http://llvm.org/releases/download.html#3.4), installed it, added the proper paths to the project file, and it just worked (which did actually surprise me).
Oh, sorry, I should have specified.  By 'worked' I was referring to linking with this plugin, and providing access to its AST and CC.

When initializing a unit with libclang, it takes flags as if compiling through the console.  As part of the flags, my plugin passes it the search paths of MinGW's headers.  Because that seems to have worked in the plugin, maybe it will work for compiling as well... I will do some tests.
Title: Re: Clang based CC, new CC interface
Post by: Ivan171 on January 18, 2014, 09:26:52 am
Hi, i'm new to the forum. First of all english is not my native language, so, bear with me.

Well, my experience with clang is not that bad. Everything i've tried to compile with it up to now, have worked pretty well (Although i use it mostly for testing purposes at the moment). Including C++, which i see a lot of people saying it doesn't work on Windows. It does work. At the moment, Clang from svn, is able to bootstrap on Windows (using Mingw 4.7+).

But it have some issues, like, dllimport/dllexport does not work for classes. It doesn't support Win64 EH (There's no exceptions at all on Windows 64). This issues is being worked on by the way.

Clang 3.4 does not work with Mingw 4.7+ due to ABI incompatibilities.
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 29, 2014, 05:46:20 am
A few glamour shots of this plugin.

Operator resolution (also right click, jump to declaration works).
(http://www.pasteall.org/pic/show.php?id=65943) (http://www.pasteall.org/pic/show.php?id=65943)

Syntax highlighting in documentation popups.
(http://www.pasteall.org/pic/show.php?id=65944) (http://www.pasteall.org/pic/show.php?id=65944)

Dynamic checking of code validity.
(http://www.pasteall.org/pic/show.php?id=65945) (http://www.pasteall.org/pic/show.php?id=65945)
Title: Re: Clang based CC, new CC interface
Post by: dmoore on January 29, 2014, 04:33:40 pm
Looks terrific (btw, you can also use the README.md or README.rst to embed screen shots  on the github page using a dir in your repo to place the images. Nice feature of github IMO)

I am sure I could figure out myself, but...

* Which docstring formats are you able to highlight code for? (What tool are you using to do the highlighting?)

* Is the dynamic check highlighting a scintilla feature (can they be hidden by clicking on them or something?) Are you making hooks in the CC manager to make this easy for all plugins to do? (Or is it easy to do without hooks). I want to do something similar for python.
Title: Re: Clang based CC, new CC interface
Post by: Folco on January 29, 2014, 05:51:44 pm
Splendid work Alpha !
Title: Re: Clang based CC, new CC interface
Post by: oBFusCATed on January 29, 2014, 09:52:51 pm
The last image is interesting. Can you try to explain how it works and what kind of API do you use?
Title: Re: Clang based CC, new CC interface
Post by: dmoore on January 29, 2014, 10:12:42 pm
The last image is interesting. Can you try to explain how it works and what kind of API do you use?

Looks like just an editor hook, then uses wxScintilla annotations:

See, esp. DiagnoseEd, in https://github.com/alpha0010/ClangLib/commit/6a0c09cf1cd39448d6fa6d022afd9f3a0bed6ebb#diff-2c47c86d730d3c95d68ec3fa39ad7e7bR719

Would be nice if some of this moved out of the plugin and into the SDK, but I haven't thought about how practical that is.
Title: Re: Clang based CC, new CC interface
Post by: oBFusCATed on January 30, 2014, 12:16:16 am
100% should be part of the SDK...
Does clang provide both CC info + errors/warnings info with a single compile/parse?
Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 30, 2014, 06:10:55 am
* Which docstring formats are you able to highlight code for? (What tool are you using to do the highlighting?)
Maybe it would have been better to try to use a hidden Scintilla control, but to simplify things, I wrote a pseudo-lexer (https://github.com/alpha0010/ClangLib/blob/master/clangproxy.cpp#L335).

The last image is interesting. Can you try to explain how it works and what kind of API do you use?
Looks like just an editor hook, then uses wxScintilla annotations:
See, esp. DiagnoseEd, in https://github.com/alpha0010/ClangLib/commit/6a0c09cf1cd39448d6fa6d022afd9f3a0bed6ebb#diff-2c47c86d730d3c95d68ec3fa39ad7e7bR719
Yes, that is it.
100% should be part of the SDK...
Hmm... I will see if I can figure out a good way to do that.
Does clang provide both CC info + errors/warnings info with a single compile/parse?
Essentially yes.  Also, when working on the root source file of a translation unit (the .cpp), each update after the first parse is significantly faster because headers included at the top have their processed version cached.

With reference to performance, while working in a single editor, this plugin works well.  However, the initialization of each new editor (a source file, not a header) can take 2-5 seconds under Linux, and upwards of 20-30 seconds with Windows (at some point, I will switch this to run in a background thread).  Also, because libclang is organized by translation units, each of these takes 50-150 MB memory.  (Of note, in this plugin, some of the files include Code::Blocks' PCH, and some do not.  The files that do not use it are nearly instantaneous to initialize.)
From my testing, Clang's comment parser seems to have some issues under Linux, and on Windows, often completely fails to return anything (although, that might be error on my part, I am not sure).

(btw, you can also use the README.md or README.rst to embed screen shots  on the github page using a dir in your repo to place the images. Nice feature of github IMO)
Thanks, I will look into that.
Title: Re: Clang based CC, new CC interface
Post by: thomas on February 07, 2014, 10:38:36 pm
So, although I should have known better, I wasted the day trying to get Clang-SVN under Windows working. Turned out it is the typical Unix-like open source shit that you expect. Why does everything that you get for free suck so much?

Install prerequisites, configure, and make. Yeah right. Python is a prerequisite for running the test suite. Of course you don't want to run the test suite, but make will fail with "python version > 2.5 required"  if you don't have Python installed. So you install Python version 3.3, and guess what, now make fails with "invalid syntax".
Searching the web for the exact error gives a solution: "Yeah, print is a function in python 3.x now, you will need the latest 2.x version".

Great. Fuck you.

Using vanilla MinGW or MinGW-TDM, configure works fine, but the build fails after 3-4 seconds because off_t is not defined (which is, of course, total bollocks).

So you install Python 2.7 and a MinGW-w64 based compiler, and behold, it configures and builds.

In fact, it builds every darn architecture that LLVM supports, which only takes like 2 hours. So you reconfigure with --enable-targets. No matter what you choose, you get a "cannot determine triplet" error. Fuck you, again.

So you reconfigure again, and rebuild all that shit. The build process has no way of running without the test suite. Anything but "all" fails, and "all" runs the test suite. Further, make hangs when you call it with -j, great. Single-threaded builds in 2014. Fuck you, again.
Surprisingly, it indeed -- finally -- generates a working 460MB compiler. Except there are no C++ or C++11 headers.

So you reconfigure for C++11 support (which you have to specify explicitly!), and guess what. It fails because i386 is not supported under 10.4 and under OSX. The fuck, what?

I'm building a Windows compiler under Windows, using a Windows compiler, targetting Windows. What do I care if i386 is supported under OSX?

EDIT:
Interesting. Looking at the SVN log, I find an entry from earlier today "Try to unbreak MinGW build". So apparently this is a known problem. Well, I'll give it another try next weekend.
Title: Re: Clang based CC, new CC interface
Post by: killerbot on February 07, 2014, 11:24:47 pm
I feel your pain Thomas, I have suffered these things too.
And what is really frustrating these clang guys go shouting all over the world how nice their compiler is a, how much better the gcc and so on. Maybe that is true, but actually as long as they don't make it usable, the only thing they have created is a big piece of shit. I really hope that one day some guy over there really uses his brain instead of their self proclaimed "oh so good compiler"  story.

But in the meantime they ship OS-X with even super old gcc, avoiding people to use gcc on their lovely overpriced mac books.
Title: Re: Clang based CC, new CC interface
Post by: oBFusCATed on February 07, 2014, 11:40:16 pm
Thomas:
1. Have you reported this problems to clang devs, no point shouting here:)
2. Have you tried the binaries, I think they build them everyday

Killerbot:
You can use newer gcc using macports as far as I know, but they are not 100% compatible :)
Title: Re: Clang based CC, new CC interface
Post by: Ceniza on February 08, 2014, 09:25:38 pm
Just for the record.

The reason why Apple stopped shipping and using newer versions of GCC is because of their current license (GPL3), same reason why they had to drop samba and roll out their own substitute.

As for LLVM and Clang, their main platform is OS X (more like BSD in general, since FreeBSD also decided to drop GCC and move to Clang/LLVM). Linux, being related to Unix and BSD, can, of course, more easily get Clang/LLVM working. Windows, as usual, is the outsider (however popular) when it comes to standards (what Microsoft managed to implement of POSIX is, at its best, laughable).

What I find kind of ironic is the eagerness to bash on Clang/LLVM for its poor support for Windows, when Code::Blocks itself (and wxWidgets for that matter) is in a rather sorry state when it comes to OS X support. The interesting thing is that both projects share the same reason for it: lack of community support and contributions for a specific platform.

By the way, I thought Eranif managed to get Clang working for his code-completion in CodeLite years ago, or am I getting confused? If so, he may be the best guy to ask about it.
Title: Re: Clang based CC, new CC interface
Post by: thomas on February 09, 2014, 01:28:28 pm
Have you tried the binaries, I think they build them everyday
Their builds don't have WinAPI, nor C++SL. That be the reason why I'd waste hours trying to do a build myself.

When you do your own build, you can either use CLang's libc++ which is a sooooo much better alternative to GCC's, or have the build process incorporate the existing host standard library from your.

Except of course, neither one works.

Telling configure to use CLang's lib results in an immediate abort (because it passes some unknown commandline option to GCC), and using the host's library "works" except it later turns out that the compiler doesn't have it. Now the obvious solution would be to first bootstrap a half-working CLang without working standard library (which nevertheless understands that command line switch!), and then build a fully functional version from that. Except of course you can't build CLang without a working standard library...

Quote
Have you reported this problems to clang devs, no point shouting here:)
To what avail? Those things must be bloody evident to them. Surely I'm not the first person in the world writing a program that contains #include <algorithm>. Surely I'm not the first to try this with CLang either. They must be aware that it doesn't work.
Title: Re: Clang based CC, new CC interface
Post by: thomas on February 09, 2014, 01:56:06 pm
As for LLVM and Clang, their main platform is OS X (more like BSD in general, since FreeBSD also decided to drop GCC and move to Clang/LLVM). Linux, being related to Unix and BSD, can, of course, more easily get Clang/LLVM working. Windows, as usual, is the outsider (however popular) when it comes to standards (what Microsoft managed to implement of POSIX is, at its best, laughable).
This is not Microsoft's fault, however. It is the typical BSD/Linux/Unix shit. Programs must be super portable to run on every system that is 67 years old, with a processor that you can only find in a museum. And configure does that, isn't it just great. Except the software fails to build on the most mainstream system because the software is not written properly, but using that configure crap.
This isn't a symptom particular to CLang. It's the same with GCC and its prerequisites as well. If you try to build GMP, you get this configure error about some typedef not having the correct size. If you just comment out the stupefied line that assumes sizeof(long) == 8, everything works perfect. Now, thing is, without this "super smart portable" shit, it would work much better.

Quote
What I find kind of ironic is the eagerness to bash on Clang/LLVM for its poor support for Windows, when Code::Blocks itself (and wxWidgets for that matter) is in a rather sorry state when it comes to OS X support. The interesting thing is that both projects share the same reason for it: lack of community support and contributions for a specific platform.
There are some notable differences, though:

Quote
By the way, I thought Eranif managed to get Clang working for his code-completion in CodeLite years ago, or am I getting confused? If so, he may be the best guy to ask about it.
I have Clang working as far as that's concerned too. Reading closely helps understanding the issue. Getting the support lib to compile (or the whole compiler suite) isn't the issue. The problem is getting a compiler that works with C++ (or Windows).
I have a perfectly working compiler for 10 or 12 different architectures with perfectly working tools and support libs. Except no single C++ header or WinAPI header (or lib) is included, nor is there a way (at least not documented) of configuring it so it will look for C++ headers in some other system path and compiling C++ will work.
Title: Re: Clang based CC, new CC interface
Post by: thomas on February 09, 2014, 09:36:57 pm
In the mean time, it seems like I've managed to do a build of an almost-working C++ compiler, which borrows the missing headers from an existing MinGW-w64 installation ("borrows" means that it looks in the other compiler's install directory, it remains to be seen how ABI compatibility will play with libs).

At least a minimum compileable C++11 example seems to compile fine. Including <algorithm> and <memory> works as well as including <windows>, further creating a unique_ptr<int> works.

On the downside, CLang fails executing its unit test suite on all std::mutex test cases. Which basically means it's a kind-of working C++11 compiler without thread support (depending on another install). Still not great, but at least getting close to something that works.
Title: Re: Clang based CC, new CC interface
Post by: ollydbg on February 19, 2014, 03:42:43 am
See what other Open Source/Free IDE's plan on Clang:

1, Qt Creator and Clang | Qt Blog (http://blog.qt.digia.com/blog/2011/10/19/qt-creator-and-clang/), I notice that clang can even support multiply PCHs, not the single one as GCC.

2, [Qt-creator] Future of Qt Creator's C++ parsing (http://lists.qt-project.org/pipermail/qt-creator/2012-February/000251.html) after their discussions [Qt-creator] QtCreator C++ code parser and clang integration status (http://lists.qt-project.org/pipermail/qt-creator/2012-February/000248.html).

3, Kate/KDevelop Sprint 2014: Let There Be Clang | Milian Wolff (http://milianw.de/blog/katekdevelop-sprint-2014-let-there-be-clang)

4, CodeLite (http://codelite.org/)already integrate Clang code completion.

5, a Clang based on-line code browser, see an example: Parser.cpp [llvm/tools/clang/lib/Parse/Parser.cpp] - Woboq Code Browser (http://code.woboq.org/userspace/llvm/tools/clang/lib/Parse/Parser.cpp.html#204), hover the mouse on the function names, you will see every thing, it is free for non-commerical project.

6, (new added), Clang Developers - Clang-based indexer and code navigator (http://clang-developers.42468.n3.nabble.com/Clang-based-indexer-and-code-navigator-td4030906.html), it's demo can be seen SourceWeb (http://rprichard.github.io/sourceweb/)


PS: Roberto Raggi, the author of the C++ parser in QtCreator and Kdevelop are now developing another parser from scratch, see: cplusplus (https://github.com/robertoraggi/cplusplus), he try to create a new tool as alternative of clang? (he posted here in Cool stuff! about the Qt (http://milianw.de/blog/katekdevelop-sprint-2014-let-there-be-clang#comment-1583))
Title: Re: Clang based CC, new CC interface
Post by: Meiner on February 28, 2014, 12:00:47 pm
I'm very happy that there is some progress in the development of a new clang-based CC-Plugin, thanks for that.

I tried to compile the source code yesterday, but I get some weired errors. Basically classes like wxStringVec, CCToken, CCProviderStatus and CCCallTip seem not to be in scope (include paths, etc. are pointing to the right locations from my point of view). I can't figure out in which files these classes are originally located, so I'm not able to fix this error at the moment. I'd appreciate some hints what I'm doing wrong.
Title: Re: Clang based CC, new CC interface
Post by: Alpha on February 28, 2014, 04:46:46 pm
[...] Basically classes like wxStringVec, CCToken, CCProviderStatus and CCCallTip seem not to be in scope [...]
It sounds like you are compiling against Code::Blocks trunk, which will not work.  I am developing this plugin against the cc_interface branch (https://github.com/alpha0010/codeblocks_sf/tree/cc_interface) (details here (http://forums.codeblocks.org/index.php/topic,18114.0.html)).
Title: Re: Clang based CC, new CC interface
Post by: scarphin on February 28, 2014, 06:03:43 pm
Do you plan on releasing some nightly or binary or something off this branch? That looks cool and I really like to test it! Btw I tried building myself without success because of reasons I have no idea. ;)
Title: Re: Clang based CC, new CC interface
Post by: Alpha on February 28, 2014, 07:44:58 pm
An 'official' nightly of the cc_interface branch will likely come soon (this is in internal discussion).  Once that is out, I plan to try packaging a binary of this plugin.
Title: Re: Clang based CC, new CC interface
Post by: Folco on February 28, 2014, 08:46:46 pm
Alpha, you are doing a great job with that branch, I'm very impatient to ues CB with the code completion provided by clang. Keep the good work, and many thanks !
Title: Re: Clang based CC, new CC interface
Post by: Meiner on March 01, 2014, 12:01:47 pm
[...] Basically classes like wxStringVec, CCToken, CCProviderStatus and CCCallTip seem not to be in scope [...]
It sounds like you are compiling against Code::Blocks trunk, which will not work.  I am developing this plugin against the cc_interface branch (https://github.com/alpha0010/codeblocks_sf/tree/cc_interface) (details here (http://forums.codeblocks.org/index.php/topic,18114.0.html)).

Yes, absolutely right, I didn't see the CC-Branch. I'll try to compile it once again, but I'm excited to hear that there may be an (official/inofficial) release in the near future.


EDIT: I have one further question about the build process. I managed it to build the cc-branch and the clanglib plugin against the cc-branch's source code, but starting (compiled) CB leads to the error message: "One or more plugin could not be loaded........clanglib.dll", because of missing symbols. Is there something more I have to do?
Title: Re: Clang based CC, new CC interface
Post by: Alpha on March 02, 2014, 02:01:07 am
EDIT: I have one further question about the build process. I managed it to build the cc-branch and the clanglib plugin against the cc-branch's source code, but starting (compiled) CB leads to the error message: "One or more plugin could not be loaded........clanglib.dll", because of missing symbols. Is there something more I have to do?
From your LLVM/Clang installation, is 'libclang.dll' (some_path\LLVM\bin\libclang.dll) in the executable's searchable path?
Title: Re: Clang based CC, new CC interface
Post by: Meiner on March 02, 2014, 09:29:41 pm
Yeah, that was it, stupid mistake of mine. Now I have a working build of CB which includes your plugin. It's so nice for me to see that there will be a working code completion for oglplus (C++ OpenGL wrapper), which seems to be a true nightmare for most existing code completion systems.

If you want to have some feedback:

1. In my case there is a huge delay when loading a project (first parsing takes place) and when changing the current active file. Is that a normal behaviour at the current development status?

2. I like the dynamic checking of code validity, it's very nice integrated, although with some code it's rather confusing (a lot of warnings^^).


Meiner
Title: Re: Clang based CC, new CC interface
Post by: Alpha on March 03, 2014, 09:25:49 pm
If you want to have some feedback:
Feedback is always welcome, thank you.  (Though, life is very busy for me right now, so no promises on how soon I will be able to address anything.)

1. In my case there is a huge delay when loading a project (first parsing takes place) and when changing the current active file. Is that a normal behaviour at the current development status?
Known problem.  Currently, my development on this plugin has been mainly to try to explore the extent of possible functionality that clang provides.  I have ignored performance... though, once I try to bring this plugin to more 'daily' use, the next steps will be to increase caching of results, and to move parsing to a separate thread, so as not to block the UI for so long (not done yet due to simplicity).
Title: Re: Clang based CC, new CC interface
Post by: maryjeck on March 10, 2014, 04:51:55 am
Can it work under a nightly build ??? ???
Title: Re: Clang based CC, new CC interface
Post by: scarphin on March 10, 2014, 10:06:15 am
I guess many are waiting for it. ;)
Title: Re: Clang based CC, new CC interface
Post by: oBFusCATed on March 10, 2014, 10:37:43 am
Not yet, but hopefully the code will be merge soon.
Anyone interested in this should try the modified version from here https://github.com/alpha0010/codeblocks_sf/tree/cc_interface
Title: Re: Clang based CC, new CC interface
Post by: Alpha on May 12, 2014, 01:00:20 am
Binary release (https://github.com/alpha0010/ClangLib/releases/tag/v0.1) available here (https://github.com/alpha0010/ClangLib/releases/download/v0.1/ClangLib_v0.1_20140511_rev9765_win32.7z) for Windows, compatible with nightly build rev9765.
Title: Re: Clang based CC, new CC interface
Post by: ollydbg on May 12, 2014, 02:04:22 am
Binary release (https://github.com/alpha0010/ClangLib/releases/tag/v0.1) available here (https://github.com/alpha0010/ClangLib/releases/download/v0.1/ClangLib_v0.1_20140511_rev9765_win32.7z) for Windows, compatible with nightly build rev9765.
Hi, Alpha, very nice work, I have just download both the nightly build and this clanglib binaries, the question is: How to use this plugin?
Do I need to disable the normal cc plugin?
Thanks.

EDIT:
This function works fine, see image shot below:
(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2014-05-12082725_zps3f542efd.png)


Q1: How to remove(hide) the tip window, see image shot below, do I need to hit some short-cut key?
(http://i683.photobucket.com/albums/vv194/ollydbg_cb/2014-05-12081710_zpsd534c5cf.png)


Title: Re: Clang based CC, new CC interface
Post by: vwdvaan on May 12, 2014, 01:06:49 pm
On my system:
Code
codeblocks.exe caused an Access Violation at location 62f86c09 in module libclang.dll Reading from location 00000014.

Registers:
eax=00000000 ebx=0000001e ecx=0022f6d0 edx=7b220a2d esi=0022f6c8 edi=075a97b0
eip=62f86c09 esp=0022f568 ebp=0022f760 iopl=0         nv up ei pl zr na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00010246

Call stack:
62F86C09  libclang.dll:62F86C09  clang_getInclusions
62447151  clanglib.dll:62447151
6CC41261  wxmsw28u_gcc_cb.dll:6CC41261  _ZNK12wxAppConsole11HandleEventEP12wxEvtHandlerMS0_FvR7wxEventES3_
6CCC436E  wxmsw28u_gcc_cb.dll:6CCC436E  _ZN12wxEvtHandler21ProcessEventIfMatchesERK21wxEventTableEntryBasePS_R7wxEvent
6CCC47A7  wxmsw28u_gcc_cb.dll:6CCC47A7  _ZN12wxEvtHandler23SearchDynamicEventTableER7wxEvent
6CCC4864  wxmsw28u_gcc_cb.dll:6CCC4864  _ZN12wxEvtHandler12ProcessEventER7wxEvent
6CDC9F08  wxmsw28u_gcc_cb.dll:6CDC9F08  _ZN11wxTimerBase6NotifyEv
6CD01CB4  wxmsw28u_gcc_cb.dll:6CD01CB4  _ZN7wxTimer4InitEv
75DBC4E7  USER32.dll:75DBC4E7  gapfnScSendMessage
75DBC5E7  USER32.dll:75DBC5E7  gapfnScSendMessage
75DBCC19  USER32.dll:75DBCC19  gapfnScSendMessage
75DBCC70  USER32.dll:75DBCC70  DispatchMessageW
75DB41EB  USER32.dll:75DB41EB  IsDialogMessageW
6CCEFD80  wxmsw28u_gcc_cb.dll:6CCEFD80  _ZN11wxEventLoop17PreProcessMessageEP6tagMSG
6CCEF852  wxmsw28u_gcc_cb.dll:6CCEF852  _ZN11wxEventLoop14ProcessMessageEP6tagMSG
6CCEFAEF  wxmsw28u_gcc_cb.dll:6CCEFAEF  _ZN11wxEventLoop8DispatchEv
Title: Re: Clang based CC, new CC interface
Post by: Alpha on May 12, 2014, 04:58:38 pm
Do I need to disable the normal cc plugin?
Technically you probably should (because otherwise you are parsing everything twice), however, I do not.  The two plugins seem to coexist nicely on my computer.

Q1: How to remove(hide) the tip window, see image shot below, do I need to hit some short-cut key?
Currently hardcoded to show warnings and errors, so I guess your two options are: fix the warning, or add -w to your compiler flags (restart C::B probably required).  Or patch the plugin :) .

On my system: [...]
Does this happen with any project?  If it is a specific file, are you able to post a (minimal) code piece that will cause this crash?
Title: Re: Clang based CC, new CC interface
Post by: Alpha on May 12, 2014, 09:28:00 pm
I have successfully compiled the ClangLib plugin but always crash at clang_getInclusions when they start to parse the project.
Odd...  Did you self-compile everything, or is this linking against a nightly?
Hi Alpha!

Compiled myself with TDM GCC 4.6.1, CB svn 9760, LLVM + Clang 3.3 and your ClangLib plugin.
On small CB projects, ex. c++ Hello World it's working but on projects with wxWidgets always crash.

Do you have a special build steps to compile LLVM with gcc on Windows?
Oh, apologies; looks like I never got back to you on this.
For Windows, I use the prebuilt LLVM/Clang (3.4), and TDM GCC 4.8.1 to compile wxWidgets and Code::Blocks.

My guess from your crash is that clang_parseTranslationUnit() is returning null (which it will do in the case of internal errors so extreme, it cannot recover), and then the null is passed to clang_getInclusions().  Most likely, this is due to a compiler flag (and not your source code itself - clang seems fairly good at handling, or gracefully backing down, on any sort of code I have sent it).  Could you post your full compile command line for the project that crashes?
Title: Re: Clang based CC, new CC interface
Post by: vwdvaan on May 12, 2014, 10:32:31 pm
I have successfully compiled the ClangLib plugin but always crash at clang_getInclusions when they start to parse the project.
Odd...  Did you self-compile everything, or is this linking against a nightly?
Hi Alpha!

Compiled myself with TDM GCC 4.6.1, CB svn 9760, LLVM + Clang 3.3 and your ClangLib plugin.
On small CB projects, ex. c++ Hello World it's working but on projects with wxWidgets always crash.

Do you have a special build steps to compile LLVM with gcc on Windows?
Oh, apologies; looks like I never got back to you on this.
For Windows, I use the prebuilt LLVM/Clang (3.4), and TDM GCC 4.8.1 to compile wxWidgets and Code::Blocks.

My guess from your crash is that clang_parseTranslationUnit() is returning null (which it will do in the case of internal errors so extreme, it cannot recover), and then the null is passed to clang_getInclusions().  Most likely, this is due to a compiler flag (and not your source code itself - clang seems fairly good at handling, or gracefully backing down, on any sort of code I have sent it).  Could you post your full compile command line for the project that crashes?
Hi Alpha!
This is the command line for g++:
Code
g++.exe -Wextra -Wall -pg -g -pipe -Wno-attributes -Winvalid-pch -include wx_pch.h -fpermissive -finput-charset=ISO-8859-2 -D__GNUWIN32__ -D__WXMSW__ -DwxUSE_UNICODE -DWX_PRECOMP -D__WXDEBUG__ -D__USE_MALLOC -IE:\Dev\C-C++\CodeBlocks\DevPacks\wx\lib\gcc_lib\mswud -IE:\Dev\C-C++\CodeBlocks\DevPacks\wx\include -Ie:\Dev\C-C++\CodeBlocks\DevPacks\litecms2\include -IControls -IE:\Dev\C-C++\CodeBlocks\DevPacks\nvwa\bin\Debug -Ie:\Dev\C-C++\CodeBlocks\DevPacks\nvwa -Ie:\Dev\C-C++\CodeBlocks\DevPacks\debug -Ie:\Dev\C-C++\CodeBlocks\DevPacks\wx\lib\gcc_dll\mswu -c E:\Dev\C-C++\Proiecte\VaanColorPicker\VCPSettings.cpp -o obj\MinGW\Debug_lcms2\VCPSettings.o
Title: Re: Clang based CC, new CC interface
Post by: Alpha on May 12, 2014, 10:58:00 pm
This is the command line for g++:
Code
g++.exe -Wextra -Wall -pg -g -pipe -Wno-attributes -Winvalid-pch -include wx_pch.h -fpermissive -finput-charset=ISO-8859-2 -D__GNUWIN32__ -D__WXMSW__ -DwxUSE_UNICODE -DWX_PRECOMP -D__WXDEBUG__ -D__USE_MALLOC -IE:\Dev\C-C++\CodeBlocks\DevPacks\wx\lib\gcc_lib\mswud -IE:\Dev\C-C++\CodeBlocks\DevPacks\wx\include -Ie:\Dev\C-C++\CodeBlocks\DevPacks\litecms2\include -IControls -IE:\Dev\C-C++\CodeBlocks\DevPacks\nvwa\bin\Debug -Ie:\Dev\C-C++\CodeBlocks\DevPacks\nvwa -Ie:\Dev\C-C++\CodeBlocks\DevPacks\debug -Ie:\Dev\C-C++\CodeBlocks\DevPacks\wx\lib\gcc_dll\mswu -c E:\Dev\C-C++\Proiecte\VaanColorPicker\VCPSettings.cpp -o obj\MinGW\Debug_lcms2\VCPSettings.o
Okay, I believe clang is choking on "-finput-charset=ISO-8859-2", could you test: disable clang plugin, remove that build flag, enable clang plugin (probably need to restart C::B).
Title: Re: Clang based CC, new CC interface
Post by: scarphin on May 13, 2014, 03:51:40 am
The plugin looks quite effective, thnx for your work. I listed some issues/features I encountered below, some might be bugs:

1- I had to disable the CC plugin to install the Clang plugin for the first time. CB crashed the first time after loading a project (CC plugin was enabled when I copied the files). After installation I could enable CC plugin.
2- I'm getting a ~1sec delay before the 1st and 3rd characters of anything I type on my old dual core cpu, will that delay be permanent?
3- The '.' and '->' kicks in for almost everything like 'for./->' 'int./->' 'some_class_name./->' even when typed on their own and the list consists of almost everything too.
4- I hope the argument tip comment for functions like 'some_func(/*! argument_type !*/)' will be optional as it doesn't play nice with my color profile.
5- I see the 'for', 'while' etc... are completed into complete blocks now and not just into keywords, is it possible to make this also optional? I use abbreviations plugin to customize these blocks.

Edit: Win7 x64
Title: Re: Clang based CC, new CC interface
Post by: vwdvaan on May 13, 2014, 09:34:41 am
This is the command line for g++:
Code
g++.exe -Wextra -Wall -pg -g -pipe -Wno-attributes -Winvalid-pch -include wx_pch.h -fpermissive -finput-charset=ISO-8859-2 -D__GNUWIN32__ -D__WXMSW__ -DwxUSE_UNICODE -DWX_PRECOMP -D__WXDEBUG__ -D__USE_MALLOC -IE:\Dev\C-C++\CodeBlocks\DevPacks\wx\lib\gcc_lib\mswud -IE:\Dev\C-C++\CodeBlocks\DevPacks\wx\include -Ie:\Dev\C-C++\CodeBlocks\DevPacks\litecms2\include -IControls -IE:\Dev\C-C++\CodeBlocks\DevPacks\nvwa\bin\Debug -Ie:\Dev\C-C++\CodeBlocks\DevPacks\nvwa -Ie:\Dev\C-C++\CodeBlocks\DevPacks\debug -Ie:\Dev\C-C++\CodeBlocks\DevPacks\wx\lib\gcc_dll\mswu -c E:\Dev\C-C++\Proiecte\VaanColorPicker\VCPSettings.cpp -o obj\MinGW\Debug_lcms2\VCPSettings.o
Okay, I believe clang is choking on "-finput-charset=ISO-8859-2", could you test: disable clang plugin, remove that build flag, enable clang plugin (probably need to restart C::B).
The same crash.
I made a simple wxFrame project and after i added a button on frame and set OnClick event for button...
Code
#0 6970B665	libclang!clang_defaultCodeCompleteOptions() (E:\Dev\C-C++\CodeBlocks\libclang.dll:??)
#1 00000040 ?? () (??:??)
#2 695F749E libclang!clang_defaultCodeCompleteOptions() (E:\Dev\C-C++\CodeBlocks\libclang.dll:??)
#3 ?? ?? () (??:??)


Title: Re: Clang based CC, new CC interface
Post by: koonschi on May 13, 2014, 11:59:07 am
Hi, first of all:
Awesome work. This really gives codeblocks a boost. I particularly enjoy clang's ability to deduce the auto type and complicated templates.

Some feedback from my side:
- It would be great to have the clang parsing in a separate thread. I'm working on a big project with 1000+ files, and nearly half of my time is spent switching between those files, so I always get the parsing delay (up to 8 seconds) when opening a new file, which is distracting.
- The (parsing?) delay also occurs pretty often when I'm editing headers. When I want to type a new member function, I get a delay of 1 - 2 seconds every 2 characters.
- Showing the diagnosis warnings/errors should be configurable in some way.
- I'd love to put the "Resolve symbol (clang)" function on a hotkey.

Keep up the great work!

Edit: Working on Kubuntu 14.04, 64 bit, latest nightly.
Title: Re: Clang based CC, new CC interface
Post by: Alpha on May 14, 2014, 04:23:02 am
Thanks all for the feedback, I have started a todo list (https://github.com/alpha0010/ClangLib#wishtodo-list).

Definitely lots of performance issues to take care of.  Most of these will be mitigated by switching to a threaded model, however, I do not plan to attempt that until all major single-threaded crashes are dealt with (no need to complicate problems).

A few specific responses:
3- The '.' and '->' kicks in for almost everything like 'for./->' 'int./->' 'some_class_name./->' even when typed on their own and the list consists of almost everything too.
Hmm, I never noticed it before; I shall be working on resolving that.
- The (parsing?) delay also occurs pretty often when I'm editing headers. When I want to type a new member function, I get a delay of 1 - 2 seconds every 2 characters.
Working editing headers disables clang's internal caching ability, so that is why the delay is even more noticeable.  (This delay is unacceptable for real work; I am thinking on how to best resolve it.)
[...]
The same crash.
I made a simple wxFrame project and after i added a button on frame and set OnClick event for button... [...]
I will be looking more in depth to try to reproduce this crash on my machine (probably this weekend, or sooner if time allows).  Could you also post (an archive of) a minimal project that causes the crash?

All feedback/ideas/bug reports/feature requests welcome.  I may not be able to respond to each point, but the more ideas, the better :) .
Title: Re: Clang based CC, new CC interface
Post by: carra on May 26, 2014, 10:32:51 am
I have installed nightly 9765 (on Windows) and currently I prefer it working with the old CC interface. The Clang one has problems when opening a project, even small ones. It sometimes makes C::B unresponsive for some seconds, and a couple of times I've had C::B crash when opening projects.

On the interface itself, I personally believe there is too much hand holding for the programmer. When your code is still being written, the errors displayed inside the code take too much room and make readability poor. Also I'd prefer not to have all parameter commas written when I am writing a function.
Title: Re: Clang based CC, new CC interface
Post by: Alpha on May 28, 2014, 01:52:35 am
It sometimes makes C::B unresponsive for some seconds, and a couple of times I've had C::B crash when opening projects.
Known issues; this plugin is still in its early stages, and not intended for general use.
When your code is still being written, the errors displayed inside the code take too much room and make readability poor.
From my usage, it currently updates too quickly for sane typing, so I think changing the default to a longer timer, or only checking on save, might be better.  Also, once there is a settings page, this will become configurable to put messages in tooltips/not display at all.
Also I'd prefer not to have all parameter commas written when I am writing a function.
It looked like a good idea at the time :P , though I agree, it mostly just gets in the way (without the ability to tab between parameters, or something).

I unfortunately am currently recovering from a minor surgery, so I am unsure when I will be able to step back into work.  Soon hopefully.
Title: Re: Clang based CC, new CC interface
Post by: carra on May 29, 2014, 01:02:24 pm
I unfortunately am currently recovering from a minor surgery, so I am unsure when I will be able to step back into work.  Soon hopefully.
Don't worry there is no hurry. I just tried the plugin and thoguht it would be nice to give some feedback. But right now just concentrate on recovering  ;)
Title: Re: Clang based CC, new CC interface
Post by: ollydbg on June 04, 2014, 03:31:20 pm
I unfortunately am currently recovering from a minor surgery, so I am unsure when I will be able to step back into work.  Soon hopefully.
Hopefully you will be recovered soon  :).

BTW: When reviewing the CC code, one idea comes to my mind: is it possible put all the operations on the TokensTree as a "threaded task", and those tasks will only be run from the thread pool. If the user need to show the code suggestion list, we assignment a task to the pool, and it will run in the worker thread, and finally when the task finishes, we can receive an event, and show the suggestion list. It makes some kind of synchronize operations. The main idea is to remove the "lockers" in the TokensTree, and we can serialize the operations on the Tree. We can even assign some priority task, which can be put in the front of the task queue.
Title: Re: Clang based CC, new CC interface
Post by: edison on September 01, 2014, 04:44:51 am
The error message is hard to read when using a dark theme.
Title: Re: Clang based CC, new CC interface
Post by: Alpha on September 01, 2014, 04:47:35 pm
The style used is currently hooked (hacked) into wxSmith highlighting; so you can modify it by changing the second to last entry for C++ under Settings->Editor->Syntax highlighting.
Title: Re: Clang based CC, new CC interface
Post by: edison on September 01, 2014, 06:11:06 pm
Thanks, it's much better after change the settings.
Title: Re: Clang based CC, new CC interface
Post by: edison on October 13, 2014, 05:40:51 am
The Clanglib does not work with last CB svn build (994x+).
Title: Re: Clang based CC, new CC interface
Post by: Alpha on October 13, 2014, 07:41:30 am
The Clanglib does not work with last CB svn build (994x+).
What exactly is not working?  (During my quick test, it built and ran against svn head.)
Title: Re: Clang based CC, new CC interface
Post by: edison on October 13, 2014, 08:54:08 am
Yes, it can be built but when I run the CB(svn 9966), there is a pop-up window:


Update:
I have found out what caused this problem: the libclang.dll from ClangOnWin build.
There is not such problem when using the offical build from http://llvm.org/builds/ .

Title: Re: Clang based CC, new CC interface
Post by: Alpha on January 29, 2016, 08:54:50 pm
Pushed a Windows binary compatible with C::B 16.01 ClangLib v0.2 (https://github.com/alpha0010/ClangLib/releases/tag/v0.2).  The main difference from previous binary is parsing is done in the background.

Note: feature development is currently here (http://forums.codeblocks.org/index.php/topic,20623.0.html).  This release is does not include most of the new changes in order for stability on Windows.