Clang CC

[yvesdm3000]

You may look at this plugin: Semantic highlight which also uses clang.

Maybe I should make a merge from this plugin into the CC-plugin, just to make sure the same file is not parsed twice...

In any case this is very low on my own priority list.

I'll look into setting up a VM that could make a binary out of the plugin automatically after each push. I guess it should follow the weekly nightly builds and on what wxWindows version should it be built ? And I guess people will want Linux+Windows? And then also a binary of Clang I suppose ?

For Linux maybe an Ubuntu 15.10 with the distro's version of Clang?

-Yves

[yvesdm3000]

Also people may opt for different syntax highlighting for different languages making cc plugins for different languages less responsive in terms of colors. I think at least cc plugins for different languages should define their own colors.

If you ask me this is not a good idea.
Generally I try to make all languages look roughly the same.
What error color are we talking about, because I've not tried this plugin yet?

For now I see that the inline Clang warning/error annotations have unconfigurable colours, style number 50. I guess Alpha0010 just chose a number ?

-Yves

[ollydbg]

I'll look into setting up a VM that could make a binary out of the plugin automatically after each push. I guess it should follow the weekly nightly builds and on what wxWindows version should it be built ?

Currently, Windows C::B nightly build use wx2.8.12, see here:
The 31 October 2015 build (10553) is out.
Which describe the compiler(currently tdm gcc) killerbot use to make the nightly build)

And I guess people will want Linux+Windows? And then also a binary of Clang I suppose ?

For Linux maybe an Ubuntu 15.10 with the distro's version of Clang?

-Yves

I think a binary of Clang is necessary. (At least under Windows OS)

[l_inc]

ollydbg

I'd like to help on this issue.
Is the test code you supplied only works under Linux system?

Thanks for taking a look. Unfortunately I am able to reproduce the bug in Fedora only.

You may look at this plugin: Semantic highlight which also uses clang.

Well, thank you for the hint. Though this would make up already two plugins doing the same intensive background parsing. It's much more preferable to have this functionality in a single one. But I see... the same thought is already present in the topic. :-)

[oBFusCATed]

Maybe I should make a merge from this plugin into the CC-plugin, just to make sure the same file is not parsed twice...

Nope don't do it. This is something that should be done in the sdk and the cc plugins should only feed it with data.
As far as I know scintilla doesn't have easy way to implement this, so this is why we've not investigated much time.

[scarphin]

For now I see that the inline Clang warning/error annotations have unconfigurable colours, style number 50. I guess Alpha0010 just chose a number ?

-Yves

I am aware of that but I couldn't locate the piece of code that configures this yet. Can someone point me to the code in question so I can take care of that too?

And I guess people will want Linux+Windows? And then also a binary of Clang I suppose ?

For Linux maybe an Ubuntu 15.10 with the distro's version of Clang?

-Yves

I think a binary of Clang is necessary. (At least under Windows OS)

MSYS2 project already provides a clang binary. Only clang.dll is needed to build and use the plugin, actually this binary is what I'm using to build and test the plugin. Link here:
http://sourceforge.net/projects/msys2/files/REPOS/MINGW/x86_64/mingw-w64-x86_64-clang-3.7.0-6-any.pkg.tar.xz/download
'i386' folder (instead of x86_64) also contains a 32-bit build for 32-bit users. Be aware that the file name changes with the version of clang in case you hit a dead link.

Maybe I should make a merge from this plugin into the CC-plugin, just to make sure the same file is not parsed twice...

Nope don't do it. This is something that should be done in the sdk and the cc plugins should only feed it with data.
As far as I know scintilla doesn't have easy way to implement this, so this is why we've not investigated much time.

Afaik the built-in cc plugin provides the semantic highlight for the data members currently. I use it and runs quite smooth and nice. I've even enhanced it a little further to color the class names also. I believe it won't hurt to implement this in clang plugin till it's implemented in sdk (which I don't think will ever happen though).

[oBFusCATed]

Afaik the built-in cc plugin provides the semantic highlight for the data members currently. I use it and runs quite smooth and nice. I've even enhanced it a little further to color the class names also. I believe it won't hurt to implement this in clang plugin till it's implemented in sdk (which I don't think will ever happen though).

The one in the CC is just a hack that breaks in too many ways (it is not context aware, it is just global).
I'm not sure what is the proper way to implement this, but I fear it is through implementing custom Scintilla lexer,
which will lead to many changes to other parts of editor handling which depend on the styles of the c/c++ lexers.

[Alpha]

Sorry everyone about the lack of comments in my code... I really had not intended for it to halt in this state.  Bad habit of writing comments after finishing a section of code, instead of with it ...

Btw clang's documentation parser doesn't seem to be as good as the built-in cc plugin's, or is it?

Yes, I ran into some annoying bugs in it (some comments fail to parse, sometimes clang dumps the entire file into a comment).  However, the styling/generating of the output in this plugin is unfinished; some of the information that clang does provide (e.g. blocks @param, @return, ...) is discarded/used incorrectly.

When I complete the keyword 'while' for example, I get something like:

Code

while(/*! condition !*/){
}

Is it the clang itself that completes the code like this? Can this behavior be disabled?

The actual text is provided by clang as optional output.  It would be a fairly small change in the code to tell this plugin to ignore clang's suggested output.

For now I see that the inline Clang warning/error annotations have unconfigurable colours, style number 50. I guess Alpha0010 just chose a number ?

I reused the wxsmith generated code entry from the C++ lexer.  Eventually it should probably get its own entry, but I did this since that style has the same meaning of "this text is not editable".

I am aware of that but I couldn't locate the piece of code that configures this yet. Can someone point me to the code in question so I can take care of that too?

stc->AnnotationSetStyle()

Maybe I should make a merge from this plugin into the CC-plugin, just to make sure the same file is not parsed twice...

Nope don't do it. This is something that should be done in the sdk and the cc plugins should only feed it with data.
As far as I know scintilla doesn't have easy way to implement this, so this is why we've not investigated much time.

Eventual goal for this plugin was to provide clang powered data that other subsystems (including highlighting, searching, refactoring,...) in addition to CC (hence the plugin name I choose).  This will have to be done in a way that data is mediated, so other future plugins can also provide these services.

Afaik the built-in cc plugin provides the semantic highlight for the data members currently. I use it and runs quite smooth and nice. I've even enhanced it a little further to color the class names also. I believe it won't hurt to implement this in clang plugin till it's implemented in sdk (which I don't think will ever happen though).

The one in the CC is just a hack that breaks in too many ways (it is not context aware, it is just global).

Yeah, it really is a hack.  But if you follow certain coding/naming conventions (similar to C::B core) it *nearly* works without mistakes.  I wrote this poor man's semantic highlighting in this technique since it (mostly) runs very fast, and requires no complicated lexer editing.

[ollydbg]

Here is the news about clang used in Kdevelop IDE
First Beta release of KDevelop 5.0.0 available | KDevelop, the page is released in 10/25/2015. The section about clang:

Clang

KDevelop always prided itself for its state of the art C++ language support. We introduced innovative code browsing functionality, semantic highlighting and advanced code completion, features that our user base has come to rely upon for their daily work. All of this was made possible by a custom C++ parser, and an extremely complex semantic analyzer for the C++ language. Adding support for all the quirky corner cases in C++, as well as maintaining compatibility with the latest C++ language standards such as C++11, drained our energy and stole time needed to improve other areas of our IDE. Furthermore, the code was so brittle, that it was close to impossible to improve its performance or add bigger new features such as proper C language support.

Now, after close to two years of work, we finally have a solution to this dilemma: A Clang based language plugin. Not only does this give us support for the the very latest C++ language standard, it also enables true C and Objective-C language support. Furthermore, you get all of the immensely useful compiler warnings directly inside your editor. Even better, fixing these warnings is now often just a matter of pressing a single button to apply a Clang provided fix-it!

There are, however, a few caveats that need to be mentioned:

    On older machines the performance may be worse than with our legacy C++ support. But the new Clang based plugin finally scales properly with the number of cores on your CPU, which can lead to significantly improved performance on more modern machines.
    Some features of our legacy C++ support have not yet been ported to the new plugin. This includes special support for Qt code, most notably signal/slot code completion using the old Qt 4 macro syntax. We will be working on improving this situation and welcome feedback from our users on what we should focus on.
    The plugin works fine with Clang 3.6 and above, but some features, such as auto-type resolution in the code browser, or argument code completion hints within a function call, require unreleased extensions to Clang. The required changes have been contributed upstream by members of our community and we intend on continuing this effort. For the best C++ experience in KDevelop, we recommend a custom build of a recent Clang. Alternatively, stay tuned for Clang 3.8 early next year.

I have looked at its source:KDE - KDevelop - Development Information, the git repo:KDE QuickGit :: kdevelop.git/summary, and I can see there are two folders under the language folder:
"clang" is the new code base, and "cpp" contains the old c++ parser.
I looked at the parser source file in the "cpp" folder parser.cpp, I see that it implement many AST kinds, but look at its change log, I see that there are few commits in recent years, the most commits are before year 2012.

This gives a question: what is the future about our native cc? maybe, all of them will be replaced by clang-cc......

[l_inc]

Alpha, yvesdm3000
I got into a couple of problems with Clang CC. First one is that clang_getDiagnosticNumRanges segfaults.

Code

        CXDiagnostic diag = clang_getDiagnosticInSet(diagSet, i); //i == 0
        //ExpandDiagnosticSet(clang_getChildDiagnostics(diag), diagnostics);
        size_t numRnges = clang_getDiagnosticNumRanges(diag); //<-- crash here

The crash happens inside of the function on the line return D->getNumRanges();, where getNumRanges does not get called because of the corrupted pointer to the virtual function table. So this seems to be a use after free inside the plugin. This happens pretty much every time when closing a tab with a header file.

Another problem is that the plugin sometimes gets into an infinite loop inside wxExecute(command, output, errors, wxEXEC_NODISABLE); . Inside the function int wxGUIAppTraits::WaitForChild(wxExecuteData& execData) to be more precise, while GTK_EndProcessDetector is always getting 0 from waitpid and g++ is somehow not present in the process list. Not sure, when exactly this happens, I had this issue twice yesterday. Seems I'll need more investigation on this.

[l_inc]

So this seems to be a use after free inside the plugin. This happens pretty much every time when closing a tab with a header file.

Just checked it again, cause the bug's too annoying. This is indeed a use after free, cause clang_getDiagnosticSetFromTU(m_ClTranslUnit); in TranslationUnit::GetDiagnostics is called after clang_disposeTranslationUnit(m_ClTranslUnit); in the TranslationUnit::~TranslationUnit() for the same m_ClTranslUnit, i.e. the TranslationUnit object is reused (by ClangProxy::GetDiagnostics) after its destructor was called as a result of ClangProxy::RemoveTranslationUnit .

[yvesdm3000]

So this seems to be a use after free inside the plugin. This happens pretty much every time when closing a tab with a header file.

Just checked it again, cause the bug's too annoying. This is indeed a use after free, cause clang_getDiagnosticSetFromTU(m_ClTranslUnit); in TranslationUnit::GetDiagnostics is called after clang_disposeTranslationUnit(m_ClTranslUnit); in the TranslationUnit::~TranslationUnit() for the same m_ClTranslUnit, i.e. the TranslationUnit object is reused (by ClangProxy::GetDiagnostics) after its destructor was called as a result of ClangProxy::RemoveTranslationUnit .

Thanks for the feedback. I'll include a fix in my next push for this.

Yves

[l_inc]

Alpha, yvesdm3000
OK. This was an easy one. Someone just doesn't know, what std::vector::assign is meant for. Btw. I hope those who wrote the copy-constructor of the TranslationUnit as simulating a move-contructor for speed know what they do, cause it might become unsafe.

Patch attached.

[yvesdm3000]

Alpha, yvesdm3000
OK. This was an easy one. Someone just doesn't know, what std::vector::assign is meant for. Btw. I hope those who wrote the copy-constructor of the TranslationUnit as simulating a move-contructor for speed know what they do, cause it might become unsafe.

Patch attached.

I noticed the construction too. As long as it doesn't give me trouble I keep it as-is, it does help to keep the lifetime of the clang TU correct without additional pointers. For the file-id list, I did not investigate yet why we need the swap there... I don't want to be too negative, the plugin is still a very good starting point!

Having real C++11 and wxWidgets 2.9 (for some thread goodies) as minimum requirements is whishful thinking I guess...

Yves

[yvesdm3000]

For the std::vector::assign, that must have been me, and I don't even understand myself how I could have written that, probably late hours and lack of sleep.

Yves