Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: yvesdm3000 on October 02, 2015, 07:44:04 pm

Title: Clang CC
Post by: yvesdm3000 on October 02, 2015, 07:44:04 pm
Hello,

Just a quick question, did development on the Clang-based CC stall? Or is there a branch somewhere with the latest code since it seems https://github.com/alpha0010/ClangLib is the latest incarnation ?

-Yves
Title: Re: Clang CC
Post by: Alpha on October 03, 2015, 01:23:03 pm
Yes, that is the most recent work on Clang based CC.  Unfortunately, a variety of events/issues in my life have stalled my ability to do large chunks of programming anywhere currently.  At whatever point I am able to return to my projects, this will be my top priority.  (In the mean time, I still will happily review any pull requests there.)
Title: Re: Clang CC
Post by: yvesdm3000 on October 06, 2015, 05:42:28 am
Great!

I'll work on it a bit, probably first fixing small bugs to improve its usability and send pull-requests when something is ready.

-Yves
Title: Re: Clang CC
Post by: yvesdm3000 on October 21, 2015, 08:32:50 pm
First pull-request is ready. For people that want faster access to bugfixes/new features, I commit them on

https://github.com/yvesdm3000/ClangLib

Constructive feedback is always welcome.

Current state is so far pretty stable, but oviously the expected lags are still there since the clang functionality should be moved to a thread.

Yves
Title: Re: Clang CC
Post by: scarphin on October 22, 2015, 12:37:02 am
yvesdm3000: Considering this plugin can differentiate between a '.' and '->', the only reason preventing me to use it over cb's own cc is the non-threaded functionality. Unfortunately I'm not skilled enough to fix this but I will sure test your commits. Thanks for that.
Title: Re: Clang CC
Post by: yvesdm3000 on October 22, 2015, 05:14:47 am
I want to make the base functionality work without freezes and crashes first before the introduction of a thread. The threading is also why I'm not using it for any other coding just yet, only using it to develop the plugin itself.

I'm also missing the "Find declaration"/"Find implementation" context menu items that are available in the basic CC functionality. There is only a "Resolve symbol (clang)" or something which is not really what you'd expect.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on October 31, 2015, 10:13:13 pm
I have a reworked version that uses multi-threading now. Works pretty smoothly, but you need a long initial wait until you get code-completion, but that doesn't block the UI ! There is this bug where, initially, the file gets parsed a couple of times and you don't get code-completion when the file is being parsed. It's only when you load the project, after that it's fine. I'm aware of the issue.

https://github.com/yvesdm3000/ClangLib

I think I'll also need a separate configuration window to add Clang/Code-Completion specific compile flags and I figure having a configurable timeout on the code-completion operation might be handy for some people too.

As always constructive feedback is always welcome!

-Yves

Title: Re: Clang CC
Post by: scarphin on November 01, 2015, 02:31:55 pm
That looks nice, thanks. I couldn't find much time to try it recently but my workload is less nowadays and I'll check it out before giving feedback.

Btw are you working on this on a 64 or 32-bit environment? Last time I checked there was a weird bug (a buffer overflow or something) that occurs only on a 64-bit build. 32-bit didn't exhibit this behavior. I'll give more information when I try it. Thanks again.
Title: Re: Clang CC
Post by: yvesdm3000 on November 01, 2015, 08:06:31 pm
64 bit with wxWindows 2.8

I'm now trying it on a Centos 5 with wxWindows 3.0 and it suffers from a couple of problems, mostly scintilla events not being received when editing the file so I advise you to use wx 2.8. It's mostly impacting the inline diagnostics not being updated. I'm looking at the c::b builtin codecompletion to see if I can take over their structure of c::b events to solve these problems.

Yves

Title: Re: Clang CC
Post by: scarphin on November 02, 2015, 01:32:44 pm
I had the time to try your changes and they work quite well. Good job. The long parsing time bug is there as you've previously stated. Apart from that I get the errors attached when it's parsing a big project (I tried it on a Qt project). I don't get these errors with small projects so I think it's some kind of a buffer limitation but I have no idea where to look, any ideas?

I may contribute some changes starting with the coloring of the information boxes but I'm not sure if each code completion plugin needs to have its own set of colors. I think they do because of the different coloring schemes for different languages. What do you think? I'll submit patches using the name 'Clang Code Completion' under color settings in 'settings->environment->colors'.
Title: Re: Clang CC
Post by: scarphin on November 02, 2015, 02:38:23 pm
Oh, one more thing I forgot about. I had to add:
Code
#include <iostream>
to 'clangplugin.cpp' and 'translationunit.cpp' to build the plugin. I don't know why iostream were missing in those 2 files initially.
Title: Re: Clang CC
Post by: yvesdm3000 on November 02, 2015, 09:03:23 pm
I'll add those includes. I guess my compiler (GCC 4.8) probably doesn't need them to be there.

I'm going to try the plugin on a 800mb sourcecode project tomorrow, a lot of files with over 5000 lines, big amount of defines+template code etc and see what happens. The builtin codecompletion plugin doesn't have any issue with the project, but it doesn't like the templating code at all, and I don't start about the defines...

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on November 02, 2015, 09:06:06 pm
What do you think? I'll submit patches using the name 'Clang Code Completion' under color settings in 'settings->environment->colors'.

No problem for me as long as Alpha0010 and the other nice C::B guys approve.

-Yves
Title: Re: Clang CC
Post by: Alpha on November 03, 2015, 01:37:47 am
Oh, one more thing I forgot about. I had to add:
Code
#include <iostream>
to 'clangplugin.cpp' and 'translationunit.cpp' to build the plugin. I don't know why iostream were missing in those 2 files initially.
Erm... why do we need iostream?  Logging/debug logging should be directed through C::B's log manager.

What do you think? I'll submit patches using the name 'Clang Code Completion' under color settings in 'settings->environment->colors'.
No problem for me as long as Alpha0010 and the other nice C::B guys approve.
I believe the main CC plugin already registers some colors; if possible, reuse those first.
Title: Re: Clang CC
Post by: oBFusCATed on November 03, 2015, 01:43:07 am
I may contribute some changes starting with the coloring of the information boxes but I'm not sure if each code completion plugin needs to have its own set of colors. I think they do because of the different coloring schemes for different languages. What do you think? I'll submit patches using the name 'Clang Code Completion' under color settings in 'settings->environment->colors'.
Can you give better details what colour do you need to define?
If you can make them generic it will be best.
Title: Re: Clang CC
Post by: yvesdm3000 on November 03, 2015, 04:55:26 am
Oh, one more thing I forgot about. I had to add:
Code
#include <iostream>
to 'clangplugin.cpp' and 'translationunit.cpp' to build the plugin. I don't know why iostream were missing in those 2 files initially.
Erm... why do we need iostream?  Logging/debug logging should be directed through C::B's log manager.

What do you think? I'll submit patches using the name 'Clang Code Completion' under color settings in 'settings->environment->colors'.
No problem for me as long as Alpha0010 and the other nice C::B guys approve.
I believe the main CC plugin already registers some colors; if possible, reuse those first.

For the logging it's just debugging stuff, especially with threads I tend to be carefull and a simple fprintf is often easy. Those should be removed at a later stage in a cleanup when everything works as expected. Intended logging should go to the log manager.

For the colours, all I have seen is that the plugin simply returns the codecompletion data back to C::B so colouring should be handled on that level, except for maybe the HTML it returns for documentation.

Yves
Title: Re: Clang CC
Post by: scarphin on November 03, 2015, 11:51:05 am
Erm... why do we need iostream?  Logging/debug logging should be directed through C::B's log manager.
I had to include them because the compiler were spitting 'cout doesn't belong to namespace std' kind of errors. I think yvesdm3000 made it clear.

I believe the main CC plugin already registers some colors; if possible, reuse those first.
As you wish. I also prefer to reuse the same colors but my reasoning was that the clang cc uses a different color for errors while the built-in cc doesn't have a designated color for that which might introduce color entries that are not used by some plugins. Also people may opt for different syntax highlighting for different languages making cc plugins for different languages less responsive in terms of colors. I think at least cc plugins for different languages should define their own colors.

@obfuscated: Stated above.
Title: Re: Clang CC
Post by: oBFusCATed on November 03, 2015, 08:37:44 pm
Also people may opt for different syntax highlighting for different languages making cc plugins for different languages less responsive in terms of colors. I think at least cc plugins for different languages should define their own colors.
If you ask me this is not a good idea.
Generally I try to make all languages look roughly the same.
What error color are we talking about, because I've not tried this plugin yet?
Title: Re: Clang CC
Post by: scarphin on November 03, 2015, 08:54:35 pm
I try to make all languages look exactly the same but that doesn't rule out other combinations users may provide.

As for the errors I provided 2 screenshots in one of my previous posts.
Title: Re: Clang CC
Post by: oBFusCATed on November 03, 2015, 09:10:50 pm
I guess I've misunderstood you, because I thought that you're talking about some colours for errors. Ignore.
Title: Re: Clang CC
Post by: yvesdm3000 on November 04, 2015, 06:07:12 am
I guess I've misunderstood you, because I thought that you're talking about some colours for errors. Ignore.

Since the clang-plugin uses a compiler, it has warnings and errors on his own. These text-strings are inlined on the line it occurs. I'm unsure if C::B has generic colours for this.

In the end the plugin should also have a dialog to toggle these features on or off, and also some options to add or filter-out compiler options with some good defaults of options it doesn't know about.

-Yves
Title: Re: Clang CC
Post by: yvesdm3000 on November 04, 2015, 10:45:45 am
I added some commits that addresses crashes. Seems to work OK now for me on very big files, but I had to add some bigger timeouts which makes it less smooth. :-(

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on November 07, 2015, 11:45:39 am
Performed a new push to GitHub addressing various usability issues and moved to using cb BackgroundThread and its job queue. I now use it on a daily basis for my professional coding and deal with cpp files with more then 15.000 lines. I can't really say it's smooth on those big files since reparsing takes a long time, but it doesn't prohibit me in typing/editing, it only results in less or delayed code completion results.

Yves
Title: Re: Clang CC
Post by: l_inc on November 09, 2015, 01:38:15 am
yvesdm3000
I had some difficulties compiling this plugin, but it's such a pleasure to use. Is there any chance to have it in codeblocks-contrib anytime soon? I mean the standard CC plugin is nothing but a source of infinite frustration.
Title: Re: Clang CC
Post by: yvesdm3000 on November 09, 2015, 07:03:45 am
Nice to see feedback. I know it's probably a bit difficult to compile, mostly because my env is not very typical so I simply don't check in my cbp file. I think you only need to fix compile options for wx and c::b ?

It's on my list of todo's, but not at the top.

More tasks to do: the class&method toolbar, goto declaration/implementation context menus and something that replaces cscope.

-Yves
Title: Re: Clang CC
Post by: ollydbg on November 09, 2015, 08:47:22 am
It would be best if you can release some prebuild binaries of this package, I remember Alpha has released such one along with the nightly build. So that people can easily test this plugin.

More tasks to do: the class&method toolbar, goto declaration/implementation context menus and something that replaces cscope.
-Yves
The current goto declaration/implementation is supplied by the build-in codecompletion plugin.


yvesdm3000
I had some difficulties compiling this plugin, but it's such a pleasure to use. Is there any chance to have it in codeblocks-contrib anytime soon? I mean the standard CC plugin is nothing but a source of infinite frustration.
The standard CC is still maintained by some people, but it is hard to parse the template or C++11 new feature related code. :(
Title: Re: Clang CC
Post by: scarphin on November 09, 2015, 10:38:20 am
Attached patch makes the documentation colors configurable using the environment color settings. The syntax highlight colors inside the document still needs work to match the users syntax highlighting colors though. Btw clang's documentation parser doesn't seem to be as good as the built-in cc plugin's, or is it?

I'm experiencing a misbehavior with the  completion. When I complete the keyword 'while' for example, I get something like:
Code
while(/*! condition !*/){
}
Is it the clang itself that completes the code like this? Can this behavior be disabled?
Title: Re: Clang CC
Post by: scarphin on November 09, 2015, 01:14:26 pm
Forgot to tell that the 'Top' link at the bottom of the documentation window doesn't work for some reason.
Title: Re: Clang CC
Post by: l_inc on November 09, 2015, 01:43:50 pm
ollydbg
The standard CC is still maintained by some people, but it is hard to parse the template or C++11 new feature related code. :(
It has problems with all sorts of other stuff not related to C++11 (like this issue (http://forums.codeblocks.org/index.php/topic,20662.0.html)). I know it's a huge piece of work, but it has just never functioned properly for me with anything beyond helloworlds, and I'm very sorry about it. Even the plain notepad++ way of suggesting whatever it sees in the document without any language parsing is more helpful. Btw. notepad++ has more features working better, but it's a subject for a different topic.

yvesdm3000
More tasks to do: the class&method toolbar, goto declaration/implementation context menus and something that replaces cscope.
If you accept feature suggestions, then I'd prefer to have context dependent syntax highlighting. Current highlighting schemes are plain based on a set of keywords. I'd like the highlighting to be able to differentiate between functions, variables, class names, preprocessor constants etc., which requires support of a syntax parsing plugin.

Another feature is to provide suggestions in a relevance-based order. I.e. when passing arguments to a function, the first suggestion should be a local variable name or an argument of a currently edited function that has the type most compatible with the currently passed argument, and further suggestions should include other castable identifiers in order of how forceful the casting should be.

These two would make it close to the level of VisualAssistX.
Title: Re: Clang CC
Post by: ollydbg on November 09, 2015, 01:58:58 pm
ollydbg
The standard CC is still maintained by some people, but it is hard to parse the template or C++11 new feature related code. :(
It has problems with all sorts of other stuff not related to C++11 (like this issue (http://forums.codeblocks.org/index.php/topic,20662.0.html)).
I'd like to help on this issue.
Is the test code you supplied only works under Linux system?
I have only Windows system at hand.
The native codecompletion is not that bad as I see.



Quote
yvesdm3000
More tasks to do: the class&method toolbar, goto declaration/implementation context menus and something that replaces cscope.
If you accept feature suggestions, then I'd prefer to have context dependent syntax highlighting. Current highlighting schemes are plain based on a set of keywords. I'd like the highlighting to be able to differentiate between functions, variables, class names, preprocessor constants etc., which requires support of a syntax parsing plugin.

Another feature is to provide suggestions in a relevance-based order. I.e. when passing arguments to a function, the first suggestion should be a local variable name or an argument of a currently edited function that has the type most compatible with the currently passed argument, and further suggestions should include other castable identifiers in order of how forceful the casting should be.

These two would make it close to the level of VisualAssistX.
You may look at this plugin: Semantic highlight (http://forums.codeblocks.org/index.php/topic,16249.0.html) which also uses clang.
Title: Re: Clang CC
Post by: yvesdm3000 on November 09, 2015, 02:45:59 pm
You may look at this plugin: Semantic highlight (http://forums.codeblocks.org/index.php/topic,16249.0.html) which also uses clang.

Maybe I should make a merge from this plugin into the CC-plugin, just to make sure the same file is not parsed twice...

In any case this is very low on my own priority list.

I'll look into setting up a VM that could make a binary out of the plugin automatically after each push. I guess it should follow the weekly nightly builds and on what wxWindows version should it be built ? And I guess people will want Linux+Windows? And then also a binary of Clang I suppose ?

For Linux maybe an Ubuntu 15.10 with the distro's version of Clang?

-Yves
Title: Re: Clang CC
Post by: yvesdm3000 on November 09, 2015, 02:48:56 pm
Also people may opt for different syntax highlighting for different languages making cc plugins for different languages less responsive in terms of colors. I think at least cc plugins for different languages should define their own colors.
If you ask me this is not a good idea.
Generally I try to make all languages look roughly the same.
What error color are we talking about, because I've not tried this plugin yet?

For now I see that the inline Clang warning/error annotations have unconfigurable colours, style number 50. I guess Alpha0010 just chose a number ?

-Yves
Title: Re: Clang CC
Post by: ollydbg on November 09, 2015, 02:52:32 pm
I'll look into setting up a VM that could make a binary out of the plugin automatically after each push. I guess it should follow the weekly nightly builds and on what wxWindows version should it be built ?
Currently, Windows C::B nightly build use wx2.8.12, see here:
The 31 October 2015 build (10553) is out. (http://forums.codeblocks.org/index.php/topic,20676.0.html)
Which describe the compiler(currently tdm gcc) killerbot use to make the nightly build)

Quote
And I guess people will want Linux+Windows? And then also a binary of Clang I suppose ?

For Linux maybe an Ubuntu 15.10 with the distro's version of Clang?

-Yves
I think a binary of Clang is necessary. (At least under Windows OS)
Title: Re: Clang CC
Post by: l_inc on November 09, 2015, 03:06:24 pm
ollydbg
Quote
I'd like to help on this issue.
Is the test code you supplied only works under Linux system?
Thanks for taking a look. Unfortunately I am able to reproduce the bug in Fedora only.

Quote
You may look at this plugin: Semantic highlight (http://forums.codeblocks.org/index.php/topic,16249.0.html) which also uses clang.
Well, thank you for the hint. Though this would make up already two plugins doing the same intensive background parsing. It's much more preferable to have this functionality in a single one. But I see... the same thought is already present in the topic. :-)
Title: Re: Clang CC
Post by: oBFusCATed on November 09, 2015, 08:10:15 pm
Maybe I should make a merge from this plugin into the CC-plugin, just to make sure the same file is not parsed twice...
Nope don't do it. This is something that should be done in the sdk and the cc plugins should only feed it with data.
As far as I know scintilla doesn't have easy way to implement this, so this is why we've not investigated much time.
Title: Re: Clang CC
Post by: scarphin on November 09, 2015, 09:22:28 pm
For now I see that the inline Clang warning/error annotations have unconfigurable colours, style number 50. I guess Alpha0010 just chose a number ?

-Yves
I am aware of that but I couldn't locate the piece of code that configures this yet. Can someone point me to the code in question so I can take care of that too?

Quote
And I guess people will want Linux+Windows? And then also a binary of Clang I suppose ?

For Linux maybe an Ubuntu 15.10 with the distro's version of Clang?

-Yves
I think a binary of Clang is necessary. (At least under Windows OS)
MSYS2 project already provides a clang binary. Only clang.dll is needed to build and use the plugin, actually this binary is what I'm using to build and test the plugin. Link here:
http://sourceforge.net/projects/msys2/files/REPOS/MINGW/x86_64/mingw-w64-x86_64-clang-3.7.0-6-any.pkg.tar.xz/download (http://sourceforge.net/projects/msys2/files/REPOS/MINGW/x86_64/mingw-w64-x86_64-clang-3.7.0-6-any.pkg.tar.xz/download)
'i386' folder (instead of x86_64) also contains a 32-bit build for 32-bit users. Be aware that the file name changes with the version of clang in case you hit a dead link.

Maybe I should make a merge from this plugin into the CC-plugin, just to make sure the same file is not parsed twice...
Nope don't do it. This is something that should be done in the sdk and the cc plugins should only feed it with data.
As far as I know scintilla doesn't have easy way to implement this, so this is why we've not investigated much time.
Afaik the built-in cc plugin provides the semantic highlight for the data members currently. I use it and runs quite smooth and nice. I've even enhanced it a little further to color the class names also. I believe it won't hurt to implement this in clang plugin till it's implemented in sdk (which I don't think will ever happen though).
Title: Re: Clang CC
Post by: oBFusCATed on November 09, 2015, 09:40:50 pm
Afaik the built-in cc plugin provides the semantic highlight for the data members currently. I use it and runs quite smooth and nice. I've even enhanced it a little further to color the class names also. I believe it won't hurt to implement this in clang plugin till it's implemented in sdk (which I don't think will ever happen though).
The one in the CC is just a hack that breaks in too many ways (it is not context aware, it is just global). :)
I'm not sure what is the proper way to implement this, but I fear it is through implementing custom Scintilla lexer,
which will lead to many changes to other parts of editor handling which depend on the styles of the c/c++ lexers.
Title: Re: Clang CC
Post by: Alpha on November 10, 2015, 06:12:15 am
Sorry everyone about the lack of comments in my code... I really had not intended for it to halt in this state.  Bad habit of writing comments after finishing a section of code, instead of with it :P ...

Btw clang's documentation parser doesn't seem to be as good as the built-in cc plugin's, or is it?
Yes, I ran into some annoying bugs in it (some comments fail to parse, sometimes clang dumps the entire file into a comment).  However, the styling/generating of the output in this plugin is unfinished; some of the information that clang does provide (e.g. blocks @param, @return, ...) is discarded/used incorrectly.

When I complete the keyword 'while' for example, I get something like:
Code
while(/*! condition !*/){
}
Is it the clang itself that completes the code like this? Can this behavior be disabled?
The actual text is provided by clang as optional output.  It would be a fairly small change in the code to tell this plugin to ignore clang's suggested output.

For now I see that the inline Clang warning/error annotations have unconfigurable colours, style number 50. I guess Alpha0010 just chose a number ?
I reused the wxsmith generated code entry from the C++ lexer.  Eventually it should probably get its own entry, but I did this since that style has the same meaning of "this text is not editable".
I am aware of that but I couldn't locate the piece of code that configures this yet. Can someone point me to the code in question so I can take care of that too?
stc->AnnotationSetStyle() (https://github.com/alpha0010/ClangLib/blob/2d64e1d35ed32e1aec5de7ec2c0112d8764d07cb/clangplugin.cpp#L971)

Maybe I should make a merge from this plugin into the CC-plugin, just to make sure the same file is not parsed twice...
Nope don't do it. This is something that should be done in the sdk and the cc plugins should only feed it with data.
As far as I know scintilla doesn't have easy way to implement this, so this is why we've not investigated much time.
Eventual goal for this plugin was to provide clang powered data that other subsystems (including highlighting, searching, refactoring,...) in addition to CC (hence the plugin name I choose).  This will have to be done in a way that data is mediated, so other future plugins can also provide these services.

Afaik the built-in cc plugin provides the semantic highlight for the data members currently. I use it and runs quite smooth and nice. I've even enhanced it a little further to color the class names also. I believe it won't hurt to implement this in clang plugin till it's implemented in sdk (which I don't think will ever happen though).
The one in the CC is just a hack that breaks in too many ways (it is not context aware, it is just global). :)
Yeah, it really is a hack.  But if you follow certain coding/naming conventions (similar to C::B core) it *nearly* works without mistakes.  I wrote this poor man's semantic highlighting in this technique since it (mostly) runs very fast, and requires no complicated lexer editing.
Title: Re: Clang CC
Post by: ollydbg on November 10, 2015, 03:44:07 pm
Here is the news about clang used in Kdevelop IDE
First Beta release of KDevelop 5.0.0 available | KDevelop (https://www.kdevelop.org/news/first-beta-release-kdevelop-500-available), the page is released in 10/25/2015. The section about clang:
Quote
Clang

KDevelop always prided itself for its state of the art C++ language support. We introduced innovative code browsing functionality, semantic highlighting and advanced code completion, features that our user base has come to rely upon for their daily work. All of this was made possible by a custom C++ parser, and an extremely complex semantic analyzer for the C++ language. Adding support for all the quirky corner cases in C++, as well as maintaining compatibility with the latest C++ language standards such as C++11, drained our energy and stole time needed to improve other areas of our IDE. Furthermore, the code was so brittle, that it was close to impossible to improve its performance or add bigger new features such as proper C language support.

Now, after close to two years of work, we finally have a solution to this dilemma: A Clang based language plugin. Not only does this give us support for the the very latest C++ language standard, it also enables true C and Objective-C language support. Furthermore, you get all of the immensely useful compiler warnings directly inside your editor. Even better, fixing these warnings is now often just a matter of pressing a single button to apply a Clang provided fix-it!

There are, however, a few caveats that need to be mentioned:

    On older machines the performance may be worse than with our legacy C++ support. But the new Clang based plugin finally scales properly with the number of cores on your CPU, which can lead to significantly improved performance on more modern machines.
    Some features of our legacy C++ support have not yet been ported to the new plugin. This includes special support for Qt code, most notably signal/slot code completion using the old Qt 4 macro syntax. We will be working on improving this situation and welcome feedback from our users on what we should focus on.
    The plugin works fine with Clang 3.6 and above, but some features, such as auto-type resolution in the code browser, or argument code completion hints within a function call, require unreleased extensions to Clang. The required changes have been contributed upstream by members of our community and we intend on continuing this effort. For the best C++ experience in KDevelop, we recommend a custom build of a recent Clang. Alternatively, stay tuned for Clang 3.8 early next year.


I have looked at its source:KDE - KDevelop - Development Information (https://www.kde.org/applications/development/kdevelop/development), the git repo:KDE QuickGit :: kdevelop.git/summary (https://quickgit.kde.org/?p=kdevelop.git), and I can see there are two folders under the language folder:
"clang" is the new code base, and "cpp" contains the old c++ parser.
I looked at the parser source file in the "cpp" folder parser.cpp (http://quickgit.kde.org/?p=kdevelop.git&a=blob&h=43fe9851dc4847fe0e1e86e733c8ca1415e49c60&hb=e03c8c789c8f5e7879eadfd4b97cc976553bdbd6&f=languages%2Fcpp%2Fparser%2Fparser.cpp), I see that it implement many AST kinds, but look at its change log, I see that there are few commits in recent years, the most commits are before year 2012.

This gives a question: what is the future about our native cc? maybe, all of them will be replaced by clang-cc......
Title: Re: Clang CC
Post by: l_inc on November 10, 2015, 03:50:32 pm
Alpha, yvesdm3000
I got into a couple of problems with Clang CC. First one is that clang_getDiagnosticNumRanges segfaults.
Code
        CXDiagnostic diag = clang_getDiagnosticInSet(diagSet, i); //i == 0
        //ExpandDiagnosticSet(clang_getChildDiagnostics(diag), diagnostics);
        size_t numRnges = clang_getDiagnosticNumRanges(diag); //<-- crash here
The crash happens inside of the function on the line return D->getNumRanges();, where getNumRanges does not get called because of the corrupted pointer to the virtual function table. So this seems to be a use after free inside the plugin. This happens pretty much every time when closing a tab with a header file.

Another problem is that the plugin sometimes gets into an infinite loop inside wxExecute(command, output, errors, wxEXEC_NODISABLE); . Inside the function int wxGUIAppTraits::WaitForChild(wxExecuteData& execData) to be more precise, while GTK_EndProcessDetector is always getting 0 from waitpid and g++ is somehow not present in the process list. Not sure, when exactly this happens, I had this issue twice yesterday. Seems I'll need more investigation on this.
Title: Re: Clang CC
Post by: l_inc on November 10, 2015, 06:19:30 pm
So this seems to be a use after free inside the plugin. This happens pretty much every time when closing a tab with a header file.
Just checked it again, cause the bug's too annoying. This is indeed a use after free, cause clang_getDiagnosticSetFromTU(m_ClTranslUnit); in TranslationUnit::GetDiagnostics is called after clang_disposeTranslationUnit(m_ClTranslUnit); in the TranslationUnit::~TranslationUnit() for the same m_ClTranslUnit, i.e. the TranslationUnit object is reused (by ClangProxy::GetDiagnostics) after its destructor was called as a result of ClangProxy::RemoveTranslationUnit .
Title: Re: Clang CC
Post by: yvesdm3000 on November 10, 2015, 08:02:42 pm
So this seems to be a use after free inside the plugin. This happens pretty much every time when closing a tab with a header file.
Just checked it again, cause the bug's too annoying. This is indeed a use after free, cause clang_getDiagnosticSetFromTU(m_ClTranslUnit); in TranslationUnit::GetDiagnostics is called after clang_disposeTranslationUnit(m_ClTranslUnit); in the TranslationUnit::~TranslationUnit() for the same m_ClTranslUnit, i.e. the TranslationUnit object is reused (by ClangProxy::GetDiagnostics) after its destructor was called as a result of ClangProxy::RemoveTranslationUnit .

Thanks for the feedback. I'll include a fix in my next push for this.

Yves
Title: Re: Clang CC
Post by: l_inc on November 10, 2015, 08:22:54 pm
Alpha, yvesdm3000
OK. This was an easy one. Someone just doesn't know, what std::vector::assign is meant for. Btw. I hope those who wrote the copy-constructor of the TranslationUnit as simulating a move-contructor for speed know what they do, cause it might become unsafe.

Patch attached.
Title: Re: Clang CC
Post by: yvesdm3000 on November 10, 2015, 10:20:13 pm
Alpha, yvesdm3000
OK. This was an easy one. Someone just doesn't know, what std::vector::assign is meant for. Btw. I hope those who wrote the copy-constructor of the TranslationUnit as simulating a move-contructor for speed know what they do, cause it might become unsafe.

Patch attached.

I noticed the construction too. As long as it doesn't give me trouble I keep it as-is, it does help to keep the lifetime of the clang TU correct without additional pointers. For the file-id list, I did not investigate yet why we need the swap there... I don't want to be too negative, the plugin is still a very good starting point!

Having real C++11 and wxWidgets 2.9 (for some thread goodies) as minimum requirements is whishful thinking I guess...

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on November 10, 2015, 10:25:42 pm
For the std::vector::assign, that must have been me, and I don't even understand myself how I could have written that, probably late hours and lack of sleep.

Yves
Title: Re: Clang CC
Post by: l_inc on November 11, 2015, 01:14:16 am
yvesdm3000
Quote
As long as it doesn't give me trouble I keep it as-is, it does help to keep the lifetime of the clang TU correct without additional pointers.
I think it's rather about avoiding additional copying of the TranslationUnit data. I'm not much into C++, and this was something new to me. It's just important to remember to copy-construct objects from temporary objects only. This includes initialization and assignment as well, because the assignment constructor of TranslationUnit gets the assigned object by value and hence will corrupt it during copy-construction.

Thing is... the compiler should do copy elision for temporary objects (which includes the m_Files vector) anyway, so it should be the same speed when doing the standard colon-initialization, but without introducing hazardous conditions for normal objects. So I would really want to know the opinion of the author of this Achtung.
Title: Re: Clang CC
Post by: yvesdm3000 on November 11, 2015, 07:09:38 am
yvesdm3000
I think it's rather about avoiding additional copying of the TranslationUnit data. I'm not much into C++, and this was something new to me. It's just important to remember to copy-construct objects from temporary objects only. This includes initialization and assignment as well, because the assignment constructor of TranslationUnit gets the assigned object by value and hence will corrupt it during copy-construction.
When it's not C++11, the elements in the vector<TranslationUnit> will always be copy-constructed, they will then always be copied again when the vector grows beyond the allocated memory. Now for the clang TU, there is only 1 call to release that memory and when you have multiple copies of that same pointer, doing a clang_releaseTranslationUnit would then be done on all copies and hence freeing the same memory multiple times. Before C++11 we would have solved this with a pointer to TranslationUnit and do memory-management ourselves, with C++11 with the move operator it becomes much easier since the lifetime can be tied to the lifetime of the container elements, and with Alpha's trick of emulating the move operator it seems to perform the same task. It works, so I don't touch it.

Yves
Title: Re: Clang CC
Post by: Alpha on November 11, 2015, 07:50:53 am
Yes, the constructor is not intended specifically for performance (though if it helps with that, any speed is welcome), but to try to make memory management as automatic as possible: simply adding to/removing from m_TranslUnits handles the necessary operations.  (Clang TU's use a lot of RAM, so leaking one is... undesirable.)

I hope those who wrote the copy-constructor of the TranslationUnit as simulating a move-contructor for speed know what they do, cause it might become unsafe.
Casting away a const and editing it is undefined if the original object was created const.  However, TranslationUnits must be mutable to interact with clang, so this is not the case.  For insurance though, I tried to do the minimum possible operations: 3 pointer copies from the vector swap(), 1 pointer copy for the clang TU.

The self swap() of m_Files frees any over allocated memory.  Since the ctor is the only place this vector is modified, it did not make sense to continue holding on to the memory.
Title: Re: Clang CC
Post by: Alpha on November 11, 2015, 09:05:29 am
Regarding RemoveTranslationUnit(), keep in mind that multiple editors for different files may utilize the same TU.  Also, they are expensive to (re)create, so keeping them around (as memory constraints allow) after closing an editor helps smooth the workflow for people who frequently (re)open and close files.
It was my intent to update CreateTranslationUnit() to treat m_TranslUnits as a cache, allowing it to discard TU's after a certain limit (probably using a variant of LRU).
Title: Re: Clang CC
Post by: yvesdm3000 on November 11, 2015, 10:05:05 am
Regarding RemoveTranslationUnit(), keep in mind that multiple editors for different files may utilize the same TU.

I think I commented-out that capability. Every file has its own TU, mostly for simplicity for now and I'm unsure if I would ever want to re-enable that, given that an include-file might look different with defines set... My focus right now is to have the basic functionality working: have CodeCompletion when you need it, no hangs but without any optimizations regarding resources and thus making many copies across threads.

I think it's even better to cache the TU to disk as a sort-of PCH when the associated file is closed. Symbol search in closed files would then reopen each PCH.

Yves
Title: Re: Clang CC
Post by: l_inc on November 11, 2015, 02:33:30 pm
yvesdm3000
Quote
the elements in the vector<TranslationUnit> will always be copy-constructed, they will then always be copied again when the vector grows beyond the allocated memory
Yes, and is there a guarantee that they won't be copy-constructed twice from the same object? I mean there is a notion of a sane implementation, but no guarantee, right?

Quote
Before C++11 we would have solved this with a pointer to TranslationUnit and do memory-management ourselves
Before C++11, I think, I would rather implement a simplified version of a shared_ptr . This seems much safer.

Alpha
Quote
Casting away a const and editing it is undefined if the original object was created const.
My concerns are rather related to attempts to accidentally do smth. like m_TranslUnits[1] = m_TranslUnits[0]; .
Title: Re: Clang CC
Post by: l_inc on November 11, 2015, 04:05:46 pm
yvesdm3000, Alpha

Another problem is that the plugin sometimes gets into an infinite loop inside wxExecute(command, output, errors, wxEXEC_NODISABLE); . Inside the function int wxGUIAppTraits::WaitForChild(wxExecuteData& execData) to be more precise, while GTK_EndProcessDetector is always getting 0 from waitpid and g++ is somehow not present in the process list.
OK. I was a bit wrong. The infinite loop in WaitForChild results from the call wxExecute(cmd,                 output, wxEXEC_NODISABLE);, which is in "sdk / globals.cpp" . The command line is "wx-config --version=2.8 --cflags". It happens on opening a project, but I don't know what conditions it needs. wxconfig started by the call keeps hanging in the process list, endProcData->pid in wxGUIAppTraits::WaitForChild(wxExecuteData& execData) is negative (not sure yet, if it's supposed to be negative) with an absolute value equal to the hanging process. Here's the call stack:
Code
#0 0x7fb2a8045590	wxGUIAppTraits::WaitForChild(wxExecuteData&) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#1 0x7fb2a8045bfb wxExecute(wchar_t**, int, wxProcess*) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#2 0x7fb2a8046345 wxExecute(wxString const&, int, wxProcess*) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#3 0x7fb2a804232b wxDoExecuteWithCapture(wxString const&, wxArrayString&, wxArrayString*, int) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#4 0x7fb2a8a00851 ExpandBackticks(wxString&) (str=...) (globals.cpp:870)
#5 0x7fb2a89a54d5 CompilerCommandGenerator::SetupCompilerOptions(Compiler*, ProjectBuildTarget*) (this=0x59b9e80, compiler=0x2e97240, target=0x59b4000) (compilercommandgenerator.cpp:937)
#6 0x7fb2a89aa3a2 CompilerCommandGenerator::Init(cbProject*) (this=this@entry=0x59b9e80, project=project@entry=0x5982ae0) (compilercommandgenerator.cpp:161)
#7 0x7fb275f511c9 CompilerMINGW::GetCommandGenerator(cbProject*) (this=<optimized out>, project=0x5982ae0) (compilerMINGW.cpp:51)
#8 0x7fb27f2e0ede ClangPlugin::UpdateCompileCommand(cbEditor*) (this=0x23b0b30, ed=0x5a16ca0) (/tmp/l.inc/ClangLib/clangplugin.cpp:995)
#9 0x7fb27f2decaa ClangPlugin::OnEditorActivate(CodeBlocksEvent&) (this=0x23b0b30, event=...) (/tmp/l.inc/ClangLib/clangplugin.cpp:675)
#10 0x7fb27f2f4f5c cbEventFunctor<ClangPlugin, CodeBlocksEvent>::Call(CodeBlocksEvent&) (this=0x2bed0e0, event=...) (/tmp/l.inc/codeblocks-svn/src/include/cbfunctor.h:49)
#11 0x7fb2a8a15930 Manager::ProcessEvent(CodeBlocksEvent&) (this=<optimized out>, event=...) (manager.cpp:263)
#12 0x7fb2a8a22e35 PluginManager::NotifyPlugins(CodeBlocksEvent&) (this=<optimized out>, event=...) (pluginmanager.cpp:1450)
#13 0x7fb2a89e3ba8 EditorManager::SetActiveEditor(EditorBase*) (this=this@entry=0x1ee2d40, ed=0x5a16ca0) (editormanager.cpp:468)
#14 0x7fb2a89e52c3 EditorManager::CheckForExternallyModifiedFiles() (this=0x1ee2d40) (editormanager.cpp:972)
#15 0x7fb2a803e47c wxEvtHandler::ProcessEventIfMatches(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#16 0x7fb2a803e533 wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#17 0x7fb2a803e8bf wxEvtHandler::ProcessEvent(wxEvent&) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#18 0x7fb2a803e3d8 wxEvtHandler::ProcessPendingEvents() () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#19 0x7fb2a7fb5e69 wxAppConsole::ProcessPendingEvents() () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#20 0x7fb2a80f54ee wxAppBase::ProcessIdle() () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#21 0x7fb2a805f705 wxapp_idle_callback() (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#22 0x7fb2a5c98a8a g_main_context_dispatch() (/lib64/libglib-2.0.so.0:??)
#23 0x7fb2a5c98e20 g_main_context_iterate.isra() (/lib64/libglib-2.0.so.0:??)
#24 0x7fb2a5c98ecc g_main_context_iteration() (/lib64/libglib-2.0.so.0:??)
#25 0x7fb2a7743151 gtk_main_iteration() (/lib64/libgtk-x11-2.0.so.0:??)
#26 0x7fb2a805fc85 wxApp::Yield(bool) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#27 0x7fb2a8045595 wxGUIAppTraits::WaitForChild(wxExecuteData&) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#28 0x7fb2a8045bfb wxExecute(wchar_t**, int, wxProcess*) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#29 0x7fb2a8046345 wxExecute(wxString const&, int, wxProcess*) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)
#30 0x7fb2a804232b wxDoExecuteWithCapture(wxString const&, wxArrayString&, wxArrayString*, int) () (/opt/wx/2.8/lib/libwx_gtk2u-2.8.so.0:??)

OK. This was an easy one. [...] Patch attached.
No, it wasn't. Now code completion related to the closed file is dead. I'll try to have a look, but I hope one of you guys would fix it, cause it seems I miss many assumptions needed to be considered.
Title: Re: Clang CC
Post by: yvesdm3000 on November 11, 2015, 08:32:18 pm

No, it wasn't. Now code completion related to the closed file is dead. I'll try to have a look, but I hope one of you guys would fix it, cause it seems I miss many assumptions needed to be considered.
Yeah I understand code completion is probably dead once you have closed (and reopened) a file. That will certainly be fixed with the upcoming push. This push will feature a duplicate TranslationUnit for the current file that is being reparsed, allowing the typing user to use Code Completion while it's being reparsed. It will also parse a TranslationUnit when the clang TU associated with it is missing, such as is the case once you've closed and reopened a file.

Yves
Title: Re: Clang CC
Post by: l_inc on November 11, 2015, 10:39:35 pm
yvesdm3000
OK. But as of now I've just restored the commented behaviour of GetTranslationUnitId for the OnEditorClose handler only. I suppose this way the translation unit is only killed when its root file is closed, which seems to me most reasonable.

Patch attached. The previous patch should be applied before.
Title: Re: Clang CC
Post by: yvesdm3000 on November 12, 2015, 03:58:10 pm
Mmh, I seem to have run into the same issue when expanding the backticks:

#0  0x00002b410eb4349b in wxPipeInputStream::CanRead() const () from /home/yves/codeblocks-1510/lib/libwx_gtk2u-2.8.so.0
#1  0x00002b410eb446f1 in wxGUIAppTraits::WaitForChild(wxExecuteData&) ()
   from /home/yves/codeblocks-1510/lib/libwx_gtk2u-2.8.so.0
#2  0x00002b410eb42e42 in wxExecute(wchar_t**, int, wxProcess*) ()
   from /home/yves/codeblocks-1510/lib/libwx_gtk2u-2.8.so.0
#3  0x00002b410eb426d7 in wxExecute(wxString const&, int, wxProcess*) ()
   from /home/yves/codeblocks-1510/lib/libwx_gtk2u-2.8.so.0
#4  0x00002b410eb4046b in wxDoExecuteWithCapture(wxString const&, wxArrayString&, wxArrayString*, int) ()
   from /home/yves/codeblocks-1510/lib/libwx_gtk2u-2.8.so.0
#5  0x00002b411ef03486 in ClangPlugin::GetCompilerInclDirs (this=0x3be3ae0, compId=...)
    at /mnt/data/ClangLib/clangplugin.cpp:856
#6  0x00002b411ef03c62 in operator= (this=0x3be3dd8) at /home/yves/wxWidgets-2.8/include/wx-2.8/wx/string.h:659
#7  ClangPlugin::UpdateCompileCommand (this=0x3be3ae0, ed=<optimized out>) at /mnt/data/ClangLib/clangplugin.cpp:1070

I fixed a similar issue before where the UpdateCompileCommand was run inside a timer, and wxExecute also ran timers which in turn ran again UpdateCompileCommand with a wxExecute...   But in this case, it's not a timer issue... Looks more like a WX issue.

Yves

Title: Re: Clang CC
Post by: yvesdm3000 on November 13, 2015, 06:00:58 pm
Regarding RemoveTranslationUnit(), keep in mind that multiple editors for different files may utilize the same TU. 

Coming back on the problem of header files (and in theory other .CPP files, although very very bad practice) linked to the same TU. The problem is pretty complex... If a header-file is self-contained, it can have it's own TU, but --- although this is a good coding practice --- we cannot assume this will always be the case. Even more, a header-file might be freshly created, not referenced in any other file so it gets his own TU and then included in another .CPP file. In this case, we should get rid of the TU since it will also be parsed as part of the CPP file.

And then we have the case of different #defines  set before the inclusion of a header file, which version do we show ? I'm thinking of having a way of preferring the TU that is currently open and prefer that TU so whenever you go to the .CPP file and you select the header file there to open it, it would start using the context of that TU. The only issue here is a desync of how Scintilla will grey out #defines and how CC operates.

For removal, in the short term I'm releasing the TU when the 'main' file that requested the creation of the TU is closed. It can always be reloaded once it's needed (upcoming functionality) and I think I should also start looking into building .PCH files. When we have PCH files from TU's, reloading them is less painful.

Yves
Title: Re: Clang CC
Post by: l_inc on November 13, 2015, 06:07:45 pm
yvesdm3000
Quote
For removal, in the short term I'm releasing the TU when the 'main' file that requested the creation of the TU is closed.
This is pretty much exactly what my little patch does.
Title: Re: Clang CC
Post by: yvesdm3000 on November 14, 2015, 04:36:06 pm
I created a branch "devel" which does not necessarily work correctly, but it should show people what is being fixed and what might allready be fixed.
The current state of that devel-branch is that the main TU is working correctly, but the alternate TU is only giving 0 CodeCompletion results back (no error!). Since each reparse swaps TU, you only get code completion on one TU, and after a reparse you get 0...

I'll have to step thru clang src to see why it behaves like that.

Regarding caching PCH files, is there a good Code::Blocks location where I should place these? Some infrastructure, or is wxStandardPaths::GetTempDir good enough ?

Yves
Title: Re: Clang CC
Post by: l_inc on November 15, 2015, 03:08:18 pm
yvesdm3000
Quote
Regarding caching PCH files, is there a good Code::Blocks location where I should place these? Some infrastructure, or is wxStandardPaths::GetTempDir good enough ?
Do you really mean precompiled headers, or is it supposed to be a code parser database (like .sdf in Visual Studio)? As for the latter I'd prefer to keep the corresponding information in memory. And you can make an option to store it periodically on disk at a user-defined location. GetTempDir is a good default for such a location.

Btw. how is it going with integrating the plugin into codeblocks-contrib?
Title: Re: Clang CC
Post by: yvesdm3000 on November 16, 2015, 07:05:41 am
yvesdm3000
Quote
Regarding caching PCH files, is there a good Code::Blocks location where I should place these? Some infrastructure, or is wxStandardPaths::GetTempDir good enough ?
Do you really mean precompiled headers, or is it supposed to be a code parser database (like .sdf in Visual Studio)? As for the latter I'd prefer to keep the corresponding information in memory. And you can make an option to store it periodically on disk at a user-defined location. GetTempDir is a good default for such a location.

Btw. how is it going with integrating the plugin into codeblocks-contrib?

For PCH, it seems like I can save a whole Translation Unit into a PCH file. Not really useful for code-completion, but for operations like 'go to implementation', 'go to declaration', 'show references' to work efficiently, it's better to have ALL files parsed into AST trees. Obviously we can't assume users will always have enough memory to store that in memory, especially for pretty large projects. So for example 'goto implementation', it would first look in any TU in memory, if it can't find it, it would go into a slower mode and open each PCH file one by one and see if it can find what it's looking for.

For codeblocks-contrib, not sure if I am going to do this immediately, I think the plugin needs restructuring first. I want to split the plugin into pieces:

1. ClangLib, just updating its internal database, parsing documents as they are loaded&changed and governing the threads for doing the tasks it publishes in its public API
2. ClangCC using ClangLib API, it will perform CodeCompletion and associated settings nothing more
3. ClangToolbar using ClangLib API, it will allow a toolbar with class & method name
4. ClangCodeRefactor using ClangLib API, it will allow context-menu's to lookup declaration, implementation and maybe higher level code refactoring stuff like reference searching and symbol renaming

Yves
Title: Re: Clang CC
Post by: oBFusCATed on November 16, 2015, 08:43:39 am
Btw. how is it going with integrating the plugin into codeblocks-contrib?
This one will land in the core someday. This is not a contrib plugin in.

For codeblocks-contrib, not sure if I am going to do this immediately, I think the plugin needs restructuring first. I want to split the plugin into pieces:
If you want to split this in multiple cb plugins, keep in mind that cb gets slower to start the more plugins there are installed (no matter if enabled or disabled).
Also do you really have added so much code you need to separate it into different pieces?
Title: Re: Clang CC
Post by: yvesdm3000 on November 16, 2015, 11:52:13 am
Btw. how is it going with integrating the plugin into codeblocks-contrib?
This one will land in the core someday. This is not a contrib plugin in.

For codeblocks-contrib, not sure if I am going to do this immediately, I think the plugin needs restructuring first. I want to split the plugin into pieces:
If you want to split this in multiple cb plugins, keep in mind that cb gets slower to start the more plugins there are installed (no matter if enabled or disabled).
Also do you really have added so much code you need to separate it into different pieces?

You do have a good point there. I would mostly do it so there would be room for other plugin developers to use Clang functionality without having to hack on the existing plugin. The best way to do that is to actually use this API ourselves and allow others replace certain behaviours.

Yves

Title: Re: Clang CC
Post by: l_inc on November 16, 2015, 12:59:02 pm
yvesdm3000
Quote
Obviously we can't assume users will always have enough memory to store that in memory, especially for pretty large projects.
Well, I have 16GBs in my work laptop, and I would appreciate an option to not drain my SSD of life with temporary stuff. Besides, you would still need to be able to handle low disk space situations, so why wouldn't you leverage the same out-of-space-avoidance mechanism in memory?

Quote
it's better to have ALL files parsed into AST trees. [...] So for example 'goto implementation', it would first look in any TU in memory, if it can't find it, it would go into a slower mode and open each PCH file one by one and see if it can find what it's looking for.
Yes, and for the option of no on-disk ASTs it would be necessary to parse all the remaining TUs (a progress bar would make sense), but that's the price for not having enough cache space, no matter if it's on disk or in memory.
Title: Re: Clang CC
Post by: Alpha on November 16, 2015, 06:41:36 pm
yvesdm3000
Quote
it's better to have ALL files parsed into AST trees. [...] So for example 'goto implementation', it would first look in any TU in memory, if it can't find it, it would go into a slower mode and open each PCH file one by one and see if it can find what it's looking for.
Yes, and for the option of no on-disk ASTs it would be necessary to parse all the remaining TUs (a progress bar would make sense), but that's the price for not having enough cache space, no matter if it's on disk or in memory.
The support I had initially attempted for in plan of this is TokenDatabase (roughly) keeps track (in memory) of where main definitions exist; then if any of those ASTs are not currently loaded, the could be found directly (not implemented), instead of having to re-parse the entire project.  The project does need to be parsed once at least, and this index probably should be serialized for searching to be useful between sessions.
Title: Re: Clang CC
Post by: l_inc on November 16, 2015, 06:54:20 pm
Alpha
This would only help with "find implementation". But "find all references" would still require to traverse the complete set of ASTs. So I support the TokenDatabase idea with respect to keeping it in memory for "find implementation" optimization, but no serialization, because re-/creation of all the non-existent ASTs is still needed for other things.
Title: Re: Clang CC
Post by: yvesdm3000 on November 17, 2015, 07:45:34 am
Yes TokenDatabase is used as a 'global' database, but does not contain everything.

When you'd look for references or anything else that would take some time, it should certainly show a dialog-box with a progress bar.

For the splitup, I've settled with an internal splitup to keep things organized and less spaghetti, so only 1 plugin but different components inside. The only drawback is that the toolbar is named 'ClangLib', which is an ugly name for it.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on November 21, 2015, 05:35:17 pm
Hi,

I've put a binary up on the GitHub page. This version is compiled for Ubuntu 15.10

Don't expect a speed-wonder, but it does work and doesn't stop the user from typing.
You'll also have to wait a little after startup since it will still be parsing the files in the background and obviously you can't have Code-Completion yet when the file is not yet parsed. One of the next steps to take is to cache Translation Units to disk so that next time you open the project, it can use cached values.

I also implemented the toolbar that is now using Clang and is mostly a copy of the toolbar from the regular CodeCompletion plugin but is obviously driven by data coming out of Clang. There are still some limitations in the toolbar but nothing that prevents you from using it.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on November 30, 2015, 10:57:34 am
I just pushed some more commits to the master branch with some more fixes  in it and also fixes for the inline diagnostics, and asynchronous documentation (this was still an operation that kept the UI hanging).

I'll put a new binary up soon, and look into building a binary for windows using MingW.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on December 01, 2015, 07:00:03 am
Hi,

I added a .cbplugin file for ubuntu 15.10 to the 0.3 release page for your convenience.

https://github.com/yvesdm3000/ClangLib/releases/tag/0.3

Yves

Title: Re: Clang CC
Post by: yvesdm3000 on December 01, 2015, 02:10:01 pm
yvesdm3000
Quote
Obviously we can't assume users will always have enough memory to store that in memory, especially for pretty large projects.
Well, I have 16GBs in my work laptop, and I would appreciate an option to not drain my SSD of life with temporary stuff. Besides, you would still need to be able to handle low disk space situations, so why wouldn't you leverage the same out-of-space-avoidance mechanism in memory?

Quote
it's better to have ALL files parsed into AST trees. [...] So for example 'goto implementation', it would first look in any TU in memory, if it can't find it, it would go into a slower mode and open each PCH file one by one and see if it can find what it's looking for.
Yes, and for the option of no on-disk ASTs it would be necessary to parse all the remaining TUs (a progress bar would make sense), but that's the price for not having enough cache space, no matter if it's on disk or in memory.

Hi l_inc,

I'm looking now at generating PCH files to accelerate the startup. I think I can manage to make this functionality optional since in any case I can never assume a PCH file is always available for any source file...

Yves
Title: Re: Clang CC
Post by: l_inc on December 01, 2015, 11:16:55 pm
yvesdm3000
Thank you for taking that into account. Thing is that I'd probably leave the functionality enabled in most cases, but the storage location would be on a RAM drive. This would allow to have accelerated startup until reboot.
Title: Re: Clang CC
Post by: yvesdm3000 on December 04, 2015, 07:04:43 am
Another push: initial configuration dialog and added some quick-exit situations for code-completion.

I'm currently wondering how I could implement the Clang 'fixit' functionality. These are warnings/errors where Clang can suggest what exactly needs to be done, like adding extra brackets or adding ';' at the end, with precise locations etc. I don't want to execute these fixits automatically since the user might have another intention, so I want something where I do insert them temporarily, but are highlighted with some backgroundcolor and if the user goes to another line, these are removed again, but then I need something visual to actually apply the fixit...

If someone has ideas, I'm open to suggestions!

Yves
Title: Re: Clang CC
Post by: oBFusCATed on December 04, 2015, 09:01:54 am
Show a popup with a button, but please add this in the C::B's SDK and post  a patch for it, thus other CC implementers can benefit from it.
Title: Re: Clang CC
Post by: l_inc on December 04, 2015, 12:59:30 pm
yvesdm3000
Quote
I want something where I do insert them temporarily, but are highlighted with some backgroundcolor and if the user goes to another line, these are removed again
This way the original code will potentially be overwritten, and the fix suggestion might mislead if not appropriate. If you don't make the fix suggestions persistently visible, then I'd prefer a popup as oBFusCATed suggested.

However the persistently visible warnings and errors is what I like a lot about ClangCC. Thus embedding a fix suggestion in-between the lines together with the actual warning/error should be even better. But I guess this way is not as compatible with multiple fix suggestions, as a popup.

Quote
but then I need something visual to actually apply the fixit
As for me a couple of shortcuts would suffice (fix, show another fix, goto next warning/error, goto previous warning/error), but I like when GUI provides multiple ways of doing something, so I would prefer to see these buttons on the ClangCC toolbar as well. Buttons inside of a popup seem to me an overkill.
Title: Re: Clang CC
Post by: oBFusCATed on December 04, 2015, 05:46:52 pm
Check how it is done in XCode, I've heard that theirs is a good solution to this problem.
Also adding shortcuts is fine, but they should be the second option, because most of the times they are not obvious and hard to find by new users of the feature.

One option is to use the scintilla's inline annotations, but I don't know if you can add buttons in it.
Another is to use some kind of an editor marker that when clicked pops up some ui.
Title: Re: Clang CC
Post by: yvesdm3000 on December 05, 2015, 09:23:23 am
OK I'll hande/implement this first in the plugin, but I'll make sure I can easily move it to C::B later on. This enables me to keep using stock code::blocks and the early adopters have less work following progress since they don't have to patch codeblocks.

Looks like XCode replaces the whole line with the fixit when not yet applied and reverts that if you don't apply it. The choice to apply it is not really a button but a context-menu similar to the CodeCompletion-context-menu.

About the confusion with the error/warning annotations, I would not change the existing behaviour, I treat these fixits as something extra, so there can be an error-annotation saying there is a missing ')' and also a fixit that pre-visualizes the ')' at the end. For now I only need something extra for the 'apply'-button/menu/whatever.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on December 05, 2015, 09:46:13 am
As for me a couple of shortcuts would suffice

I'm aware we lack operations on these warning/errors. I would at least need goto next/previous error/warning with good shortcuts. It's however pretty confusing since there are also the build error/warnings which are not necessarily from the same compiler.
I also think it would be a good idea to have a separate output view that outputs there errors, similar to the build messages. Maybe some people don't like them to be inlined, it would also give a more global overview of the amount of warning/errors.

Yves
Title: Re: Clang CC
Post by: oBFusCATed on December 05, 2015, 10:41:36 am
I also think it would be a good idea to have a separate output view that outputs there errors, similar to the build messages.
Please don't do this, we have too many logs already, and I'm sure more people won't like the new log pane, then the integrated annotations!

Maybe some people don't like them to be inlined, it would also give a more global overview of the amount of warning/errors.
You'll have to find a way to reuse one of the compiler logs or even both. Probably we should move them to the sdk somehow.
Title: Re: Clang CC
Post by: yvesdm3000 on December 07, 2015, 10:46:48 am
I compiled a version for Windows now. It seems to work fine but makes CodeBlocks very unstable, disappearing widgets etc... Still need to figure out why that happens before I publish a binary.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on December 07, 2015, 02:45:29 pm
I released an alpha-version for Windows here:

https://github.com/yvesdm3000/ClangLib/releases/tag/0.3.1

All constructive feedback is welcome.

Yves
Title: Re: Clang CC
Post by: MortenMacFly on December 07, 2015, 08:22:57 pm
All constructive feedback is welcome.
I've added a tiny note on commit:

https://github.com/yvesdm3000/ClangLib/commit/d0d558a1bf5ffe00b3a6dfc4ee816ea6fb48be2a
Title: Re: Clang CC
Post by: yvesdm3000 on December 08, 2015, 07:23:49 am
All constructive feedback is welcome.
I've added a tiny note on commit:

https://github.com/yvesdm3000/ClangLib/commit/d0d558a1bf5ffe00b3a6dfc4ee816ea6fb48be2a

I now incorporated your suggestion.

Too bad the tabjump doesn't work on Windows, the way I implemented it is probably a little bit too hackish.

Maybe I should add this to the Scintilla wrapper class instead, so that the other CC implementations can use it too ? It would solve the "hack"-issue too...

The feature allows you to jump to the next parameter of a function that was inserted with CC, using the Tab-key. This also selects the placeholder text so that the user can easily start typing the parameter value.
This works nicely on Linux.

Yves
Title: Re: Clang CC
Post by: l_inc on December 08, 2015, 07:57:33 pm
yvesdm3000
You made a whole bunch of methods public, which rendered the plugin unusable, cause the vector of TUs now works with them as with copyable objects, which results in exceptions (("Illegal copy attempted of TranslationUnit object.")). Btw. the "-ansi" option must be removed, cause it disables many features of C++11.

Another problem is that the plugin works for a single TU only if there are multiple open.
Title: Re: Clang CC
Post by: yvesdm3000 on December 08, 2015, 09:14:35 pm
yvesdm3000
You made a whole bunch of methods public, which rendered the plugin unusable, cause the vector of TUs now works with them as with copyable objects, which results in exceptions (("Illegal copy attempted of TranslationUnit object.")). Btw. the "-ansi" option must be removed, cause it disables many features of C++11.

Another problem is that the plugin works for a single TU only if there are multiple open.

Can you be a little bit more specific? Do you use the precompiled version or do you compile it yourself? The error you are complaining about is only enabled when C++11 is enabled so I assume you compile the plugin yourself.
C++11 is not a requrement since that is not what C::B uses...  I even don't test it for C++11 since I want to focus on implementing features first and make the changes for C++11 later (it's easier to move from old to new C++). I know there is the move-operator and stuff that is defined away, but currently that is not supposed to be enabled for now. I can even imagine it doesn't work at all because I swap away a TU that is being reparsed, to move it out from under the lock, you get huge congestion otherwise, blocking everything while parsing/reparsing.

For me the plugin does not work with a single TU, it switches TU when the associated document is opened. And I happen to push a change a couple of hours ago that fixes the "Find imlementation" to search across multiple TU's. Obviously those TU's need to be opened first... Code-completion only needs 1 TU. If you want symbols from non-included headers, better include the header then.

Regarding broadening the scope for the "Find implementation" to the full project, that is still a TODO and I think it's better to implement persistency of the internal database first and then do any indexing of all the documents in the project.

I remind you this is only a prerelease and by all means not a final product, although I do use it for my daily main job now.

Yves
Title: Re: Clang CC
Post by: l_inc on December 08, 2015, 11:11:07 pm
yvesdm3000
Quote
Can you be a little bit more specific?
Sorry, I was in a hurry. I do indeed compile the plugin. Precompiled versions of CodeBlocks plugins seem to me only good when being part of a distribution. Otherwise their usefulness scope is too limited.

Quote
I even don't test it for C++11 since I want to focus on implementing features first and make the changes for C++11 later
Wouldn't you be willing to consider dropping support for C++ older than C++11 at all? C++11 is safer in many ways. E.g. much safer would be to have a vector of unique_ptr's to TUs with enforced move-semantics than having these dirty swap-hacks in a copy-constructor. I see no reason to continue suffering in supporting the older standards. I would even expect the "mysterious crashes" you are talking about in your todo-list to go away after conscientiously switching to C++11.

Quote
I remind you this is only a prerelease and by all means not a final product, although I do use it for my daily main job now.
I understand that. I was just shocked to see the commented private replaced with public and the private methods being used outside of the class scope. This made me scared about the project becoming a heap of crutches even before the first release would come out.
Title: Re: Clang CC
Post by: yvesdm3000 on December 09, 2015, 01:13:49 am

Quote
I remind you this is only a prerelease and by all means not a final product, although I do use it for my daily main job now.
I understand that. I was just shocked to see the commented private replaced with public and the private methods being used outside of the class scope. This made me scared about the project becoming a heap of crutches even before the first release would come out.

Thanks for pointing this out. I've put it back to private. This was most certainly only a debugging measure.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on December 09, 2015, 01:19:26 am
The mysterious crashes can be removed from the list for my part. I haven't had a 'mysterious' crash anymore for over a month now... There might still be in the Windows-port but I can't do any user-testing with it using my daily job...

Yves

Title: Re: Clang CC
Post by: yvesdm3000 on December 09, 2015, 01:30:38 am
For the C++11 requirement please ask the C::B developers. They seem to be open to include this plugin into mainstream so I don't want to bother them with restricting requirements from my part... And even for myself, I need to compile this thing in RHEL5/CentOS5 although I do think I can manage with C++11.

Yves
Title: Re: Clang CC
Post by: l_inc on December 09, 2015, 02:38:49 am
yvesdm3000
Quote
For me the plugin does not work with a single TU, it switches TU when the associated document is opened.
I was talking about the diagnostics. I've just debugged it a little. And the problem seems to be the GetCurrentTranslationUnitId method, cause it always returns 0. So only the TU that happened to have gotten the m_TranslUnitId == 0 has the diagnostics displayed, which is ensured at the beginning of OnDiagnostics . I removed the if ( m_TranslUnitId == -1 ) condition to always resolve the current TU in GetCurrentTranslationUnitId, and it works OK now.
Title: Re: Clang CC
Post by: yvesdm3000 on December 09, 2015, 06:54:11 am
yvesdm3000
Quote
For me the plugin does not work with a single TU, it switches TU when the associated document is opened.
I was talking about the diagnostics. I've just debugged it a little. And the problem seems to be the GetCurrentTranslationUnitId method, cause it always returns 0. So only the TU that happened to have gotten the m_TranslUnitId == 0 has the diagnostics displayed, which is ensured at the beginning of OnDiagnostics . I removed the if ( m_TranslUnitId == -1 ) condition to always resolve the current TU in GetCurrentTranslationUnitId, and it works OK now.
You seem to have found a bug there. Thanks for sharing !

Yves
Title: Re: Clang CC
Post by: l_inc on December 09, 2015, 01:36:04 pm
yvesdm3000
My pleasure. I want the plugin to work correctly and efficiently almost as much as you do, I guess.
Title: Re: Clang CC
Post by: Calmarius on December 17, 2015, 03:24:35 pm
Will this clang CC solve the build configuration problem?
Most open source projects out there are compiled by the "./configure; make" combo.
And makefiles and configurations can set bazilions preprocessor directives and extra include paths. Can it go through these and still parse correctly?

Title: Re: Clang CC
Post by: l_inc on December 17, 2015, 03:35:10 pm
Calmarius
The configuration you use for editing the source code is not necessarily the one derived by a particular invocation of configure on your system. So it's not the perfect choice to directly rely on it.

As for Clang CC, it relies on the configuration specified in the CB project. If you want the code completion and diagnostics to reflect the full configuration used for building the project, you need to setup the project build options accordingly.
Title: Re: Clang CC
Post by: MortenMacFly on December 17, 2015, 09:14:12 pm
Will this clang CC solve the build configuration problem?
Most open source projects out there are compiled by the "./configure; make" combo.
And makefiles and configurations can set bazilions preprocessor directives and extra include paths. Can it go through these and still parse correctly?
How should it work? Configure is there to setup the code for your platform and to your needs / options / compilers... So unless you provide magic balls how on earth should any parser know and resolve this?

The only right thing to do it to actually run configure and have the right setup in place.
Title: Re: Clang CC
Post by: Alpha on December 20, 2015, 10:46:24 pm
@yvesdm3000 I have pulled your recent work, and am running into an odd issue.  If I type ; or if I press enter, C::B freezes after about 0.5 seconds.  However, if C::B is running under gdb, there are no freezes.  Is there some additional needed configuration step?  (The only local changes I added are for correct paths in the .cbp.)
Title: Re: Clang CC
Post by: l_inc on December 21, 2015, 12:06:19 am
Alpha
You can attach to the running CB when it's already frozen and at least to lookup the call stacks.
Title: Re: Clang CC
Post by: Alpha on December 21, 2015, 01:32:23 am
Thanks.  Turns out it was the debug printing attempting send to the console when C::B was started full GUI mode that caused the freeze.  (Starting C::B inside a terminal works just fine.)
Title: Re: Clang CC
Post by: yvesdm3000 on December 21, 2015, 03:02:50 pm
Pretty interesting, I always run it from the terminal to get the debugging output... Glad to see you're up and running again. Maybe we should move the debug-output to a macro or something.

I'm currently tackling the TokenDatabase to span the whole project/workspace with persistence ondisk. This is needed for useful 'go to implementation' and most refactoring operations. For normal Code-completion this is not really necessary, but some results-filtering is currently still disabled because I don't want to make it another thread-context-switch, I think it should be done in the same flow where code-completion was requested. It's not hard, but I don't really miss it myself.

Yves
Title: Re: Clang CC
Post by: Jenna on December 21, 2015, 04:52:48 pm
Why not use the logmanager ?
Code: cpp
Manager::Get()->GetLogManager()->DebugLog(F(_("")));
or
Code: cpp
Manager::Get()->GetLogManager()->DebugLogError(F(_("")));

This logs to debug-log if available and/or to console (if open).
Title: Re: Clang CC
Post by: yvesdm3000 on December 22, 2015, 06:06:59 am
Is it threadsafe ?

Yves
Title: Re: Clang CC
Post by: ollydbg on December 22, 2015, 06:17:55 am
Is it threadsafe ?

Yves
NO, please see the whole thread discussion here: F() function is not thread safe? (http://forums.codeblocks.org/index.php/topic,18773.msg128612.html#msg128612), also the logger function is not thread safe.

EDIT: That's why the native cc plugin use it's own logger.
Title: Re: Clang CC
Post by: yvesdm3000 on December 22, 2015, 06:40:35 am
fprintf was a good choice then. I will probably copy the logger from the native CC, no need to reinvent the wheel.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on December 22, 2015, 08:29:37 am
The devel-branch now uses the CCLogger from the regular CodeComplete plugin.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on January 23, 2016, 03:44:15 pm
Did another push to the 'devel'-branch that makes the tokendatabase persistent and reindexes at project load. I'm  now stuck on a situation where Clang sometimes crashes when reading a PCH file so I'll probably skip using PCH files and reparse the file when needed and implement the PCH-optimization later.

My current goal is to make "Goto implementation" work correctly when the file is not yet open.

Yves
Title: Re: Clang CC
Post by: MortenMacFly on January 24, 2016, 03:33:13 pm
I also noticed some fatal crash bug: If you disable the plugin and then open an editor, C::B will crash immediately. The reason is that you don't check whether or not the plugin is still enabled. Therefore, the pointer to m_pClangPlugin will be invalid when accessed in clangdiagnostics.cpp here:
(line 145:)         m_TranslUnitId = m_pClangPlugin->GetTranslationUnitId(fn);
Probably that is true on other places, as well... be careful when developing...

The proper implementation is also to detach from the event queue and event sinks, if the plugin is disabled, so pleae add:
Code
    // remove registered event sinks
    Manager::Get()->RemoveAllEventSinksFor(this);
...in ClangDiagnostics::OnRelease( IClangPlugin* pClangPlugin ).
Title: Re: Clang CC
Post by: yvesdm3000 on January 24, 2016, 08:06:54 pm
Hi MortenMacFly,

I committed your proposed fix on master. Looks like this was only overlooked in clangdiagnostics.cpp, the other places has the "RemoveAllEventSinksFor(this);".

Many thanks for pointing this out!

Yves
Title: Re: Clang CC
Post by: MortenMacFly on February 06, 2016, 08:44:33 pm
I committed your proposed fix on master. Looks like this was only overlooked in clangdiagnostics.cpp, the other places has the "RemoveAllEventSinksFor(this);".
OK, thank you. BTW: There are many more crash candidates. Unfortunately if you use this plugin on Windows it will definitely crash sooner or later. I've never been able to run a complete C::B session w/o crash. :-/

Another thing: Please don't use code that is not portable. alloca for example is very bad and should never be used. Also, currently your git HEAD is broken because it seems you did commit a file (tokendatabase.cpp) w/o resolving conflicts first.

If possible, I would recommend that you test this plugin from time to time on Windows. Be careful with hacks, they usually don't work on other platforms. If it compiles on Windows, I can help with that from time to time... just guide me what to do.
Title: Re: Clang CC
Post by: MortenMacFly on February 06, 2016, 09:00:50 pm
Here comes another reproducible crash:
* have Clang plugin enabled
* disable the plugin in plugin manager
* search in files so that the result (listlog) has some entries
* try to open a file from the log (double-click)
-> Crash
Title: Re: Clang CC
Post by: yvesdm3000 on February 07, 2016, 03:31:01 pm
OK thank you for the bug report, I'll look into it as soon as I have some time for it.

I mostly test the plugin using my day-to-day work which is Linux-based ...

Maybe I should have some sessions on windows just so that I can try things, or have some unit-tests to run, it would certainly improve reliablity...

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on February 08, 2016, 09:50:29 am
I was able to reproduce the problem and I fixed it in master. Thanks for the bugreport !

There is however something fishy in Manager::Get()->RemoveAllEventSinksFor() that I'll investigate later.

Yves

Title: Re: Clang CC
Post by: yvesdm3000 on February 09, 2016, 11:51:36 am
Another thing: Please don't use code that is not portable. alloca for example is very bad and should never be used.
A question, where did you see the use of alloca ?

Please always use the "master" branch. The "develop" branch is mostly playground for new features and is hopelessly unstable right now due to some problems inside the clang-library itself.

Yves
Title: Re: Clang CC
Post by: Alpha on February 09, 2016, 03:30:02 pm
By the way, I would like to help clean up code when I have time.  Do you think you will have a set of commits ready for me to pull upstream anytime soon?
Title: Re: Clang CC
Post by: yvesdm3000 on February 09, 2016, 04:24:45 pm
Hi Alpha,

I'm not ready to push this upstream yet:

I'm mostly working on the tokendatabase right now, using an indexing job to parse all files within the workspace. It's a complex matter, with tokens parsed from unsaved files going into the database but you don't want half of the tokens available to the point where it's parsing in any db lookup from the editor, and we only want it persisted when the file is saved (otherwise you type some symbols, they get parsed, inserted in the database and then you close without saving ==> tokens still in the database) etc...

I first tried with a revision number, but that became hopelessly complex so I dropped this in favour of different instances of tokendatabases that can be merged when needed or dropped on failure or cancellation. It has a superior advantage that a thread can update it's own tokendatabase and it gets merged into the main database after the file is successfully parsed.

Yves
Title: Re: Clang CC
Post by: MortenMacFly on February 09, 2016, 08:07:20 pm
By the way, I would like to help clean up code when I have time.
You could also just fork from Yves repo and send pull requests to Yves...
Title: Re: Clang CC
Post by: Alpha on February 10, 2016, 01:49:11 am
By the way, I would like to help clean up code when I have time.
You could also just fork from Yves repo and send pull requests to Yves...
I have done that already.  However, since it would likely include a decent number of whitespace edits, I was trying to avoid merging troubles.
Title: Re: Clang CC
Post by: yvesdm3000 on February 10, 2016, 11:50:51 am
OK, thank you. BTW: There are many more crash candidates. Unfortunately if you use this plugin on Windows it will definitely crash sooner or later. I've never been able to run a complete C::B session w/o crash. :-/

Another thing: Please don't use code that is not portable. alloca for example is very bad and should never be used. Also, currently your git HEAD is broken because it seems you did commit a file (tokendatabase.cpp) w/o resolving conflicts first.

If possible, I would recommend that you test this plugin from time to time on Windows. Be careful with hacks, they usually don't work on other platforms. If it compiles on Windows, I can help with that from time to time... just guide me what to do.

OK, it certainly looks like you are viewing the 'devel' branch since the alloca and merge-conflict stuff are in there. It should actually be named 'experimental' and contains a lot of hacks and other things that certainly should not be in master (like you pointed out).
To cope with this misunderstanding I'll create a new branch which is named 'experimental' and I hope it should be more clear for people trying stuff.

I now used the build from the 'master' brand on 16.01 on windows for a couple of hours without any crashing-issue. There is however some desync issue where *rarely* the chosen code-completion is not the one inserted in the editor that I'll look into shortly.

Yves
Title: Re: Clang CC
Post by: MortenMacFly on February 10, 2016, 08:00:30 pm
OK, it certainly looks like you are viewing the 'devel' branch since the alloca and merge-conflict stuff are in there. It should actually be named 'experimental' and contains a lot of hacks and other things that certainly should not be in master (like you pointed out).
Ah - ok, that might be my mistake. I though "devel" is where development occurs and master you only use to sync with Alpha. Good to know. So I'll switch to master and try again...
Title: Re: Clang CC
Post by: yvesdm3000 on February 20, 2016, 09:12:33 am
I made another push to master: Mostly speed and stability improvements.

https://github.com/yvesdm3000/ClangLib

Regarding code-completion, I think that is feature complete. I'm moving on to code-refactoring things.

Yves
Title: Re: Clang CC
Post by: ollydbg on February 21, 2016, 03:23:38 pm
I have some questions:
1, what are the m_UnsavedFiles means in all the Jobs? (in clangproxy.h file), does every job need such parameter?
2, about SyncJob, you just want to run the task in worker thread, but the mainthread is just blocked, is it possible our ccmanager supply some interface which support synchronized request? I mean if ccmanager ask a code completion list, it just returned, and let a specific cc plugin to do the job to fill something(through synchronized api), and this way, we won't let the main gui thread get blocked

EDIT: I would suggest you can add some doxygen style document to the source code, so that people(include me) can read and understand or debug the code easily.
Title: Re: Clang CC
Post by: yvesdm3000 on February 21, 2016, 05:00:48 pm
Hi Ollydbg,

I have some questions:
1, what are the m_UnsavedFiles means in all the Jobs? (in clangproxy.h file), does every job need such parameter?
This is to give Clang access to the files that are open in the editor and are not saved, e.g. the changes that are in the buffer of the editor.
These are duplicated since Clang runs in a thread and the user might start changing them. It results in some nice asserts in libclang when not deep-copied. One might optimize this by only passing the ones that are needed, but it's more complicated than at first sight:
The user might have added an #include statement to an Unsaved header file and we haven't parsed the #include statement and thus we are unaware there is an UnsavedFile we need to pass, so for now I just chose to pass them all and postpone optimizations for later. It has a low impact and a lot less that I would have guessed at the start, so this would have been a premature optimization.

Quote
2, about SyncJob, you just want to run the task in worker thread, but the mainthread is just blocked, is it possible our ccmanager supply some interface which support synchronized request? I mean if ccmanager ask a code completion list, it just returned, and let a specific cc plugin to do the job to fill something(through synchronized api), and this way, we won't let the main gui thread get blocked
SyncJob is mostly 'optionally' synchronous, e.g. the caller can wait for it with a timeout. Initially I didn't understand how the C::B 'GetAutocomplist' worked, it seemed like a synchronous operation to me so I figured I'd start a code-completion and when it took too long, just return empty and be ready for the next request. Later on I saw you can actually do the same code-completion request once the threaded one is finished so I figured I keep the timeout-implementation since it keeps the total operation simple&fast when the clang-operation is faster than 20 milliseconds. So theoretically the 'SyncJob' could be removed from the project again, or at least for the CodeCompleteAtJob.
On the other hand, I do think the 'GetAutocomplist'-call in the cc-plugin framework looks like an afterthought, something with a request event and a response event would probably have been much nicer, but I'm unaware of the lowlevel choices and scintilla if that would even be possible.

Quote
EDIT: I would suggest you can add some doxygen style document to the source code, so that people(include me) can read and understand or debug the code easily.
Sure thing. I'll add it to the top of my todo list for you.

Yves

Title: Re: Clang CC
Post by: ollydbg on February 23, 2016, 08:08:58 am
@yves:
I think the 20 ms lock time can be removed, at least I see a time delay in the ccmanager, see below:
Code
void CCManager::OnEditorHook(cbEditor* ed, wxScintillaEvent& event)
{
    wxEventType evtType = event.GetEventType();
    if (evtType == wxEVT_SCI_CHARADDED)
    {
        const wxChar ch = event.GetKey();
        // get the char set the active ccplugin is interest to trigger the call tip
        CCPluginCharMap::const_iterator ctChars = m_CallTipChars.find(GetProviderFor(ed));
        if (ctChars == m_CallTipChars.end())
            ctChars = m_CallTipChars.find(nullptr); // default char set

        // Are there any characters which could trigger the call tip?
        if (ctChars->second.find(ch) != ctChars->second.end())
        {
            int tooltipMode = Manager::Get()->GetConfigManager(wxT("ccmanager"))->ReadInt(wxT("/tooltip_mode"), 1);
            if (   tooltipMode != 3 // keybound only
                || m_CallTipActive != wxSCI_INVALID_POSITION )
            {
                wxCommandEvent pendingShow(cbEVT_DEFERRED_CALLTIP_SHOW);
                AddPendingEvent(pendingShow);
            }
        }
        else
        {
            cbStyledTextCtrl* stc = ed->GetControl();
            const int pos = stc->GetCurrentPos();
            const int wordStartPos = stc->WordStartPosition(pos, true);
            // get the char set the active ccplugin is interest to trigger the suggestion list
            CCPluginCharMap::const_iterator alChars = m_AutoLaunchChars.find(GetProviderFor(ed));
            if (alChars == m_AutoLaunchChars.end())
                alChars = m_AutoLaunchChars.find(nullptr); // default char set

            // auto suggest list can be triggered either:
            // 1, some number of chars are entered
            // 2, an interested char belong to alChars is entered
            int autolaunchCt = Manager::Get()->GetConfigManager(wxT("ccmanager"))->ReadInt(wxT("/auto_launch_count"), 3);
            /* TODO (ollydbg#1#02/22/16): What does the magic number 4 means? */
            if (   (pos - wordStartPos >= autolaunchCt && !stc->AutoCompActive())
                || pos - wordStartPos == autolaunchCt + 4 )
            {
                CodeBlocksEvent evt(cbEVT_COMPLETE_CODE);
                Manager::Get()->ProcessEvent(evt);
            }
            else if (alChars->second.find(ch) != alChars->second.end())
            {
            /* TODO (ollydbg#1#02/22/16): What does the 10 ms used for? Can we just ask plugin to start a code completion session? */
                m_AutoLaunchTimer.Start(10, wxTIMER_ONE_SHOT);
                m_AutocompPosition = pos; // remember the current caret position
            }
        }
    }
...

You can see, there are 10 ms there.

Quote
On the other hand, I do think the 'GetAutocomplist'-call in the cc-plugin framework looks like an afterthought, something with a request event and a response event would probably have been much nicer, but I'm unaware of the lowlevel choices and scintilla if that would even be possible.
I think this is a good method, and we can go with this direction.
Title: Re: Clang CC
Post by: yvesdm3000 on February 23, 2016, 08:39:08 am
@yves:
I think the 20 ms lock time can be removed, at least I see a time delay in the ccmanager, see below:

You can see, there are 10 ms there.
This is not the same timeout. The 20 ms comes after the 10 ms you identified and gives Clang 20 ms to calculate the CC results. If it can do it in 20ms, the code-completion results are returned immediately, otherwise the clang-plugin will post a new code-completion request when the results have arrived.

Quote
Quote
On the other hand, I do think the 'GetAutocomplist'-call in the cc-plugin framework looks like an afterthought, something with a request event and a response event would probably have been much nicer, but I'm unaware of the lowlevel choices and scintilla if that would even be possible.
I think this is a good method, and we can go with this direction.
Beware that with the current code-completion support in C::B, it is very complex where a code-completion is requested, handled in a thread, the editor is then modified and then when the operation is finished, the plugin needs to figure out if the code-completion still applies because another code completion might have come in, it might be identical, it might not be, the same request might be pending, and then you do another one in our own callback handler etc... The amount of cases where it can do something else than the user expects is very high.
Nevertheless it's there, in 16.01 so we'll live with it.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on February 23, 2016, 08:40:20 am
Oh, and I pushed some doxygen documentation to the 'staging' branch. It's not everything, but it's a start.

Yves
Title: Re: Clang CC
Post by: ollydbg on February 23, 2016, 08:48:23 am
@yves:
I think the 20 ms lock time can be removed, at least I see a time delay in the ccmanager, see below:

You can see, there are 10 ms there.
This is not the same timeout. The 20 ms comes after the 10 ms you identified and gives Clang 20 ms to calculate the CC results. If it can do it in 20ms, the code-completion results are returned immediately, otherwise the clang-plugin will post a new code-completion request when the results have arrived.
You are right, the 20 ms lock is from clangcc, but 10 ms delay is from ccmanager, I mean we do not need the 10ms delay(note that under windows 10ms could be 16ms, because the system time tick), and we just start the clangcc, thus we can save a lot of time.

Oh, and I pushed some doxygen documentation to the 'staging' branch. It's not everything, but it's a start.

Yves
Great job!
I think I will add some doxygen style document to ccmanager class either. :)
Title: Re: Clang CC
Post by: ollydbg on February 23, 2016, 09:06:16 am
Beware that with the current code-completion support in C::B, it is very complex where a code-completion is requested, handled in a thread, the editor is then modified and then when the operation is finished, the plugin needs to figure out if the code-completion still applies because another code completion might have come in, it might be identical, it might not be, the same request might be pending, and then you do another one in our own callback handler etc... The amount of cases where it can do something else than the user expects is very high.
Nevertheless it's there, in 16.01 so we'll live with it.
The current way when code-completion is requested
1, ccmanager call a function in ccplugin, thus a request is send to ccplugin
2, ccplugin directly return some token list back to ccmanager, thus ccmanager show the results.


EDIT: the asynchronized request is just my idea.
An asynchronized request:
1, ccmanager call a function in ccplugin, thus a request is send to ccplugin
2, ccplugin return a special value(maybe a number), and ccmanager know it will return the results later.
3, ccplugin can do the dirty work in a worker thread
4, when result is out, ccplugin can send a message or call a sdk function from ccplugin
5, the result is shown.

If there are another request in between those steps, we can cancel some steps and do it again, a request can have a number, so they don't conflict.


Title: Re: Clang CC
Post by: yvesdm3000 on February 23, 2016, 11:26:20 am
Can you show me where that number is? I tried setting a number on the event but that doesn't get used and doesn't come back in the event that is a result of the ccplugin indicating it has some results.

For what it looks like now, I think I managed to make it work correctly by saving the line/column position where code-completion is requested and when ccplugin is ready, it checks if that position is still valid, if not, drops/cancels the request.

Yves
Title: Re: Clang CC
Post by: ollydbg on February 23, 2016, 12:06:26 pm
Can you show me where that number is? I tried setting a number on the event but that doesn't get used and doesn't come back in the event that is a result of the ccplugin indicating it has some results.
Oh, sorry, this is my idea, not implemented yet, I just put there for discussion. Not sure it is good or bad. :)

Quote
For what it looks like now, I think I managed to make it work correctly by saving the line/column position where code-completion is requested and when ccplugin is ready, it checks if that position is still valid, if not, drops/cancels the request.

Yves
Yes, it is also a good choice, if you looked at the GDB's MI interface, it use a number to synchronize a request and a response.
Title: Re: Clang CC
Post by: ollydbg on February 23, 2016, 02:57:48 pm

EDIT: the asynchronized request is just my idea.
An asynchronized request:
1, ccmanager call a function in ccplugin, thus a request is send to ccplugin
2, ccplugin return a special value(maybe a number), and ccmanager know it will return the results later.
3, ccplugin can do the dirty work in a worker thread
4, when result is out, ccplugin can send a message or call a sdk function from ccplugin
5, the result is shown.

If there are another request in between those steps, we can cancel some steps and do it again, a request can have a number, so they don't conflict.

Hi, yvesdm3000, I think you have already implement this kind of feature.
The code is here:
Code
std::vector<cbCodeCompletionPlugin::CCToken> ClangCodeCompletion::GetAutocompList(bool isAuto, cbEditor* ed, int& tknStart, int& tknEnd)
{
    // you just use the m_CCOutstandingResults to generate a returned tokens
}

And in the code:
Code
void ClangCodeCompletion::OnCodeCompleteFinished( ClangEvent& event )
{
    //CCLogger::Get()->DebugLog( wxT("OnCodeCompleteFinished") );
    if ( event.GetTranslationUnitId() != m_TranslUnitId )
    {
        return;
    }
    if (m_CCOutstanding > 0)
    {
        if ( event.GetLocation() != m_CCOutstandingLoc )
        {
            CCLogger::Get()->DebugLog( wxT("Discard old CodeCompletion request result") );
            return;
        }
        EditorManager* edMgr = Manager::Get()->GetEditorManager();
        cbEditor* ed = edMgr->GetBuiltinActiveEditor();
        if (ed)
        {
            m_CCOutstandingResults = event.GetCodeCompletionResults();
            if ( m_CCOutstandingResults.size() > 0 )
            {
                CodeBlocksEvent evt(cbEVT_COMPLETE_CODE);
                evt.SetInt(1);
                Manager::Get()->ProcessEvent(evt);
                return;
            }
        }
        m_CCOutstanding--;
    }
}
This event is send from the thread to the plugin, and you manually fire a cbEVT_COMPLETE_CODE.

Pay attachment to the value
Code
evt.SetInt(1);
I think in the ccmanager, this int value only have two meanings:
FROM_TIMER or not FROM_TIMER.
I'm not quite sure whether FROM_TIMER means, but we may add another option like "delayed call".
By reading the doxygen docs of this function:
Code
virtual std::vector<CCToken> GetAutocompList(bool isAuto, cbEditor* ed, int& tknStart, int& tknEnd) = 0;
I see that the returned value could be
1, an empty vector, in this case, no code suggestion will list
2, a non empty vector, thus a code suggestion list will prompted.
So, maybe we can add another returned value, indicates that the plugin will returned the vector later (asynchronized way).
Title: Re: Clang CC
Post by: ollydbg on February 23, 2016, 02:59:24 pm
A bug report: I think not every source file(cpp or h files) were included in the clanglib.cbp.
Title: Re: Clang CC
Post by: yvesdm3000 on February 23, 2016, 04:30:27 pm
A bug report: I think not every source file(cpp or h files) were included in the clanglib.cbp.

use clanglib-unix.cbp or clanglib-msw_64.cbp

Yves
Title: Re: Clang CC
Post by: iceant on March 07, 2016, 01:23:30 pm
Yvesdm3000, thanks for your great work.
I got an problem to run this plugin with latest svn version(10798). Which is wx2.8.12 32bit windows, unicode version

I build Clang CC with clanglib-msw_64.cbp, and changed some folders. Clang is downloaded from official web site(3.7.1 version 32bit)
Here is the installation steps:
1) First, build the clanglib.dll and clanglib.zip
2) copy clanglib.dll from c:\git directory to CODEBLOCKS/share/CodeBlocks/plugins
3) copy clanglib.zip to CODEBLOCKS/share/CodeBlocks/
4) copy libclang.dll to CODEBLOCKS/, the same directory with codeblocks.exe

But I can not run it success.

Here is the error message:
-------------------------------------------------------------
One or more plugins were not loaded.
This usually happens when a plugin is built for
a different version of the Code::Blocks SDK.
Check the application log for more info.

List of failed plugins:

clanglib.dll
-------------------------------------------------------------

Can you help me on this? Thanks

A bug report: I think not every source file(cpp or h files) were included in the clanglib.cbp.

use clanglib-unix.cbp or clanglib-msw_64.cbp

Yves
Title: Re: Clang CC
Post by: scarphin on March 07, 2016, 01:33:58 pm
You have to link to the 'codeblocks.dll' in your installation folder. Check the following link for more information:
http://wiki.codeblocks.org/index.php/Linking_the_plugin_to_a_Nightly_Build (http://wiki.codeblocks.org/index.php/Linking_the_plugin_to_a_Nightly_Build)
Title: Re: Clang CC
Post by: iceant on March 07, 2016, 01:51:37 pm
scarphin, thanks a lot. It works now.

You have to link to the 'codeblocks.dll' in your installation folder. Check the following link for more information:
http://wiki.codeblocks.org/index.php/Linking_the_plugin_to_a_Nightly_Build (http://wiki.codeblocks.org/index.php/Linking_the_plugin_to_a_Nightly_Build)
Title: Re: Clang CC
Post by: yvesdm3000 on March 22, 2016, 06:48:42 pm
I pushed new code on master that fixes the clang-based-occurrences highlighting for people that are keen on that and some other stability improvements and some very small feature enhancements available in the configuration panel like adding compile-options to clang.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on March 28, 2016, 09:20:52 am
I pushed new functionality on the 'staging' branch that implements the 'fixit' functionality. It's not the way you suggested but I add a marker in the gutter-bar and when you click it, the fixit is applied.

I can't immediately see how I could do it with the annotations since I can't find any event that fires when you click an annotation. The marker-way is how eclipse does it.

Yves
Title: Re: Clang CC
Post by: teto on April 02, 2016, 10:04:42 pm
Very interesting work. Wanted to give it a try: cloned on linux and opened the cbp project. It seems configured for windows only. Anyone tried on linux? else I will wait for it to land into the core. keep up the good work :)
Title: Re: Clang CC
Post by: yvesdm3000 on April 04, 2016, 08:02:00 am
You need to use the 'unix' project file.

Main development is done on Linux btw and I should test more on Windows...

Yves
Title: Re: Clang CC
Post by: teto on April 04, 2016, 05:29:11 pm
Thanks. Thread is long so it might be a good idea to sum up some info in the first message or in the README. From git log, I gather that master is the active branch.
I had to comment out jus tthis line to compile: "assert( first.m_id == second.m_Id );". When I tried to load the plugin, codeblocks said the SDK was incompatible (I got cb 16.01 with SDK 1.29 while Clang expects 1.30).  Thus I  tried to compile the plugin against codeblocks  3b88b40f9f818607795a90cfd2120a202a8ccec3 but I get this warning even though I compile with "-std=c++11" (tried with C++0x, which was already enabled in the .cbp):
Code
g++ -DBUILDING_PLUGIN -DNOPCH -Wextra -Wall -std=c++11 -ansi -fPIC -g -I/usr/lib/x86_64-linux-gnu/wx/include/gtk2-unicode-release-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXGTK__ -pthread -I/usr/lib/llvm-3.6/include -fmessage-length=0 -fexceptions -Winvalid-pch -pthread -I/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks -I/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/tinyxml -I/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/scripting/include -I/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/scripting/bindings -I/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/scripting/sqplus -I/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/wxscintilla/include -DcbDEBUG -I. -c /home/teto/ClangLib/cclogger.cpp -o .objs/plugins/clanglib/cclogger.o
In file included from /mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/sdk_events.h:12:0,
                 from /mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/manager.h:21,
                 from /mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/logmanager.h:9,
                 from /home/teto/ClangLib/cclogger.h:15,
                 from /home/teto/ClangLib/cclogger.cpp:10:
/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/prep.h:96:1: warning: identifier ‘nullptr’ is a keyword in C++11 [-Wc++0x-compat]
 template<typename T>inline void Delete(T*& p){delete p; p = nullptr;}
 ^
/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/prep.h: In function ‘void Delete(T*&)’:
/mnt/ntfs/teto/codeblocks_sf/install/include/codeblocks/prep.h:96:61: error: ‘nullptr’ was not declared in this scope
 template<typename T>inline void Delete(T*& p){delete p; p = nullptr;}
Title: Re: Clang CC
Post by: yvesdm3000 on April 04, 2016, 08:36:30 pm
Maybe  we should revert back to using NULL ?

Yves
Title: Re: Clang CC
Post by: oBFusCATed on April 04, 2016, 08:38:00 pm
No, use nullptr and build in c++11 mode.
Since sometime master is using c++11 and a few days ago, I've removed our old implementation of nullptr/nullptr_t.
Title: Re: Clang CC
Post by: l_inc on April 04, 2016, 09:55:47 pm
Quote
No, use nullptr and build in c++11 mode.
Since sometime master is using c++11
Hooray :-). I'm looking forward to seeing good use of shared_ptr's, unique_ptr's, compiler-enforced move semantics, etc. and a cleanup from the nasty support of older standards. Some things in the plugin just seemed too counter-natural without C++11.
Title: Re: Clang CC
Post by: teto on April 04, 2016, 11:11:57 pm
If I move the -std=c++0x flags to the end of the compilation line, it works. Thus I had to ruse with Codeblocks options to append -std=C++0x to compiler options without checking the box. Same with clang++.
Then I hit another error:
Code
/home/teto/ClangLib/clangproxy.cpp: In member function ‘void ClangProxy::RemoveTranslationUnit(ClTranslUnitId)’:
/home/teto/ClangLib/clangproxy.cpp:761:33: error: use of deleted function ‘ClTranslationUnit& ClTranslationUnit::operator=(const ClTranslationUnit&)’
     m_TranslUnits[translUnitId] = ClTranslationUnit(translUnitId, nullptr);
Is master supposed to compile at all times or should I check a specific commit ?
Title: Re: Clang CC
Post by: oBFusCATed on April 04, 2016, 11:48:47 pm
Please use -std=c++11 or gnu++11, c++0x is not supported!
If your compiler doesn't have full c++11 support, then this compiler is not supported by us (you won't get any help for problems you encounter, nor we'll apply patches to support your compiler).
You'll have to upgrade it!
Title: Re: Clang CC
Post by: teto on April 05, 2016, 12:22:06 am
I use gcc version 5.2.1 (full support for C++11 as claimed here https://gcc.gnu.org/gcc-5/changes.html) which supports std=C++11 but even with it I have the same error. Could you share your setup ? If you use llvm I will switch to llvm though if g++ claims there is an error, it may be best to fix it (I shall even propose a patch if confirmed) rather than ignore it.
Title: Re: Clang CC
Post by: oBFusCATed on April 05, 2016, 12:40:20 am
I'm not talking about the plugin, but codeblocks in general. I've not tried the plugin yet.
Title: Re: Clang CC
Post by: teto on April 05, 2016, 12:43:28 am
My codeblocks compiles fine (from your git remote), the error appears while compiling the plugin.
Title: Re: Clang CC
Post by: yvesdm3000 on April 05, 2016, 06:47:23 am
If CodeBlocks is now moved to C++11, then I'll move the plugin to C++11 too and git rid of some ugly code in the process.

I pushed some compile fixes for C++11

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on April 05, 2016, 01:51:04 pm
I however don't want to bypass the original author Alpha about going to C++11 as minimum requirement since he did the original code.
Maybe Alpha can give his point of view on this before I start removing code ?

Yves
Title: Re: Clang CC
Post by: teto on April 05, 2016, 03:50:58 pm
Thansk for the fix yves, I managed to compile and load it.

I haven't looked at the code yet, do I need to do anything else apart from installing the plugin to enable it (like disabling old CC) ? Any advice on how to stresstest it ?

Cheers
Title: Re: Clang CC
Post by: Alpha on April 05, 2016, 04:34:47 pm
I however don't want to bypass the original author Alpha about going to C++11 as minimum requirement since he did the original code.
Maybe Alpha can give his point of view on this before I start removing code ?
I had intended to migrate the code to C++11 once core C::B made the switch.  So, green light to go ahead on the change now.
Title: Re: Clang CC
Post by: yvesdm3000 on April 05, 2016, 08:37:58 pm
Thansk for the fix yves, I managed to compile and load it.

I haven't looked at the code yet, do I need to do anything else apart from installing the plugin to enable it (like disabling old CC) ? Any advice on how to stresstest it ?


I always disable the old CC. I don't know at all how well they play together.

Just know that 'goto implementation' currently only works on loaded Translation Units and you can have a maximum of 5 loaded at the same time. It's a tokendatabase thing and been working to solve that for a while... It's not easy and if I would load the whole database in its current implementation for all files for my customers project, it takes a couple of gigabytes of memory, which is not acceptable.
Know that I use this plugin daily for some good coding on a pretty big project.

Also the CodeCompletion seems to be invoked a bit too often which makes it 'a little noticeable' slow on a virtual machine (not on a real PC) and sometimes CC-event gets triggered when they shouldn't, but it's not something that bothers me right now.

Class browser is not implemented at all and it looks I should try to learn from the mistakes the regular CC-plugin made. I haven't started this work at all yet.

Yves
Title: Re: Clang CC
Post by: teto on April 06, 2016, 03:35:08 pm
Ok I have it working. A codeblocks reboot might be necessary or maybe it's because I disabled the regular CC-plugin but I now see the "tooltips" unlike yesterday.
My project is a custom project, hence the .cbp file does not contain the compilation flags etc... is there a way to pass on a compile_commands.json ?

I was curious to test this plugin compared to YouCompleteMe + Vim which is often not capable of finding the definition. The number of translation unit seems indeed to be the crux of the matter.
I was curious how to fix it and I stumbled on this topic that could be of interest to you:
https://github.com/Valloric/ycmd/pull/180

To sum up, clang devs think of providing a standard tool for cross TU interrogation, so maybe you should not spend too much time on this (or do it upstream :p).
Title: Re: Clang CC
Post by: yvesdm3000 on April 06, 2016, 05:16:12 pm
My project is a custom project, hence the .cbp file does not contain the compilation flags etc... is there a way to pass on a compile_commands.json ?

A trick that I use is to setup the .cbp file to not use an external makefile, do a theoretical configuration with mostly the C++ language and the header search paths and then switch back to using an external makefile.

The Clang-CC plugin doesn't check if the makefile is extenal and can still query the part of the settings that are theoretically disabled.

Quote
I was curious to test this plugin compared to YouCompleteMe + Vim which is often not capable of finding the definition. The number of translation unit seems indeed to be the crux of the matter.
I was curious how to fix it and I stumbled on this topic that could be of interest to you:
https://github.com/Valloric/ycmd/pull/180

To sum up, clang devs think of providing a standard tool for cross TU interrogation, so maybe you should not spend too much time on this (or do it upstream :p).
I used to use YouCompleteMe for a while but it had the same issue that I had with the initial implementation of this plugin: It blocks the UI too often preventing me to type. I will be doing essentially the same thing like they suggest: Parse every file in the project and build a token-to-file relationship. This way the plugin knows which file to parse to go find the implementation.
There is actually not a lot that Clang can do about it, in the end it needs to parse every file anyway to know what implementations it has. The intention is however to keep this indexing information to the bare minimum: Token->file and some flags that indicates if the token has a declaration, a definition or reference. This comes to a single list of token strings and 3 ints for each token-file relationship and this information will be serialized so that you can have a kick-start when initially starting up the project. (an optional 4'th int might be added later with a checksum for better fine-grained lookups.

Yves
Title: Re: Clang CC
Post by: teto on April 06, 2016, 05:23:15 pm
The blocking UI problem is in general solved by neovim (a great vim fork), though I dunno for YCM in peculiar, my computer is just too fast to notice :)

I've given https://github.com/Andersbakken/rtags a try this afternoon. Out of the 3 vim plugins mentioned on the webpage, I could only make https://github.com/lyuts/vim-rtags work. I've not had the time to see how it compares with YCM but  from the description it looks promising.
Title: Re: Clang CC
Post by: yvesdm3000 on April 13, 2016, 02:13:05 pm
I pushed a small fix on master for when code-completion seems to be invoked when you don't want to.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on May 02, 2016, 09:02:50 am
I just pushed the changes for the tokendatabase to make 'goto implementation' work much better to the 'staging' branch so whoever wants to test is free to do so.

The tokendatabase is however not yet persistent so each time you load a project, the plugin will do a complete reindexation.
What prohibits me to implement this persistence is that the tokendatabase should be linked to a project and not the workspace because you could (theoretically) have 2 separate projects each have a class that share the name but not the implementation. In further stages when can start renaming symbols this might even become more important.

Yves
Title: Re: Clang CC
Post by: MortenMacFly on May 14, 2016, 07:11:22 am
What prohibits me to implement this persistence is that the tokendatabase should be linked to a project and not the workspace because you could (theoretically) have 2 separate projects each have a class that share the name but not the implementation.
Please keep in mind that it could also be that two targets in a project share the same class with different implementations. there are quite some projects around where that is the case...
Title: Re: Clang CC
Post by: yvesdm3000 on May 14, 2016, 07:40:14 am
Are you referring to having a #define set on a target that selects some class/interface/code?
If that's the case, that is just covered by clang itself and me passing its compile options. I also reparse the translation units when configuration options are changed. I should however check if this is also done when the target is changed, so good point.

If it's something else that I'm not aware of, please specify.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on May 20, 2016, 11:29:15 pm
I just landed the persistent tokenindex database, split out per loaded project.

It's pushed on the "staging" branch right now. I'll leave it stabilize for now until I merge it onto the master branch.

Next step will be the class-browser, but I need to figure out how I can get this information out of Clangs AST.

Yves
Title: Re: Clang CC
Post by: MortenMacFly on May 27, 2016, 09:10:40 pm
If it's something else that I'm not aware of, please specify.
I meant that if you say you'll need a token database per projects I say its better per target. Because the very same reason why you don't do it per workspace also applies to projects due to targets.

If you can manage that differently somehow I am happy.
Title: Re: Clang CC
Post by: yvesdm3000 on May 28, 2016, 08:05:10 am
If it's something else that I'm not aware of, please specify.
I meant that if you say you'll need a token database per projects I say its better per target. Because the very same reason why you don't do it per workspace also applies to projects due to targets.

If you can manage that differently somehow I am happy.

I think I understand why. You could have a different set of files associated on a target, but that does not necessarily mean it's a problem since the indexdb will contain symbols of all files unless you want to use ClangCC to hint for upcoming linking problems e.g. you're code-completing on a class that is not defined for your target.

It becomes a real problem:

And to make things worse, if I would have to multiply the token-databases with the different targets possible, it would make the memory-footprint huge.

However there is 1 distinct difference with the project-splitup: There can only be 1 target selected at the same time so to cope with the memory-footprint, we could unload and load a different tokendatabase when the target is switched (and flush the in-memory Translation Units since they might need new compile command arguments).

Yves
Title: Re: Clang CC
Post by: l_inc on June 04, 2016, 01:50:30 am
yvesdm3000
Hi. I like the plugin a lot, but there are some problems I encountered so far that annoy me (present in this week's staging branch as well):

1) Code completion doesn't work for labels: no matter forward or backward referenced

2) Code completion doesn't work for fields of unions/structures inside compound literals (in C):
Code
typedef union mystruct_t_
{
  uint32_t raw;
  struct
  {
    uint32_t offset:12;
    uint32_t reserved:24;
  } fields;
} mystruct_t;

uint32_t const value =
      ((mystruct_t) { .fields.reserved = 1 }).raw;
//CC not working here ^

3) Header files are often parsed incorrectly as main files in their own translation unit. Every file in a project has a property (Properties... -> Build -> Compile file) specifying whether it is a separate translation unit. This property could be a basis for the decision. Besides it would be nice to be able to quickly switch every such header between existing translation units, in which the header file is included, through a quickly accessible menu, because it's not uncommon to have (very) different parsing of the same header depending on the translation unit.

As a side note, unfortunately clang doesn't compile things like static size_t const tmp = "Hello"-"Hello";, which results in a spurious error-diagnostic. But I guess that should rather be reported to the clang developers: even though the standard doesn't guarantee it compiles, it better would as in gcc.
Title: Re: Clang CC
Post by: yvesdm3000 on June 04, 2016, 08:06:58 pm
yvesdm3000
Hi. I like the plugin a lot, but there are some problems I encountered so far that annoy me (present in this week's staging branch as well):
Hi l_inc,

I highly appreciate the feedback! Beware the staging is still a bit 'wet' in the way that the index-database still has issues.

Quote
1) Code completion doesn't work for labels: no matter forward or backward referenced
Beside the generic nagging about gotos etc (but I'm also realistic: I should not care about what someone does or even have to do at times!), I fear that the current code-completion support comes completely from the libclang library. I give it the translation unit and a position and the library returns me a list of code-completions where we only filter out constructors and operators when they are not appropriate. Nevertheless I think we should be able to retrieve the labels within the function scope and detect the goto statement and do code-completion ourselves.

Quote
2) Code completion doesn't work for fields of unions/structures inside compound literals (in C):
Code
typedef union mystruct_t_
{
  uint32_t raw;
  struct
  {
    uint32_t offset:12;
    uint32_t reserved:24;
  } fields;
} mystruct_t;

uint32_t const value =
      ((mystruct_t) { .fields.reserved = 1 }).raw;
//CC not working here ^
It might be libclangs fault, or maybe we are not giving it the correct position. We always go back to the start of the word, that is what libclang wants, but I'm unsure if we do it correctly in this case. I'll try to look at this case later next week.

Quote
3) Header files are often parsed incorrectly as main files in their own translation unit. Every file in a project has a property (Properties... -> Build -> Compile file) specifying whether it is a separate translation unit. This property could be a basis for the decision. Besides it would be nice to be able to quickly switch every such header between existing translation units, in which the header file is included, through a quickly accessible menu, because it's not uncommon to have (very) different parsing of the same header depending on the translation unit.
Yep, I'm completely aware of this problem. There is some code that should cope with this but it's buggy and disabled. The way I see it is that the currently focussed translation unit will be checked first if it handles the newly focussed header file. This way you could force the 'view' of the header file by going to the header/source file that does some defines before the #include statement and when you would then go to the header file, you'd see it trough that translation unit.

Quote
As a side note, unfortunately clang doesn't compile things like static size_t const tmp = "Hello"-"Hello";, which results in a spurious error-diagnostic. But I guess that should rather be reported to the clang developers: even though the standard doesn't guarantee it compiles, it better would as in gcc.
That's a drawback of using libclang for code-completion and gcc as a compiler. It would be very hard to fix this since you're mixing up 2 compilers... Best would be to use clang as a compiler too, but often you don't have that option. (Same for me for my other projects. GCC is the only compiler where the product compiles, nevertheless I find clangcc very usefull for code-completion.

Yves
Title: Re: Clang CC
Post by: l_inc on June 05, 2016, 12:49:06 am
yvesdm3000
Quote
Beside the generic nagging about gotos etc (but I'm also realistic: I should not care about what someone does or even have to do at times!)
I'm very fastidious with respect to coding style, and I always try to find the cleanest and clearest way of doing things. Starting to use goto's was (once) a result of a critical analysis of my C code. While I totally support most of the critics of goto, there's one single case where it does a better job in supporting clean and clear code in C than anything else. And that's error handling. Especially when you handle resource allocation errors while allocating three or more resources in a function. That's a generally accepted application of goto's and the only one I make use of in my code. In C++ RAII is in many cases a better alternative, but we don't have that in C.

Aside from the justification of goto's used in a very limited way, there's indeed another argument, which you've already mentioned. That is, code completion should support possible language constructs rather than some specific coding styles. Supporting the latter could only be an optional additional bonus.

Quote
Nevertheless I think we should be able to retrieve the labels within the function scope and detect the goto statement and do code-completion ourselves.
Well, thank you for considering that option. However I expected libclang to be somewhat more clever in that respect. If it doesn't support code completion for labels, then I'd first bugreport that to the libclang developers.

Quote
I'll try to look at this case later next week.
OK. Thanks for taking your time looking into that.

Quote
This way you could force the 'view' of the header file by going to the header/source file that does some defines before the #include statement and when you would then go to the header file, you'd see it trough that translation unit.
I see this kind of implementation as a possible temporary workaround. But a drop-down or a quick popup with an assignable hotkey (like the one you have on Ctrl+Tab) seems to me more user-friendly.

Quote
GCC is the only compiler where the product compiles, nevertheless I find clangcc very usefull for code-completion
Totally. And in general you're right regarding the issues of using different compilers. However clang developers target broad compatibility with gcc, and I'm gonna find some time to report at least this little issue to the clang developers, as supporting string merging in this specific case is arguably better than not supporting, even though this goes beyond the requirements imposed by the C standard.
Title: Re: Clang CC
Post by: yvesdm3000 on June 05, 2016, 09:04:58 am
yvesdm3000
I'm very fastidious with respect to coding style, and I always try to find the cleanest and clearest way of doing things. Starting to use goto's was (once) a result of a critical analysis of my C code. While I totally support most of the critics of goto, there's one single case where it does a better job in supporting clean and clear code in C than anything else. And that's error handling. Especially when you handle resource allocation errors while allocating three or more resources in a function. That's a generally accepted application of goto's and the only one I make use of in my code. In C++ RAII is in many cases a better alternative, but we don't have that in C.
It looks like I've been doing too much C++ the last 2 years. I actually agree about using goto's and resources in C. It's not an ideal solution but there is no other clean alternative. Some do the for( ; ; ) and break thing but I find it cluttering and misuse of the for loop. I've also seen some code that have defines to hide that for-loop trick, but had side-effects on its own, especially around the warnings/errors it emits when doing something wrong.

Quote
Well, thank you for considering that option. However I expected libclang to be somewhat more clever in that respect. If it doesn't support code completion for labels, then I'd first bugreport that to the libclang developers.
I'll try to file a bugreport for this.

Quote
I see this kind of implementation as a possible temporary workaround. But a drop-down or a quick popup with an assignable hotkey (like the one you have on Ctrl+Tab) seems to me more user-friendly.
I'm not sure about the details on what you propose in the sub-menu. I guess it should have a list of filenames (e.g. the top-parent of the translation unit) to select under which context you wish to view the header file?

Yves
Title: Re: Clang CC
Post by: l_inc on June 05, 2016, 03:34:00 pm
yvesdm3000
Quote
Some do the for( ; ; ) and break thing but I find it cluttering and misuse of the for loop
Yes, that's using a more complicated mechanism to simulate a better fitting primitive one. This consideration also allows to view a do { /*...*/ } while(0); construct as a more appropriate replacement, although still ugly. I actually preferred nesting of the if (/*...*/) {/*...*/} construct. But in all cases that only stops further proceeding and therefore is only half of the story. The second half is to deallocate the resources, which needs more clutter of constructs. I used a switch with all fall-through cases:
Code
NTSTATUS HighlyNestedFun()
{
NTSTATUS ntStatus = STATUS_SUCCESS;
enum failure_stage {success, res1_failure, res2_failure, res3_failure, res4_failure, res5_failure}
failureStage = success;
HANDLE hRes1, hRes2, hRes3, hRes4, hRes5;

hRes1 = AcquireResource1();
if (!hRes1)
ntStatus = STATUS_INSUFFICIENT_RESOURCES1, failureStage = res1_failure;
else
{
hRes2 = AcquireResource2();
if (!hRes2)
ntStatus = STATUS_INSUFFICIENT_RESOURCES2, failureStage = res2_failure;
else
{
hRes3 = AcquireResource3();
if (!hRes3)
ntStatus = STATUS_INSUFFICIENT_RESOURCES3, failureStage = res3_failure;
else
{
hRes4 = AcquireResource4();
if (!hRes4)
ntStatus = STATUS_INSUFFICIENT_RESOURCES4, failureStage = res4_failure;
else
{
hRes5 = AcquireResource5();
if (!hRes5)
ntStatus = STATUS_INSUFFICIENT_RESOURCES5, failureStage = res5_failure;
else
{
//using acquired resources
}
}
}
}
}

switch (failureStage)
{
case success:
FreeResource5(hRes5);
case res5_failure:
FreeResource4(hRes4);
case res4_failure:
FreeResource3(hRes3);
case res3_failure:
FreeResource2(hRes2);
case res2_failure:
FreeResource1(hRes1);
case res1_failure:
}

return ntStatus;
}
But eventually I came to the conclusion that all this clutter is nothing but an odd simulation of a much clearer use of goto's. Limiting goto use to these cases only also contributes to code clearness, because the semantics of resource allocation error handling is then quickly recognized from the sole appearance of goto in the code.

Quote
I guess it should have a list of filenames (e.g. the top-parent of the translation unit) to select under which context you wish to view the header file?
Yes. In contrast to your idea to switch back and forth between the file and possible roots of the corresponding translation units this also allows to cover some convoluted cases, in which the same file within the same project serves both as a root of a translation unit as well as an included file.
Title: Re: Clang CC
Post by: yvesdm3000 on June 05, 2016, 09:21:51 pm
You can always post bugs and feature requests here:

https://github.com/yvesdm3000/ClangLib/issues

I've allready done it for the compound literals issue. I'll also add the header-tu-scope feature request too if you don't. ;-)

Yves

Title: Re: Clang CC
Post by: l_inc on June 05, 2016, 10:42:42 pm
yvesdm3000
Quote
You can always post bugs and feature requests here
Thank you. I have to think about creating an account on github first. :-)
Title: Re: Clang CC
Post by: yvesdm3000 on June 16, 2016, 10:26:56 pm
For those that like bleeding edge, I pushed a new feature on the 'staging' branch that allows you to apply the fixits automatically, but only if your cursor is at least 2 lines away from the fixit line. It is disabled by default so you'll have to enable it in the settings.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on August 21, 2016, 03:21:07 pm
Hi,

I pushed a new feature to the 'staging' branch that lets the user configure if they want the 'if/else/for/while' constructs in code-completion or not. While I was at it, I also added the feature where you can toggle if you want macros in code-completion or not.

Yves
Title: Re: Clang CC
Post by: l_inc on August 21, 2016, 04:17:28 pm
yvesdm3000
Hi. Did you have a look at the issue with header files parsed out of translation unit context? This issue results in literally every header file in my current main project being full of red diagnostics and no code completion works in the header files. I saw you had created an issue for that, just asking if you'll work on this in anytime soon.
Title: Re: Clang CC
Post by: yvesdm3000 on August 21, 2016, 07:56:27 pm
Hi l_inc,

Yes I briefly looked at that and tried enabling it again. The problem is that reparsing doesn't work correctly in that setup. This is because Clang caches all header files in a PCH (the preamble files in /tmp) and this obviously includes the header file we're modifying. Disabling this cache has a big impact on performance and I currently don't know (yet) how I should fix this. A quick fix might be that we disable this caching behaviour when we're working in a header file, but then again the performance degradation will still be there.

Yves
Title: Re: Clang CC
Post by: l_inc on August 21, 2016, 08:19:19 pm
yvesdm3000
Quote
Yes I briefly looked at that and tried enabling it again
What exactly do you enable?
Quote
The problem is that reparsing doesn't work correctly in that setup.
If I understand the problem correctly, it should be possible to make clang create the PCH cache for an include subtree of an include tree of a particular selectable translation unit. Meaning that you view the header file as a root of a virtual include tree that is terminated at the point of first inclusion of the header file in the original tree of the chosen translation unit. Essentially this way of parsing is a generalization of your current parsing for the case when the virtual include tree does not match the actual include tree that is built starting directly from the header file as the root.
Title: Re: Clang CC
Post by: yvesdm3000 on August 21, 2016, 08:36:44 pm
yvesdm3000
Quote
Yes I briefly looked at that and tried enabling it again
What exactly do you enable?
There is some lookup code where given a specific FileId it searches for an existing translation unit. This is currently disabled
Quote
Quote
The problem is that reparsing doesn't work correctly in that setup.
If I understand the problem correctly, it should be possible to make clang create the PCH cache for an include subtree of an include tree of a particular selectable translation unit. Meaning that you view the header file as a root of a virtual include tree that is terminated at the point of first inclusion of the header file in the original tree of the chosen translation unit. Essentially this way of parsing is a generalization of your current parsing for the case when the virtual include tree does not match the actual include tree that is built starting directly from the header file as the root.
<-- Really needed to read this a couple of times... The PCH cache creation is completely handled by Clang and is simply enabled by an option flag. Maybe we should find a way to handle this PCH cache ourselves?

Yves
Title: Re: Clang CC
Post by: l_inc on August 21, 2016, 09:54:27 pm
yvesdm3000
Quote
<-- Really needed to read this a couple of times... The PCH cache creation is completely handled by Clang and is simply enabled by an option flag.
I'm sorry for unclear expressions. I must have given an example of what interface I expect libclang to provide to achieve this. Ideally I expect libclang to allow to setup a callback. This callback should be called every time libclang processes the include directive. And the callback should allow to stop PCH generation. The resulting PCH then represents the result of processing of the subtree I was talking about. In case such a callback interface is not provided, it shouldn't be too complicated to add such an interface.

To say it in different words. You let libclang generate a PCH for translation unit of user's choice. You hook processing of the include directive. During PCH generation your hook/callback checks the file included by the directive. If this is the header file you are interested in, the hook should make libclang finalize PCH creation. This way created PCH is then used for code completion in the header file.

I know it sounds like a foolish speculation, but I can try to have a look at libclang's code and to add such a callback if clang really doesn't provide it or any equivalent interface.
Title: Re: Clang CC
Post by: yvesdm3000 on August 22, 2016, 01:53:00 pm
Sadly it's not that simple...

http://clang.llvm.org/doxygen/group__CINDEX__TRANSLATION__UNIT.html

However I need to experiment a little more and possibly look a bit at the clang code to see if I do pass the header-file as an "unsaved_file" it detects that and reparses the header file(s), maybe it would only work if the header-file is a first-level header file (e.g. specified directly in the .cpp file). Maybe it needs to be in the unsaved_file list at the initial parse to work or something like that...

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on September 28, 2016, 11:06:24 am
Hi,

Function overrides for the 'goto implementation' operation is now implemented on the 'staging' branch.

So if you go to a function implementation that points to a pure virtual function, it will also search for the implementation(s) in subclasses.

I'm still pondering if I should also do the same for non-pure virtual functions and offer the user a list, or if this should be a separate operation like 'Find function overrides'.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on October 26, 2016, 05:54:10 pm
Just pushed fixes and a .cbp for wxWidgets 3.0 support on the 'staging' branch.

Compiling for wxWidgets 3.0 should now work correctly and I'll be continuously testing it, just like I do for wxWidgets 2.8.

Yves
Title: Re: Clang CC
Post by: BlueHazzard on October 27, 2016, 11:36:56 am
honestly: your project files (for windows) are a danger for the commonality. Everywhere hard coded paths (THE OUTPUT IS TO C:\git ????????? honestly??????) ...
i have cleaned up and will make a pull request, but i am not able to run the plugin... Codeblocks gives me errors about symbols not loaded and depwalker shows me unresolved symbols for libclang.dll i thought clang is linked in statically?

[edit:] my problem is that the official build of clang is for msvc and i need a build for mingw...
Title: Re: Clang CC
Post by: yvesdm3000 on November 01, 2016, 10:53:38 am
honestly: your project files (for windows) are a danger for the commonality. Everywhere hard coded paths (THE OUTPUT IS TO C:\git ????????? honestly??????) ...
i have cleaned up and will make a pull request, but i am not able to run the plugin... Codeblocks gives me errors about symbols not loaded and depwalker shows me unresolved symbols for libclang.dll i thought clang is linked in statically?

[edit:] my problem is that the official build of clang is for msvc and i need a build for mingw...
Hi,
No clang is not linked in statically.
Sorry that the project file for windows isn't of high quality yet. Patches are welcome. It is pretty hard for me to test this on windows since that is not my main platform. Please indicate if you are willing to clean up the project file since I don't want to do duplicate work. If not, I'll take a look at it.

Yves
Title: Re: Clang CC
Post by: BlueHazzard on November 01, 2016, 12:17:40 pm
i made the clean up, but i can't test it because i am not able to compiler clang probably into a shared monolithic library... Still working on it, but with not much hope...


greetings
Title: Re: Clang CC
Post by: yvesdm3000 on December 09, 2016, 01:30:29 pm
I pushed a cleanedup version of the codeblocks project for Windows + fixed some path issues for Windows.

Been using the plugin for a while on the windows version and no issues encountered for now.

Yves
Title: Re: Clang CC
Post by: oBFusCATed on December 09, 2016, 09:06:07 pm
How does this plugin gather and feed the files and flags to clang?
Can it use a compiler database?
Title: Re: Clang CC
Post by: yvesdm3000 on December 09, 2016, 09:42:33 pm
Hi Obfuscated,

It gathers the compile command from codeblocks for GCC or clang.
I'm currently working on a new feature to output the 'compiler_commands.json' file so that we can use the clang tools without the need to rebuild this database with external tools.

Yves
Title: Re: Clang CC
Post by: oBFusCATed on December 10, 2016, 12:58:13 am
I don't have a proper project, but a cmake ninja based one. Cmake can generate the correct database. So it will be useful if the cc plugin can use it to do its parsing.

p.s. it is not a good idea to put the generation of the compile_command.json in the clangcc plugin. It is best done as script when the scripting api is expanded or as another plugin or even as a core feature.
Title: Re: Clang CC
Post by: yvesdm3000 on December 10, 2016, 08:47:34 am
Yes I was allready thinking about splitting off this feature since it's not directly using anything from libclang itself, it's currently only recycling some plugin internal utility function. I'm glad you agree.

While using the clang codecompletion I noticed the plugin doesn't handle paths with spaces correctly yet, so that will be fixed soon.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on December 10, 2016, 01:23:08 pm
How does this plugin gather and feed the files and flags to clang?
Can it use a compiler database?

I guess you are need the compilation flags to come out of the compiler_commands.json that is generated from ninja ?

It can't do that right now, but I can't imagine it's difficult to add.

Yves
Title: Re: Clang CC
Post by: oBFusCATed on December 10, 2016, 02:02:44 pm
I guess you are need the compilation flags to come out of the compiler_commands.json that is generated from ninja ?
Yes, the cb project generated by cmake doesn't contain flags (it is custom makefile based project), so the cc will have very hard time gathering them.
Probably we can add a field in the project with the path of the compilation database and then the CC could use it.
We can also teach other tools and plugins to use it.
Title: Re: Clang CC
Post by: yvesdm3000 on December 10, 2016, 05:39:50 pm
I guess you are need the compilation flags to come out of the compiler_commands.json that is generated from ninja ?
Yes, the cb project generated by cmake doesn't contain flags (it is custom makefile based project), so the cc will have very hard time gathering them.
Probably we can add a field in the project with the path of the compilation database and then the CC could use it.
We can also teach other tools and plugins to use it.
My main job's project is also using a custom makefile. I simply mirror the compiler options enough so that ClangCC is working fine. To change options, i switch from custom makefile to cb, set an option and then toggle the custom makefile option back to 'on'. I'm not sure this is a bug in cb to be able to query the compile options when the custom makefile option is on but it sure is handy for me.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on January 09, 2017, 07:42:50 am
Pushed a new feature on the 'staging' branch where for each #include, it now adds a warning/error in case the specified header has a warning/error.

I doesn't work (yet) with multiple levels though (a header that includes a header that includes a header that has an error). I specifically didn't do this yet because it should filter out system headers for warnings (passed with -isystem) and the programmer isn't supposed to fix warnings in system headers..

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on January 10, 2017, 09:42:57 pm
I guess you are need the compilation flags to come out of the compiler_commands.json that is generated from ninja ?
Yes, the cb project generated by cmake doesn't contain flags (it is custom makefile based project), so the cc will have very hard time gathering them.
Probably we can add a field in the project with the path of the compilation database and then the CC could use it.
We can also teach other tools and plugins to use it.

FYI next thing I'm implementing is support for this compiler_commands.json file

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on January 20, 2017, 07:54:26 am
I pushed a new feature to support the use of the json file. It is a project-level setting that is compiler-target specific (last tab under project settings, might be hidden by the other tabs so have to scroll to it)

Yves
Title: Re: Clang CC
Post by: BlueHazzard on January 27, 2017, 12:56:29 am
I was finally able to build the plugin under windows. Here are my steps:
1) Download LLVM from http://releases.llvm.org/3.9.1/LLVM-3.9.1-win32.exe
1.1) Install LLVM
2) Download the "reimp" tool from http://wyw.dcweb.cn/download.asp?path=&file=reimp_new.zip
2.1) Extract reimp
2.2) Build reimp with the terminal by navigating into the src subfolder and typing "mingw32-make.exe -f Makefile"
3) copy reimp.exe to INSTALL_PATH_LLVM\lib
3.1) open terminal in this folder and run "reimp libclang.lib" This should generate a liblibclang.a file.
3.2) Rename liblibclang.a to libclang.a
4) Download the source code of the cclang plugin
5) I had to make the changes to the project file according the patch i attached
6) Open the project file and set the CLANG global variable to the INSTALL_PATH_LLVM set the other global variables accordingly
7) Build target "plugin"
8) Copy the libclang.dll from INSTALL_PATH_LLVM/bin to your codeblocks directory
9) Open Codeblocks Plugins->Manage Plugins
9.1) Disable the code completition plugin
9.2) Install new and select the clanglib.cbplugin
10) Have fun
Title: Re: Clang CC
Post by: yvesdm3000 on January 28, 2017, 11:57:53 am
Hi BlueHazzard,

Many thanks for this howto!

It looks like you solved the symbols-issue better than I did: I modified the llvm/clang Makefiles to output what I needed, it is certainly simpler and better with the reimp tool.

Any reason why you added a new target? Can't we just update the existing targets?

Yves
Title: Re: Clang CC
Post by: BlueHazzard on January 28, 2017, 12:08:01 pm
Hi, i tested the plugin a bit and it is really nice.
Here some things i noted in this short period of testing:

1) The name of the binary is somehow unlucky... ClangLib sounds as it is something from clang but not a code completion plugin. I think ccClang or something like this would be a better name..
2) On the start of projects i get the sometimes a error message about some database could not be found? Is this only because i didn't used cclang on this project earlier?
3) I opened the codeblocks project file and it loaded successfully, but the cursor turned in a spinning circle (busy symbol) and never returned from it. I can click everywhere and it works, but the circle does not go away. And this blocks the right klick on the project tree. Also i can't close the project via menu, so it seems that the loading process is not competed?
4) I get plenty of messages like this:
Code
ClangLib: translUnitId == wxNOT_FOUND, cannot complete in file \codeblocks_sf\src\sdk\cbplugin.cpp
Does this mean the parser is not complete on this file?
5) One time the plugin prevented a file to save. I was not able to save the file anymore (a error message appeared that this file could not be saved) I was not able to reproduce this in the debugger, but this seems to be a really bad bug...

thank you all for this great work!
Title: Re: Clang CC
Post by: BlueHazzard on January 28, 2017, 12:10:14 pm
Quote
Any reason why you added a new target? Can't we just update the existing targets?
With this target you can't run it trough the run button. But it creates a "install able" file you can distribute. The other targets let you to run codeblocks with the plugin installed there? I did want to modify this targets, because modifying them would may be break your workflow?
Title: Re: Clang CC
Post by: yvesdm3000 on January 28, 2017, 02:36:47 pm
Hi,

I'm fine with it if the existing targets would make the plugin. I have my own version anyway since my installation always seem to be different than the generic one anyway.

Yves
Title: Re: Clang CC
Post by: oBFusCATed on January 28, 2017, 02:47:19 pm
1) The name of the binary is somehow unlucky... ClangLib sounds as it is something from clang but not a code completion plugin. I think ccClang or something like this would be a
cb_clang_complete or something in this style is event better name.
Title: Re: Clang CC
Post by: yvesdm3000 on January 28, 2017, 03:09:22 pm
I really haven't chosen the name, I'm finishing the plugin that Alpha started. Reality is that this plugin does more than code-completion, it also does code-analysis and has the potential to do a lot more where I simply start the next feature when the previous is finished.

Yves
Title: Re: Clang CC
Post by: Alpha on January 28, 2017, 04:26:36 pm
I probably could have done a better job choosing the name; I had not considered "lib" being in important prefix/suffix...
The intent of the name was that this plugin would also support other functionality retrievable via clang, such as true semantic highlighting, auto suggest (+/- apply) fixes for warnings and errors, refactoring, reformatting, etc.
Title: Re: Clang CC
Post by: yvesdm3000 on January 28, 2017, 07:37:22 pm
I probably could have done a better job choosing the name; I had not considered "lib" being in important prefix/suffix...
The intent of the name was that this plugin would also support other functionality retrievable via clang, such as true semantic highlighting, auto suggest (+/- apply) fixes for warnings and errors, refactoring, reformatting, etc.

I actually haven't found reformatting functionality in libclang, only using a cmd-line tool which is not based on libclang but on the 'continuously-changing' c++ classes...  I think if it is based on an external tool, it should have its own C::B plugin.

Yves
Title: Re: Clang CC
Post by: oBFusCATed on January 28, 2017, 08:29:40 pm
Yes, clang-format doesn't require a compiler it is standalone tool or library...
Title: Re: Clang CC
Post by: yvesdm3000 on January 30, 2017, 02:16:01 pm
For the users of ClangCC, I just pushed a fix for an issue when a file that is not part of the project would be saved, caused a crash on ClangCC.

(on the staging branch)

Yves
Title: Re: Clang CC
Post by: MortenMacFly on February 19, 2017, 07:53:57 pm
Dear Yves,

I've tried to compile this plugin on a more recent compiler. I suggest to apply the patch that is attached to:
- get rid of the deprecated auto_ptr
- simplify the singleton pattern
- fix an compiler error in non-PCH builds due to missing #include <iostream>

The patch applies to 2 files on the staging branch only: cclogger{.h/.cpp}.
Title: Re: Clang CC
Post by: oBFusCATed on February 19, 2017, 08:01:54 pm
Another note: if you're using CC_LOCKER_TRACK_TT_MTX_LOCK I guess you'll have to fix its implementation ;)
Title: Re: Clang CC
Post by: ollydbg on February 21, 2017, 02:33:30 am
Another note: if you're using CC_LOCKER_TRACK_TT_MTX_LOCK I guess you'll have to fix its implementation ;)
We should totally remove those lockers.  :)
Title: Re: Clang CC
Post by: oBFusCATed on February 21, 2017, 09:14:00 am
I've not said that. Just said that in some configurations they are broken (hint no locking is happening)!
Title: Re: Clang CC
Post by: BlueHazzard on February 22, 2017, 11:51:47 pm
Hi, just i crash report i got lately very often:
Code
codeblocks.exe caused an Access Violation at location 4309DC03 Reading from location 4309DC03.

Registers:
eax=4309dc03 ebx=00000001 ecx=28c0e370 edx=28c0e370 esi=0028bad4 edi=00000000
eip=4309dc03 esp=0028ba50 ebp=0028baec iopl=0         nv up ei pl nz ac po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00210216

AddrPC   Params
4309DC03 00000001 00000001 00000001
0A92A530 28C0E370 0028BA94 0028BB30  clanglib.dll!BeginHighlightOccurrences  [cclang-plugin/src/ClangLib/src/clangrefactoring.cpp @ 227]
   225:     ClTokenPosition loc(line + 1, pos - stc->PositionFromLine(line) + 1);
   226:
>  227:     m_pClangPlugin->RequestOccurrencesOf( translId,  ed->GetFilename(), loc );
   228: }
   229:
0A92A171 0028BC4C 02EFD728 0028BC7C  clanglib.dll!OnTimer  [cclang-plugin/src/ClangLib/src/clangrefactoring.cpp @ 181]
   179:
   180:     if (evId == idHighlightTimer)
>  181:         BeginHighlightOccurrences(ed);
   182:     else
   183:         event.Skip();

i get the crashes randomly, don't know if you can do something with it...
greetings
Title: Re: Clang CC
Post by: yvesdm3000 on February 23, 2017, 08:28:02 am
I fixed a memory corruption crash yesterday in TreeMap::Remove() that might be the cause. Been reworking the index database lately and also built a tool that can dump the indexdb and now fixing many issues found by the tool. The goal is to depend less on libclang for things like the toolbar (scope menus) and goto-implementation and documentation-lookup. The problem with libclang is that you often need a translation unit to find various info and that takes a long time to create (essentially parsing/compiling the needed cpp file).

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on February 23, 2017, 08:30:14 am
Another note: if you're using CC_LOCKER_TRACK_TT_MTX_LOCK I guess you'll have to fix its implementation ;)
That's a file that I grabbed as-is from the regular CC plugin? I guess it has been updated there too?

Yves
Title: Re: Clang CC
Post by: oBFusCATed on February 23, 2017, 08:48:26 am
No, it has not been updated.
Title: Re: Clang CC
Post by: yvesdm3000 on February 23, 2017, 09:10:15 am
Dear Yves,

I've tried to compile this plugin on a more recent compiler. I suggest to apply the patch that is attached to:
- get rid of the deprecated auto_ptr
- simplify the singleton pattern
- fix an compiler error in non-PCH builds due to missing #include <iostream>

The patch applies to 2 files on the staging branch only: cclogger{.h/.cpp}.

I applied your patch. Thank you!

Maybe apply it also on the builltin codecompletion plugin since it has the same file?

Yves
Title: Re: Clang CC
Post by: BlueHazzard on February 27, 2017, 01:17:55 am
Hi, your last commit seems to have fixed my previous crash.
But on project closing (like the codeblocks project file) i get sometimes this crash report:

Code
AddrPC   Params
02FEABF6 0000002E FFFFFFFF 0028BAD4  wxmsw28u_gcc_custom.dll!wxStringBase::rfind
02FED382 0000002E 00000001 0028BB64  wxmsw28u_gcc_custom.dll!wxString::Find
02FED3DB 2F772C98 0000002E 0028E488  wxmsw28u_gcc_custom.dll!wxString::AfterLast
618734B6 2F772C98 00000001 00000000  codeblocks.dll!FileTypeOf
0BA78572 2D171550 0E8AA3B0 2B0D5A50  compiler.dll!SetupIncludeDirs  [codeblocks_sf/src/plugins/compilergcc/compilerMINGWgenerator.cpp @ 55]
    53:         {
    54:             ProjectFile* f = *it;
>   55:             if (FileTypeOf(f->relativeFilename) == ftHeader &&
    56:                 f->compile)
    57:             {
6181B4CA 2AA20288 0A84315C 00000024  codeblocks.dll!CompilerCommandGenerator::Init
0BA75442 2AA20288 0028E488 00000000  compiler.dll!GetCommandGenerator  [codeblocks_sf/src/plugins/compilergcc/compilerMINGW.cpp @ 51]
    49: {
    50:     CompilerMINGWGenerator *generator = new CompilerMINGWGenerator;
>   51:     generator->Init(project);
    52:     return generator;
    53: }
0A756A6E 063E0198 2F789598 2D29D1C8  clanglib.dll!GetCompileCommand  [ClangLib/src/clangplugin.cpp @ 1186]
  1184:     if (compileCommandStr.IsEmpty())
  1185:         compileCommandStr = wxT("$options $includes");
> 1186:     CompilerCommandGenerator* gen = comp->GetCommandGenerator(proj);
  1187:     if (gen)
  1188:         gen->GenerateCommandLine(compileCommandStr, target, pf, filename,
0A757958 2D29D070 FFFFFFFC 0028C484  clanglib.dll!UpdateCompileCommand  [ClangLib/src/clangplugin.cpp @ 1262]
  1260: int ClangPlugin::UpdateCompileCommand(cbEditor* ed)
  1261: {
> 1262:     std::vector<wxString> compileCommand = GetCompileCommand( ed->GetProjectFile(), ed->GetFilename() );
  1263:
  1264:     if (compileCommand.empty())
0A753B54 0028C518 0028C484 0028C3D4  clanglib.dll!OnEditorActivate  [ClangLib/src/clangplugin.cpp @ 656]
   654:         if(m_TranslUnitId == wxNOT_FOUND)
   655:             m_TranslUnitId = GetTranslationUnitId(filename);
>  656:         UpdateCompileCommand(ed);
   657:         if (m_TranslUnitId == wxNOT_FOUND)
   658:         {
0A7B008F 0028C518 0028C418 06332D18  clanglib.dll!Call  [wxWidgets-2.8.12/include/wx/buffer.h @ 127]
   125: #endif // wxABI_VERSION >= 20804
   126:
>  127: DEFINE_BUFFER(wxCharBuffer, char, wxStrdupA);
   128: #if wxABI_VERSION >= 20804
   129: DEFINE_WRITABLE_BUFFER(wxWritableCharBuffer, wxCharBuffer, char);
6188D58E 0028C518 FFFFFFFC 0028C454  codeblocks.dll!Manager::ProcessEvent
618A321B 0028C518 FFFFFFFF 00000000  codeblocks.dll!PluginManager::NotifyPlugins
61856F4D 0028D044 2D3508C8 0028C5D4  codeblocks.dll!EditorManager::OnPageChanged
02FA1261 06331094 61856DBA FFFFFFFC  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 61C32430 06331094 0028D044  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028D044 06331094 030243E3  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028D044 06332D1C 030243E3  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
 --- repeats 16 times ---
03024829 0028D044 0028C828 030243E3  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028D044 06585ED0 0028C954  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028D044 0641E648 525A6C97  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 54 times ---
03024829 0028D044 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028D044 05DFD9A8 0323C913  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028D044 000000B3 033B31D5  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
03247331 0000002B 0028D0C4 0028D0C8  wxmsw28u_gcc_custom.dll!wxAuiNotebook::SetSelection
03247B2A 0000001E 0028D168 033B9FCC  wxmsw28u_gcc_custom.dll!wxAuiNotebook::RemovePage
03247BC3 0000001E 2D3508C8 0028D1E0  wxmsw28u_gcc_custom.dll!wxAuiNotebook::DeletePage
617C7E84 0000001E 00000003 2CC706A4  codeblocks.dll!cbAuiNotebook::DeletePage
618531BA 2D3508C8 00000001 0028D1C4  codeblocks.dll!EditorManager::Close
61853140 0028D22C 00000001 0028D224  codeblocks.dll!EditorManager::Close
617F27E4 00000001 00000000 00000000  codeblocks.dll!cbProject::CloseAllFiles
618C1F53 2AA20288 00000000 00000001  codeblocks.dll!ProjectManager::CloseProject
0049091E 0028DF84 0000000D 0000001A  codeblocks.exe!ProjectManagerUI::OnCloseProject
02FA1261 05ECBAE8 004906FC 00000000  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 00590460 05ECBAE8 0028DF84  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028DF84 05ECBAE8 0028D4E4  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028DF84 05E18F48 0028D504  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
 --- repeats 16 times ---
03024829 0028DF84 00000000 0028D724  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028DF84 06585ED0 00000040  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028DF84 0641E648 0028D784  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 53 times ---
03024829 0028DF84 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028DF84 062EDBD8 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028DF84 04102D6C 2F7B7CB0  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028DF84 00000366 00000084  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
0310DA81 00000366 00000000 00000000  wxmsw28u_gcc_custom.dll!wxMenuBase::SendEvent
0309A32B 00000000 00000366 00000000  wxmsw28u_gcc_custom.dll!wxMenu::MSWCommand
0306DAB0 00000366 00000000 00000000  wxmsw28u_gcc_custom.dll!wxWindow::HandleCommand
0307134C 00000111 00000366 00000000  wxmsw28u_gcc_custom.dll!wxWindow::MSWWindowProc
030BF047 00000111 00000366 00000000  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWWindowProc
0306ADAE 00040AA8 00000111 00000366  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00040AA8 00000111  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00040AA8  USER32.dll!_UserCallWinProcCheckWow@32
770E77C4 0306AD60 00000000 0028E588  USER32.dll!_DispatchMessageWorker@8
770E788A 0028E554 00000000 00000111  USER32.dll!_DispatchMessageW@4
0306FBB9 0028E664 0000007E 00000066  wxmsw28u_gcc_custom.dll!wxWindow::DoPopupMenu
004F7C06 0028E664 0028E8B8 0028E800  codeblocks.exe!wxWindowBase::PopupMenu
0048AC05 2E0A2378 0028E8B8 00770020  codeblocks.exe!ProjectManagerUI::ShowMenu
0048DB76 0028F468 00000000 00000000  codeblocks.exe!ProjectManagerUI::OnTreeItemRightClick
02FA1261 05ECBAE8 0048DAD2 00000000  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 00590178 05ECBAE8 0028F468  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028F468 05ECBAE8 00000000  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028F468 05E18F48 00000000  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 16 times ---
03024829 0028F468 00000000 00000000  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028F468 06585ED0 2FD76F80  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028F468 0641E648 6DBFBC4E  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
---- repeats 54 times -----
03024829 0028F468 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028F468 062EDBD8 0028F3B8  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028F468 770E965E 00000014  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028F468 00001111 00000000  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
030BDF15 00000361 0028F968 0028F618  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWOnNotify
0307240C 0000004E 00000361 0028F968  wxmsw28u_gcc_custom.dll!wxWindow::MSWWindowProc
0306ADAE 00090BAE 0000004E 00000361  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00090BAE 0000004E  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00090BAE  USER32.dll!_UserCallWinProcCheckWow@32
770E965E 04512E20 00000000 0306AD60  USER32.dll!_SendMessageWorker@24
770E96C5 00090BAE 0000004E 00000361  USER32.dll!_SendMessageW@16
030BFBE4 00000204 00000002 0066007E  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWWindowProc
0306ADAE 00040AA8 00000204 00000002  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00040AA8 00000204  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00040AA8  USER32.dll!_UserCallWinProcCheckWow@32
770E77C4 0306AD60 00000000 0028FBE8  USER32.dll!_DispatchMessageWorker@8
770E788A 0028FBC4 00000000 00000000  USER32.dll!_DispatchMessageW@4
0304FAEF 2A7D0B58 00000001 0028FD40  wxmsw28u_gcc_custom.dll!wxEventLoop::Dispatch

seems to be a recursive loop? Hope this helps.
greetings keep up the good work!!!
Title: Re: Clang CC
Post by: MortenMacFly on February 27, 2017, 07:06:00 am
>   55:             if (FileTypeOf(f->relativeFilename) == ftHeader &&
I would also verify "f"not being NULL:
if (f && FileTypeOf(f->relativeFilename) == ftHeader &&
...especially on shut-down.
Title: Re: Clang CC
Post by: oBFusCATed on February 27, 2017, 08:51:19 am
And reversing the conditions speeds things quite a bit. :)
Checking f->compile is quite a lot faster than calling FileTypeOf and most of the times it is false.
This change will land in master/head soon.
Title: Re: Clang CC
Post by: BlueHazzard on March 01, 2017, 05:01:06 pm
And an other crash report:
Code
77B9E49B 2B27A350 0ABEF9C0 0FBA7650  ntdll.dll!0x2e49b
77B9E0A3 040C0000 00000000 2B27A350  ntdll.dll!_RtlFreeHeap@12
754C98CD 2B27A350 2B27A350 0004F980  msvcrt.dll!_free
033C793D 2DDE13EC 034FDDAC FFFFFFFF  wxmsw28u_gcc_custom.dll!wxWCharBuffer::~wxWCharBuffer
0A3039BF 0ABEF928 2DDE13EC 7FCE500D  clanglib.dll!FromUTF8  [wxWidgets-2.8.12/include/wx/buffer.h @ 127]
   125: #endif // wxABI_VERSION >= 20804
   126:
>  127: DEFINE_BUFFER(wxCharBuffer, char, wxStrdupA);
   128: #if wxABI_VERSION >= 20804
   129: DEFINE_WRITABLE_BUFFER(wxWritableCharBuffer, wxCharBuffer, char);
0A2C834E 2EA192D8 0ABEFA2C 00000001  clanglib.dll!UpdateIncludeDiagnosticsVisitor  [ClangLib/src/translationunit.cpp @ 630]
   628:
   629:     str = clang_getFileName(included_file);
>  630:     wxString includedFileName = wxString::FromUTF8(clang_getCString(str));
   631:     clang_disposeString(str);
   632:     if (data->errorIncludes.find( includedFileName ) != data->errorIncludes.end())
Title: Re: Clang CC
Post by: yvesdm3000 on March 06, 2017, 06:24:27 am
>   55:             if (FileTypeOf(f->relativeFilename) == ftHeader &&
I would also verify "f"not being NULL:
if (f && FileTypeOf(f->relativeFilename) == ftHeader &&
...especially on shut-down.

Sorry this is in compiler.dll and not in the clanglib plugin.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on March 07, 2017, 02:15:09 pm
And an other crash report:
Code
77B9E49B 2B27A350 0ABEF9C0 0FBA7650  ntdll.dll!0x2e49b
77B9E0A3 040C0000 00000000 2B27A350  ntdll.dll!_RtlFreeHeap@12
754C98CD 2B27A350 2B27A350 0004F980  msvcrt.dll!_free
033C793D 2DDE13EC 034FDDAC FFFFFFFF  wxmsw28u_gcc_custom.dll!wxWCharBuffer::~wxWCharBuffer
0A3039BF 0ABEF928 2DDE13EC 7FCE500D  clanglib.dll!FromUTF8  [wxWidgets-2.8.12/include/wx/buffer.h @ 127]
   125: #endif // wxABI_VERSION >= 20804
   126:
>  127: DEFINE_BUFFER(wxCharBuffer, char, wxStrdupA);
   128: #if wxABI_VERSION >= 20804
   129: DEFINE_WRITABLE_BUFFER(wxWritableCharBuffer, wxCharBuffer, char);
0A2C834E 2EA192D8 0ABEFA2C 00000001  clanglib.dll!UpdateIncludeDiagnosticsVisitor  [ClangLib/src/translationunit.cpp @ 630]
   628:
   629:     str = clang_getFileName(included_file);
>  630:     wxString includedFileName = wxString::FromUTF8(clang_getCString(str));
   631:     clang_disposeString(str);
   632:     if (data->errorIncludes.find( includedFileName ) != data->errorIncludes.end())

I wrote a little sikuliscript to reproduce this issue by continuously opening and closing projects and the same trace as you show here is there on a CentOS7 system. I think the problem comes from the fact that wxString is not thread safe since it is doing a double free of the wxString data and I pass these wxString objects between threads. Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety. I'll make a POC and see if that improves reliability.

Yves
Title: Re: Clang CC
Post by: ollydbg on March 07, 2017, 02:52:50 pm
It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety. I'll make a POC and see if that improves reliability.
wxString in wxWidgets 2.8.12 is not thread safe, and it should be thread safe in wxWidgets 3.0+, because it use std::string, and std::string is atomic reference counted.  There is same issue in the native CC, and it's hard to solve unless we move to wx3.0+.

What does POC mean?  Prove of Concept?
Title: Re: Clang CC
Post by: BlueHazzard on March 07, 2017, 03:17:56 pm
Hi, sorry i forget to answer and report my findings:

Code
77B9E0A3 040C0000 00000000 2B27A350  ntdll.dll!_RtlFreeHeap@12
754C98CD 2B27A350 2B27A350 0004F980  msvcrt.dll!_free
033C793D 2DDE13EC 034FDDAC FFFFFFFF  wxmsw28u_gcc_custom.dll!wxWCharBuffer::~wxWCharBuffer
0A3039BF 0ABEF928 2DDE13EC 7FCE500D  clanglib.dll!FromUTF8  [wxWidgets-2.8.12/include/wx/buffer.h @ 127]
   125: #endif // wxABI_VERSION >= 20804
   126:
>  127: DEFINE_BUFFER(wxCharBuffer, char, wxStrdupA);
   128: #if wxABI_VERSION >= 20804
   129: DEFINE_WRITABLE_BUFFER(wxWritableCharBuffer, wxCharBuffer, char);
0A2C834E 2EA192D8 0ABEFA2C 00000001  clanglib.dll!UpdateIncludeDiagnosticsVisitor  [ClangLib/src/translationunit.cpp @ 630]
   628:
   629:     str = clang_getFileName(included_file);
>  630:     wxString includedFileName = wxString::FromUTF8(clang_getCString(str));
   631:     clang_disposeString(str);
   632:     if (data->errorIncludes.find( includedFileName ) != data->errorIncludes.end())
about this crash: I replaced the function
Code
wxString::FromUTF8(clang_getCString(str))
with (pseudocode, have the actual code at home)
Code
wxString clang_to_wxString(clangstring* str)
{
if(str == nullptr)
return wxEmptyString;
else
return wxString::FromUTF8(clang_getCString(str));
}

and i have no crashes with this bt so far, but i tested it not that much.

Title: Re: Clang CC
Post by: BlueHazzard on March 07, 2017, 03:23:00 pm
regarding this crash:
Quote
Quote from: MortenMacFly on 27-02-2017, 07:06:00

    Quote from: BlueHazzard on 27-02-2017, 01:17:55

        >   55:             if (FileTypeOf(f->relativeFilename) == ftHeader &&

    I would also verify "f"not being NULL:
    if (f && FileTypeOf(f->relativeFilename) == ftHeader &&
    ...especially on shut-down.


Sorry this is in compiler.dll and not in the clanglib plugin.
with the bt:
Code
AddrPC   Params
02FEABF6 0000002E FFFFFFFF 0028BAD4  wxmsw28u_gcc_custom.dll!wxStringBase::rfind
02FED382 0000002E 00000001 0028BB64  wxmsw28u_gcc_custom.dll!wxString::Find
02FED3DB 2F772C98 0000002E 0028E488  wxmsw28u_gcc_custom.dll!wxString::AfterLast
618734B6 2F772C98 00000001 00000000  codeblocks.dll!FileTypeOf
0BA78572 2D171550 0E8AA3B0 2B0D5A50  compiler.dll!SetupIncludeDirs  [codeblocks_sf/src/plugins/compilergcc/compilerMINGWgenerator.cpp @ 55]
    53:         {
    54:             ProjectFile* f = *it;
>   55:             if (FileTypeOf(f->relativeFilename) == ftHeader &&
    56:                 f->compile)
    57:             {
6181B4CA 2AA20288 0A84315C 00000024  codeblocks.dll!CompilerCommandGenerator::Init
0BA75442 2AA20288 0028E488 00000000  compiler.dll!GetCommandGenerator  [codeblocks_sf/src/plugins/compilergcc/compilerMINGW.cpp @ 51]
    49: {
    50:     CompilerMINGWGenerator *generator = new CompilerMINGWGenerator;
>   51:     generator->Init(project);
    52:     return generator;
    53: }
0A756A6E 063E0198 2F789598 2D29D1C8  clanglib.dll!GetCompileCommand  [ClangLib/src/clangplugin.cpp @ 1186]
  1184:     if (compileCommandStr.IsEmpty())
  1185:         compileCommandStr = wxT("$options $includes");
> 1186:     CompilerCommandGenerator* gen = comp->GetCommandGenerator(proj);
  1187:     if (gen)
  1188:         gen->GenerateCommandLine(compileCommandStr, target, pf, filename,
0A757958 2D29D070 FFFFFFFC 0028C484  clanglib.dll!UpdateCompileCommand  [ClangLib/src/clangplugin.cpp @ 1262]
  1260: int ClangPlugin::UpdateCompileCommand(cbEditor* ed)
  1261: {
> 1262:     std::vector<wxString> compileCommand = GetCompileCommand( ed->GetProjectFile(), ed->GetFilename() );
  1263:
  1264:     if (compileCommand.empty())
0A753B54 0028C518 0028C484 0028C3D4  clanglib.dll!OnEditorActivate  [ClangLib/src/clangplugin.cpp @ 656]
   654:         if(m_TranslUnitId == wxNOT_FOUND)
   655:             m_TranslUnitId = GetTranslationUnitId(filename);
>  656:         UpdateCompileCommand(ed);
   657:         if (m_TranslUnitId == wxNOT_FOUND)
   658:         {
0A7B008F 0028C518 0028C418 06332D18  clanglib.dll!Call  [wxWidgets-2.8.12/include/wx/buffer.h @ 127]
   125: #endif // wxABI_VERSION >= 20804
   126:
>  127: DEFINE_BUFFER(wxCharBuffer, char, wxStrdupA);
   128: #if wxABI_VERSION >= 20804
   129: DEFINE_WRITABLE_BUFFER(wxWritableCharBuffer, wxCharBuffer, char);
6188D58E 0028C518 FFFFFFFC 0028C454  codeblocks.dll!Manager::ProcessEvent
618A321B 0028C518 FFFFFFFF 00000000  codeblocks.dll!PluginManager::NotifyPlugins
61856F4D 0028D044 2D3508C8 0028C5D4  codeblocks.dll!EditorManager::OnPageChanged
02FA1261 06331094 61856DBA FFFFFFFC  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 61C32430 06331094 0028D044  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028D044 06331094 030243E3  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028D044 06332D1C 030243E3  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
 --- repeats 16 times ---
03024829 0028D044 0028C828 030243E3  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028D044 06585ED0 0028C954  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028D044 0641E648 525A6C97  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 54 times ---
03024829 0028D044 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028D044 05DFD9A8 0323C913  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028D044 000000B3 033B31D5  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
03247331 0000002B 0028D0C4 0028D0C8  wxmsw28u_gcc_custom.dll!wxAuiNotebook::SetSelection
03247B2A 0000001E 0028D168 033B9FCC  wxmsw28u_gcc_custom.dll!wxAuiNotebook::RemovePage
03247BC3 0000001E 2D3508C8 0028D1E0  wxmsw28u_gcc_custom.dll!wxAuiNotebook::DeletePage
617C7E84 0000001E 00000003 2CC706A4  codeblocks.dll!cbAuiNotebook::DeletePage
618531BA 2D3508C8 00000001 0028D1C4  codeblocks.dll!EditorManager::Close
61853140 0028D22C 00000001 0028D224  codeblocks.dll!EditorManager::Close
617F27E4 00000001 00000000 00000000  codeblocks.dll!cbProject::CloseAllFiles
618C1F53 2AA20288 00000000 00000001  codeblocks.dll!ProjectManager::CloseProject
0049091E 0028DF84 0000000D 0000001A  codeblocks.exe!ProjectManagerUI::OnCloseProject
02FA1261 05ECBAE8 004906FC 00000000  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 00590460 05ECBAE8 0028DF84  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028DF84 05ECBAE8 0028D4E4  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028DF84 05E18F48 0028D504  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
 --- repeats 16 times ---
03024829 0028DF84 00000000 0028D724  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028DF84 06585ED0 00000040  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028DF84 0641E648 0028D784  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 53 times ---
03024829 0028DF84 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028DF84 062EDBD8 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028DF84 04102D6C 2F7B7CB0  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028DF84 00000366 00000084  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
0310DA81 00000366 00000000 00000000  wxmsw28u_gcc_custom.dll!wxMenuBase::SendEvent
0309A32B 00000000 00000366 00000000  wxmsw28u_gcc_custom.dll!wxMenu::MSWCommand
0306DAB0 00000366 00000000 00000000  wxmsw28u_gcc_custom.dll!wxWindow::HandleCommand
0307134C 00000111 00000366 00000000  wxmsw28u_gcc_custom.dll!wxWindow::MSWWindowProc
030BF047 00000111 00000366 00000000  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWWindowProc
0306ADAE 00040AA8 00000111 00000366  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00040AA8 00000111  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00040AA8  USER32.dll!_UserCallWinProcCheckWow@32
770E77C4 0306AD60 00000000 0028E588  USER32.dll!_DispatchMessageWorker@8
770E788A 0028E554 00000000 00000111  USER32.dll!_DispatchMessageW@4
0306FBB9 0028E664 0000007E 00000066  wxmsw28u_gcc_custom.dll!wxWindow::DoPopupMenu
004F7C06 0028E664 0028E8B8 0028E800  codeblocks.exe!wxWindowBase::PopupMenu
0048AC05 2E0A2378 0028E8B8 00770020  codeblocks.exe!ProjectManagerUI::ShowMenu
0048DB76 0028F468 00000000 00000000  codeblocks.exe!ProjectManagerUI::OnTreeItemRightClick
02FA1261 05ECBAE8 0048DAD2 00000000  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 00590178 05ECBAE8 0028F468  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028F468 05ECBAE8 00000000  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028F468 05E18F48 00000000  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 16 times ---
03024829 0028F468 00000000 00000000  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028F468 06585ED0 2FD76F80  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028F468 0641E648 6DBFBC4E  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
---- repeats 54 times -----
03024829 0028F468 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028F468 062EDBD8 0028F3B8  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028F468 770E965E 00000014  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028F468 00001111 00000000  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
030BDF15 00000361 0028F968 0028F618  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWOnNotify
0307240C 0000004E 00000361 0028F968  wxmsw28u_gcc_custom.dll!wxWindow::MSWWindowProc
0306ADAE 00090BAE 0000004E 00000361  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00090BAE 0000004E  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00090BAE  USER32.dll!_UserCallWinProcCheckWow@32
770E965E 04512E20 00000000 0306AD60  USER32.dll!_SendMessageWorker@24
770E96C5 00090BAE 0000004E 00000361  USER32.dll!_SendMessageW@16
030BFBE4 00000204 00000002 0066007E  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWWindowProc
0306ADAE 00040AA8 00000204 00000002  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00040AA8 00000204  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00040AA8  USER32.dll!_UserCallWinProcCheckWow@32
770E77C4 0306AD60 00000000 0028FBE8  USER32.dll!_DispatchMessageWorker@8
770E788A 0028FBC4 00000000 00000000  USER32.dll!_DispatchMessageW@4
0304FAEF 2A7D0B58 00000001 0028FD40  wxmsw28u_gcc_custom.dll!wxEventLoop::Dispatch

you will note that this is a stack overflow crash. And the stack is nearly 210 calls in the clang library. Actually it is in the event handling code of wx 210 calls. So i think there is somewhere a recursive event loop. So this is a clang problem.
I am not this familiar with the event system so i didn't looked over it to much...
This is underlined by this messages from the bt I think this is a clang problem because of this returning bt information:
Code
0A751716 0028F468 06585ED0 2FD76F80  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
[Edit:] correcting the hypothesis
Title: Re: Clang CC
Post by: BlueHazzard on March 07, 2017, 03:30:07 pm
A other problem i noted quite frequently is if you open a new cpp file, enter code at the end of the file (no line text after the last line code) and this code has errors, so they get reported by the instant error reporting wxScilite will report some error about writing past the buffer, or writing non printable characters or something like this. I am actively investigating in this if i find some time.
This is no crash, but a assert()

I don't recall how i can reproduce this, but i will report as soon i am able to reproduce it...
Title: Re: Clang CC
Post by: yvesdm3000 on March 07, 2017, 03:48:33 pm
It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety. I'll make a POC and see if that improves reliability.
wxString in wxWidgets 2.8.12 is not thread safe, and it should be thread safe in wxWidgets 3.0+, because it use std::string, and std::string is atomic reference counted.  There is same issue in the native CC, and it's hard to solve unless we move to wx3.0+.

What does POC mean?  Prove of Concept?
Yes, Prove of Concept, I adapt some operations that are used in the project load&unload scenario that I was using and see if this specific problem is fixed.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on March 07, 2017, 03:52:23 pm
A other problem i noted quite frequently is if you open a new cpp file, enter code at the end of the file (no line text after the last line code) and this code has errors, so they get reported by the instant error reporting wxScilite will report some error about writing past the buffer, or writing non printable characters or something like this. I am actively investigating in this if i find some time.
This is no crash, but a assert()

I don't recall how i can reproduce this, but i will report as soon i am able to reproduce it...
Sounds familiar, I fixed something similar in the past, but it was only reproducible on Windows. Sadly I don't test it often on Windows. It's hard enough as it is to test on wx28, wx30, ubuntu, centos, ...

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on March 07, 2017, 09:15:47 pm
regarding this crash:
Quote
Quote from: MortenMacFly on 27-02-2017, 07:06:00

    Quote from: BlueHazzard on 27-02-2017, 01:17:55

        >   55:             if (FileTypeOf(f->relativeFilename) == ftHeader &&

    I would also verify "f"not being NULL:
    if (f && FileTypeOf(f->relativeFilename) == ftHeader &&
    ...especially on shut-down.


Sorry this is in compiler.dll and not in the clanglib plugin.
with the bt:
Code
AddrPC   Params
02FEABF6 0000002E FFFFFFFF 0028BAD4  wxmsw28u_gcc_custom.dll!wxStringBase::rfind
02FED382 0000002E 00000001 0028BB64  wxmsw28u_gcc_custom.dll!wxString::Find
02FED3DB 2F772C98 0000002E 0028E488  wxmsw28u_gcc_custom.dll!wxString::AfterLast
618734B6 2F772C98 00000001 00000000  codeblocks.dll!FileTypeOf
0BA78572 2D171550 0E8AA3B0 2B0D5A50  compiler.dll!SetupIncludeDirs  [codeblocks_sf/src/plugins/compilergcc/compilerMINGWgenerator.cpp @ 55]
    53:         {
    54:             ProjectFile* f = *it;
>   55:             if (FileTypeOf(f->relativeFilename) == ftHeader &&
    56:                 f->compile)
    57:             {
6181B4CA 2AA20288 0A84315C 00000024  codeblocks.dll!CompilerCommandGenerator::Init
0BA75442 2AA20288 0028E488 00000000  compiler.dll!GetCommandGenerator  [codeblocks_sf/src/plugins/compilergcc/compilerMINGW.cpp @ 51]
    49: {
    50:     CompilerMINGWGenerator *generator = new CompilerMINGWGenerator;
>   51:     generator->Init(project);
    52:     return generator;
    53: }
0A756A6E 063E0198 2F789598 2D29D1C8  clanglib.dll!GetCompileCommand  [ClangLib/src/clangplugin.cpp @ 1186]
  1184:     if (compileCommandStr.IsEmpty())
  1185:         compileCommandStr = wxT("$options $includes");
> 1186:     CompilerCommandGenerator* gen = comp->GetCommandGenerator(proj);
  1187:     if (gen)
  1188:         gen->GenerateCommandLine(compileCommandStr, target, pf, filename,
0A757958 2D29D070 FFFFFFFC 0028C484  clanglib.dll!UpdateCompileCommand  [ClangLib/src/clangplugin.cpp @ 1262]
  1260: int ClangPlugin::UpdateCompileCommand(cbEditor* ed)
  1261: {
> 1262:     std::vector<wxString> compileCommand = GetCompileCommand( ed->GetProjectFile(), ed->GetFilename() );
  1263:
  1264:     if (compileCommand.empty())
0A753B54 0028C518 0028C484 0028C3D4  clanglib.dll!OnEditorActivate  [ClangLib/src/clangplugin.cpp @ 656]
   654:         if(m_TranslUnitId == wxNOT_FOUND)
   655:             m_TranslUnitId = GetTranslationUnitId(filename);
>  656:         UpdateCompileCommand(ed);
   657:         if (m_TranslUnitId == wxNOT_FOUND)
   658:         {
0A7B008F 0028C518 0028C418 06332D18  clanglib.dll!Call  [wxWidgets-2.8.12/include/wx/buffer.h @ 127]
   125: #endif // wxABI_VERSION >= 20804
   126:
>  127: DEFINE_BUFFER(wxCharBuffer, char, wxStrdupA);
   128: #if wxABI_VERSION >= 20804
   129: DEFINE_WRITABLE_BUFFER(wxWritableCharBuffer, wxCharBuffer, char);
6188D58E 0028C518 FFFFFFFC 0028C454  codeblocks.dll!Manager::ProcessEvent
618A321B 0028C518 FFFFFFFF 00000000  codeblocks.dll!PluginManager::NotifyPlugins
61856F4D 0028D044 2D3508C8 0028C5D4  codeblocks.dll!EditorManager::OnPageChanged
02FA1261 06331094 61856DBA FFFFFFFC  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 61C32430 06331094 0028D044  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028D044 06331094 030243E3  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028D044 06332D1C 030243E3  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
 --- repeats 16 times ---
03024829 0028D044 0028C828 030243E3  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028D044 06585ED0 0028C954  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028D044 0641E648 525A6C97  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 54 times ---
03024829 0028D044 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028D044 05DFD9A8 0323C913  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028D044 000000B3 033B31D5  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
03247331 0000002B 0028D0C4 0028D0C8  wxmsw28u_gcc_custom.dll!wxAuiNotebook::SetSelection
03247B2A 0000001E 0028D168 033B9FCC  wxmsw28u_gcc_custom.dll!wxAuiNotebook::RemovePage
03247BC3 0000001E 2D3508C8 0028D1E0  wxmsw28u_gcc_custom.dll!wxAuiNotebook::DeletePage
617C7E84 0000001E 00000003 2CC706A4  codeblocks.dll!cbAuiNotebook::DeletePage
618531BA 2D3508C8 00000001 0028D1C4  codeblocks.dll!EditorManager::Close
61853140 0028D22C 00000001 0028D224  codeblocks.dll!EditorManager::Close
617F27E4 00000001 00000000 00000000  codeblocks.dll!cbProject::CloseAllFiles
618C1F53 2AA20288 00000000 00000001  codeblocks.dll!ProjectManager::CloseProject
0049091E 0028DF84 0000000D 0000001A  codeblocks.exe!ProjectManagerUI::OnCloseProject
02FA1261 05ECBAE8 004906FC 00000000  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 00590460 05ECBAE8 0028DF84  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028DF84 05ECBAE8 0028D4E4  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028DF84 05E18F48 0028D504  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
 --- repeats 16 times ---
03024829 0028DF84 00000000 0028D724  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028DF84 06585ED0 00000040  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028DF84 0641E648 0028D784  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 53 times ---
03024829 0028DF84 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028DF84 062EDBD8 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028DF84 04102D6C 2F7B7CB0  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028DF84 00000366 00000084  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
0310DA81 00000366 00000000 00000000  wxmsw28u_gcc_custom.dll!wxMenuBase::SendEvent
0309A32B 00000000 00000366 00000000  wxmsw28u_gcc_custom.dll!wxMenu::MSWCommand
0306DAB0 00000366 00000000 00000000  wxmsw28u_gcc_custom.dll!wxWindow::HandleCommand
0307134C 00000111 00000366 00000000  wxmsw28u_gcc_custom.dll!wxWindow::MSWWindowProc
030BF047 00000111 00000366 00000000  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWWindowProc
0306ADAE 00040AA8 00000111 00000366  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00040AA8 00000111  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00040AA8  USER32.dll!_UserCallWinProcCheckWow@32
770E77C4 0306AD60 00000000 0028E588  USER32.dll!_DispatchMessageWorker@8
770E788A 0028E554 00000000 00000111  USER32.dll!_DispatchMessageW@4
0306FBB9 0028E664 0000007E 00000066  wxmsw28u_gcc_custom.dll!wxWindow::DoPopupMenu
004F7C06 0028E664 0028E8B8 0028E800  codeblocks.exe!wxWindowBase::PopupMenu
0048AC05 2E0A2378 0028E8B8 00770020  codeblocks.exe!ProjectManagerUI::ShowMenu
0048DB76 0028F468 00000000 00000000  codeblocks.exe!ProjectManagerUI::OnTreeItemRightClick
02FA1261 05ECBAE8 0048DAD2 00000000  wxmsw28u_gcc_custom.dll!wxAppConsole::HandleEvent
0302436E 00590178 05ECBAE8 0028F468  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEventIfMatches
0302443A 0028F468 05ECBAE8 00000000  wxmsw28u_gcc_custom.dll!wxEventHashTable::HandleEvent
03024895 0028F468 05E18F48 00000000  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
--- repeats 16 times ---
03024829 0028F468 00000000 00000000  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
0A751716 0028F468 06585ED0 2FD76F80  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
03024829 0028F468 0641E648 6DBFBC4E  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
---- repeats 54 times -----
03024829 0028F468 00000000 FEEDBAB1  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028F468 062EDBD8 0028F3B8  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
03024829 0028F468 770E965E 00000014  wxmsw28u_gcc_custom.dll!wxEvtHandler::ProcessEvent
031338F2 0028F468 00001111 00000000  wxmsw28u_gcc_custom.dll!wxWindowBase::TryParent
030BDF15 00000361 0028F968 0028F618  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWOnNotify
0307240C 0000004E 00000361 0028F968  wxmsw28u_gcc_custom.dll!wxWindow::MSWWindowProc
0306ADAE 00090BAE 0000004E 00000361  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00090BAE 0000004E  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00090BAE  USER32.dll!_UserCallWinProcCheckWow@32
770E965E 04512E20 00000000 0306AD60  USER32.dll!_SendMessageWorker@24
770E96C5 00090BAE 0000004E 00000361  USER32.dll!_SendMessageW@16
030BFBE4 00000204 00000002 0066007E  wxmsw28u_gcc_custom.dll!wxTreeCtrl::MSWWindowProc
0306ADAE 00040AA8 00000204 00000002  wxmsw28u_gcc_custom.dll!wxWndProc
770E62FA 0306AD60 00040AA8 00000204  USER32.dll!_InternalCallWinProc@20
770E6D3A 00000000 0306AD60 00040AA8  USER32.dll!_UserCallWinProcCheckWow@32
770E77C4 0306AD60 00000000 0028FBE8  USER32.dll!_DispatchMessageWorker@8
770E788A 0028FBC4 00000000 00000000  USER32.dll!_DispatchMessageW@4
0304FAEF 2A7D0B58 00000001 0028FD40  wxmsw28u_gcc_custom.dll!wxEventLoop::Dispatch

you will note that this is a stack overflow crash. And the stack is nearly 210 calls in the clang library. Actually it is in the event handling code of wx 210 calls. So i think there is somewhere a recursive event loop. So this is a clang problem.
I am not this familiar with the event system so i didn't looked over it to much...
This is underlined by this messages from the bt I think this is a clang problem because of this returning bt information:
Code
0A751716 0028F468 06585ED0 2FD76F80  clanglib.dll!ProcessEvent  [ClangLib/src/clangplugin.cpp @ 222]
   220: bool ClangPlugin::ProcessEvent( wxEvent& event )
   221: {
>  222:     if (cbPlugin::ProcessEvent(event))
   223:         return true;
   224:     for (std::vector<ClangPluginComponent*>::iterator it = m_ActiveComponentList.begin(); it != m_ActiveComponentList.end(); ++it)
[Edit:] correcting the hypothesis

No it is not a recursing problem. On some operations wx is processing events for a short while, thats why you see them stacking up.

Yves
Title: Re: Clang CC
Post by: oBFusCATed on March 07, 2017, 09:54:06 pm
Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.
How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).
Title: Re: Clang CC
Post by: BlueHazzard on March 07, 2017, 11:58:19 pm
A other problem i noted quite frequently is if you open a new cpp file, enter code at the end of the file (no line text after the last line code) and this code has errors, so they get reported by the instant error reporting wxScilite will report some error about writing past the buffer, or writing non printable characters or something like this. I am actively investigating in this if i find some time.
This is no crash, but a assert()

I don't recall how i can reproduce this, but i will report as soon i am able to reproduce it...
Sounds familiar, I fixed something similar in the past, but it was only reproducible on Windows. Sadly I don't test it often on Windows. It's hard enough as it is to test on wx28, wx30, ubuntu, centos, ...

Yves

This seems to fix the error:

Code
diff --git a/src/clangrefactoring.cpp b/src/clangrefactoring.cpp
index b3913ac..f88b76d 100644
--- a/src/clangrefactoring.cpp
+++ b/src/clangrefactoring.cpp
@@ -217,8 +217,8 @@ void ClangRefactoring::BeginHighlightOccurrences(cbEditor* ed)
     // Set Styling:
     // clear all style indications set in a previous run (is also done once after text gets unselected)
     stc->IndicatorClearRange(0, stc->GetLength());
-
-    if (stc->GetTextRange(pos - 1, pos + 1).Strip().IsEmpty())
+    int max_length =  std::min(pos + 1 ,stc->GetLength());
+    if (stc->GetTextRange(pos - 1, max_length).Strip().IsEmpty())
         return;

     const int line = stc->LineFromPosition(pos);
i honestly don't know if it changes the behavior of the function, but at least it does not give this annoying message ;) . The decision if this is the right fix i let for the developer...
Title: Re: Clang CC
Post by: yvesdm3000 on March 08, 2017, 06:47:48 am
Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.
How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).

I do a c_str() on the source wxString when the operation or event is copied. I also avoided using wxCommandEvent and built a ClangEvent object that does this deep copy of every member.

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on March 08, 2017, 07:21:07 am
A other problem i noted quite frequently is if you open a new cpp file, enter code at the end of the file (no line text after the last line code) and this code has errors, so they get reported by the instant error reporting wxScilite will report some error about writing past the buffer, or writing non printable characters or something like this. I am actively investigating in this if i find some time.
This is no crash, but a assert()

I don't recall how i can reproduce this, but i will report as soon i am able to reproduce it...
Sounds familiar, I fixed something similar in the past, but it was only reproducible on Windows. Sadly I don't test it often on Windows. It's hard enough as it is to test on wx28, wx30, ubuntu, centos, ...

Yves

This seems to fix the error:

Code
diff --git a/src/clangrefactoring.cpp b/src/clangrefactoring.cpp
index b3913ac..f88b76d 100644
--- a/src/clangrefactoring.cpp
+++ b/src/clangrefactoring.cpp
@@ -217,8 +217,8 @@ void ClangRefactoring::BeginHighlightOccurrences(cbEditor* ed)
     // Set Styling:
     // clear all style indications set in a previous run (is also done once after text gets unselected)
     stc->IndicatorClearRange(0, stc->GetLength());
-
-    if (stc->GetTextRange(pos - 1, pos + 1).Strip().IsEmpty())
+    int max_length =  std::min(pos + 1 ,stc->GetLength());
+    if (stc->GetTextRange(pos - 1, max_length).Strip().IsEmpty())
         return;

     const int line = stc->LineFromPosition(pos);
i honestly don't know if it changes the behavior of the function, but at least it does not give this annoying message ;) . The decision if this is the right fix i let for the developer...

I applied the fix. Thanks !

Yves
Title: Re: Clang CC
Post by: yvesdm3000 on March 08, 2017, 08:14:48 am
Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.
How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).

I want to use std::string because Clang uses utf8, otherwise i'm stuck with a conversion of 16 bits into 8 bits (for most of the cases). I know std::string isn't ideal for utf8 but when you know the limitations, it's doable since I don't process the strings all that much.

Yves
Title: Re: Clang CC
Post by: ollydbg on March 11, 2017, 09:40:44 am
In the file "include\sdk_events.h", there is a class named "CodeBlocksThreadEvent" which I introduced years ago to implement deep copy of it's wxString member. It is for wx 2.8.12, since wx 3.x does not such issue, and this class is used for thread safety in some plugins.
Title: Re: Clang CC
Post by: oBFusCATed on March 11, 2017, 10:25:38 am
In wx3.0 in most implementations (gcc until 5.x) the string is still reference counted, but it is protected by an atomic. So deep copying is useful there, too.

Also in wx3.0 they've added a clone method: http://docs.wxwidgets.org/3.1.0/classwx_string.html#afe63f53ecaa197333c405ca985f733fe wxString::Clone.
Title: Re: Clang CC
Post by: ollydbg on March 11, 2017, 02:46:09 pm
In wx3.0 in most implementations (gcc until 5.x) the string is still reference counted, but it is protected by an atomic. So deep copying is useful there, too.

Also in wx3.0 they've added a clone method: http://docs.wxwidgets.org/3.1.0/classwx_string.html#afe63f53ecaa197333c405ca985f733fe wxString::Clone.
If I remember correctly, if the wxString(std::string) is atomic reference counted, it is theadsafe. I mean the wxCommandEvent should be threadsafe. What's what the atomic reference counter used for, in "copy on write" mode.
Title: Re: Clang CC
Post by: yvesdm3000 on March 18, 2017, 03:18:46 pm
I pushed a new feature called "Call hierarchy" that allows you to lookup call references and where a specific method/function is being called. And then each function that referenced a function can simply be looked up again in a new level in the tree. It's very usefull for understanding large codebases and is more accurate than the thread-search plugin.

There are still a lot of limitations that will be solved in the next updates:
- Doesn't list overrides
- Doesn't list callers that are calling a pure or not function that the selected call overrides
- Doesn't work on anything else than functions/methods. Member variables, global variables, class instances all come to mind tho be useful for this feature.
- List to the right should become optional

And obviously small bugs since it's all new code that will be solved every time I come across one (or someone indicates one).

As always open to feedback, suggestions or other excellent ideas.

Yves
Title: Re: Clang CC
Post by: BlueHazzard on March 18, 2017, 09:42:27 pm
Hi!
good work!

i tested your plugin on linux and have an assert from wx:

Code
ASSERT INFO:
/usr/include/wx-3.0/wx/datetime.h(876): assert "IsValid() && dt.IsValid()" failed in operator!=(): invalid wxDateTime

BACKTRACE:
[1] ClangToolbar::OnEditorHook(cbEditor*, wxScintillaEvent&)
[2] EditorHooks::CallHooks(cbEditor*, wxScintillaEvent&)
[3] cbEditor::OnScintillaEvent(wxScintillaEvent&)
[4] cbEditor::OnEditorUpdateUI(wxScintillaEvent&)
[5] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[6] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[7] wxEvtHandler::SearchDynamicEventTable(wxEvent&)
[8] wxEvtHandler::TryHereOnly(wxEvent&)
[9] wxEvtHandler::ProcessEventLocally(wxEvent&)
[10] wxEvtHandler::ProcessEvent(wxEvent&)
[11] wxWindowBase::TryAfter(wxEvent&)
[12] wxScintilla::NotifyParent(SCNotification*)
[13] ScintillaWX::NotifyParent(SCNotification)
[14] Editor::NotifyUpdateUI()
[15] Editor::Paint(Surface*, PRectangle)
[16] ScintillaWX::DoPaint(wxDC*, wxRect)
[17] wxScintilla::OnPaint(wxPaintEvent&)
[18] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[19] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[20] wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*)
[21] wxEvtHandler::TryHereOnly(wxEvent&)
[22] wxEvtHandler::DoTryChain(wxEvent&)
[23] wxEvtHandler::ProcessEvent(wxEvent&)
[24] wxEvtHandler::SafelyProcessEvent(wxEvent&)
[25] wxWindow::GTKSendPaintEvents(_GdkRegion const*)
[26] g_closure_invoke
[27] g_signal_emit_valist
[28] g_signal_emit
[29] gtk_main_do_event
[30] gdk_window_process_all_updates
[31] g_main_context_dispatch
[32] g_main_context_iteration
[33] gtk_main_iteration
[34] wxWindow::DoPopupMenu(wxMenu*, int, int)
[35] wxWindowBase::PopupMenu(wxMenu*, int, int)
[36] wxWindowBase::PopupMenu(wxMenu*, wxPoint const&) /usr/include/wx-3.0/wx/window.h:1216
[37] EditorBase::DisplayContextMenu(wxPoint const&, ModuleType)
[38] cbStyledTextCtrl::OnContextMenu(wxContextMenuEvent&)
[39] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[40] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[41] wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*)
[42] wxEvtHandler::TryHereOnly(wxEvent&)
[43] wxEvtHandler::DoTryChain(wxEvent&)
[44] wxEvtHandler::ProcessEvent(wxEvent&)
[45] wxEvtHandler::SafelyProcessEvent(wxEvent&)
[46] g_closure_invoke
[47] g_signal_emit_valist
[48] g_signal_emit
[49] gtk_propagate_event
[50] gtk_main_do_event
[51] g_main_context_dispatch
[52] g_main_loop_run
[53] gtk_main
[54] wxGUIEventLoop::DoRun()
[55] wxEventLoopBase::Run()
[56] wxAppConsoleBase::MainLoop()
[57] CodeBlocksApp::OnRun() codeblocks_sf/src/src/app.cpp:850
[58] wxEntry(int&, wchar_t**)
[59] CodeBlocks codeblocks_sf/src/src/app.cpp:322
[60] __libc_start_main
[61] _start

The source is from here:
clangtoolbar.cpp:196
Code
if ( (ed->GetLastModificationTime() != m_CurrentState.m_CurrentEditorModificationTime)||(m_Function&&(m_Function->GetCount()==0)))

it seems that m_CurrentState.m_CurrentEditorModificationTime is not initialized by inspecting with the debugger...

Also i can't test your new function? How do i call it? In the right click menu i can't find a entry?
[edit:] Found it... Somehow the menu entry was not present, but after a restart all worked

greetings
Title: Re: Clang CC
Post by: yvesdm3000 on March 19, 2017, 04:40:11 am
CurrentState.m_CurrentEditorModificationTime  initialization should now be fixed

Yves
Title: Re: Clang CC
Post by: BlueHazzard on April 14, 2017, 10:25:08 pm
How is the update algorithm of the cc? If i open the codeblocks project file one of my processors is at 100%  and the console outputs
Code
Reindex on tokenindexdb 0x3831db0 project=codeblocks_sf/src/CodeBlocks_wx30-unix.cbp /codeblocks_sf/src/include/configurationpanel.h
ClTranslationUnit::Parse id=127
[Thread 0x7fffb6732700 (LWP 20185) exited]
ClTranslationUnit::UpdateTokenDatabase 127 finished: 74779 tokens processed

could this be a recursive loop?
Does the parser indexes all files all the time, or does it stop after all project files are parsed?

thank you for your work!

greetings
Title: Re: Clang CC
Post by: yvesdm3000 on April 15, 2017, 12:08:46 pm
How is the update algorithm of the cc? If i open the codeblocks project file one of my processors is at 100%  and the console outputs
Code
Reindex on tokenindexdb 0x3831db0 project=codeblocks_sf/src/CodeBlocks_wx30-unix.cbp /codeblocks_sf/src/include/configurationpanel.h
ClTranslationUnit::Parse id=127
[Thread 0x7fffb6732700 (LWP 20185) exited]
ClTranslationUnit::UpdateTokenDatabase 127 finished: 74779 tokens processed

could this be a recursive loop?
Does the parser indexes all files all the time, or does it stop after all project files are parsed?

thank you for your work!

greetings

The indexer checks if the timestamp of a file is more recent than the timestamp in the database. If it does, it reindexes and updates the timestamp in the database. Someday I want to extend this with the filesize just in case a file has been reverted (a checksum is probably overkill) .

I actually fixed an issue on this in commit d0c4f433b0b1e619f7fd14794c05d84379db7c1b on 21 of march for an issue where it continuously tried to reindex the same file. I don't know if your build has this fix yet?

Yves

Title: Re: Clang CC
Post by: BlueHazzard on August 01, 2017, 02:48:40 pm
Code
#0 0xa6ed440	__cxa_throw() ( output\share\codeblocks\plugins\clanglib.dll:??)
#1 0xa6ec266 operator new(unsigned int) () ( output\share\codeblocks\plugins\clanglib.dll:??)
#2 0xa7bae18 typeinfo for std::time_put<wchar_t, std::ostreambuf_iterator<wchar_t, std::char_traits<wchar_t> > >() ( output\share\codeblocks\plugins\clanglib.dll:??)
#3 0xa6f0200 std::bad_alloc::what() const() ( output\share\codeblocks\plugins\clanglib.dll:??)
#4 0xa7784c9 std::allocator_traits<std::allocator<ClAbstractToken> >::max_size(__a=...) (mingw32/4.9.2/include/c++/bits/alloc_traits.h:422)
#5 0xa7784a8 std::allocator_traits<std::allocator<ClAbstractToken> >::allocate(__a=..., __n=131072) (mingw32/4.9.2/include/c++/bits/alloc_traits.h:357)
#6 0xa762a99 std::_Vector_base<ClAbstractToken, std::allocator<ClAbstractToken> >::_M_allocate(this=0x55a1d0bc, __n=131072) ( mingw32/4.9.2/include/c++/bits/stl_vector.h:170)
#7 0xa786642 std::vector<ClAbstractToken, std::allocator<ClAbstractToken> >::_M_emplace_back_aux<ClAbstractToken const&>(this=0x55a1d0bc) ( mingw32/4.9.2/include/c++/bits/vector.tcc:412)
#8 0xa786a06 std::vector<ClAbstractToken, std::allocator<ClAbstractToken> >::push_back(this=0x55a1d0bc, __x=...) ( mingw32/4.9.2/include/c++/bits/stl_vector.h:923)
#9 0xa723c8c ClTreeMap<ClAbstractToken>::Insert(this=0x55a1d0b8, key=..., value=...) ( cclang-plugin/src/ClangLib/src/treemap.h:37)
#10 0xa6e3c77 ClTokenDatabase::InsertToken(this=0xb00fdc8, token=...) ( cclang-plugin\src\ClangLib\src\tokendatabase.cpp:960)
#11 0xa6e9bf0 ClImportClangToken(cursor=..., scopeCursor=..., typ=ClTokenType_FuncDecl, client_data=0xb00fcc8) ( cclang-plugin\src\ClangLib\src\translationunit.cpp:886)
#12 0xa6ea1ea ClAST_Visitor(cursor=..., parent=..., client_data=0xb00fcc8) ( cclang-plugin\src\ClangLib\src\translationunit.cpp:983)
#13 0xf212b76 libclang!clang_disposeTranslationUnit() ( output\libclang.dll:??)
#14 0x42f42028 ?? () (??:??)
#15 0xf215041 libclang!clang_isTranslationUnit() ( output\libclang.dll:??)
#16 0x8 ?? () (??:??)
#17 ?? ?? () (??:??)

a other crash back trace. It happened after some time idle running