Clang CC on CentOS 7.3

[sodev]

Beeing annoyed by the fact that as soon as i use C++11 features (especially auto) CodeBlocks is pretty much degraded to a text editor i thought i try out this new CC plugin. To not clutter up the other thread i made this one to share my experiences on CentOS because it doesn't work out-of-the box there by following the scarce documentation .


Prequisites

• CentOS 7.3 1611
• CodeBlocks r10958 compiled against wxWidgets 2.8.12
• yum install clang-devel llvm-devel (this pulls in all required dependencies)
• ClangLib master e27fca4 and/or ClangLib staging e914a8c

Compiling ClangLib

• First you need to compile and install CodeBlocks, i won't cover this here
• For some fancy reason pkg-config doesn't look by default where CodeBlocks installs its configuration file, so you need to do something like export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
• Load the ClangLib Project File into an empty CodeBlocks Workspace
• Remove the option --ansi from the common compiler options to get rid of "nullptr was not declared" messages
• Comment the #include <clang-c/Documentation.h> line in the file translationunit.h, this header isn't present and it compiles without it
• Select your favorite configuration and compile it
• Install the plugin into CodeBlocks by either using the installation function from the Plugin Menu or manually copying the necessary files into the CodeBlocks installation
• Restart CodeBlocks, don't open any workspace/project and disable the integrated CodeCompletion plugin and close CodeBlocks again, see further down why

Common Issues

• I used this plugin as a drop-in replacement for the integrated CodeCompletion, i used the same settings, i especially had the option to update the parser on saving only enabled
• I tested the master branch only for a short time because the toolbar that usually lets you select the namespace and the present methods only had one segment and this was always empty, so this version was unusable for me
• As soon as you load a workspace CodeBlocks will crash if you load another workspace or close it, so all settings you made will be gone, if you want to make any changes to the configuration don't load a workspace
• With the integrated CodeCompletion plugin enabled the popups didn't contain any Doxygen comments, this wasn't the case when it was disabled
• For some unkown reason the Doxygen comments disappear from the popups at a later time
  
• Pretty much half of the settings of the main configuration page don't get saved, all the corresponding code is disabled because these settings seem to be shared with the integrated CodeCompletion plugin that i had to disable (and isn't it a bad idea anyway to have to parsers running over all files?)

Staging Issues

• After loading a workspace CodeBlocks eats next to 100% CPU which never goes down again and CodeBlocks feels slow, especially after opening new files everything was very slow and it took quite long (in comparision to the integrated CodeCompletion) to populate the toolbar
• Clang complains about some simple external includes, e.g. it fails to to process #include <boost/shared_ptr.hpp>, and all code completion is dead in that file
• The CodeCompletion popups for methods don't list arguments that have default parameters, inserting the method doesn't add these parameters as well, however the popup thats appears when you press TAB while going through the inserted code displays the argument, but it doesn't display its default value
• Although i think i disabled the option that inserts brief doxygen comments around the inserted arguments, i still get the comments inserted
• The inserted method arguments are space padded from the method parantheses, it is very annoying that this cannot be disabled because this violates my coding style that is correctly setup in the source formatter plugin

TLDR
Sadly in the current stage this plugin is not usable for me, it hogs the CPU, is too slow, doesn't work in cases where the integrated one does and randomly breaks in cases where it does. The developer has written in the other thread that it works for him without problems, i cannot confirm that and don't know if i'm doing anything wrong, i would be glad if someone else shares his experiences and maybe knows how to fix my issues .

[yvesdm3000]

Too bad it doesn't work out too well for you. Lets try to understand what's wrong and I'll try to fix the issues.
I don't really look at the 'master' branch anymore, focussing on developing new features that should be pushed to the master at a point where most of the features are stable.

With this 'staging' version, the whole project is indexed and especially at initial startup it might take a long time until all files are parsed. That's just the nature of things when you want to lookup a definition of a symbol... What can be improved here is how the user is informed that files are being indexed. Currently I log this in an extra output pane that is shown when codeblocks is run with the "-d" option.
Any ideas are welcome here since C::B doesn't offer anything for this.

A main source of performance issues are Clang compiler errors. When there are way too many, it becomes very sluggish. The trick is to make it happy first, and keep it happy while developing your project. Since there is a problem with #include <boost/shared_ptr.hpp>, you have a header-search-path problem. Fix it, it will make ClangCC so much happier. Search-paths are retrieved from the C::B project settings.

Yves

[yvesdm3000]

I also use CentOS7 (next to CentOS5), but both my compiler (GCC6) and Clang ( 3.8 ) are probably newer than what you use.
It might be a good idea I add an environment that matches more closely your setup since that is what more users will probably will want to use anyway.

I also notice that it is very sluggish if I allocate my CentOS7 VM only 1 core to work on. Giving it 4 cores greatly improves its performance. The way it's designed is that Clang runs on it' own thread, separated from the GUI. I specifically designed it this way so that you should always be able to type even when Clang is (re)parsing, doing code completion, indexing, analyzing etc and this was the main problem in Alpha's excellent initial code. Am I safe to assume you also run it on 1 core?
Also the latest feature where the #include <> statement shows an error when the include-file itself contains errors seem to introduce some latency in code-completion that I'm tackling now.

Yves

[sodev]

I think i should give some more details about my platform and project.

I am running on basically stock CentOS, every package comes from the official CentOS or EPEL repositories, the only exception are CodeBlocks and some libraries the project uses like boost, these i compile myself. So as of now i'm using gcc 4.8.5 and clang 3.4.2.

The project is not some testing playground but a real-life application which i am working on for multiple years now. The workspace contains 12 projects, according to the CodeBlocks statistics plugin around 570 files, 140K lines with 80K lines beeing code. There are no search path issues, the workspace compiles and runs fine, the integrated CC has no problem with boost. I'm pretty sure the problem is not related to the compiler configuration, but for the record, the include paths are comprised of absolute paths, relative paths and paths containing CodeBlocks Global Variables.

My VM was indeed running on only one core so i added another one and gave it another shot with a clean source tree. This time CodeBlocks was much more responsive during the initial parsing phase and after about 5 minutes the CPU load dropped to a normal level. However my problems are still the same and the toolbar doesn't seem to get populated faster.

It's not a general problem that boost doesn't get detected, other files using boost don't show the error, and i don't think its related to boost at all. I think its related to the fact that my includes are ordered - c/c++ library headers, system headers, external library headers, internal library headers, project headers - in the files where the inline diagnostics marks the boost include as an error, this is just the first include after the c/c++ library headers. It looks like there is an indexdb file created for each project, however i get these files for only 5 of the 12 projects, no matter how many files of the projects with a missing indexdb file i open, it doesn't get created. And the files where the diagnostics complains about the boost headers are part of projects for which this indexdb file is missing.

I checked the CodeBlocks debug console and recognized a pattern, clang is complaining on a number of c/c++ library heades from gcc, from the names i assume they contain compiler intrinsics. I don't know if its a big help if i post some of these messages or the whole log, but because this is a real project corporate rules require that i remove some information like paths and names from the logs which is quite some work i'd like to save.

[yvesdm3000]

Hi,

Thanks for the feedback, it is much appreciated.

I test this on our business app (don't know statistics right now, high number of developers, with various teams from around the globe etc, the code is in factors of gigabytes), the only difference is that I only have 1 huge cbproject for it. Since it uses a makefile, I try to recreate the paths&defines so that ClangCC is happy, at least for the parts I need to work in. I think it's a good idea if I try using multiple projects for this and try to spot the issues you are seeing.

Regarding the paths, there might still be unknown issues with the translation from the codeblocks include-paths to clang. It would help if you could pinpoint if it's a relative path or a path with an environment variable in it. The ClangCC plugin itself allready uses a path from the C::B global-variables functionality and also uses the backtick way for pkg-config without issues. In my daytime project, relative paths are used.

Also be aware, you are trying Clang for code completion, but under the hood you are also using Clang as a kind-of compiler so in theory the code should meet the criteria to be compiled using Clang. I know my daytime-work-project isn't, but the parts I work in are fine (and includes boost).

Yves

[sodev]

Looks like i have been a little bit too lazy, i relied on svn to display these indexdb files as unrevisioned to spot them, too bad some directories had wildcard ignores, so i was wrong, i'm not missing any indexdb files, each project has one, so thats not the issue . Search paths are also not the problem, i have files where boost does get picked up and get auto-completed, and every project includes boost by the very same absolute path.

So i took a closer look at the debug messages, and it looks like all my error messages are the same:

Code

+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/ia32intrin.h to errorIncludes, scope: /my_path/my_header.h
[snip]
+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/ia32intrin.h to errorIncludes, scope: /my_path/my_header.h

+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/xmmintrin.h to errorIncludes, scope: /my_path/my_header.h
[snip]
+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/xmmintrin.h to errorIncludes, scope: /my_path/my_header.h

+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/emmintrin.h to errorIncludes, scope: /my_path/my_header.h
[snip]
+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/emmintrin.h to errorIncludes, scope: /my_path/my_header.h

+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/adxintrin.h to errorIncludes, scope: /my_path/my_header.h

[snip]

Visit include statement /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/emmintrin.h
+++ Inserting error at 13,10
Visit include statement /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/xmmintrin.h
+++ Inserting error at 13,10
Visit include statement /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/immintrin.h
No warnings or errors for this include file
Visit include statement /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/adxintrin.h
+++ Inserting error at 13,10

In the terminal attached to the codeblocks process i can see clang spills out error messages about undeclared identifier mostly. I verified by example that whenever i open a file that produces these messages, code completion is dead in that file. Files that produce only warnings like unused parameters, type conversion and whatever do have a working code completion.

The second issue i have in some files is that the diagnostics marks some headers with an error like this:

Code

#include <cstdint>
#include <type_traits>

#include <map>
#include <set>
#include <sstream>
#include <string>
#include <vector>

#include <boost/optional.hpp>
 Errors present
#include <boost/shared_ptr.hpp>
#include <boost/scoped_ptr.hpp>

I can't say for sure if that happens only or always with files that produce the error messages above, but in some examples this was the case.

Playing around with the CC i discovered the following things:

• Using the context menu action "Open include file" on #include statements more or less randomly doesn't work the first time, i have to click on the very same line twice in a row to open the include file
• Clicking in the method body for the first time selects the scope and function in the toolbar, however when opening the function combobox it only contains this very function and the scope name on top. Clicking on it a second time populates this combobox with (almost, see next point) all methods and removes the scope
• The method combobox never contains any constructor or destructor, clicking on the body of any of these unselects everything in the toolbar
• The function "Goto declaration" doesn't work when i click on the method name in the implementation file of that method, only when i click on that methode name in a different file that calls this method. However "Find implementation" on the method name in the header file of that method works, so basically i can switch from the header to the implementation but not the reverse way

Also be aware, you are trying Clang for code completion, but under the hood you are also using Clang as a kind-of compiler so in theory the code should meet the criteria to be compiled using Clang. I know my daytime-work-project isn't, but the parts I work in are fine (and includes boost).

I don't understand what you mean with this, my project gets compiled with gcc only.

[yvesdm3000]

Looks like i have been a little bit too lazy, i relied on svn to display these indexdb files as unrevisioned to spot them, too bad some directories had wildcard ignores, so i was wrong, i'm not missing any indexdb files, each project has one, so thats not the issue .

Good, something I don't have to look into then.

Search paths are also not the problem, i have files where boost does get picked up and get auto-completed, and every project includes boost by the very same absolute path.

So i took a closer look at the debug messages, and it looks like all my error messages are the same:

Code

+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/ia32intrin.h to errorIncludes, scope: /my_path/my_header.h
[snip]
+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/ia32intrin.h to errorIncludes, scope: /my_path/my_header.h

+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/xmmintrin.h to errorIncludes, scope: /my_path/my_header.h
[snip]
+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/xmmintrin.h to errorIncludes, scope: /my_path/my_header.h

+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/emmintrin.h to errorIncludes, scope: /my_path/my_header.h
[snip]
+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/emmintrin.h to errorIncludes, scope: /my_path/my_header.h

+++ Adding /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/adxintrin.h to errorIncludes, scope: /my_path/my_header.h

[snip]

Visit include statement /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/emmintrin.h
+++ Inserting error at 13,10
Visit include statement /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/xmmintrin.h
+++ Inserting error at 13,10
Visit include statement /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/immintrin.h
No warnings or errors for this include file
Visit include statement /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/adxintrin.h
+++ Inserting error at 13,10

OK this is debug output mostly from one of the latest features where a header that contains errors will result in a diagnostic error where the #include statement is. This way, when an error is really triggered by this file, you can spot the source of the error more easily.

In the terminal attached to the codeblocks process i can see clang spills out error messages about undeclared identifier mostly. I verified by example that whenever i open a file that produces these messages, code completion is dead in that file. Files that produce only warnings like unused parameters, type conversion and whatever do have a working code completion.

The second issue i have in some files is that the diagnostics marks some headers with an error like this:

Code

#include <cstdint>
#include <type_traits>

#include <map>
#include <set>
#include <sstream>
#include <string>
#include <vector>

#include <boost/optional.hpp>
 Errors present
#include <boost/shared_ptr.hpp>
#include <boost/scoped_ptr.hpp>

Although it mostly works, after using this a bit more extensively, I did notice some false positives e.g. the include file is marked with errors, I did not find errors in it. It's the next issue I'm about to fix.

Playing around with the CC i discovered the following things:

• Using the context menu action "Open include file" on #include statements more or less randomly doesn't work the first time, i have to click on the very same line twice in a row to open the include file

Yep, I also noticed this. It's somewhere on the list to be fixed.

Also be aware, you are trying Clang for code completion, but under the hood you are also using Clang as a kind-of compiler so in theory the code should meet the criteria to be compiled using Clang. I know my daytime-work-project isn't, but the parts I work in are fine (and includes boost).

I don't understand what you mean with this, my project gets compiled with gcc only.

It means that this plugin is based on Clang and under-the-hood your project is being compiled using clang, but only the early stages of compilation, just enough so that an AST tree is created to perform code-completion and diagnostics on it, no executable output.

Your intrinsics-problem is probably because clang can't compile it and gcc can. It's one of the downsides of using Clang for this without an immediate solution altough the Clang developers try to be compatible with gcc as much as they can.

Yves

[oBFusCATed]

yvesdm3000: Isn't clang working in a mode where it is tolerant to errors as much as possible?

[yvesdm3000]

yvesdm3000: Isn't clang working in a mode where it is tolerant to errors as much as possible?

"As much as possible"  --> Yes, but it remains in the first place a compiler. The C::B built-in code-completion parser is probably better at it: it can just skip what looks like a function, a declaration or anything it doesn't know about. Clang will require knowledge on everything to build its AST tree for the compiler part, even if it's only needed for diagnostics or making things pretty...

Yves

[yvesdm3000]

I just fixed the "Update parser when typing" setting, it seems it wasnn't implemented properly.
I also moved the setting to the 'Diagnostics'-tab since it really only has an effect on diagnostics (and also for goto-implementation things). For CC a reparse is needed in about 90% of the cases anyway since CC is about modifying code and Clang will implicitly do that for you anyway...

Yves