Author Topic: ClangComplete plugin  (Read 74315 times)

Offline Lalaland

  • Multiple posting newcomer
  • *
  • Posts: 23
ClangComplete plugin
« on: November 10, 2011, 03:51:45 pm »
V 0.1 is now complete.

Compilation Instructions
For windows you will need "Clang Binaries for Mingw32/x86" at http://llvm.org/releases/download.html plus whatever it takes to normally compile plugins for codeblocks. You will need to add clang.dll, found in the bin folder and the include folder to the build options

For linux you will need a package that provides libclang.so, the debian package is libclang1.  You will also need the headers, which can be found in the debian libclang-dev package.


The github directory can be found here:
https://github.com/Lalaland/ClangComplete

Just click on downloads and download the zip file. In the zip file is the codeblocks project file + the source files.


Usage Instructions
Create a simple project and restart codeblocks.

Load the .cbplugin plugin.
Open the simple project.
When you type 3 letters of a word, '.', '->' or '::' , or press the keys Cntr-F1 codecompletion will start

Note, this plugin will probably fail on advanced projects(at least it fails on this project), just use it on simple stuff for now



I am announcing that I am starting work on an alternate code completion plugin that uses libclang to parse the code.

With very simplistic code I am starting out with this:
http://i.imgur.com/JAs6w.png

With some images and alphabetical sorting I am here:


With correct accessibility images I am here:


No more showing of things you cannot use:


Code completion in the middle of words + local variables:




Goals for V1:
  • Add support for code completion without context(no '.', '->' or '::')
  • Start code completion after 3 letters
  • Add alternative characters(::, ->) to start code completion
  • Detect and apply compilation settings so as to find included directories and whatnot
  • Run first(AKA slow) parseTranslation on opening of file
  • Have the first(slow) parseTranslation run in a seperate thread
  • Handle switching files well(probably a map of file names to Translation Units)
  • Have icons like the current code completion
  • Inherit from cbCodeComepltionPlugin
  • Detect location of compiler's libstd++ headers, so I do not have to rely on clang's internal detection
  • Somehow inform the user that the files are still being parsed

Now I need to move into providing auxillary features such as: calltip help, find declaration, etc

Current goals include:
  • Have options including disabling images for results
  • Have tooltip to help users type in parameters for functions
  • Filter results by priority(going to have to use custom AutoComplete)
  • Have icons show accesability(public, private, protected) as well(need libclang patch)

Bugs
  • When reloading the plugin on an already open file, it will not load the file
  • Any error in the proceding input will cause it to fail(and show no completion results)

NOTE: I am no longer using a patched libclang
« Last Edit: November 23, 2011, 03:39:51 pm by Lalaland »

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: ClangComplete plugin
« Reply #1 on: November 10, 2011, 07:53:01 pm »
Lalaland: libclang looks *VERY* powerful. That's a stunningly simple example.

A useful thing that could happen is for you to figure out a way of pulling out the common UI pieces that all CC plugins would share from the current C::B CC plugin (much like ObFuScAtEd is doing for the debugger).

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: ClangComplete plugin
« Reply #2 on: November 10, 2011, 08:11:23 pm »
btw, is clang fast enough to parse the buffer on the fly, as in your simple example, or do you need to get clang to pre-scan all of the files in order to get acceptable responsiveness?

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: ClangComplete plugin
« Reply #3 on: November 10, 2011, 10:47:25 pm »
Yay, go on Lalaland, we are with you :)
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ClangComplete plugin
« Reply #4 on: November 11, 2011, 01:40:57 am »
Great Job. I'm thinking what I can contribute :D.
It looks like you are directly link to "libclang", this is different with codelite's implementation (it call clang.exe from command line).

By the way, this topic should be moved to CodeCompletion redesign, we have a sub-forum dedicated to CC, there I have some clang stuff posted some days ago.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Lalaland

  • Multiple posting newcomer
  • *
  • Posts: 23
Re: ClangComplete plugin
« Reply #5 on: November 11, 2011, 05:27:41 am »
odmoore: It does seem to lag a bit with a large number of includes. This
Code
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/array.hpp>
#include <boost/signal.hpp>
#include <boost/function.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/noncopyable.hpp>
#include <boost/aligned_storage.hpp>
#include <boost/enable_shared_from_this.hpp>

for example adds about a 2 second delay.

However, by enabling some flags that supposedly create some sort of cache, it only has the long delay the first time code completion is requested.

All the rest of the requests complete almost instantaneously.

Thank you for pointing out codelite's implementation.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ClangComplete plugin
« Reply #6 on: November 11, 2011, 05:47:07 am »
However, by enabling some flags that supposedly create some sort of cache, it only has the long delay the first time code completion is requested.
All the rest of the requests complete almost instantaneously.
Once I have seen in the clang's API document, there is an function parameter which can let clang to automatically create a pch file in memory. Then the user don't need to specify which file are PCH file, clang can create and maintain the cached PCH.

The other thing is that we need to grab all the "compiler search paths" and "user definitions" from the current target in c::b project.
BTW: QTcreator now has a branch to use clang too.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Lalaland

  • Multiple posting newcomer
  • *
  • Posts: 23
Re: ClangComplete plugin
« Reply #7 on: November 11, 2011, 06:59:28 am »
I have already started looking into grabbing all of the search paths.
This code is what I am using to do it.
Code
            ProjectBuildTarget *target = Manager::Get()->GetProjectManager()->GetActiveProject()->GetBuildTarget(0);
            wxString test = target->GetCompilerID();
            Compiler * comp = CompilerFactory::GetCompiler(test);

            wxArrayString next = comp->GetCompilerSearchDirs(target);

            wxString pray = GetStringFromArray(next, _(" "));

            Manager::Get()->GetLogManager()->Log(pray);

However, I have not looked into user definitions and other command line settings yet.
I just wish the documentation on the Compiler::GenerateCommandLine function was better.

EDIT: I got the GenerateCommandLine function working(just copied the code found in the compiler plugin)
« Last Edit: November 11, 2011, 07:31:03 am by Lalaland »

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: ClangComplete plugin
« Reply #8 on: November 11, 2011, 07:27:53 am »
odmoore: It does seem to lag a bit with a large number of includes. This
Code
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/array.hpp>
#include <boost/signal.hpp>
#include <boost/function.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/noncopyable.hpp>
#include <boost/aligned_storage.hpp>
#include <boost/enable_shared_from_this.hpp>


Okay, I'm getting a bit paranoid with boost. Last time I tried to install boost, my distro screwed up the dependencies.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: ClangComplete plugin
« Reply #9 on: November 11, 2011, 08:16:21 am »
I am announcing that I am starting work on an alternate code completion plugin that uses libclang to parse the code.
It looks very promising. One note: Please keep in mind that the plugin should derive from cbCodeCompletionPlugin, not cbPlugin to work properly in the end.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline daniloz

  • Regular
  • ***
  • Posts: 268
Re: ClangComplete plugin
« Reply #10 on: November 11, 2011, 08:19:29 am »
Hey this really looks great and promising...

I've tried to compile it on Windows and had to change some project definitions, maybe I can contribute with a windows version of the project file (as soon as I get it running here).

However, I could not yet compile the libclang... So, here my questions:

- is there a precompiled libclang.dll for windows already somewhere ?
- any directions of how to compile it? :-)

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: ClangComplete plugin
« Reply #11 on: November 11, 2011, 08:35:41 am »
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: ClangComplete plugin
« Reply #12 on: November 11, 2011, 08:54:36 am »
- is there a precompiled libclang.dll for windows already somewhere ?
- any directions of how to compile it? :-)
We have discussed this on the forum.
You can use either CMake+mingw or Cmake+VC or MSYS+mingw.
see:Re: Clang command line support for codecompletion and the following posts. :D

@
Lalaland
I'm not sure how you collect the GCC/MSVC's default compiler paths, for GCC, I know this is a function in CC to collect those paths, see:

Code
const wxArrayString& NativeParser::GetGCCCompilerDirs(const wxString &cpp_compiler)
{
    // keep the gcc compiler path's once if found across C::B session
    // makes opening workspaces a *lot* faster by avoiding endless calls to the compiler
    static std::map<wxString, wxArrayString> dirs;
    if (!dirs[cpp_compiler].IsEmpty())
        return dirs[cpp_compiler];

    // for starters , only do this for gnu compiler
    //CCLogger::Get()->DebugLog(_T("CompilerID ") + CompilerID);
    //
    //   Windows: mingw32-g++ -v -E -x c++ nul
    //   Linux  : g++ -v -E -x c++ /dev/null
    // do the trick only for c++, not needed then for C (since this is a subset of C++)


    // let's construct the command
    // use a null file handler
    // both works fine in Windows and Linux

#ifdef __WXMSW__
    wxString Command = cpp_compiler + _T(" -v -E -x c++ nul");
#else
    wxString Command = cpp_compiler + _T(" -v -E -x c++ /dev/null");
#endif

    // wxExecute can be a long action and C::B might have been shutdown in the meantime...
    if (Manager::IsAppShuttingDown())
        return dirs[cpp_compiler];

    static bool flag = false;
    if (flag)
        return dirs[cpp_compiler];

    // action time  (everything shows up on the error stream
    wxArrayString Output, Errors;
    flag = true;
    if (wxExecute(Command, Output, Errors, wxEXEC_SYNC | wxEXEC_NODISABLE) == -1)
    {
        TRACE(_T("GetGCCCompilerDirs::wxExecute failed!"));
        flag = false;
        return dirs[cpp_compiler];
    }
    flag = false;

    // start from "#include <...>", and the path followed
    // let's hope this does not change too quickly, otherwise we need
    // to adjust our search code (for several versions ...)
    bool start = false;
    for (size_t idxCount = 0; idxCount < Errors.GetCount(); ++idxCount)
    {
        wxString path = Errors[idxCount].Trim(true).Trim(false);
        if (!start)
        {
            if (!path.StartsWith(_T("#include <...>")))
                continue;
            path = Errors[++idxCount].Trim(true).Trim(false);
            start = true;
        }

        wxFileName fname(path, wxEmptyString);
        fname.Normalize();
        fname.SetVolume(fname.GetVolume().MakeUpper());
        if (!fname.DirExists())
            break;

        CCLogger::Get()->DebugLog(_T("Caching GCC dir: ") + fname.GetPath());
        dirs[cpp_compiler].Add(fname.GetPath());
    }

    return dirs[cpp_compiler];
}

void NativeParser::AddGCCCompilerDirs(Compiler* compiler, ParserBase* parser)
{
    wxFileName fn(wxEmptyString, compiler->GetPrograms().CPP);
    wxString masterPath = compiler->GetMasterPath();
    Manager::Get()->GetMacrosManager()->ReplaceMacros(masterPath);
    fn.SetPath(masterPath);
    fn.AppendDir(_T("bin"));

    const wxArrayString& gccDirs = GetGCCCompilerDirs(fn.GetFullPath());
    TRACE(_T("Adding %d cached gcc dirs to parser..."), gccDirs.GetCount());
    for (size_t i=0; i<gccDirs.GetCount(); ++i)
    {
        parser->AddIncludeDir(gccDirs[i]);
        TRACE(_T("AddCompilerDirs() : Adding cached compiler dir to parser: ") + gccDirs[i]);
    }
}
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Lalaland

  • Multiple posting newcomer
  • *
  • Posts: 23
Re: ClangComplete plugin
« Reply #13 on: November 11, 2011, 05:25:39 pm »
Hey this really looks great and promising...

I've tried to compile it on Windows and had to change some project definitions, maybe I can contribute with a windows version of the project file (as soon as I get it running here).

However, I could not yet compile the libclang... So, here my questions:

- is there a precompiled libclang.dll for windows already somewhere ?
- any directions of how to compile it? :-)

If you are in linux, sudo apt-get install libclang does the trick. I do not know for windows.

@
Lalaland
I'm not sure how you collect the GCC/MSVC's default compiler paths, for GCC, I know this is a function in CC to collect those paths, see:


I am using this (almost a direct copy and paste from the gcc compiler plugin)

         
Code
  ProjectFile* pf = editor->GetProjectFile();
            ProjectBuildTarget *target = Manager::Get()->GetProjectManager()->GetActiveProject()->GetBuildTarget(0);
            wxString test = target->GetCompilerID();
            Compiler * comp = CompilerFactory::GetCompiler(test);

            const pfDetails& pfd = pf->GetFileDetails(target);


            wxString Object = (comp->GetSwitches().UseFlatObjects)?pfd.object_file_flat:pfd.object_file;

            const CompilerTool &tool = comp->GetCompilerTool(ctCompileObjectCmd,_(".cpp"));
            wxString tempCommand = _("$options $includes");
            comp->GenerateCommandLine(tempCommand,target,pf,UnixFilename(pfd.source_file_absolute_native),Object,pfd.object_file_flat,
                                         pfd.dep_file);




            Manager::Get()->GetLogManager()->Log(tempCommand);

Which seems to work works, and has given me this for my current project.
Code
 -g -I/usr/include/codeblocks -I/usr/include/codeblocks/tinyxml -I/usr/include/codeblocks/scripting/include -I/usr/include/codeblocks/scripting/bindings -I/usr/include/codeblocks/scripting/sqplus -I/usr/include/codeblocks/wxscintilla/include    -I/usr/lib/wx/include/gtk2-unicode-release-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXGTK__ -pthread  -fPIC    -I../../../../usr/include/codeblocks 
So this problem appears to be solved(however,  the GenerateCommandLine functions should be refactored. Instead of all those ending arguments it should just take a pfDetails, or just a ProjectFile)

Offline Lalaland

  • Multiple posting newcomer
  • *
  • Posts: 23
Re: ClangComplete plugin
« Reply #14 on: November 11, 2011, 06:10:58 pm »
I finally have an improvement over the official version of code completion.
The current code completion cannot handle the macro mess also known as wx/buffer.h, so it cannot complete wxCharBuffer.

However, clang has no problem completing it :)