Author Topic: Clang command line support for codecompletion  (Read 70153 times)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Clang command line support for codecompletion
« on: October 25, 2010, 09:04:29 am »
I just find a useful file (the file can be enable codecompletion for emace)

http://stackoverflow.com/questions/2487931/llvm-c-ide-for-windows
and
https://llvm.org/svn/llvm-project/cfe/trunk/utils/clang-completion-mode.el

http://mike.struct.cn/blogs/entry/15/

I have never used emace.

Can some emace guys give some comments on this?

So that we can use the same command line?
« Last Edit: October 25, 2010, 09:27:19 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Clang command line support for codecompletion
« Reply #1 on: October 25, 2010, 10:11:50 am »
ollydbg: if you plan to integrate clang, please, don't do it with creating another process and communicating with some IPC mechanism.
The proper way is to link directly to clangs libs. It will save you many problems.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #2 on: October 25, 2010, 10:17:20 am »
ollydbg: if you plan to integrate clang, please, don't do it with creating another process and communicating with some IPC mechanism.
The proper way is to link directly to clangs libs. It will save you many problems.
thanks for the hint.

currently, I'm checking Clang's features (command line mode) listed in

Examples of using Clang section of http://clang.llvm.org/get_started.html

Tested by this package
http://llvm.org/releases/2.8/llvm-gcc4.2-2.8-x86-mingw32.tar.bz2

It seems I can't get the result from
Code
$ clang -cc1 ~/t.c -ast-print

I have the error like:
Code
F:\cb\test_code\test_clang>llvm-c++ main.cpp
llvm-c++: CreateProcess: No such file or directory

F:\cb\test_code\test_clang>llvm-c++ main.cpp -E
# 1 "main.cpp"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "main.cpp"
int main()
{

}

F:\cb\test_code\test_clang>llvm-c++ main.cpp -cc1 -ast-print
llvm-c++: unrecognized option '-cc1'
cc1plus.exe: error: unrecognized command line option "-ast-print"

F:\cb\test_code\test_clang>llvm-c++ -cc1 main.cpp -ast-print
llvm-c++: unrecognized option '-cc1'
cc1plus.exe: error: unrecognized command line option "-ast-print"

So, I have to firstly figure it out. :D
« Last Edit: October 25, 2010, 10:19:24 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline eranif

  • Regular
  • ***
  • Posts: 256
Re: Clang command line support for codecompletion
« Reply #3 on: October 25, 2010, 11:02:02 am »
You might find this useful:

http://codelite.svn.sourceforge.net/viewvc/codelite/trunk/LiteEditor/clang_code_completion.h?revision=4207&view=markup
http://codelite.svn.sourceforge.net/viewvc/codelite/trunk/LiteEditor/clang_code_completion.cpp?revision=4207&view=markup

It is working (currently disabled)
Although it uses the command line invocation of clang executable.

Note that clang itself provides a lib named 'libIndex' to perform what I am doing in the linked code. However, they too are invoking clang from the command line.

Eran

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #4 on: October 26, 2010, 02:33:46 am »
thanks eranif.
I will look into your links.

btw: did you know some Clang binaries which works under windows.
http://llvm.org/releases/2.8/llvm-gcc4.2-2.8-x86-mingw32.tar.bz2 seems is not a full package

thanks.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #5 on: October 26, 2010, 10:45:15 am »
updated:

Today, I have tried the method listed in
http://clang.llvm.org/get_started.html

to build a clang.exe, but no luck, all failed  :( :( :(.

1, one failed method.
under Msys mingw, check out the llvm and clang svn code, and configure. make. but the error occurred when I run the make,  such as:
Quote
$ make
Makefile:130: /Makefile.rules: No such file or directory
make: *** No rule to make target `/Makefile.rules'.  Stop.



2. failed method two:

I just use the cmake, then generate the makefile for mingw, then just run the make command.
when at build stage 97%, I get these message:
Quote
Linking CXX executable ..\..\..\..\bin\c-index-test.exe
CMakeFiles\c-index-test.dir\c-index-test.c.obj:c-index-test.c:(.text+0x47a4): un
defined reference to `_imp__pthread_create'
CMakeFiles\c-index-test.dir\c-index-test.c.obj:c-index-test.c:(.text+0x47e0): un
defined reference to `_imp__pthread_join'
collect2: ld returned 1 exit status
make[2]: *** [bin/c-index-test.exe] Error 1
make[1]: *** [tools/clang/tools/c-index-test/CMakeFiles/c-index-test.dir/all] Er
ror 2
make: *** [all] Error 2

E:\clang_build\build\tools\clang>
I have struggleed with the pthread for several hour,( copy some libs or shared libs...), and finally I still get stuck. :(


I hope some brave guy can help to build a clang distribution (windows). thanks.
currently, I can't find any clang binaries for windows.

thanks.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Clang command line support for codecompletion
« Reply #6 on: October 26, 2010, 12:27:54 pm »
Probably you should post on their mailing list :)
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #7 on: October 26, 2010, 03:08:16 pm »
Probably you should post on their mailing list :)
I asked there, but no replies. seems the devs are not interested in windows build.

my friend xunxun has build on for me (under mingw 4.5.1), he just some customized configure.

Code
./configure --prefix=/llvm --enable-optimized --enable-targets=host

then manually change the file
Makefile.config

Quote
-O2

to

Code
-pipe -O2 -DPTW32_STATIC_LIB

But the built clang seems can't work correctly, it even crashed in a quite dummy test code

a.c
Code
main()
{
}

then I run:
Code
clang.exe a.c -E

then clang crashed....Too bad. :(

I don't want to test any more until it becomes stable and useful enough. :shock:
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline eranif

  • Regular
  • ***
  • Posts: 256
Re: Clang command line support for codecompletion
« Reply #8 on: October 26, 2010, 03:21:47 pm »
thanks eranif.
I will look into your links.

btw: did you know some Clang binaries which works under windows.
http://llvm.org/releases/2.8/llvm-gcc4.2-2.8-x86-mingw32.tar.bz2 seems is not a full package

thanks.
I compiled clang myself on my machine using SVN / MinGW - so I am not using those.

Eran

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #9 on: October 26, 2010, 03:29:57 pm »
thanks eranif.
I will look into your links.

btw: did you know some Clang binaries which works under windows.
http://llvm.org/releases/2.8/llvm-gcc4.2-2.8-x86-mingw32.tar.bz2 seems is not a full package

thanks.
I compiled clang myself on my machine using SVN / MinGW - so I am not using those.

Eran

Hi, eran. thanks.
 
could you tell me how to build the clang svn under mingw? I have tried this nearly half of the day, but no success.  :(
what the plan if clang becomes more powerful? I guess it could replace the codecompletion part of codeblocks or codelite (if clang becomes stable)

if possible, could you upload the whole package of the clang to some site?

Currently, I can't find any clang binaries on the internet.

thanks.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline eranif

  • Regular
  • ***
  • Posts: 256
Re: Clang command line support for codecompletion
« Reply #10 on: October 26, 2010, 03:38:46 pm »
I already integrated this code into codelite and it seems to work very well.
The problem is that it is too slow.

Most of the time the parsing is done using an in-complete source file so you will need to provide a string buffer containing the input string up to the point where the completion should appear.

The problem is that, I tried to use the completion using codelite's project itself on the file frame.cpp (one of the largest files in codelite project) it took 4 seconds for the completion to show (or even more, can't remember now it was 3-4 months ago)

It also lacks some crucial information (which I could not seem to get an access to, like an exact position of the match), I can probably obtain more information if I will link directly with the clang libraries - but for the POC (prove of concept) it seems too much for me.

I had several ideas of how to improve the responsiveness of the code-completion by using a pre-compiled header, but it seems like that PCH is not yet supported for C++ (at least it was not available couple of months ago)

You can search their mailing list and see the discussion I had with DGregor (he is the person that knows most about code-completion in clang)


About the binaries, I remembers that enabling optimization causes a crash or ICE using MinGW. Also, make sure you build clang in release mode (which is NOT the default) it will boost its performance a lot.

I will have a look at home of how I built clang and will see if I can upload the binaries to codelite's site

Eran
 

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Clang command line support for codecompletion
« Reply #11 on: October 26, 2010, 05:37:33 pm »
@ollydbg:

you can try this one: http://apt.jenslody.de/downloads/clang_win32.7z

Cross-build on linux, at least simple hello world works on my win-xp machine.
Pretty printing also seems to work.

You will most likely find a broken clang++.exe in the bin subfolder, this is just a symlink to clang.exe on linux, but neither 7z, nor windows can handle (linux-)symlinks.
To use clang++.exe, you can just copy clang.exe to clang++.exe .

Offline ironhead

  • Almost regular
  • **
  • Posts: 210
Re: Clang command line support for codecompletion
« Reply #12 on: October 26, 2010, 06:43:02 pm »
You will most likely find a broken clang++.exe in the bin subfolder, this is just a symlink to clang.exe on linux, but neither 7z, nor windows can handle (linux-)symlinks.
To use clang++.exe, you can just copy clang.exe to clang++.exe .

Not that it matters, but as a quick FYI, you can use tar to de-reference symlinks:
Code
-h, --dereference            follow symlinks; archive and dump the files they point to
    --hard-dereference       follow hard links; archive and dump the files they refer to

I use these options when preparing MinGW packages that I've compiled under Cygwin.

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Clang command line support for codecompletion
« Reply #13 on: October 26, 2010, 07:05:27 pm »
Not that it matters, but as a quick FYI, you can use tar to de-reference symlinks:

I know that, but I cros-compiled it on linux, and windows can not follow symlinks (at least not linux-symlinks, and the windows links are not real links, just link-files).

By the way, 7z seems to work correctly with symlinks (just access control and ownership is not copied without tar).

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #14 on: October 27, 2010, 02:41:21 am »
I already integrated this code into codelite and it seems to work very well.
The problem is that it is too slow.

Most of the time the parsing is done using an in-complete source file so you will need to provide a string buffer containing the input string up to the point where the completion should appear.

The problem is that, I tried to use the completion using codelite's project itself on the file frame.cpp (one of the largest files in codelite project) it took 4 seconds for the completion to show (or even more, can't remember now it was 3-4 months ago)

It also lacks some crucial information (which I could not seem to get an access to, like an exact position of the match), I can probably obtain more information if I will link directly with the clang libraries - but for the POC (prove of concept) it seems too much for me.

I had several ideas of how to improve the responsiveness of the code-completion by using a pre-compiled header, but it seems like that PCH is not yet supported for C++ (at least it was not available couple of months ago)

You can search their mailing list and see the discussion I had with DGregor (he is the person that knows most about code-completion in clang)


About the binaries, I remembers that enabling optimization causes a crash or ICE using MinGW. Also, make sure you build clang in release mode (which is NOT the default) it will boost its performance a lot.

I will have a look at home of how I built clang and will see if I can upload the binaries to codelite's site

Eran
 

thanks eran for the explanation and help.
I just do a simple search and found that clang now support PCH, see:  Precompiled Headers (PCH)

Quote
Using Precompiled Headers with clang

The Clang compiler frontend, clang -cc1, supports two command line options for generating and using PCH files.

To generate PCH files using clang -cc1, use the option -emit-pch:

 $ clang -cc1 test.h -emit-pch -o test.h.pch

This option is transparently used by clang when generating PCH files. The resulting PCH file contains the serialized form of the compiler's internal representation after it has completed parsing and semantic analysis. The PCH file can then be used as a prefix header with the -include-pch option:

  $ clang -cc1 -include-pch test.h.pch test.c -o test.s


@jens:
thanks for your time and effort to build the win32 version of clang for me, it works quite well.

for the symlink issue, under windows, I use 7-zip to extract the files, but some files like clang++.exe was less than 10k. So, as you suggest, I need to copy and renamed the clang.exe.

I will do more test on clang today.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #15 on: October 27, 2010, 03:37:13 am »
Here is my detailed test for codecompletion in clang (Windows)

prerequisite:

You need to download the package supplied by jens: clang_win32
Then add the path of clang.exe in your PATH.

test

create a file like:
E:\clang_win32\main.cpp
the file contains some code snippet like:
Code

class AAA
{
public:
   int aaa;
   int bbb;
};


int myfunction()
{
  AAA a;
  a.
  return 0;
}


Now, you can see, the  codecompletion list should generated after "a." .

You just run the command: pay attention to the :13:5, this this the line and column number of the caret after "a."
Code
clang -w -fsyntax-only -Xclang -code-completion-at=E:\clang_win32\main.cpp:13:5 E:\clang_win32\main.cpp

Then here is the wonder result:
Code
E:\clang_win32>clang -w -fsyntax-only -Xclang -code-completion-at=E:\clang_win32
\main.cpp:13:5 E:\clang_win32\main.cpp
COMPLETION: AAA : AAA::
COMPLETION: aaa : [#int#]aaa
COMPLETION: bbb : [#int#]bbb
COMPLETION: operator= : [#AAA &#]operator=(<#const AAA &#>)
COMPLETION: ~AAA : [#void#]~AAA()

 :D

then main reference is eranif's codelite source (thanks eranif):
http://codelite.svn.sourceforge.net/viewvc/codelite/trunk/LiteEditor/clang_code_completion.cpp?revision=4207&view=markup


If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #16 on: October 27, 2010, 05:24:39 am »
about the pch. suppose I have two files: testcpp.h and main.cpp

testcpp.h
Code

class BBB
{
public:
int f1();
int f2();
int f3();
};

main.cpp

Code


#include "testcpp.h"

int myfunction()
{
  BBB a;
  a.
  return 0;
}

Now,
first I need to generate the pch file using command:
Code
E:\clang_win32>clang++ -cc1 -x c++-header testcpp.h -emit-pch -o testcpp.h.pch

Then, do the code completion command.
Code
E:\clang_win32>clang++ -cc1 -w -fsyntax-only -include-pch testcpp.h.pch -code-co
mpletion-at=E:\clang_win32\main.cpp:8:5  E:\clang_win32\main.cpp
In file included from E:\clang_win32\main.cpp:3:
E:\clang_win32\testcpp.h:3:7: error: redefinition of 'BBB'
class BBB
      ^
E:\clang_win32\testcpp.h:3:7: note: previous definition is here
class BBB
      ^
1 error generated.
COMPLETION: BBB : BBB::
COMPLETION: f1 : [#int#]f1()
COMPLETION: f2 : [#int#]f2()
COMPLETION: f3 : [#int#]f3()

Strange that there is an error about "redefinition".

So, I just comment the #include line like, so the modified main.cpp
Code


//#include "testcpp.h"

int myfunction()
{
  BBB a;
  a.
  return 0;
}

Now, everything works fine.
Code
E:\clang_win32>clang++ -cc1 -w -fsyntax-only -include-pch testcpp.h.pch -code-co
mpletion-at=E:\clang_win32\main.cpp:8:5  E:\clang_win32\main.cpp
COMPLETION: BBB : BBB::
COMPLETION: f1 : [#int#]f1()
COMPLETION: f2 : [#int#]f2()
COMPLETION: f3 : [#int#]f3()

Any comments about how to solve the redefinition issue???

thanks.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #17 on: October 27, 2010, 06:17:05 am »
Sorry guys, I have made a mistake in my previous post, now add the include guard solved all the problem of redefinition.

modified testcpp.h
Code

#ifndef TESTCPP_H
#define TESTCPP_H

class BBB
{
public:
int f1();
int f2();
int f3();
};

#endif




@eranif

it seems the clang++ and PCH works fine. You can try it to see if it works much faster.


If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Clang command line support for codecompletion
« Reply #18 on: October 27, 2010, 07:39:36 am »
@jens:
thanks for your time and effort to build the win32 version of clang for me, it works quite well.

Nice to hear, for me it's much easier (the most time) to do a cross-build for windows in my linux, than do the build on windows itself.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #19 on: October 27, 2010, 08:15:35 am »
I have just briefly read the Clang's source code about
lexer
parser
And I found that it is definitely all hand-written code (Note our CC's parser is also hand-written), but it is quite modular.

It's lexer's source code was located in
Code
llvm\tools\clang\include\clang\Lex
llvm\tools\clang\lib\Lex
parser's source is located in
Code
llvm\tools\clang\include\clang\Parse
llvm\tools\clang\lib\Parse

for example:
In the parser.cpp
Code
/// \brief Determine whether the current token, if it occurs after a
/// declarator, continues a declaration or declaration list.
bool Parser::isDeclarationAfterDeclarator() const {
  return Tok.is(tok::equal) ||      // int X()=  -> not a function def
    Tok.is(tok::comma) ||           // int X(),  -> not a function def
    Tok.is(tok::semi)  ||           // int X();  -> not a function def
    Tok.is(tok::kw_asm) ||          // int X() __asm__ -> not a function def
    Tok.is(tok::kw___attribute) ||  // int X() __attr__ -> not a function def
    (getLang().CPlusPlus &&
     Tok.is(tok::l_paren));         // int X(0) -> not a function def [C++]
}

there are even skip functions in lexer ( we have skip XXX functions in CC either) :D
Code
/// SkipEscapedNewLines - If P points to an escaped newline (or a series of
/// them), skip over them and return the first non-escaped-newline found,
/// otherwise return P.
const char *Lexer::SkipEscapedNewLines(const char *P) {
  while (1) {
    const char *AfterEscape;
    if (*P == '\\') {
      AfterEscape = P+1;
    } else if (*P == '?') {
      // If not a trigraph for escape, bail out.
      if (P[1] != '?' || P[2] != '/')
        return P;
      AfterEscape = P+3;
    } else {
      return P;
    }

    unsigned NewLineSize = Lexer::getEscapedNewLineSize(AfterEscape);
    if (NewLineSize == 0) return P;
    P = AfterEscape+NewLineSize;
  }
}

BTW: seems all the tokens were handled as char type.(one byte). Hope our CC can use char type instead. because wxChar seems waste a lot of space. (Unicode wxChar seems equal two bytes, not sure how long in X86-64 system).

Another similaity is that Clang do the pre-processors and lexer in the same stage, this is the conceptional same as in CC's Tokenizer.

« Last Edit: October 27, 2010, 08:18:23 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ironhead

  • Almost regular
  • **
  • Posts: 210
Re: Clang command line support for codecompletion
« Reply #20 on: October 27, 2010, 04:14:42 pm »
BTW: seems all the tokens were handled as char type.(one byte). Hope our CC can use char type instead. because wxChar seems waste a lot of space. (Unicode wxChar seems equal two bytes, not sure how long in X86-64 system).

This is limiting, isn't it?  It would limit CC to be UTF-8 compliant at best (i.e. not truly Unicode).

As far as I know, Unicode is a two-byte sequence irregardless of architecture.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Clang command line support for codecompletion
« Reply #21 on: October 27, 2010, 04:26:36 pm »
sizeof(wchar_t) is 4 on MacOSX.
Using non ASCII characters in the sourcecode/sourcefiles is highly discouraged anyway...
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline eranif

  • Regular
  • ***
  • Posts: 256
Re: Clang command line support for codecompletion
« Reply #22 on: October 27, 2010, 04:52:56 pm »
BTW: As promised, here are the build instruction for building clang under Windows / MinGW
Quote
Important notes:
----------------
MinGW\bin directory must be in the PATH environment variable

Install CMAKE from here:
------------------------
http://www.cmake.org/cmake/resources/software.html

Building clang using MinGW:
--------------------------

- Checkout llvm sources             : svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
- cd llvm/tools directory           : svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
- create a build directory under the root llvm directory (.e.g mkdir build_release)
- cd to the build directory (e.g. cd build_release) and run : cmake -G "MinGW Makefiles" ../ -DCMAKE_BUILD_TYPE=Release
- open the file CMakeCache.txt and reduce the optimizatin level from -O3 to -O2 (replace all -O3 to -O2)
- Open the file llvm\tools\clang\tools\c-index-test\CMakeLists.txt and comment out the following lines:

#add_clang_executable(c-index-test
#  c-index-test.c
#  )

#set_target_properties(c-index-test
#  PROPERTIES
#  LINKER_LANGUAGE CXX)

- Run                               : mingw32-make

To update sources from svn, you need to update both repositories, so:
--------------------------------------------------------------------
from the root directory, run: svn up
cd to tools/clang and run   : svn up


Eran
« Last Edit: October 27, 2010, 04:55:07 pm by eranif »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #23 on: October 28, 2010, 03:09:49 am »
thanks eranif for the information!!!

seems c-index-test has some problems ( I failed building this target the day before yesterday :D) see build failed method two..
It seems we don't have perl to build clang. (as the http://llvm.org/docs/GettingStarted.html#requirements said, under MSYS, perl is need), but the cmake+mingw method, perl is not need.


BTW:
As this post said:
http://lists.trolltech.com/pipermail/qt-creator/2009-January/001329.html
In QT creator, they use two kind of parser.
let's say, a simple parsing and a detailed parsing.

When QTcreator analysis the whole project files, it just do the simple parsing. this is the same way as we do in C::B's CC parser. or the Codelite's ctags parser. it just correct tags.

But QTcreator will do a detailed parsing on the current active translation unit. including the type checking, macro expansion...So, the error information can be shown, and a more precise code-completion list can be generated.

As this way: we can just use clang do the "detailed parsing" and left the other thing still to CC's parser or ctags. Any ideas?

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline eranif

  • Regular
  • ***
  • Posts: 256
Re: Clang command line support for codecompletion
« Reply #24 on: October 28, 2010, 06:39:10 am »
Actually, for codelite I took another path:
- All code completion goes through the current parser code
- In case the current parser fails to parse / yield results, the clang code completion "kicks in"

I think that this is the best option in terms of stability and backward compatibilty
I believe in making evolution, not a revolution :D


Eran
 

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #25 on: November 05, 2010, 08:00:38 am »
I have successfully build the clang under msys/mingw with some clang maillist guy's help.



The problem:

updated:

Today, I have tried the method listed in
http://clang.llvm.org/get_started.html

to build a clang.exe, but no luck, all failed  :( :( :(.

1, one failed method.
under Msys mingw, check out the llvm and clang svn code, and configure. make. but the error occurred when I run the make,  such as:
Quote
$ make
Makefile:130: /Makefile.rules: No such file or directory
make: *** No rule to make target `/Makefile.rules'.  Stop.


Was solved because I have made a big mistake.

There are two kind of make.exe. One is under mingw/bin, the other is under msys/bin
The error caused because msys use the make.exe under mingw/bin. this was totally wrong, we should use make.exe under msys/bin.
So, you should delete make.exe under mingw/bin or rename it.


The final report:
it takes one hour to compile the total target..( too long..)
it seem the total build file was quite large : 3.5G !!!

and the folder \build\Debug+Asserts is 2.5G.
especially, under the folder: \build\Debug+Asserts\bin

clang++.exe   430M
clang.exe       430M
llc.exe             170M
....

striping the clang++.exe will reduce its size to 21M.

Testing the code-completion feature works fine!!!
« Last Edit: November 05, 2010, 08:03:43 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline infinigon

  • Multiple posting newcomer
  • *
  • Posts: 10
Re: Clang command line support for codecompletion
« Reply #26 on: November 06, 2010, 01:32:20 pm »

it seem the total build file was quite large : 3.5G !!!

and the folder \build\Debug+Asserts is 2.5G.
especially, under the folder: \build\Debug+Asserts\bin

clang++.exe   430M
clang.exe       430M
llc.exe             170M
....

striping the clang++.exe will reduce its size to 21M.

According to http://llvm.org/docs/MakefileGuide.html#variables you can set ENABLE_OPTIMIZED=1 when make'ing in order to disable debugging symbols and enable optimization (should have no need to strip after this, and clang runs much faster). Also, AFAIK clang.exe and clang++.exe are exactly the same - you only need one copy of it.

Also I tried building clang, but did not realise until too late that if using MSVC to build, then clang would be configured to use the MSVC headers, which it cannot parse. Now I am also trying to build it with MinGW...
« Last Edit: November 06, 2010, 01:35:52 pm by infinigon »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #27 on: November 06, 2010, 01:44:16 pm »

it seem the total build file was quite large : 3.5G !!!

and the folder \build\Debug+Asserts is 2.5G.
especially, under the folder: \build\Debug+Asserts\bin

clang++.exe   430M
clang.exe       430M
llc.exe             170M
....

striping the clang++.exe will reduce its size to 21M.

According to http://llvm.org/docs/MakefileGuide.html#variables you can set ENABLE_OPTIMIZED=1 when make'ing in order to disable debugging symbols and enable optimization (should have no need to strip after this, and clang runs much faster). Also, AFAIK clang.exe and clang++.exe are exactly the same - you only need one copy of it.

Also I tried building clang, but did not realise until too late that if using MSVC to build, then clang would be configured to use the MSVC headers, which it cannot parse. Now I am also trying to build it with MinGW...

Hi, thanks for your help. I will try to build a ENABLE_OPTIMIZED=1 version if I have free time.
 currently, not sure how "easy/difficult" to add the clang to codecompletion plugin. or making another codecompletion plugin(difficult too).
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #28 on: December 09, 2010, 02:37:40 am »
Information for those who would like to use Clang for codecompletion and code indexer.
There are two ways:
1, using the clang from the command line. this is currently done in vim, Emacs, codelite by calling the clang.exe from command line.
2, directly link to the libraries supplied by libclang. this is done only in Apples Xcode, but it is suggest the better way. You can take the file:
Code
\llvm\tools\clang\tools\c-index-test\c-index-test.c
, this file contains all the information you need.

BTW: to build a release version of clang, the options can be found here:
How To Release LLVM To The Public
Code
Building the Release
The build of llvm, llvm-gcc, and clang must be free of errors and warnings in both debug, release+asserts, and release builds. If all builds are clean, then the release passes build qualification.

   1. debug: ENABLE_OPTIMIZED=0
   2. release+asserts: ENABLE_OPTIMIZED=1
   3. release: ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1


Edit:
The common feature like "find the definition of a function" or "find the reference" can be found there:
The Index Library
see below:
Quote
index-test tool
Usage
A command-line tool that exercises the libIndex API, useful for testing its features. As input it accepts multiple AST files (representing multiple translation units) and a few options:

   -point-at  [file:line:column]

Resolves a [file:line:column] triplet into a ASTLocation from the first AST file. If no other option is specified, it prints the ASTLocation. It also prints a declaration's associated doxygen comment, if one is available.

   -print-refs

Prints the ASTLocations that reference the declaration that was resolved out of the [file:line:column] triplet

   -print-defs

Prints the ASTLocations that define the resolved declaration

   -print-decls

Prints the ASTLocations that declare the resolved declaration

Examples

Here's an example of using index-test:

We have 3 files,

foo.h:

extern int global_var;

void foo_func(int param1);
void bar_func(void);

t1.c:

#include "foo.h"

void foo_func(int param1) {
  int local_var = global_var;
  for (int for_var = 100; for_var < 500; ++for_var) {
    local_var = param1 + for_var;
  }
  bar_func();
}

t2.c:

#include "foo.h"

int global_var = 10;

void bar_func(void) {
  global_var += 100;
  foo_func(global_var);
}

You first get AST files out of t1.c and t2.c:

$ clang -emit-ast t1.c -o t1.ast
$ clang -emit-ast t2.c -o t2.ast

Find the ASTLocation under this position of t1.c:

[...]
void foo_func(int param1) {
  int local_var = global_var;
                      ^
[...]

$ index-test t1.ast -point-at t1.c:4:23
> [Decl: Var local_var | Stmt: DeclRefExpr global_var] <t1.c:4:19, t1.c:4:19>

Find the declaration:

$ index-test t1.ast -point-at t1.c:4:23 -print-decls
> [Decl: Var global_var] <foo.h:1:12, foo.h:1:12>

Find the references:

$ index-test t1.ast t2.ast -point-at t1.c:4:23 -print-refs
> [Decl: Var local_var | Stmt: DeclRefExpr global_var] <t1.c:4:19, t1.c:4:19>
> [Decl: Function bar_func | Stmt: DeclRefExpr global_var] <t2.c:6:3, t2.c:6:3>
> [Decl: Function bar_func | Stmt: DeclRefExpr global_var] <t2.c:7:12, t2.c:7:12>

Find definitions:

$ index-test t1.ast t2.ast -point-at t1.c:4:23 -print-defs
> [Decl: Var global_var] <t2.c:3:5, t2.c:3:18>


All the features was done in the code:
Code
\llvm\tools\clang\tools\c-index-test\c-index-test.c

This is quite GOOD for an IDE!!!! :D :D :D
« Last Edit: December 09, 2010, 02:43:18 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5906
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang command line support for codecompletion
« Reply #29 on: January 16, 2011, 08:02:10 am »
It also lacks some crucial information (which I could not seem to get an access to, like an exact position of the match), I can probably obtain more information if I will link directly with the clang libraries - but for the POC (prove of concept) it seems too much for me.
Hi, eranif, I have just dig into the libclang, especially the c-index-test.c file to see if is fit your desire.

From the latest svn code of clang and llvm, I found that:

F:\llvm_build\llvm\tools\clang\include\clang-c\Index.h

Code
/**
 * \brief A single result of code completion.
 */
typedef struct {
  /**
   * \brief The kind of entity that this completion refers to.
   *
   * The cursor kind will be a macro, keyword, or a declaration (one of the
   * *Decl cursor kinds), describing the entity that the completion is
   * referring to.
   *
   * \todo In the future, we would like to provide a full cursor, to allow
   * the client to extract additional information from declaration.
   */
  enum CXCursorKind CursorKind;

  /**
   * \brief The code-completion string that describes how to insert this
   * code-completion result into the editing buffer.
   */
  CXCompletionString CompletionString;
} CXCompletionResult;

You can see, the code completion  result (one entry) only contains two members:
one is the CursorKind. and if it supply a "full Cursor" information, you can get the Location information about this code completion entry.
Maybe, the clang developers will improved this in the future. :D

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.