New parser model for Code completion

[Game_Ender]

Alright so why don't we just take a stab at making the plugin?

The Code Completion plugin  is already made, the issue is that it does not support every part of the C++ langauge because it is so very hard to fully parse C++.  I  have been looking through the CodeCompletion Plugin code for a little while and I am still working out the best way to drop in another system in place of Mandrav's tokenizer/parser combination.

I forgot to mention that Elsa supports XML serialization of its AST, so with a little work you can turn this into a quickly loadable file for code completion.

[anonuser]

Why work with the old one, why not tear it out ?

[takeshimiya]

As I've said, Elsa and ANTLR C++ are the most complete C++ parsers I could find.
The ANTLR C++ is being made from 1997 and was completely rewritten 3 times, so now is very mature.

I attach two text files describing the current situation of both parsers and what things are missing.

Additionaly, I couldn't compile Elsa on mingw32 (it's not ported yet), but ANTRL compiles succesfully.
What I liked about ANTLR C++ is that the code of the parser is very small, it takes seconds to compile, despite it being very powerful.

[attachment deleted by admin]

[rickg22]

Why work with the old one, why not tear it out ?

Actually we're searching for a brave warrior who will start the quest of implementing the new parser. But so far everybody seems busy... I want to redo Yiannis' parser with Finite State machines, but I don't know if I'll be able to do it.

If you seem so enthusiastic, you're totally welcome to dive in and start coding with ANTLR or whatever you like

[anonuser]

I'm going to see what this would take, I doubt I'll use any of what mandrav did. Other's should try too. See who gets it done better and faster
But Yeah I'll try it.
Can anyone show me what I need to tear out?

[anonuser]

First I need Yiannis to put in those FreeBSD changes I submitted    :shock:
So I can go ahead and resubmit my port.

[grv575]

The problem with elsa (which is an alkhound-based parser) is if you look at the source, it's absolutely huge.  It looks like getting elkhound to work with a new desired parser (say to parse java or python or whatever) would be a lot of work.  Plus it uses external tools like ast to produce the abstract syntax tree.  This looks all very complex and not modular and pluggable for anything other than c++.

The antlr compiler compiler looks like it might be a better bet.  The tutorials show that's it's easy to specify both the lexer, the parser, and an AST walker all with antlr.  This would make adding different parsers much more painless as it does it all (including generating the AST instead of using a seperate tool for that).

There's already a c++, java, c#, etc specifications for antlr as well.  So, it looks overall like a better fit.

[anonuser]

Alright then I'll start with looking over antler making a few test projects seeing how it holds up.

[takeshimiya]

Improving the current one, or creating a new parser, it doesn't matter, as in the end the best one would be used.

But for ANTLR C++, which is a complete C++ parser and generates an AST, the entire code weights 150KB and are only 14 files of pure c++ standard code.
Unfortunately, I can't say the same for other parsers like Elsa.

[anonuser]

You're forgetting this all needs to integrate with the editor.

[takeshimiya]

No, that's why I said to cooperate with the CodeStore author, which is doing exactly that: integrating ANTLR with a CodeCompletion plugin.

[anonuser]

Alright I'll look into it.

[grv575]

cool, there is a standalone exe: antlr exe (http://www.antlr.org/download.html)
"antlr-2.7.5.exe (Win32 executable made with mingw)"

which works without java installed and is the latest version.  Not sure if this is better or not vs. using the c++ antlr port (which is like 2.7.4 apparently).

[anonuser]

I believe the idea is to have antlr generate the language parser for you and you just use that code.
You provide language grammer it spits out a parser.

[grv575]

yeah but we can just provide a link to antlr.exe and say use this to generate a parser for language X if you want X to be used as a code completion plugin.  makes modular expansion of the parsing framework that much easier.