Author Topic: New parser model for Code completion  (Read 61415 times)

Offline Game_Ender

  • Lives here!
  • ****
  • Posts: 551
Re: New parser model for Code completion
« Reply #45 on: December 21, 2005, 04:44:17 am »
Alright so why don't we just take a stab at making the plugin?

The Code Completion plugin  is already made, the issue is that it does not support every part of the C++ langauge because it is so very hard to fully parse C++.  I  have been looking through the CodeCompletion Plugin code for a little while and I am still working out the best way to drop in another system in place of Mandrav's tokenizer/parser combination.

I forgot to mention that Elsa supports XML serialization of its AST, so with a little work you can turn this into a quickly loadable file for code completion.

Offline anonuser

  • Multiple posting newcomer
  • *
  • Posts: 44
Re: New parser model for Code completion
« Reply #46 on: December 21, 2005, 04:45:24 am »
Why work with the old one, why not tear it out ?

Offline takeshi miya

  • Lives here!
  • ****
  • Posts: 1487
Re: New parser model for Code completion
« Reply #47 on: December 21, 2005, 04:58:31 am »
As I've said, Elsa and ANTLR C++ are the most complete C++ parsers I could find.
The ANTLR C++ is being made from 1997 and was completely rewritten 3 times, so now is very mature.

I attach two text files describing the current situation of both parsers and what things are missing.

Additionaly, I couldn't compile Elsa on mingw32 (it's not ported yet), but ANTRL compiles succesfully.
What I liked about ANTLR C++ is that the code of the parser is very small, it takes seconds to compile, despite it being very powerful.

[attachment deleted by admin]

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: New parser model for Code completion
« Reply #48 on: December 21, 2005, 05:00:06 am »
Why work with the old one, why not tear it out ?
Actually we're searching for a brave warrior who will start the quest of implementing the new parser. But so far everybody seems busy... I want to redo Yiannis' parser with Finite State machines, but I don't know if I'll be able to do it.

If you seem so enthusiastic, you're totally welcome to dive in and start coding with ANTLR or whatever you like :)

Offline anonuser

  • Multiple posting newcomer
  • *
  • Posts: 44
Re: New parser model for Code completion
« Reply #49 on: December 21, 2005, 05:01:23 am »
I'm going to see what this would take, I doubt I'll use any of what mandrav did. Other's should try too. See who gets it done better and faster ;D
But Yeah I'll try it.
Can anyone show me what I need to tear out?

Offline anonuser

  • Multiple posting newcomer
  • *
  • Posts: 44
Re: New parser model for Code completion
« Reply #50 on: December 21, 2005, 05:03:18 am »
First I need Yiannis to put in those FreeBSD changes I submitted    :shock:
So I can go ahead and resubmit my port.

Offline grv575

  • Official tester
  • Regular
  • ***
  • Posts: 381
Re: New parser model for Code completion
« Reply #51 on: December 21, 2005, 05:09:14 am »
The problem with elsa (which is an alkhound-based parser) is if you look at the source, it's absolutely huge.  It looks like getting elkhound to work with a new desired parser (say to parse java or python or whatever) would be a lot of work.  Plus it uses external tools like ast to produce the abstract syntax tree.  This looks all very complex and not modular and pluggable for anything other than c++.

The antlr compiler compiler looks like it might be a better bet.  The tutorials show that's it's easy to specify both the lexer, the parser, and an AST walker all with antlr.  This would make adding different parsers much more painless as it does it all (including generating the AST instead of using a seperate tool for that).

There's already a c++, java, c#, etc specifications for antlr as well.  So, it looks overall like a better fit.

Offline anonuser

  • Multiple posting newcomer
  • *
  • Posts: 44
Re: New parser model for Code completion
« Reply #52 on: December 21, 2005, 05:14:41 am »
Alright then I'll start with looking over antler making a few test projects seeing how it holds up.

Offline takeshi miya

  • Lives here!
  • ****
  • Posts: 1487
Re: New parser model for Code completion
« Reply #53 on: December 21, 2005, 05:15:30 am »
Improving the current one, or creating a new parser, it doesn't matter, as in the end the best one would be used. :)

But for ANTLR C++, which is a complete C++ parser and generates an AST, the entire code weights 150KB and are only 14 files of pure c++ standard code.
Unfortunately, I can't say the same for other parsers like Elsa. :)

Offline anonuser

  • Multiple posting newcomer
  • *
  • Posts: 44
Re: New parser model for Code completion
« Reply #54 on: December 21, 2005, 05:19:21 am »
You're forgetting this all needs to integrate with the editor.

Offline takeshi miya

  • Lives here!
  • ****
  • Posts: 1487
Re: New parser model for Code completion
« Reply #55 on: December 21, 2005, 05:30:09 am »
No, that's why I said to cooperate with the CodeStore author, which is doing exactly that: integrating ANTLR with a CodeCompletion plugin.

Offline anonuser

  • Multiple posting newcomer
  • *
  • Posts: 44
Re: New parser model for Code completion
« Reply #56 on: December 21, 2005, 05:48:11 am »
Alright I'll look into it.

Offline grv575

  • Official tester
  • Regular
  • ***
  • Posts: 381
Re: New parser model for Code completion
« Reply #57 on: December 21, 2005, 06:00:39 am »
cool, there is a standalone exe: antlr exe (http://www.antlr.org/download.html)
"antlr-2.7.5.exe (Win32 executable made with mingw)"

which works without java installed and is the latest version.  Not sure if this is better or not vs. using the c++ antlr port (which is like 2.7.4 apparently).

Offline anonuser

  • Multiple posting newcomer
  • *
  • Posts: 44
Re: New parser model for Code completion
« Reply #58 on: December 21, 2005, 06:04:33 am »
I believe the idea is to have antlr generate the language parser for you and you just use that code.
You provide language grammer it spits out a parser.

Offline grv575

  • Official tester
  • Regular
  • ***
  • Posts: 381
Re: New parser model for Code completion
« Reply #59 on: December 21, 2005, 06:25:21 am »
yeah but we can just provide a link to antlr.exe and say use this to generate a parser for language X if you want X to be used as a code completion plugin.  makes modular expansion of the parsing framework that much easier.