'Visual Assist' style syntax highlighting

[Disch]

I used to use VS.NET with the "Visual Assist" plugin by Wholetomato, and I've sort of gotten spoiled by it.  After moving to a Linux box and using Code::Blocks, its syntax highlighting is nice, but not quite as nice.  I was thinking about making a plugin for C::B to provide similar functionality for C++, however I thought I'd ask a few things here before jumping into it.

Specifically, what I'm looking for is more of a dynamic keyword identifier.  Rather than having a list of keywords and highlighting them, additional keywords can be identified by parsing some of the text.  For example:

Code

int var;  // would identify 'var' as a keyword

class Cls
{
};     // identify 'Cls' as a keyword

typedef unsigned long u32;  // identify 'u32'

typedef void (*funcptr)(int); // identify 'funcptr'

#define AMACRO  // etc

The thing that makes this particularly tricky is that the keywords should only be identified and highlighted when they're in scope.  This requires reparsing when braces are changed, or when additional files are #included.

Something like this might seem extravagant, but after using it for a little while it becomes very easy to get used to and lets you spot simple typing errors (like misspelling a var name) immediately rather than waiting for compile time.  Plus, highlighting function names, variable names, and literal constants all differently works wonders for code clarity.

Is there anything like this already (or in the works) for C::B?  If not I'd like to think that I might be up to the task of trying it -- although I don't know if I'd be willing to put in the work (depending on how hard it would be -- I'm not at all familiar with how C::B works internally or how you'd go about making a plugin for it).

Here's a page with screens showing what I'm talking about:
http://wholetomato.com/tour.asp?syntax

Ideas/thoughts/suggestions welcome and appreciated!

Thanks in advance.

[TDragon]

Just check out C::B's Code Completion Redesign forum. This kind of thing is *extremely* hard to get right.

[ollydbg]

The current Code Completion parses all the files (include header files) involved in the project. And all the tokens were stored in a global Patric tree. Then the context code statement will be parsed to give a correct tooltip by querying the Patric tree.

I wrote a wiki page after reading some source code of code completion several days ago.

http://wiki.codeblocks.org/index.php?title=Code_Completion_Design

I do think the highlight of keyword dynamically can be implemented. But you should communicate with the wxscintilla editor control.

[Disch]

Most of that wiki article is stuff I was already aware of (I've written some simplistic assemblers in the past, though nothing for something as comlicated as C++, although I'm confident I have the skills and experience to undertake such a task -- my biggest problem is motivation and/or butting heads with someone else who's already working on it).  Although parts of it were very interesting and are things I hadn't thought of.  A good read -- thanks.

I skimmed the requirements in this thread as well as the structure/organization guidelines outlined in this wiki article.  I think this design strategy is sound.  Here's my take on it:

Many of these enhancements (code completion, intellisense, smarter syntax highlighting, etc) all revolve around the same ideas.  They all need a certain level of awareness of the symbols in a program.  Rather than all these different plugins doing this individually, it makes much more sense for it to be done once by a seperate system which all the plugins feed off of.  So everything starts at the code parser/token identifier/whatever you want to call it (I suck at technical terms).  I don't know if this is how it's currently handled by C::B and its plugins, but that certainly should be how it's handled for any future redesigns, imo (makes no sense for each plugin to parse the file on its own -- waste of CPU, leads to potential confusion/conflicts by files being parsed differently, etc).

To make these enhancements work, all you really need is a list of known symbols (and details about them) that are in scope on any given line in a file.  To accomplish this, the parser can generate a list of tokens.  Something like what ollydbg's wiki article describes here is a pretty good overview, although I'd imagine it might be more complex.

Ideally, whatever plugin would be able to get token info by supplying a name (or partial name in the case of code completion) and a line number (for scope purposes).  An intellisense plugin could use this to print argument lists, relevent documentation comments and whatnot.  Syntax highlighting could use this to determine how to highlight whatever token.  Etc, etc.  Alternatively, a plugin could be able to get all symbols available at file scope (this could be used for something like C::B's symbol table).

Parsing could be done on a specific file, and identified tokens and symbol info saved to disk so that the same file need not be parsed multiple times.  This could probably saved in the form of XML files since that seems to be standard operating procedure for this kind of thing.  On #include, the parser can load symbols from the included file.  When symbols are saved, symbols defined in the file could be saved along with that file's dependencies (includes) so that when the file is loaded, all necessary symbols can be loaded recursively.

Runtime changes are probably the hardest thing to accomidate.  I'm still kicking around ideas about this.  Small changes to individual lines would probably only require reparsing the changed lines (at best) or from the changed lines up until the next block closure / closed brace (at worst).  I'm thinking lines could be marked as "dirty" and the parser could step through each changed line.  Larger changes (such as a multi-line cut/paste) might be able to be handled the same way.  I'll have to think about it more.  In any event, in the case of a runtime parsing mishap, an option to reparse the current file should exist anyway (VAX had something like this as well)

Also, of course for runtime changes, the file can't be read off the disk.  I have no idea how C::B works with source files at runtime -- are they stored as just a big wxString?  Or a wxTextFile?  Whatever the case, the parser would need access to runtime info.

---------------------------------------

In any event, what it boils down to is pretty simple:  text in->symbols out.  The parser should be seperated from C::B and any of its plugins.  This was going to be my starting point -- parsing a few simple cpp files and dumping a symbol list.  Wrap a plugin friendly API around that and you've got most of the job done for these enhancements.  Granted it would be quite a big job, but I think I'd have fun with it.

The problem is -- is something like this already underway?  From the looks of things it seems everyone is focusing on code completion (ironically the one enhancement I couldn't care less about).  I don't want to go through all the work of starting something like this if it's already exists, or is being undertaken by someone else and is nearing completion.  And coding the actual plugins is something I'm not all that interested in (not at all familiar with wxScintilla or C::B code)

[ollydbg]

In any event, what it boils down to is pretty simple:  text in->symbols out.  The parser should be seperated from C::B and any of its plugins.  This was going to be my starting point -- parsing a few simple cpp files and dumping a symbol list.  Wrap a plugin friendly API around that and you've got most of the job done for these enhancements.  Granted it would be quite a big job, but I think I'd have fun with it.

The problem is -- is something like this already underway?  From the looks of things it seems everyone is focusing on code completion (ironically the one enhancement I couldn't care less about).  I don't want to go through all the work of starting something like this if it's already exists, or is being undertaken by someone else and is nearing completion.  And coding the actual plugins is something I'm not all that interested in (not at all familiar with wxScintilla or C::B code)

If I understand your idea, there is a topic talking that:

http://forums.codeblocks.org/index.php/topic,10087.msg70875.html#msg70875

CodeLite(another IDE) use Ctage as a index service, and it can save the symbols list.