Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: Alpha on February 16, 2013, 01:12:30 am

Title: CC based semantic highlight
Post by: Alpha on February 16, 2013, 01:12:30 am
I have begun exploration of full semantic highlighting, provided by the knowledge from CC's parser.  The attached patch is able to show colors, however it is very temperamental.  Also, if you type anything into the editor, it will break.
The colors were chosen at random; suggestions for these would be appreciated.

This code currently uses some operations which are potentially unsafe.  You have been warned.

(http://www.pasteall.org/pic/show.php?id=45557) (http://www.pasteall.org/pic/show.php?id=45557)
Title: Re: CC based semantic highlight
Post by: ollydbg on February 16, 2013, 02:18:16 am
Great work. Look at the screen shot, it is so beautiful.
I'm going to test it right now.
Title: Re: CC based semantic highlight
Post by: ollydbg on February 16, 2013, 04:52:45 pm
Testing result: sometimes, C::B hangs. :)
Title: Re: CC based semantic highlight
Post by: mistar on February 20, 2013, 10:12:48 am
Have a look also on my plugin ;) shouldn't hang ;)
http://forums.codeblocks.org/index.php/topic,16249.msg120205.html#msg120205
Title: Re: CC based semantic highlight
Post by: ollydbg on February 20, 2013, 03:07:48 pm
Quote from: mistar on February 20, 2013, 10:12:48 am
Have a look also on my plugin ;) shouldn't hang ;)
http://forums.codeblocks.org/index.php/topic,16249.msg120205.html#msg120205
Both you and Alpha did good job, thanks, will take time to test those.

@Alpha, about the hang, I think the locker(there are many lockers in the CC code), but from my point of view, we should remove mostly of them.

TokenTree are the shared resource, sometimes, the worker thread (Parserthread are adding tokens to it), in the other time, the main GUI thread(like the one you implement the semantic highlight should query the TokenTree). I think they should not happen concurrently. I mean if the parser is working, the TokenTree should not accessed by main thread. Either main GUI thread or worker thread can access the TokenTree.

Title: Re: CC based semantic highlight
Post by: p2rkw on March 30, 2013, 01:31:34 am
I see it's in trunk now: http://sourceforge.net/p/codeblocks/code/8920/
but as I can see it's impossible to disable this feature.
My proposition is to detect whenever third keyword set have exactly the same colour as default keyword set: if those colours are equal then all token processing inside CodeCompletion::UpdateEditorSyntax is unnecessary.
Title: Re: CC based semantic highlight
Post by: oBFusCATed on March 30, 2013, 11:48:51 am
Alpha:
Do you plan to fix this issue:
Code
class Test
{
    void func()
    {
        int a = var;
    }
    int var;
};

void func()
{
    int var;
}
Hint: the issue is that the var is highlighted everywhere, but it shouldn't inside the func.

If not please revert your patch!
Title: Re: CC based semantic highlight
Post by: Alpha on March 30, 2013, 07:13:50 pm
The one that is in trunk is actually this (http://forums.codeblocks.org/index.php/topic,17392.0.html).  But I will add an enable/disable checkbox now.

Quote from: oBFusCATed on March 30, 2013, 11:48:51 am
Do you plan to fix this issue: [...]
Impossible with the current method of implementation (see other thread).  Just disable it, if it bothers you, when I get the checkbox uploaded.
Title: Re: CC based semantic highlight
Post by: oBFusCATed on March 30, 2013, 07:49:32 pm
Quote from: Alpha on March 30, 2013, 07:13:50 pm
Impossible with the current method of implementation (see other thread).  Just disable it, if it bothers you, when I get the checkbox uploaded.
I'll just revert it, don't worry. There is no point in having a fake feature in trunk. If you want it in do it properly, please.

@other devs: please state your opinion here, I won't revert if for a couple of days.
Title: Re: CC based semantic highlight
Post by: ollydbg on March 31, 2013, 08:16:53 am
C++ is too complex and its grammar is context dependant, the way Alpha use was set the identifier to a group, then let the scintilla to colorize the group in the same color. I think we can follow what the way clang sematic highlight plugin use, I mean write our own lexer for scintilla, colorize each piece of text by the token type. The token type info can be queryed from the tokentree. But this may still cause error, because our parser does not do sematic check.

EDIT: It looks that it is impossible to query every text info from the tokentree, our tokentree only hold the variable definition info, but for other variable usage(reference), it need the AI() function, this will make the custom lexer much slower.
Title: Re: CC based semantic highlight
Post by: MortenMacFly on March 31, 2013, 05:23:50 pm
Quote from: oBFusCATed on March 30, 2013, 07:49:32 pm
@other devs: please state your opinion here, I won't revert if for a couple of days.
Well I like and in fact use it although I realise its not working 100%. Maybe this should be turned off by default and an explanation in the settings points out that this is not meant to work perfectly.
Title: Re: CC based semantic highlight
Post by: oBFusCATed on March 31, 2013, 05:49:01 pm
Quote from: MortenMacFly on March 31, 2013, 05:23:50 pm
Well I like and in fact use it although I realise its not working 100%. Maybe this should be turned off by default and an explanation in the settings points out that this is not meant to work perfectly.
For sure it should be turned off by default if it stays.
But the problem is that this is a half feature - broken and hard to extend.

The thing is that the semantic part of the name of the feature implies that it uses the CC and so it should work 100% of the time (ignoring CC parser bugs).
But this is not the case. And we have a prototype (I've not looked at it, so I cannot tell how good it is) that does better job.

Also this feature is almost the same as the broken highlight for stl stuff, but the brokenness is a bit more hidden.

I know that implementing this properly is hard, because of the need to modify scintilla, but I'm sure it is doable.
Also the semantic highlight should not be part of the CC plugin! It should be part of the SDK or it should be separated in a plugin.
Title: Re: CC based semantic highlight
Post by: Alpha on April 04, 2013, 12:58:07 am
It is off by default now.  If you would like to revert it completely, go ahead, however from previous discussions of this patch, it sounded as though there are at least several people who do find it useful in its current (flawed) state.

Quote from: oBFusCATed on March 31, 2013, 05:49:01 pm
Also the semantic highlight should not be part of the CC plugin! It should be part of the SDK or it should be separated in a plugin.
Agreed, but do we yet have a way for a seperate plugin to access the data in CC's parser?
Title: Re: CC based semantic highlight
Post by: oBFusCATed on April 04, 2013, 08:19:06 am
Quote from: Alpha on April 04, 2013, 12:58:07 am
It is off by default now.  If you would like to revert it completely, go ahead, however from previous discussions of this patch, it sounded as though there are at least several
people who do find it useful in its current (flawed) state.
OK, but what do you do when they ask you to highlight the member functions differently and then the local variables?

Quote from: Alpha on April 04, 2013, 12:58:07 am
Agreed, but do we yet have a way for a seperate plugin to access the data in CC's parser?
Nope, but we should start from somewhere otherwise we will never have it.

BTW: The current implementation is quite laggy and thus quite annoying. Something happens but I don't know when.

p.s. Anyone else from the devs which wants to express opinion. Currently we are 1:1.
Title: Re: CC based semantic highlight
Post by: MortenMacFly on April 04, 2013, 07:40:23 pm
Quote from: oBFusCATed on April 04, 2013, 08:19:06 am
p.s. Anyone else from the devs which wants to express opinion. Currently we are 1:1.
Express opinion on what exactly?

For semantic highlight and access to CC internals my preferences are to use clang, maybe through Eran's engine. I think we can never be better than that - its amazing how fast the code base and functional support grows. So having a nice abstraction layer for clang will serve way more than providing interfaces to access (our) CC internals IMHO...
Title: Re: CC based semantic highlight
Post by: oBFusCATed on April 04, 2013, 08:08:08 pm
Morten: If we should keep the feature or if we should drop it by reverting the patch.

No matter what method we use we have to provide some CC API. Currently we have no such thing.