Code::Blocks

Developer forums (C::B DEVELOPMENT STRICTLY!) => Plugins development => Topic started by: mistar on April 25, 2012, 07:34:09 pm

Title: Semantic highlight
Post by: mistar on April 25, 2012, 07:34:09 pm
Hi guys.

I'm working on C::B plugin for semantic highlight based on clang.
For a long time I've used Eclipse CDT but its support for the new c++11 standard (which I'm using extensively) is quite poor...

Currently I'm designing a framework for SemanticHighlight plugin and have following problems:

1. The member function CodeBlocksEvent::GetEditor() returns a pointer to EditorBase, not to cbEditor; hence the static_cast in the code below;
2. when I handle cbEVT_EDITOR_OPEN event:
Code: [Select]
void SemanticHighlight::OnEditorOpen(CodeBlocksEvent & evt)
{
    cbEditor * ed = static_cast<cbEditor *>(evt.GetEditor());
    ProjectFile * pf = ed->GetProjectFile();
    wxString projTitle = pf ? pf->GetParentProject()->GetTitle() : wxT("[None]");
    Manager::Get()->GetLogManager()->Log(wxT("\tSH: processing cbEVT_EDITOR_OPEN: ")+evt.GetEditor()->GetTitle()+wxT("@")+projTitle);
}
the variable 'ed' is always set to null pointer regardless of whether the opened file has parent project or not.

Isn't it a bug?

EDIT: I'm using the latest nightly build on Gentoo Linux
Title: Re: Semantic highlight
Post by: oBFusCATed on April 25, 2012, 08:45:03 pm
Sorry to disappoint you but currently you won't be able to implement such plugin.
And the reason is that the editor control we are using doesn't support it.
We are using Scintilla (wxScintilla to be exact).
If you look at their mailing list you'll find info of other people trying to do it and you'll be able to read about their failures.

Here is the link to the discussion: http://groups.google.com/group/scintilla-interest/browse_thread/thread/35a1ba3c99488d8f/b7365d210b6aca39?hl=en&lnk=gst&q=semantic+highlight#

For the cast, you can test the EditorBase if it is a built in editor, then the cast is safe.
Title: Re: Semantic highlight
Post by: mistar on April 25, 2012, 10:33:21 pm
Sorry to disappoint you but currently you won't be able to implement such plugin.
And the reason is that the editor control we are using doesn't support it.
We are using Scintilla (wxScintilla to be exact).
If you look at their mailing list you'll find info of other people trying to do it and you'll be able to read about their failures.

Here is the link to the discussion: http://groups.google.com/group/scintilla-interest/browse_thread/thread/35a1ba3c99488d8f/b7365d210b6aca39?hl=en&lnk=gst&q=semantic+highlight#

For the cast, you can test the EditorBase if it is a built in editor, then the cast is safe.

I know you are using Scintilla. I believe it has ability to manually set styles (without use of any lexer),
so the only thing is to correctly assign semantic information to tokens.
It is by no means the cleanest way to do it but the effect can be even better than that of SH in Eclipse CDT.
Nota bene, semantic highlighting _cannot_ use lexers as a token might have different semantics in different contexts (e.g., namespaces).

What about cbEVT_EDITOR_OPEN? In cbEVT_EDITOR_CLOSE everything is ok...
Besides, why is cbEVT_EDITOR_OPEN posted before cbEVT_PROJECT_OPEN when loading recently-opened files? This is really weird...

Thanks for pointing out EditorBase::IsBuiltinEditor :)
Title: Re: Semantic highlight
Post by: mistar on April 27, 2012, 10:19:28 am
OK, I've found the reason...

Why is cbEVT_EDITOR_OPEN event posted in EditorManager::New instead of EditorManager::Open (after cbEditor::SetProjectFile is called)?
BTW, can someone explain why are cbEVT_EDITOR_* events to be posted for non-builtin editors?
Thanks in advance!
Title: Re: Semantic highlight
Post by: ollydbg on April 27, 2012, 11:48:49 am
Your work is very promising. Great! I'm also interest on this. I see there are some document of scintilla about writing custom lexers which can set colors of the text by clients. Maybe I'm wrong.  :)

Here are some resources/links you may interest:
1, Clang code completion: ClangComplete plugin (http://forums.codeblocks.org/index.php/topic,15521.0.html)
2, Display Events: DisplayEvents plugin - CodeBlocks (http://wiki.codeblocks.org/index.php?title=DisplayEvents_plugin)
Title: Re: Semantic highlight
Post by: mistar on April 27, 2012, 01:59:19 pm
For semantic highlight one cannot use lexers, but there is an option to style a given text manually in Scintilla.
I don't suspect it to be efficient but it should be anyway more efficient than semantic highlight in Eclipse CDT.

BTW, C::B is great IDE but its sources are messy... It lacks a good framework (design).
Some examples I've mentioned above and another one I encountered when I wanted to get a project's active build target:
GetActiveBuildTarget returns a wxString and there is no option to get an index or pointer...
IHMO, the right way is to get a pointer to a build target object and then to obtain a wxString just call one of the BT methods...

So... Did you, C::B guys, consider redesigning C::B SDK API and then redesigning plugins? ;)

More complaints to come ;P
Title: Re: Semantic highlight
Post by: oBFusCATed on April 27, 2012, 02:20:40 pm
If you want to invest the time and do it and you show us that your changes improve the API,
why not, but don't expect anyone to take the job.

p.s. You can get a target by name, I think, if think there is a better way, post a patch :)
Title: Re: Semantic highlight
Post by: mistar on May 11, 2012, 06:16:38 pm
Ok, guys.
If you are interested, in two or three weeks I'll be done with SemanticHighlight plugin provided the problem with cbEVT_EDITOR_OPEN will be solved.
To finish SH plugin, I _really_ need to get the project a file belongs to because I need to get all the project's include directories and defines to get SH plugin work properly.
Since I am new to C::B sdk, I'd like someone to take a look at this proposal of change in the file editormanager.cpp:

remove these lines from EditorManager::New():
Code: [Select]
CodeBlocksEvent evt(cbEVT_EDITOR_OPEN, -1, 0, ed);
Manager::Get()->GetPluginManager()->NotifyPlugins(evt);
and insert these lines to EditorManager::Open(LoaderBase*, const wxString &, int, ProjectFile *):
Code: [Select]
if (!eb) // was not open
{
  CodeBlocksEvent evt(cbEVT_EDITOR_OPEN, -1, 0, ed);
  Manager::Get()->GetPluginManager()->NotifyPlugins(evt);
}
just before "s_CanShutdown = true". This way, cbEVT_EDITOR_OPEN is posted after all editor's data is initialized.
Title: Re: Semantic highlight
Post by: MortenMacFly on May 11, 2012, 09:33:31 pm
Why is cbEVT_EDITOR_OPEN event posted in EditorManager::New instead of EditorManager::Open (after cbEditor::SetProjectFile is called)?
Because it is defined as "about to be opened" if you like and plugins can intercept and interrupt this event in case they handle the file itself. So this won't change.

If you need another event after the file has been opened and added to the project, you would need another (new) SDK event, like cbEVT_EDITOR_OPENED. Implement this and provide a patch and it may make it into C::B's core.
Title: Re: Semantic highlight
Post by: mistar on May 12, 2012, 03:23:11 pm
Thanks for your reply.

Since I've never participate in a big project like C::B, please guide me what to do to post a patch (I've searched the forum but dodn't find anything),
or point to some description.

My proposal:
1. insert the code below to EditorManager::Open(LoaderBase*, const wxString &, int, ProjectFile *) in editormanager.cpp:
Code: [Select]
if (!eb) // was not open
{
  CodeBlocksEvent evt(cbEVT_EDITOR_OPENED, -1, 0, ed);
  Manager::Get()->GetPluginManager()->NotifyPlugins(evt);
}
just before
Code: [Select]
s_CanShutdown = true;
2. Declare new sdk editor event cbEVT_EDITOR_OPENED (or maybe _LOADED?) in sdk_events.h:
Code: [Select]
extern EVTIMPORT const wxEventType cbEVT_EDITOR_OPENED;
#define EVT_EDITOR_OPENED(fn) DECLARE_EVENT_TABLE_ENTRY( cbEVT_EDITOR_OPENED, -1, -1, (wxObjectEventFunction)(wxEventFunction)(CodeBlocksEventFunction)&fn, (wxObject *) NULL ),
3. Define cbEVT_EDITOR_OPENED in sdk_events.cpp
Code: [Select]
const wxEventType cbEVT_EDITOR_OPENED = wxNewEventType();
4. Add
Code: [Select]
NotifyPlugins(cbEVT_EDITOR_OPENED);to cbEditor::Open() in cbeditor.cpp.
Title: Re: Semantic highlight
Post by: Alpha on May 12, 2012, 03:39:02 pm
Since I've never participate in a big project like C::B, please guide me what to do to post a patch (I've searched the forum but dodn't find anything),
or point to some description.
Creating a patch to submit to BerliOS (Patch Tracker) (http://wiki.codeblocks.org/index.php?title=Creating_a_patch_to_submit_to_BerliOS_%28Patch_Tracker%29)
Title: Re: Semantic highlight
Post by: mistar on May 12, 2012, 11:24:57 pm
Since I've never participate in a big project like C::B, please guide me what to do to post a patch (I've searched the forum but dodn't find anything),
or point to some description.
Creating a patch to submit to BerliOS (Patch Tracker) (http://wiki.codeblocks.org/index.php?title=Creating_a_patch_to_submit_to_BerliOS_%28Patch_Tracker%29)

Thanks!
Patch submitted. How long does it take on average to incorporate the changes?
Title: Re: Semantic highlight
Post by: perento on May 15, 2012, 07:36:37 am
Hello mistar, may i ask what functions the plugin will be offered?
Title: Re: Semantic highlight
Post by: MortenMacFly on May 15, 2012, 08:41:54 am
Patch submitted. How long does it take on average to incorporate the changes?
Well, I had a look at this.

I think there is a better way: Instead of introducing another event, you could move the portion:
Code: [Select]
   if (can_updateui)
    {
        if (ed)
        {
            SetActiveEditor(ed);
            ed->GetControl()->SetFocus();
        }
    }
...to after:
Code: [Select]
   // check for ProjectFile
    if (ed && !ed->GetProjectFile())
    {
        // First checks if we're already being passed a ProjectFile as a parameter
        if (data)
            Manager::Get()->GetLogManager()->DebugLog(_T("Project data set for ") + data->file.GetFullPath());
        else
            Manager::Get()->GetProjectManager()->FindProjectForFile(ed->GetFilename(), &data, false, false);
        if (data)
            ed->SetProjectFile(data,true);
    }
This way, you can use the already existing cbEVT_EDITOR_ACTIVATED event and don't screw anything.

I'll check this in my local copy and look for side-effects (but I cannot think of any for the moment...).
Title: Re: Semantic highlight
Post by: MortenMacFly on May 15, 2012, 08:55:48 am
Why is cbEVT_EDITOR_OPEN event posted in EditorManager::New instead of EditorManager::Open (after cbEditor::SetProjectFile is called)?
Because it is defined as "about to be opened" if you like and plugins can intercept and interrupt this event in case they handle the file itself. So this won't change.
BTW: This was wrong. This event fires when a file has either been created (so it is new from scratch) or newly opened, like File -> Open -> "...". The latter is the reason why it is called OPEN. Internally it means C::B creates a new editor and also loads the file into memory (as needed).
Title: Re: Semantic highlight
Post by: mistar on May 15, 2012, 11:03:03 am
Hello mistar, may i ask what functions the plugin will be offered?

Basically, it should highlight the c++ syntax in a semantic way.
For example, in highlighting based on lexers, all identifiers have the same color.
In semantic highlight on the other hand, variables, types, functions, etc. are highlighted in different ways.

If you work for some time with semantic highlight and then switch back to the usual, lexer-based highlight,
you notice the difference as if you switch from lexer-based highlight to no highlight at all ;).

Some tools provide semantic highlight, eg. Eclipse CDT or KDevelop, but they lack a support for the new C++11 standard.
Also, both have internal parsers that are not developed to support the new standard.
Since libclang (see http://clang.llvm.org (http://clang.llvm.org); nb. version 3.1 is about to be released) has the best support for the new standard
and it provides parser API, it is relatively easy to write a plugin for C::B.

So far I have written a very simple tool that highlights a source in a semantic way (so it can be done using manual styling in Scintilla).
Now I'm working on retrieving compiler options (include directories, defines, etc.) so that all declarations are accessible to the libclang parser
(otherwise it highlights fragments of code as invalid).
Since some options are given implicitly (eg. via `wx-config --cxxflags`) I'm using wxExecute to retrieve the options, but it messes up the C::B user interface
on C::B sturtup (it seems that wxAuiManager and wxExecute interfere in some way)... any ideas how to do this in another way?
(The output of C::B on stdout contains "caching results from `wx-config --cxxflags`", so maybe I can use this cache?)
I've tried also to do this in a separate thread but the thread freezes (perhaps wxExecute in not thread-safe).

@MortenMacFly: Thanks, I'll have a look.
Title: Re: Semantic highlight
Post by: perento on May 15, 2012, 04:44:36 pm
Nice nice, i was looking for something to highlight user functions, classes, structures etc. and i was leaded to you thread.
Would like to see that plugin how it works, keep the work please :)
Title: Re: Semantic highlight
Post by: MortenMacFly on May 15, 2012, 08:49:02 pm
(it seems that wxAuiManager and wxExecute interfere in some way)...
They have nothing to do with each other.

any ideas how to do this in another way?
Have a look at other plugins that run external tools (e.g. CppCheck, CScope or alike), you can see how its done there.

(The output of C::B on stdout contains "caching results from `wx-config --cxxflags`", so maybe I can use this cache?)
This information is not available via a public interface.

I've tried also to do this in a separate thread but the thread freezes (perhaps wxExecute in not thread-safe).
We use threads a lot in C::B if yours freezes then you did something wrong. You can even use the thread abstractions available via the SDK for such purposes. Again: Have a look how other plugins do it (i.e. ThreadSearch).
Title: Re: Semantic highlight
Post by: mistar on May 16, 2012, 08:11:45 pm
Hi guys!

Before my work on SemanticHighlight plugin goes on, let me introduce simple SHTool plugin I've just written
to demonstrate what can be achieved. The sources are attached.

This is a very simple tool plugin that colorize the syntax in a semantic way using hardcoded styles that I like
(see the sources).
To revert the styles to original ones just reopen the file.
I've tested it with C::B 10.05 and with latest svn version (with change of sdk version in manifest.xml to 1.13.2).
The plugin works best with project files (with all compiler options and search directories set up correctly).
The plugin links against libclang.so so you have to install clang first.
Also note that parsing a file from scratch takes some time (when editing a file there are some optimizations
so in the final version it should take no more than a second).

Have fun!
Title: Re: Semantic highlight
Post by: oBFusCATed on May 16, 2012, 10:21:37 pm
Also note that parsing a file from scratch takes some time (when editing a file there are some optimizations
so in the final version it should take no more than a second).
A second for every key press? :)
Title: Re: Semantic highlight
Post by: mistar on May 16, 2012, 11:30:34 pm
A second for every key press? :)
A second after last key press.
In eclipse CDT it's not a problem; if it is done in separate thread then only visual results are delayed a bit.
Besides, there is no way to do this immediately. ;)
Anyway, in my opinion benefits surpass all the inconveniences.
Title: Re: Semantic highlight
Post by: ollydbg on May 17, 2012, 03:29:14 am
Hi guys!

Before my work on SemanticHighlight plugin goes on, let me introduce simple SHTool plugin I've just written
to demonstrate what can be achieved. The sources are attached.

This is a very simple tool plugin that colorize the syntax in a semantic way using hardcoded styles that I like
(see the sources).
To revert the styles to original ones just reopen the file.
I've tested it with C::B 10.05 and with latest svn version (with change of sdk version in manifest.xml to 1.13.2).
The plugin works best with project files (with all compiler options and search directories set up correctly).
The plugin links against libclang.so so you have to install clang first.
Also note that parsing a file from scratch takes some time (when editing a file there are some optimizations
so in the final version it should take no more than a second).

Have fun!
Hi, nice work.
I just build it under Windows. Here is what I do:
1, Put your source and package under:
Code: [Select]
E:\code\cb\cb_trunk\src\plugins\contrib\SHTool
2, I borrow the clang from Codelite source repo:
Code: [Select]
https://codelite.svn.sourceforge.net/svnroot/codelite/trunk/sdk/clangAnd put the folder "clang" in the folder "SHTool"

3, I have create a cbp file for Windows called:SHTool_Win.cbp (see attachment)

4, open the SHTool_Win.cbp, and build it. (the output will be put in E:\code\cb\cb_trunk\src\devel)

5, run the Codeblocks under E:\code\cb\cb_trunk\src\devel, Make sure the clang.dll is in your PATH.

The problem is: I can't see this plugin works when I open a new project, I can see the SHTool plugin is loaded. :(
The editor is not highlighted by SHTool.
Title: Re: Semantic highlight
Post by: ollydbg on May 17, 2012, 03:40:15 am
The problem is: I can't see this plugin works when I open a new project, I can see the SHTool plugin is loaded. :(
The editor is not highlighted by SHTool.
Ok, it works now, I need to click the Menu->Plugins->SHTool, it works really NICE.

See the screen shot with semantic highlight below:
(http://i683.photobucket.com/albums/vv194/ollydbg_cb/clang_sematic_highlight.png)
Compared with normal editor
(http://i683.photobucket.com/albums/vv194/ollydbg_cb/normal_editor.png)
Title: Re: Semantic highlight
Post by: oBFusCATed on May 17, 2012, 08:52:52 am
Ollydbg: This is not a demonstration of semantic highlight... you have to show a function's body...
Title: Re: Semantic highlight
Post by: mistar on May 17, 2012, 09:28:32 am
The problem is: I can't see this plugin works when I open a new project, I can see the SHTool plugin is loaded. :(
The editor is not highlighted by SHTool.
Ok, it works now, I need to click the Menu->Plugins->SHTool, it works really NICE.
Yes, now this is only a simple tool (when you want to highlight sources just Menu->Plugins->SHTool; I forgot to write it explicitly), not real-time highlighter.
Now it is time to design a framework for real-time SemanticHighlight plugin.
Just a couple of event handlers, provided the events are fired in right order and with all required data... ;)
Title: Re: Semantic highlight
Post by: oBFusCATed on May 17, 2012, 10:22:02 am
mistar:
Is it possible to make the required changes in the SDK, so multiple plugins can be semantic highlight providers?
I guess if we make a CC plugin based on clang we have to implement the semantic highlight in it in order to prevent calling clang to parse the code twice.

Is the parsing required for CC and semantic parsing the same?
Title: Re: Semantic highlight
Post by: mistar on May 17, 2012, 10:31:48 am
mistar:
Is it possible to make the required changes in the SDK, so multiple plugins can be semantic highlight providers?
I guess if we make a CC plugin based on clang we have to implement the semantic highlight in it in order to prevent calling clang to parse the code twice.

Is the parsing required for CC and semantic parsing the same?
AFAIK, clang_codeCompleteAt calls clang_reparseTranslationUnit, see the description of clang_defaultEditingTranslationUnitOptions here: http://clang.llvm.org/doxygen/group__CINDEX__TRANSLATION__UNIT.html (http://clang.llvm.org/doxygen/group__CINDEX__TRANSLATION__UNIT.html)
Title: Re: Semantic highlight
Post by: oBFusCATed on May 17, 2012, 10:40:43 am
I've never looked at the clang's API and I restrain myself from doing so, because I'd be caught in a nasty CC refactoring.  :P
Title: Re: Semantic highlight
Post by: mistar on May 19, 2012, 11:28:57 am
I think there is a better way: Instead of introducing another event, you could move the portion:
Code: [Select]
   if (can_updateui)
    {
        if (ed)
        {
            SetActiveEditor(ed);
            ed->GetControl()->SetFocus();
        }
    }
...to after:
Code: [Select]
   // check for ProjectFile
    if (ed && !ed->GetProjectFile())
    {
        // First checks if we're already being passed a ProjectFile as a parameter
        if (data)
            Manager::Get()->GetLogManager()->DebugLog(_T("Project data set for ") + data->file.GetFullPath());
        else
            Manager::Get()->GetProjectManager()->FindProjectForFile(ed->GetFilename(), &data, false, false);
        if (data)
            ed->SetProjectFile(data,true);
    }
This way, you can use the already existing cbEVT_EDITOR_ACTIVATED event and don't screw anything.

I'll check this in my local copy and look for side-effects (but I cannot think of any for the moment...).

Well, this is not what I need... My understanding of cbEVT_EDITOR_ACTIVATED is that it is posted whenever an editor gets focus...
What I really need is an event that is posted when a file is opened in C::B to set up clang data (index, translation unit, ...) for that file
AND containing all necessary data like the project the file belongs to (to read the project's compiler options needed to correctly set up the index).

BTW, it is good practice to name things adequately so that others can understand our code ;) Therefore, if cbEVT_EDITOR_OPEN is reserved for
"is about to be opened" (and there is no extra data set up at the moment), then I believe that there might (or even should) be another event, e.g. cbEVT_EDITOR_LOADED
(this is probably better name) or whatever, signaling that the opening action is completed with all the data set up correctly.
Observe also, that my implementation shouldn't screw anything: it is just a piece of code added to what exists and this code shouldn't interfere with
the existing code.

Last but not least, cbEVT_PROJECT_OPEN event is not semantically "is about to be opened" ;)

EDIT: I would need also an event notifying that options in a build target or in a project have changed to update the highlighter's data...
The existing cbEVT_PROJECT_TARGETS_MODIFIED is not posted when I change compiler (or other) options in a target or project (in fact no event is posted at all).
The solution of reading compiler options from the active build target every time highlighter is invoked is not elegant and poses problems with wxExecute that I already mentioned
(that is, on C::B startup, wxExecute in OnEditorOpen event handler messes up the UI: image attached; I observed that loading some plugin or changing a little bit UI fixes the layout).

[attachment deleted by admin]
Title: Re: Semantic highlight
Post by: MortenMacFly on May 19, 2012, 01:52:20 pm
I'm afraid I don't really understand. When you load a project, C::B does not load all files, so there is no such event you are asking for. If you are interested in parsing all files when the project is opened, you need the EVT_PROJECT_OPEN or better EVT_PROJECT_ACTIVATE. Then, in addition you can use EVT_PROJECT_TARGETS_MODIFIED, EVT_PROJECT_FILE_CHANGED, EVT_PROJECT_FILE_ADDED and EVT_PROJECT_END_REMOVE_FILES to check for changes in the setup. There is EVT_SETTINGS_CHANGED for checking, if the project's settings have been changed.

However, it makes no sense to prepare semantic highlighting for files that are actually not really opened, a.k.a. shown as an editor. Imagine you have workspaces with several 1000 files - how long shall the user wait? So semantic highlight IMHO gets interesting, once you see a file in an editor, that's why I pointed you to the EVT_EDITOR_ACTIVATED event.

Have a look at the Code::Completion plugin. It does all that: Checking for project settings, checking for file changes, running external tools (like the compiler to query the compiler's internal settings), workign once a file is opened and so on. CC would not work if all that was not possible.

If you still believe you need more, provide patches. The one you provided does not help you much, it is the same as the EVT_EDITOR_ACTIVATED event, once you did the changes as I suggested. Maybe I'll understand better your needs if you show what you have in mind.

For the wxExecute issue: Run it as an external process, use the C::B SDK to do so, maybe even in a thread. then you won't have such issues. It works really nice in several plugins we already ave, so I see no reason why it should not work for you, except you might have done something wrong. Again: in that case provide a test case / sample to reproduce.
Title: Re: Semantic highlight
Post by: mistar on May 19, 2012, 02:17:50 pm
I'm afraid I don't really understand. When you load a project, C::B does not load all files, so there is no such event you are asking for. If you are interested in parsing all files when the project is opened, you need the EVT_PROJECT_OPEN or better EVT_PROJECT_ACTIVATE. Then, in addition you can use EVT_PROJECT_TARGETS_MODIFIED, EVT_PROJECT_FILE_CHANGED, EVT_PROJECT_FILE_ADDED and EVT_PROJECT_END_REMOVE_FILES to check for changes in the setup. There is EVT_SETTINGS_CHANGED for checking, if the project's settings have been changed.

However, it makes no sense to prepare semantic highlighting for files that are actually not really opened, a.k.a. shown as an editor. Imagine you have workspaces with several 1000 files - how long shall the user wait? So semantic highlight IMHO gets interesting, once you see a file in an editor, that's why I pointed you to the EVT_EDITOR_ACTIVATED event.

Have a look at the Code::Completion plugin. It does all that: Checking for project settings, checking for file changes, running external tools (like the compiler to query the compiler's internal settings), workign once a file is opened and so on. CC would not work if all that was not possible.

If you still believe you need more, provide patches. The one you provided does not help you much, it is the same as the EVT_EDITOR_ACTIVATED event, once you did the changes as I suggested. Maybe I'll understand better your needs if you show what you have in mind.

For the wxExecute issue: Run it as an external process, use the C::B SDK to do so, maybe even in a thread. then you won't have such issues. It works really nice in several plugins we already ave, so I see no reason why it should not work for you, except you might have done something wrong. Again: in that case provide a test case / sample to reproduce.

I do not want to scan all project's files ;)
But I really don't see why is EVT_EDITOR_ACTIVATED the same as EVT_EDITOR_OPENED/LOADED...
I need to 'process' a file whenever a tab for this file appears in the C::B tab bar and 'process' this file when its tab disappears.
This is exactly what you said: I do not need to process files that are not displayed as tabs (aka opened in my terminology).
EVT_EDITOR_ACTIVATED is posted every time an editor gets focus and this is not what I need (or I really don't understand you...).
By 'process' I mean (among others) getting the project the file belongs to (which cannot be done using EVT_EDITOR_OPEN)
to get compile options.

But you are right, I'll have a look at CC plugin...

PS. I've checked that EVT_SETTINGS_CHANGED is not posted when a target's settings are changed...
Title: Re: Semantic highlight
Post by: oBFusCATed on May 19, 2012, 02:28:00 pm
Why do you need this?
I think you need something like EVT_EDITOR_MODIFIED event*, so you get a notification, when the contents of the editor gets modified.

At the start you'll do the first pass on EVT_EDITOR_ACTIVATED and mark it as highlighted, then when you get a modified event you'll do the re-highlighting...

* If there is no such event you can add it easily I think, just make sure you don't fire it for every key press.
Probably you can use a timer and group multiple changes together...

p.s. and don't forget to test the performance with massive files 30-40k lines of code :)
Title: Re: Semantic highlight
Post by: mistar on May 19, 2012, 02:46:09 pm
Why do you need this?
I think you need something like EVT_EDITOR_MODIFIED event*, so you get a notification, when the contents of the editor gets modified.

At the start you'll do the first pass on EVT_EDITOR_ACTIVATED and mark it as highlighted, then when you get a modified event you'll do the re-highlighting...
This is an idea, but I don't like it because every time I catch EVT_EDITOR_ACTIVATED I search some set of editors for the one I am processing...
I really don't understand your reluctance in introducing new events ;)
If some things are natural and allows for better performance why not introduce them?
BTW: why don't you program in C? Why people introduced C++? After all, you can do everything in pure C or better, assembly or pure machine code ;P
Title: Re: Semantic highlight
Post by: MortenMacFly on May 19, 2012, 02:46:59 pm
* If there is no such event you can add it easily I think, just make sure you don't fire it for every key press.
Probably you can use a timer and group multiple changes together...
cbEVT_EDITOR_MODIFIED

We have them all... ;-)
Title: Re: Semantic highlight
Post by: oBFusCATed on May 19, 2012, 02:54:21 pm
BTW: why don't you program in C? Why people introduced C++? After all, you can do everything in pure C or better, assembly or pure machine code ;P
What is this comment? Something serious or just a joke?

BTW: I have no problem adding new events, but I don't understand the logic behind the current system, so I can't comment if your proposal is good or not, I'm just telling you what would I do if I'm implementing you.
BTW2: If you use a hash table there would be no performance problems :)
Title: Re: Semantic highlight
Post by: MortenMacFly on May 19, 2012, 03:07:59 pm
I do not want to scan all project's files ;)
But I really don't see why is EVT_EDITOR_ACTIVATED the same as EVT_EDITOR_OPENED/LOADED...
I need to 'process' a file whenever a tab for this file appears in the C::B tab bar and 'process' this file when its tab disappears.
This is exactly what you said: I do not need to process files that are not displayed as tabs (aka opened in my terminology).
EVT_EDITOR_ACTIVATED is posted every time an editor gets focus and this is not what I need (or I really don't understand you...).
By 'process' I mean (among others) getting the project the file belongs to (which cannot be done using EVT_EDITOR_OPEN)
to get compile options.
Aha - I guess I understood what you mean now. But keepin mind, that the file might have changed externally and this is checked only at the EVT_EDITOR_ACTIVATED IMHO. So to make sure you are getting the up-to-date file content, this event is still right for you. You could use some caching to avoid scanning if not really needed. However, I still think adding another event would be an overkill as you can do what you want with the events we have. But if you really insist... nag me again and I'll see.

PS. I've checked that EVT_SETTINGS_CHANGED is not posted when a target's settings are changed...
That is true, it fires only when the global setting are changed. In your use case, you might want to check EVT_BUILDTARGET_SELECTED or EVT_PROJECT_TARGETS_MODIFIED.
Title: Re: Semantic highlight
Post by: MortenMacFly on May 19, 2012, 03:09:19 pm
BTW: I have no problem adding new events, but I don't understand the logic behind the current system, so I can't comment if your proposal is good or not, I'm just telling you what would I do if I'm implementing you.
Do we actually have a good documentation about the events present? WiKi / SDK help?

BTW: there is the EventsDisplay plugin somewhere in the forums that shows them all...
Title: Re: Semantic highlight
Post by: mistar on May 19, 2012, 03:36:23 pm
BTW: why don't you program in C? Why people introduced C++? After all, you can do everything in pure C or better, assembly or pure machine code ;P
What is this comment? Something serious or just a joke?

BTW: I have no problem adding new events, but I don't understand the logic behind the current system, so I can't comment if your proposal is good or not, I'm just telling you what would I do if I'm implementing you.
BTW2: If you use a hash table there would be no performance problems :)
Note the ";P" at the end.

I'd just like to propose some changes that facilitate some actions.
By introducing an event that is well suited for some things, you (people) make my (as well as other's) code more maintainable.
Otherwise more and more code will be just a workaround, the entire logic will eventually be lost and you end up with redesigning some things
(which is far more time-consuming than adding new, natural events).

As for hash tables, they have expected constant time complexity; but this doesn't mean that I refrain from using them.

My logic behind SH plugin is:
Editor is opened => if c++ file, create an index and highlight the text (with compile options of the editor's project's active build target via 'Target->CompileOptions' map described below)
Editor is closing => if c++ file, destroy an index
BuildTarget is created or modified => get compile options of the target, execute those in `...` and add it to a map 'Target->CompileOptions'
BuildTarget is removed => remove options from the map.
Source file changed (I have a framework for this already) => reparse index (here enter some optimizations from clang, e.g. precompiled headers) and highlight the text.
That's all. Just a simple logic.

Aha - I guess I understood what you mean now. But keepin mind, that the file might have changed externally and this is checked only at the EVT_EDITOR_ACTIVATED IMHO. So to make sure you are getting the up-to-date file content, this event is still right for you.
This is a place to add EVT_EDITOR_EXTERNAL_CHANGE or-something-like-this event. C::B should responsible for reloading the text...
PS. I've checked that EVT_SETTINGS_CHANGED is not posted when a target's settings are changed...
That is true, it fires only when the global setting are changed. In your use case, you might want to check EVT_BUILDTARGET_SELECTED or EVT_PROJECT_TARGETS_MODIFIED.
This is similar to using EVT_EDITOR_ACTIVATED instead of EVT_EDITOR_OPENED... just a workaround (or 'ugly hack' ;))
And the real problem is that no event is posted at all after changing compiler options.

BTW: I have no problem adding new events, but I don't understand the logic behind the current system, so I can't comment if your proposal is good or not, I'm just telling you what would I do if I'm implementing you.
Do we actually have a good documentation about the events present? WiKi / SDK help?

BTW: there is the EventsDisplay plugin somewhere in the forums that shows them all...
Wiki is not up-to-date (hence I didn't realize that EVT_SETTINGS_CHANGED exists) but the sources are sufficient.

I am using DisplayEvents after updating its events database :)
Title: Re: Semantic highlight
Post by: MortenMacFly on May 19, 2012, 04:30:05 pm
This is a place to add EVT_EDITOR_EXTERNAL_CHANGE or-something-like-this event. C::B should responsible for reloading the text...
Sure, and it does exactly that. What I meant is if you only listen to editor open, then this is not enough for such cases, including the user edited the file a lot (including "hard-core" like selecting all text and removing it). So how will you track these changes? I strongly believe EVT_EDITOR_OPENED is not enough to cover all these cases, however, EVT_EDITOR_ACTIVATED in addition to EVT_EDITOR_MODIFIED will do. So I don't think this is an ugly hack, but the way to go. In the end, if a user opened a file (EVT_EDITOR_OPENED is fired) it is because in 99% of the time the content of the editor will change. So whatever you get at the time EVT_EDITOR_OPENED fires is obsolete sooner or later.

My proposal therefore: Add the event you desire in your local copy as needed and provide it as a patch with the plugin sources. If all works out nicely we / I will add it officially to the SDK.
Title: Re: Semantic highlight
Post by: mistar on May 19, 2012, 05:01:00 pm
This is a place to add EVT_EDITOR_EXTERNAL_CHANGE or-something-like-this event. C::B should responsible for reloading the text...
Sure, and it does exactly that. What I meant is if you only listen to editor open, then this is not enough for such cases, including the user edited the file a lot (including "hard-core" like selecting all text and removing it). So how will you track these changes? I strongly believe EVT_EDITOR_OPENED is not enough to cover all these cases, however, EVT_EDITOR_ACTIVATED in addition to EVT_EDITOR_MODIFIED will do. So I don't think this is an ugly hack, but the way to go. In the end, if a user opened a file (EVT_EDITOR_OPENED is fired) it is because in 99% of the time the content of the editor will change. So whatever you get at the time EVT_EDITOR_OPENED fires is obsolete sooner or later.

My proposal therefore: Add the event you desire in your local copy as needed and provide it as a patch with the plugin sources. If all works out nicely we / I will add it officially to the SDK.
I am using EVT_EDITOR_OPENED only to set up data for that editor, like libclang's index and translation unit and to do initial highlighting.
It is better to do it during opening a file (since it may take longer than just reparsing text being edited, even 5-10 seconds with huge amount of headers pulled in)
than during first key press...
I understand that I can do it using EVT_EDITOR_ACTIVATED, but this way, first I must check whether it is first activation of the editor or not and this is code pollution IHMO :)
I'm far from forcing you to add new events. I just think that adding adequate events will make C::B sources more clear...
BTW, there are plenty of events like ...BEFORE_SAVE, ...SAVE, and so on... so why not BEFORE_OPEN and OPEN?
And, what is a purpose of EVT_EDITOR_OPEN then? why not to use EVT_EDITOR_ACTIVATED instead with workarounds?
Title: Re: Semantic highlight
Post by: MortenMacFly on May 19, 2012, 08:21:09 pm
And, what is a purpose of EVT_EDITOR_OPEN then? why not to use EVT_EDITOR_ACTIVATED instead with workarounds?
I told you already, what this event is for. A lot plugins make use of this that need to track editors, like the ReopenEditor plugin (have a look). Here, for a file list for example, the ACTIVATED event is not important. It is also important to intercept opening editors that are no text editors.

Again: I am not against adding another event, its just that I am not fully convinced it is the right thing to do for your purposes. I might be wrong though - so proof me wrong with your implementation / patch. 8)
Title: Re: Semantic highlight
Post by: mistar on May 19, 2012, 10:16:19 pm
Again: I am not against adding another event, its just that I am not fully convinced it is the right thing to do for your purposes. I might be wrong though - so proof me wrong with your implementation / patch. 8)
OK, I'll try ;)
Title: Re: Semantic highlight
Post by: mistar on May 20, 2012, 11:19:54 am
I have read all your posts, guys, and come to a conclusion that you don't read my posts at large :)
Just an example:
What I meant is if you only listen to editor open, then this is not enough for such cases, including the user edited the file a lot (including "hard-core" like selecting all text and removing it). So how will you track these changes? I strongly believe EVT_EDITOR_OPENED is not enough to cover all these cases, however, EVT_EDITOR_ACTIVATED in addition to EVT_EDITOR_MODIFIED will do. So I don't think this is an ugly hack, but the way to go. In the end, if a user opened a file (EVT_EDITOR_OPENED is fired) it is because in 99% of the time the content of the editor will change. So whatever you get at the time EVT_EDITOR_OPENED fires is obsolete sooner or later.
just after I wrote
My logic behind SH plugin is:
Editor is opened => if c++ file, create an index and highlight the text (with compile options of the editor's project's active build target via 'Target->CompileOptions' map described below)
Editor is closing => if c++ file, destroy an index
BuildTarget is created or modified => get compile options of the target, execute those in `...` and add it to a map 'Target->CompileOptions'
BuildTarget is removed => remove options from the map.
Source file changed (I have a framework for this already) => reparse index (here enter some optimizations from clang, e.g. precompiled headers) and highlight the text.
That's all. Just a simple logic.
I haven't written anywhere that I will use EVT_EDITOR_OPENED to track changes in source code...
I want to use it to do some initializations for an editor that has just been opened then some event(s) to track changes and then EVT_EDITOR_CLOSE to clean up SH data for the editor.
Just as simple as Constructor/Destructor logic.

BTW, EVT_EDITOR_MODIFIED is not for tracking changes... it is fired when an editor flips its state to/from modified (with an asterisk prepended to the file name);
neither is EVT_EDITOR_UPDATE_UI good for this.
I do have a framework for tracking changes, and all I need is to do initializations.
Now I'll try to use EVT_EDITOR_ACTIVATED though I still believe this is not the right event to do this...

And please, don't get we wrong, I'm here not to criticize you. You've made great work to get C::B to work and I really appreciate it!
I just believe that couple of things I propose make the sources of C::B and also of my plugin more clear.
Title: Re: Semantic highlight
Post by: MortenMacFly on May 20, 2012, 02:18:35 pm
I just believe that couple of things I propose make the sources of C::B and also of my plugin more clear.
Sure thing - that's why I encouraged you do it the way you believe is best for you (and clean code) and provide patches to the core if necessary. :D
Title: Re: Semantic highlight
Post by: mistar on June 06, 2012, 12:02:29 am
Hi guys!
I'm almost done with the very basic real-time semantic highlighter (first parsing is quite slow, that is, 4s but next ones highlight sources quite fast, i.e., below 0.5s on my Pentium Dual-Core machine). The plugin crashes sometimes so I'm not going to attach the sources, but I'm working on it ;)

I'm using EDITOR_ACTIVATED event (I realized that it is necessary to re-highlight an editor on each editor switch to ensure changes made in other files take effect) so there is no big need for EDITOR_OPENED patch as for now ;)

What is left:
* fix bugs so that the plugin does not crash;
* add events that are posted on compiler options change (I'll need a little bit of your help here);
* write configuration panel (as for now I'm using fixed colors);
Title: Re: Semantic highlight
Post by: daniloz on June 06, 2012, 08:26:53 am
Hi guys!
I'm almost done with the very basic real-time semantic highlighter (first parsing is quite slow, that is, 4s but next ones highlight sources quite fast, i.e., below 0.5s on my Pentium Dual-Core machine). The plugin crashes sometimes so I'm not going to attach the sources, but I'm working on it ;)
...

I'm looking forward to try it...
Title: Re: Semantic highlight
Post by: perento on June 08, 2012, 02:23:36 am
Its still very earlier to ask but this just pop in my mind and wondered what do you think.
So..what you think on highlighting a member variables and is it possible? Keep the work, please.
Title: Re: Semantic highlight
Post by: mistar on June 13, 2012, 12:54:41 pm
Its still very earlier to ask but this just pop in my mind and wondered what do you think.
So..what you think on highlighting a member variables and is it possible? Keep the work, please.
It will be possible to highlight differently all different (semantic) kinds of variables, i.e., global, local, member (public/private/static/non-static/...), parameter variables, and similarly for functions.
I have little time now, so my work on SH will be delayed a little bit.
Title: Re: Semantic highlight
Post by: mistar on June 14, 2012, 12:25:55 am
Simple question: how to check whether C::B startup is completed? Is there any function for this?
I know there is EVT_APP_STARTUP_DONE event but I need to check whether I am after this event.
For example when I load my plugin via Plugins->Manage plugins, C::B is ready in my OnAttach, but when C::B is loading my plugin on startup, C::B is not ready.

I need this check because:
1. the current C::B version doesn't post EVT_EDITOR_ACTIVATED for all recently-opened editors
2. wxExecute messes up GUI layout when called from within OnEditorActivated (I don't know why) on C::B startup.
Title: Re: Semantic highlight
Post by: mistar on June 15, 2012, 11:57:49 pm
OK ;)
Sources of the very-very-very preliminary version of SemanticHighlight plugin are attached.
But be warned:
1. The plugin crashes sometimes as I've already mentioned. I suspect that this happens when SH does highlighting on a control and simultaneously the user types some text to that control. I don't know whether there is some way to lock the control during highlighting process, but maybe you know ;) So please, do not use the plugin in its present form while doing important things in C::B!
2. If you change compile options of some target and/or project, SH doesn't notice those changes. It will when we add an event posted on such changes (EVT_PROJECT_TARGETS_MODIFIED is NOT posted on such changes though, in fact there is currently no event posted!).
3. There is no configuration panel yet.
4. On C::B startup, the GUI can be messed a little bit (wxExecute on EVT_EDITOR_ACTIVATED), but if you click on a boundary between panels everything returns to the correct layout.
5. Lines 301 and 302 of SemanticHighlight.cpp have to be adjusted according to your gcc configuration. Otherwise the plugin will work but some identifiers won't be highlighted correctly. These lines are needed in some gcc configurations as libclang doesn't include them in default search directories and then some fatal errors prevent libclang's parser to do entire job.
6. The plugin should work under Windows, but I didn't test it.
7. Please, check your compiler and linker options (libraries may have different names on your system).

Any questions and comments will be appreciated.

EDIT: I forgot: recall that the first parsing process may be quite long (about 4s on my machine).
Title: Re: Semantic highlight
Post by: MortenMacFly on June 16, 2012, 05:33:43 am
Sources of the very-very-very preliminary version of SemanticHighlight plugin are attached.
To reach more tester, you should probably explain what dependencies you have, where they are available and how you "install" them.
Title: Re: Semantic highlight
Post by: MortenMacFly on June 16, 2012, 06:42:44 am
2. If you change compile options of some target and/or project, SH doesn't notice those changes. It will when we add an event posted on such changes (EVT_PROJECT_TARGETS_MODIFIED is NOT posted on such changes though, in fact there is currently no event posted!).
I could think of a project options changed and compiler settings changed events, in case the project/compiler options are changed in the UI dialogs and the user hit "OK". For the compiler options this is, however, tricky. For example: You can attach build scripts to projects or even single files that evaluate at run-time. They may even using system calls to obtain specific settings. So you don't know the full compiler option for a single file until you build the file. Here, you should try to obtain these from the editor -> project file -> active target -> compiler options. You would, however, receive the current compiler options in the mentioned events which should work in 95 percent of the cases.
However, in the end that should be the same as you would obtain in editor activated event when the settings dialog was shown (same chain: editor -> project file -> active target -> compiler options).
Title: Re: Semantic highlight
Post by: mistar on June 16, 2012, 11:05:59 am
Sources of the very-very-very preliminary version of SemanticHighlight plugin are attached.
To reach more tester, you should probably explain what dependencies you have, where they are available and how you "install" them.

To compile the plugin you need clang and llvm (see llvm.org) dynamic libraries installed.
The installation process should be quite easy, just install clang compiler and everything else should be installed as well ;)
Then, adjust SH project's paths and libraries to match those just installed and everything should work.

2. If you change compile options of some target and/or project, SH doesn't notice those changes. It will when we add an event posted on such changes (EVT_PROJECT_TARGETS_MODIFIED is NOT posted on such changes though, in fact there is currently no event posted!).
I could think of a project options changed and compiler settings changed events, in case the project/compiler options are changed in the UI dialogs and the user hit "OK". For the compiler options this is, however, tricky. For example: You can attach build scripts to projects or even single files that evaluate at run-time. They may even using system calls to obtain specific settings. So you don't know the full compiler option for a single file until you build the file. Here, you should try to obtain these from the editor -> project file -> active target -> compiler options. You would, however, receive the current compiler options in the mentioned events which should work in 95 percent of the cases.
I know there can be very tricky ways for specifying compiler options. I use for example `wx-config --cflags` to obtain the compiler/linker options for wxWidgets and such tricky ways are supported in SH already.

However, in the end that should be the same as you would obtain in editor activated event when the settings dialog was shown (same chain: editor -> project file -> active target -> compiler options).
Calling my GetCompileOptions each time editor is activated is highly inefficient (several wxExecute calls to get options from commands like `wx-config --cflags`). Moreover, I avoid to process EVT_EDITOR_ACTIVATED each time the last active editor equals the current (windows switching for example), and there is (at present) no way for checking whether the options change at all, so I don't know whether to reparse sources or not ;) I can, of course, reparse them each time EVT_EDITOR_ACTIVATED occurs but nevertheless it is worse solution IMHO.

I also debate whether to lock editor during first (several seconds long) reparsing, but there are some difficulties, for example I have no access to the percentage of the elapsed process, so I could only display some information box.
Please, share your opinions on that and similar things.
Title: Re: Semantic highlight
Post by: MortenMacFly on June 16, 2012, 05:15:45 pm
Moreover, I avoid to process EVT_EDITOR_ACTIVATED each time the last active editor equals the current (windows switching for example), and there is (at present) no way for checking whether the options change at all, so I don't know whether to reparse sources or not ;) I can, of course, reparse them each time EVT_EDITOR_ACTIVATED occurs but nevertheless it is worse solution IMHO.
Yes, I think this is not really good. However, what about the other option(s) I suggested?
Title: Re: Semantic highlight
Post by: mistar on June 16, 2012, 06:19:27 pm
Moreover, I avoid to process EVT_EDITOR_ACTIVATED each time the last active editor equals the current (windows switching for example), and there is (at present) no way for checking whether the options change at all, so I don't know whether to reparse sources or not ;) I can, of course, reparse them each time EVT_EDITOR_ACTIVATED occurs but nevertheless it is worse solution IMHO.
Yes, I think this is not really good. However, what about the other option(s) I suggested?
The events for project/target options changed and compiler options changed are IMHO the best solution here.
Since projects and targets are treated identically here (they are derived from CompileOptionsBase class) there might be one event kind.

And how about function returning true when C::B is starting and false otherwise? You have Manager::IsAppShuttingDown but no counterpart on startup.
If such a counterpart, say IsAppStartingUp, existed I could do all initializations in an event handler for EVT_APP_STARTUP when C::B is starting, in OnAttach when the plugin is loaded and in OnEditorActivated otherwise (I could then check in OnEditorActivated if IsAppStartingUp and do no initializations if so, as then all initializations would be done in OnAppStartup). This would solve, hopefully, the problem of messing up the GUI at C::B startup.

Also, did you consider adding a mutex for each wxScintilla control that would be publicly available? When I do highlighting I could aquire the mutex and then release it, and the same (internally in the control) when user types into the control. Now I have no tools to synchronize user input (which changes the layout of the control) and highlighting...
Title: Re: Semantic highlight
Post by: MortenMacFly on June 17, 2012, 12:20:40 pm
And how about function returning true when C::B is starting and false otherwise? You have Manager::IsAppShuttingDown but no counterpart on startup.
You can do it yourself easily: Have a global var "startupdone" that is false initially, connect to the "cbEVT_APP_STARTUP_DONE" event and set it to true if this fires. Combined with IsAppShuttingDown you get a method that "bools" if it is safe to operate or not.

Also, did you consider adding a mutex for each wxScintilla control that would be publicly available?
Well - that is not so easy. But what is wrong if you operate inside an event and call Skip() only if you have finished. This way you have the control exclusively during that time.
Title: Re: Semantic highlight
Post by: mistar on June 17, 2012, 09:03:09 pm
And how about function returning true when C::B is starting and false otherwise? You have Manager::IsAppShuttingDown but no counterpart on startup.
You can do it yourself easily: Have a global var "startupdone" that is false initially, connect to the "cbEVT_APP_STARTUP_DONE" event and set it to true if this fires. Combined with IsAppShuttingDown you get a method that "bools" if it is safe to operate or not.
I cannot do it in the plugin, it has to be provided from outside, that is from the main application. The reason is that attaching a plugin may occur after EVT_APP_STARTUP_DONE is fired (loading plugin via Plugins->Manage plugins).

Also, did you consider adding a mutex for each wxScintilla control that would be publicly available?
Well - that is not so easy. But what is wrong if you operate inside an event and call Skip() only if you have finished. This way you have the control exclusively during that time.
The problem is that I do not want to block application until reparse-highlight process is completed, which may happen long time after OnTextChanged handler returns (I have threads, one for each editor for parsing/highlighting). Second, reparsing is much longer than highlighting and I need to lock the editor's control only for highlighting.
IMHO, Scintilla should take care of synchronizing access to the control, so I'll try to use one function (wxScintilla::SetStyleBytes) to do highlighting and maybe this helps.
Title: Re: Semantic highlight
Post by: mistar on June 18, 2012, 12:42:38 pm
There are tools that should help in locking a mutex for an editor while updating the editor's control, but they do not work properly...
The solution would be to handle wxEVT_SCI_MODIFIED in the following way:
* if GetModificationType() is SC_MOD_BEFOREINSERT/SC_MOD_BEFOREDELETE (-> http://www.scintilla.org/ScintillaDoc.html#Notifications) then Lock the editor's mutex;
* if GetModificationType() is SC_MOD_INSERTTEXT/SC_MOD_DELETETEXT (which, according to the documentation, should occur after the control's modification) then Unlock the editor's mutex and do reparsing-highlighting;

However, I observed that updating control occurs after SC_MOD_INSERTTEXT/SC_MOD_DELETETEXT is handled (except for the first modification after startup...). Maybe I'm wrong and this is only delay in GUI redrawing...

I'm attaching sources. Note the lines 388-390 and 401-402 with wxSleep time-margins. To reproduce the behavior, just open C::B from command line so that you see the stderr output, place C::B with the terminal visible behind. The sequence in the terminal should be:
Code: [Select]
*** before
(500 millisecond pause)
**** update
(control update)
(2000 millisecond pause)
*** after
but is
Code: [Select]
*** before
(500 millisecond pause)
**** update
(2000 millisecond pause)
*** after and control update simultaneously

As for 'bool IsAppStartingUp()' function, shall I provide a patch?
If so, please guide me where to define the global variable 'bool isAppStarted' or so.
I understand that it should be set to true at the end of C::B's OnInit().
But it would be easier and faster if someone of C::B team could do it for me ;)
Title: Re: Semantic highlight
Post by: MortenMacFly on June 18, 2012, 08:03:12 pm
As for 'bool IsAppStartingUp()' function, shall I provide a patch?
Yes, just do it similar to the IsShuttingDown method. I'm off the next few days - so you have enough time for a patch... Otherwise I can do if I'm back ;-)
Title: Re: Semantic highlight
Post by: mistar on June 18, 2012, 08:27:05 pm
As for 'bool IsAppStartingUp()' function, shall I provide a patch?
Yes, just do it similar to the IsShuttingDown method. I'm off the next few days - so you have enough time for a patch... Otherwise I can do if I'm back ;-)
Ok, I'll try to do it myself.

I also checked the exact order of scintilla's events and UI update and it is
Code: [Select]
wxEVT_SCI_MODIFIED(BEFORE_INSERT/DELETE)
UI (control) update
wxEVT_SCI_MODIFIED(TEXTINSERT/DELETE)
the first time editor is activated and
Code: [Select]
wxEVT_SCI_MODIFIED(BEFORE_INSERT/DELETE)
wxEVT_SCI_MODIFIED(TEXTINSERT/DELETE)
UI (control) update
each subsequent time. Really weird...
Is this scintilla's bug, or C::B? Or maybe scintilla buffers graphical updates on several inserts/deletes?

Nevertheless the plugin hasn't crashed since yesterday after adding some synchronization :)

Now I'm going to do the most stupid part, that is, to write configuration panel... ::)
Title: Re: Semantic highlight
Post by: mistar on June 21, 2012, 07:08:35 pm
As for 'bool IsAppStartingUp()' function, shall I provide a patch?
Yes, just do it similar to the IsShuttingDown method. I'm off the next few days - so you have enough time for a patch... Otherwise I can do if I'm back ;-)
Ok, I'll try to do it myself.
@Morten: I'm affraid I've lost in C::B architecture. Please guide me how to do that or do it for me (I think it would take you 5 min or so :)).
Title: Re: Semantic highlight
Post by: MortenMacFly on June 28, 2012, 08:34:03 am
@Morten: I'm affraid I've lost in C::B architecture. Please guide me how to do that or do it for me (I think it would take you 5 min or so :)).
OK - I've done it and will commit after a few tests... (as this changes the SDK version).
Title: Re: Semantic highlight
Post by: mistar on June 29, 2012, 08:59:56 pm
@Morten: I'm affraid I've lost in C::B architecture. Please guide me how to do that or do it for me (I think it would take you 5 min or so :)).
OK - I've done it and will commit after a few tests... (as this changes the SDK version).
Thanks!
Title: Re: Semantic highlight
Post by: mistar on July 08, 2012, 08:39:36 pm
Hi there.

I observed that whenever I close configuration dialog of C::B all Scintilla's styles are set to those for the built-in syntactic highlighter.
Is there any way to turn off the built-in highlighter when SH is loaded?
Title: Re: Semantic highlight
Post by: mistar on August 17, 2012, 09:04:25 pm
@Morten: I'm affraid I've lost in C::B architecture. Please guide me how to do that or do it for me (I think it would take you 5 min or so :)).
OK - I've done it and will commit after a few tests... (as this changes the SDK version).
How about these changes? When do you expect them to be commited?
Title: Re: Semantic highlight
Post by: MortenMacFly on August 22, 2012, 08:51:41 am
How about these changes? When do you expect them to be commited?
They are in head now, with the limitations I mentioned. Tell me when thee is new stuff to test.
Title: Re: Semantic highlight
Post by: mistar on February 19, 2013, 04:32:51 pm
Hi guys!
It's a long time since my last post ;)
Sources of not-crashing plugin attached (maybe I'll find time to make some hg repository for future;))
Play with it and tell me what you think.
BUT!
 * no config dialog so far
 * if you go to editor settings and then back, the color scheme will revert to the C::B settings
    (how to turn this off???)
 * I don't handle changes in project/build target settings and active build target changes (work in progress)
 * due to some clang issues, there could be still places not colored properly ;) (I've filed bugs for this)
 * as usual, you need libclang installed (llvm and clang svn versions work fine for me; also remember to change
    linker options!)
 * tested with C::B (rev 8836) SDK v1.15.0.
 * [EDIT:] you need gcc >=4.6 with -std=c++0x or -std=c++11, or some other compiler supporting basic C++11 ('auto' and simple lambdas)

To handle options changes I need to know
 * when project options are changed (what if I modify only build target options? Do I get cbEVT_PROJECT_OPTIONS_MODIFIED also?)
 * when active build target is changed (that is set, not modified) -- what event should I use?

Enjoy!
Title: Re: Semantic highlight
Post by: Alpha on February 19, 2013, 11:07:01 pm
* no config dialog so far
I recently tested (http://forums.codeblocks.org/index.php/topic,17526.0.html) with providing semantic highlighting by hooking into the Scintilla lexer.  Pros: seamlessly integrates into the pre-existing highlighting settings.  Cons: the patch there is currently not usable (due to some sort of thread lock, I think, when you start typing).
Title: Re: Semantic highlight
Post by: mistar on February 20, 2013, 10:07:45 am
Great!
But... how about the new C++11 standard? My plugin inherits C++11 support from clang,
which is quite decent. And what about errors in code? As you type, there are plenty of them...
Colors is a minor issue in my plugin; but clearly I need my own color config panel
because it has to be richer (semantic highlight has a lot more styles).
I think of using color scheme extending syntactic one, but I don't know whether it is possible
to control syntactic scheme in SDK.
Title: Re: Semantic highlight
Post by: daniloz on February 20, 2013, 10:11:21 am
Yes, please, I want to see screenshots!  ;)
Title: Re: Semantic highlight
Post by: mistar on February 20, 2013, 10:26:18 am
Oh... I forgot: you need gcc >=4.6 with -std=c++0x or -std=c++11 to compile the plugin (or other compiler supporting 'auto' and simple lambdas).
Title: Re: Semantic highlight
Post by: oBFusCATed on February 20, 2013, 11:41:51 am
Oh... I forgot: you need gcc >=4.6 with -std=c++0x or -std=c++11 to compile the plugin (or other compiler supporting 'auto' and simple lambdas).
Do you have any plans to remove all these c++11 things in the future, because some of us are stuck on older distros, which have only gcc-4.1 or gcc-4.4?
Title: Re: Semantic highlight
Post by: mistar on February 20, 2013, 01:32:32 pm
Oh... I forgot: you need gcc >=4.6 with -std=c++0x or -std=c++11 to compile the plugin (or other compiler supporting 'auto' and simple lambdas).
Do you have any plans to remove all these c++11 things in the future, because some of us are stuck on older distros, which have only gcc-4.1 or gcc-4.4?

Hmm... I probably could... and I'll try.

But... Do you have any plans to migrate to *newer* versions of gcc in the future? ;)
4.6 is already stable ;)
I recommend using C++11 features (e.g. rvalue references) that allow for cleaner and faster code.
Title: Re: Semantic highlight
Post by: oBFusCATed on February 20, 2013, 02:07:09 pm
But... Do you have any plans to migrate to *newer* versions of gcc in the future? ;)
Yes, gcc4.4.6 will be my next gcc. Unfortunately if you're doing commercial software you have to work on older distros.
Title: Re: Semantic highlight
Post by: mistar on February 20, 2013, 04:33:53 pm
The code without c++0x features attached.
One more thing to config: in file CompileOptions.cpp change
#define GCC_VERSION_STR "4.6.3"
to your version (it is used in a workaround for clang using libstdc++).

And don't try to close editors while there are pending changes...
I'm working on it ;)

EDIT: fixed crashes on editor close.
Title: Re: Semantic highlight
Post by: carra on March 06, 2014, 02:30:41 pm
Hi, I have just found about this plugin. This is very interesting.

I want to ask, is there any plan to add the Semantic Highlight plugin as part of the Code::Block release pack? It really is a feature that I miss in C::B compared to some other editors...
Title: Re: Semantic highlight
Post by: apolloneo on February 17, 2015, 05:18:40 pm
The code without c++0x features attached.
One more thing to config: in file CompileOptions.cpp change
#define GCC_VERSION_STR "4.6.3"
to your version (it is used in a workaround for clang using libstdc++).

And don't try to close editors while there are pending changes...
I'm working on it ;)

EDIT: fixed crashes on editor close.

I am using CB for imagecraft AVR firmware coding. I'm very interesting in your plug-in to hightlight user defined functions. Does your plug in support C programing for ImageCraft compiler? If yes, how can I install it on CB? Thanks!