Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: Alpha on July 03, 2013, 03:23:16 pm

Title: CC plugin interface redesign
Post by: Alpha on July 03, 2013, 03:23:16 pm
One of the main blockers against having multiple CC plugins (for example, integrating Clang) is the current state of the sdk.  I would like to (at least begin) work on improving the interface, but first we should brainstorm what changes belong in the sdk, and what belongs in individual CC plugins.

My thoughts so far:
The sdk should:
A CC plugin should:

What ideas do others have?
Also, there was some recent effort to remove GUI code from the sdk, however this CC redesign would do the opposite.  Solutions?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 03, 2013, 04:06:14 pm
My thoughts so far:
The sdk should:
  • query plugins if they are active on a file
  • ask the active plugin for updates to a shared CC scope toolbar
  • have a unified keyboard shortcut/autolaunch, from which it asks the active plugin for the text to fill the autocomp box
  • ask, at the correct time, the plugin for the html content of a documentation popup (if the plugin returns nothing, the popup stays hidden)
  • keep track of dwell events, and ask the active plugin for tooltip contents
All these are spot on.

A CC plugin should:
  • manage all parsing
  • provide its own specialized refactoring/searching tools
  • provide its own symbol explorer
1. 100% done in the CC plugin.
2. Some parts might be extracted in the sdk, but this is a future task. First we need to have them :)
3. The UI should be unified, but it should be filled by the CC plugin. Because if you leave it to plugins
   then will have clang symbol browser, symbol browser, fortran symbol browser, python symbol browser, etc, etc.

One thing you've missed is an API for getting the full symbol at one position in a file.
For example if you select a struct member and want to see it in the debugger tooltip, the debugger plugin needs to
expand the string to instance of the object, so the variable could be evaluated correctly by the debugger.

What ideas do others have?
Also, there was some recent effort to remove GUI code from the sdk, however this CC redesign would do the opposite.  Solutions?
No, they won't. See the cbdebugger_interface.h file. The SDK could be used just as a bridge, between plugins and the main app.
Title: Re: CC plugin interface redesign
Post by: ollydbg on July 04, 2013, 03:38:38 am
...
3. The UI should be unified, but it should be filled by the CC plugin. Because if you leave it to plugins
   then will have clang symbol browser, symbol browser, fortran symbol browser, python symbol browser, etc, etc.
...
Agree.
But if we have only one shared symbol browser tree, it may take times to switch from one tree to another. (building current CC's symbol tree takes several minutes).

Quote
One thing you've missed is an API for getting the full symbol at one position in a file.
For example if you select a struct member and want to see it in the debugger tooltip, the debugger plugin needs to
expand the string to instance of the object, so the variable could be evaluated correctly by the debugger.
GDB has the ability to solve/parse the complex statement. Here is the example:
Code
instantA.memberB.memberC();
If you hover on memberB, you can directly pass string "instantA.memberB" to GDB.
If you hover on memberC, you can pass string "instantA.memberB.memberC" to GDB.

I'm OK with all the changes.

Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 04, 2013, 01:11:59 pm
Agree.
But if we have only one shared symbol browser tree, it may take times to switch from one tree to another. (building current CC's symbol tree takes several minutes).
Have I said something about rebuilding? :)

Code
instantA.memberB.memberC();
If you hover on memberB, you can directly pass string "instantA.memberB" to GDB.
If you hover on memberC, you can pass string "instantA.memberB.memberC" to GDB.
I know gdb can do it, but currently there is no method in SDK to ask CC for the full token.

p.s. Alpha if you're not afraid of using git, we can work on this using my git-svn clone on github
Title: Re: CC plugin interface redesign
Post by: Alpha on July 04, 2013, 06:27:38 pm
The SDK could be used just as a bridge, between plugins and the main app.
Okay, I see.

Agree.
But if we have only one shared symbol browser tree, it may take times to switch from one tree to another. (building current CC's symbol tree takes several minutes).
Have I said something about rebuilding? :)
Having a single tab that various plugins can swap the contents with makes sense.  The question is, when to swap?  Each time the active editor changes?  If there are split editors?  No editor open?  Swap on demand (with a combo box)?

p.s. Alpha if you're not afraid of using git, we can work on this using my git-svn clone on github
I have not used git before, but I am willing to try.  Do you have a quick overview of normal git workflow?

What I want to do is first delete most of the CC plugin, except for the parser.  Then add on/rewrite each feature as the changes to the SDK are finished.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 04, 2013, 07:28:51 pm
Having a single tab that various plugins can swap the contents with makes sense.  The question is, when to swap?  Each time the active editor changes?  If there are split editors?  No editor open?  Swap on demand (with a combo box)?
Have I said something about swapping. :) For me there is no problem to show symbol info from all plugins at once.
You can do some filters and stuff. But I suppose this is a feature that should be done at a later stage.
(btw I don't use symbols browsers, so I don't know the workflows).

I have not used git before, but I am willing to try.  Do you have a quick overview of normal git workflow?
Hm, not that easy. :)
Probably start from here: http://stackoverflow.com/questions/315911/git-for-beginners-the-definitive-practical-guide
For you the steps would be:
1. clone the master
2. create new branch
3. commit, commit, commit
4. push to your github accout,
5 ask for pull in github, so I can upload your changes to the repo.

I'm not really sure about the 4th step, because I've never done multi-people-workflow in github.

What I want to do is first delete most of the CC plugin, except for the parser.  Then add on/rewrite each feature as the changes to the SDK are finished.
If I were you I won't delete any code, but start extracting parts and making sure everything else still works.
And remember to leave as many as needed // FIXME comments. :)
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 04, 2013, 10:08:43 pm
Another task I have in mind is one very simple keyword-like cc plugin that works for unsupported files (php, python, lua, etc).
I have to edit such files from time to time and I feel the need for such CC. :)
Title: Re: CC plugin interface redesign
Post by: Folco on July 05, 2013, 06:31:56 pm
You think to a cc based on the xml files of Scintilla lexers ?
Title: Re: CC plugin interface redesign
Post by: Josh @ Dreamland on July 05, 2013, 06:36:38 pm
Hi there; I hope no one minds an "outsider" weighing in.  :)

From where I'm standing, it looks like you're asking quite a lot of a generic CC plugin. A parser is a very easily isolated data operation, while the symbol explorer sounds pretty UI-heavy, and a lot of the refactoring and search-related functionality is universal regardless of who is doing the parsing. I'm of the opinion that there should be two plugin types, here; a parsing plugin (based on Clang, on GCC-XML (http://gccxml.github.io/HTML/Index.html), or hand-rolled, or what have you), and then a code-completion plugin on top of that to handle the refactoring and symbol browsing (assuming that should not likewise be broken up).

Clang's API, as well as a thin layer over GCC-XML, would both be able to perform tasks such as enumerating contents of a scope, giving occurrences of an identifier which refer to the same object, enumerating parameters to a given function, resolving an overload, and evaluating/coercing expressions. These are fundamentals of any compiler, and all the tools that a bigger code-completion plugin would need to do code completion, refactoring, method/class extraction, among other helpful features.

If there was a plugin for code completion, and a separate plugin for code parsing, the same refactoring and code completion plugin could be used with any decently capable parser plugin.

Quote
One thing you've missed is an API for getting the full symbol at one position in a file.
For example if you select a struct member and want to see it in the debugger tooltip, the debugger plugin needs to
expand the string to instance of the object, so the variable could be evaluated correctly by the debugger.
This is also needed for refactor:rename. If it's even slightly useful in the debugger plugin, that's all the more reason to include this in either API.

Just my thoughts.
Cheers
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 05, 2013, 06:48:43 pm
From where I'm standing, it looks like you're asking quite a lot of a generic CC plugin. A parser is a very easily isolated data operation, while the symbol explorer sounds pretty UI-heavy, and a lot of the refactoring and search-related functionality is universal regardless of who is doing the parsing. I'm of the opinion that there should be two plugin types, here; a parsing plugin (based on Clang, on GCC-XML (http://gccxml.github.io/HTML/Index.html), or hand-rolled, or what have you), and then a code-completion plugin on top of that to handle the refactoring and symbol browsing (assuming that should not likewise be broken up).

......

If there was a plugin for code completion, and a separate plugin for code parsing, the same refactoring and code completion plugin could be used with any decently capable parser plugin.
But then what happens if the CC plugin is unloaded and the parser plugin tries to use/access it. The common parts should be in the SDK or in the Main application. It is just simpler to design/implement it this way.
If we have to have plugins for the plugins, then something is wrong. :)
Title: Re: CC plugin interface redesign
Post by: Alpha on July 05, 2013, 10:34:14 pm
Have I said something about swapping. :)
Of course you did.
3. The UI should be unified, but it should be filled by the CC plugin. Because if you leave it to plugins
   then will have clang symbol browser, symbol browser, fortran symbol browser, python symbol browser, etc, etc,
   and we should swap everything.
:D

Probably start from here: http://stackoverflow.com/questions/315911/git-for-beginners-the-definitive-practical-guide
Thanks; I have read through most of that now.
@devs: I can work with either git or on a branch in the main svn repository; what would be most useful to you?

Hi there; I hope no one minds an "outsider" weighing in.  :)
In the design stage, my opinion is: the more thoughts, the better.

Another task I have in mind is one very simple keyword-like cc plugin that works for unsupported files (php, python, lua, etc).
You think to a cc based on the xml files of Scintilla lexers ?
Currently CC has a fallback of providing suggestions from the keyword sets in the lexers for non-C/C++.  After the sdk is modified, I would like to extract this code into a simple keyword-CC plugin.  Keyword-CC could also tokenize the active file, and store terms for autocompletion when the word length is longer than, for example, 5 characters.

If we have to have plugins for the plugins, then something is wrong. :)
Agreed, but some day when another attempt is made a semantic highlighting, we will have to revisit this problem.  (Semantic highlight would need the knowledge of the parser, but should be a separate plugin from CC.)

I will try to post some code later today or tomorrow with a candidate new plugin interface.  (Apologies; my programming is going slowly due to personal life issues.)
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 05, 2013, 10:49:14 pm
Of course you did.
I meant something totally different here, I think.

Agreed, but some day when another attempt is made a semantic highlighting, we will have to revisit this problem.  (Semantic highlight would need the knowledge of the parser, but should be a separate plugin from CC.)
I'd implement it in the SDK or in the Main app, again.
Title: Re: CC plugin interface redesign
Post by: Alpha on July 06, 2013, 12:13:28 am
Of course you did.
I meant something totally different here, I think.
*facepalm* ... I was joking; look back at your original post, I modified the quote.
Title: Re: CC plugin interface redesign
Post by: Alpha on July 07, 2013, 08:10:58 pm
Code
class PLUGIN_EXPORT cbCodeCompletionPlugin : public cbPlugin
{
    public:
        struct FunctionRange
        {
            FunctionRange(const wxString& nm, int startLn, int endLn) : name(nm), startLine(startLn), endLine(endLn) {}

            wxString name;
            int startLine;
            int endLine;
        };

        struct ScopeRange
        {
            ScopeRange(const wxString& nm) : name(nm) {}

            wxString name;
            std::vector<FunctionRange> functions;
        };

        typedef std::vector<ScopeRange> ScopeRangesVec;

        cbCodeCompletionPlugin();

        virtual void CodeComplete(int tokenStartPos, int index, const wxString& token) = 0;

        virtual wxArrayString GetAutocompList(int& tokenStartPos, bool isAutoLaunch) = 0;
        virtual wxArrayString GetAutoLaunchStrings() = 0;
        virtual wxString GetAutocompDoc(int index, const wxString& token) = 0;
        virtual wxString GetDocAt(int pos) = 0;
        virtual wxArrayString GetCallTips(int pos) = 0;
        virtual wxArrayString GetToolTips(int pos) = 0;
        virtual wxString GetFullSymbolAt(int pos) = 0;
        virtual ScopeRangesVec GetScopes() = 0;

        virtual bool IsProviderFor(cbEditor* ed) = 0;
};

Thoughts?
Title: Re: CC plugin interface redesign
Post by: golgepapaz on July 07, 2013, 09:24:54 pm
care to elaborate further? what is function range? scope range?
What does GetScopes(), GetFullSymbolAt() do ? What is index?

Some traits classes should be added. Instead of just using simple
string arrays. Don't you want to know what kind of a completion it is
(function, variable,type, macro etc..). What context is it in?  (member access,
function scope, translation unit level,file level  etc.)

Since no filename is mentioned I am thinking one supposed to use
active editor right? This should be more generic and pass the filename
or better yet buffer as well
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 07, 2013, 10:41:54 pm
Thoughts?
Hm, why don't you do a live refactoring, one step at a time?

I doubt you'll be able to predict what we'll be needed from the start.
Just move some code around and see how the interface is coming along.

For example - extract the autocompletion list to be in cbplugin.cpp and to be common for all cc plugins.
Then continue with the rest of it - toolbar, symbol browser, tooltips, etc.

For the autocompletion list you can take two approaches:
1. define an autocompletion list item structure
Code
struct Item{ 
  wxstring text, type;
  cbAutocompletionTokenType tokenType;
  cbAutocompletionScope scope;
};
2. allow the CC plugin to tell the core, what kind of tokens can it produce and then make a struct with ints
Code
struct Item{
  wxstring text; // contains the full text visible in the list
  int iconIndex; // used for the icon
}

Also you'll define a function which returns the icons:
CC::GetTokenIcons() or CC::GetTokenIcon(int index)

If you'd ask me I'd use the second approach, because it is more flexible and it will work for dynamic languages, where the type is not always available.
Title: Re: CC plugin interface redesign
Post by: Alpha on July 07, 2013, 11:49:10 pm
My reason behind using simply a wxArrayString (and, now that I think about it, it could just be a wxString, because the next step would be to concatenate it anyway) is that a CC plugin would register the icons it wants, and just embed those values in the returned string(s).
Alternatively, should the main app be responsible for registration?

Since no filename is mentioned I am thinking one supposed to use
active editor right? This should be more generic and pass the filename
or better yet buffer as well
Active editor, because I can think of no circumstance in which one would want autocompletion on a non-active editor.  Passing a filename or cbEditor* would, in my opinion, be an unnecessary level of indirection.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 08, 2013, 01:33:00 am
What about scripts or the watch window when typing an expression?

p.s. Please don't use wxArrayXXXX they are bloody dangerous to use, because the operator[] is defined in the maddest possible way! Use std::vector or std::deque instead!
Title: Re: CC plugin interface redesign
Post by: golgepapaz on July 08, 2013, 03:55:47 am
My reason behind using simply a wxArrayString (and, now that I think about it, it could just be a wxString, because the next step would be to concatenate it anyway) is that a CC plugin would register the icons it wants, and just embed those values in the returned string(s).
Alternatively, should the main app be responsible for registration?
IMHO, that's a bad idea because using strings as a container object and marshalling, demarshalling necessary information is absurd.
I am sure there will be more information that needs to be conveyed than a simple string. First thing that comes to mind is
displayed vs inserted text. For example the code completion result for function int foo (int param) you can display it in the
completion list as foo(int):int or any variation thereof but you only need to insert the foo (maybe with parentheses)
also there is other information that I've mentioned before. I am sure you are not proposing to embed all this information in one string
which is very hard to use. Some data structures like the ones obfuscated gave examples of and I've mentioned are absolutely necessary.

Active editor, because I can think of no circumstance in which one would want autocompletion on a non-active editor.  Passing a filename or cbEditor* would, in my opinion, be an unnecessary level of indirection.
Yeah, probably but, IsProviderFor() needs it anyway so there is no need to be stingy. Thing is it can help decouple the CC plugin from the Manager to some extent and
because singletons are evil  :D and there is an attempt in this direction(discussed http://forums.codeblocks.org/index.php/topic,17750.0.html). Also every method that is using active editor need boilerplate code something like this;
Code
    cbEditor* editor = Manager::Get()->GetEditorManager()->GetBuiltinActiveEditor();
    if (!editor)
        return;

    if (IsProviderFor(editor))
    {
          //Do stuff probably involving further indirection to cbStyledTextCtrl.
    }
Pass a viable and healthy editor and save the plugin developer from some suffering. :)
Title: Re: CC plugin interface redesign
Post by: Alpha on July 08, 2013, 03:53:33 pm
(This forum begins to annoy me: I spent a little over an hour yesterday, trying to submit this post, but gave up and am trying again now.)

It has occurred to me (and I am very surprised it did not occur earlier) that the modifications I plan for the sdk are larger, and should receive their own class; however, I am unsure where to hook it into the main application (as I have never changed something like this before in Code::Blocks).  @devs: Suggestions?
The creation of, for example, a CCManager would fit... but I would prefer an option other than a new manager.

IMHO, that's a bad idea because using strings as a container object and marshalling, demarshalling necessary information is absurd.
I am sure there will be more information that needs to be conveyed than a simple string.
Actually, not exactly; for the specific purpose I had in mind, communicating with Scintilla, no other information is required.
From (wx)Scintilla api:
Code
    // Display a auto-completion list.
    // The lenEntered parameter indicates how many characters before
    // the caret should be used to provide context.
    void AutoCompShow(int lenEntered, const wxString& itemList);
However, you have made me think ;).  Having the additional information available will be useful in the future if, ex., we someday hook CC plugins into dialogue boxes, entry forms, etc.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 08, 2013, 05:41:03 pm
What do you want to store in this class? What functions will it has?
Title: Re: CC plugin interface redesign
Post by: Alpha on July 08, 2013, 05:50:15 pm
Instead of each CC plugin subscribing to editor events, I want this class to register the common ones that most (all) CC plugins would use.  This class would then call the appropriate function from the appropriate CC plugin.  For example: asking to rebuild the scope toolbar, asking for tooltips, asking for call tips, asking to launch/get data for the autocomple popup, asking for documentation on a symbol.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 08, 2013, 06:03:28 pm
Ok, then you'll need a manager :)
Another option is to define an interface class and to set the implementation in the Manager.

Code
struct cbCCManagerInterface
{
};

struct CCManager : cbCCManagerInterface {};

Manager::Get()->SetCCManager(new CCManager)

Manager::Get()->GetCCManager();
Title: Re: CC plugin interface redesign
Post by: p2rkw on July 08, 2013, 11:03:23 pm
Last time I rarely use codeblocks, but some time ago I spend some time with CC source code, so I share my thoughts with you :)
Most modern IDEs collects and caches all token's data when it gets selected. Take a look at netbeans or eclipse: you selects token, and in next few moments
all matchig occurences are hightlighted. Rest of token data are probably cached, Why CB doesn't go this way?  This may speed up token related operations such as autocomplete, go to definition/declaration, etc Also some kind of "find usages" are faster in those IDEs.

So here is my proposiotnion (I dont have to much time to write rich description, I hope you got the idea):
Code
struct TokenID;
struct TypeID;

struct TokenInfo{
  std::string name;
  
  TypeID typeID;
  TypeID parentTypeID;
};

struct TypeInfo{
  std::string name;
  
  //std::vector<TypeID> childs, bases, etc...
};

struct CodeComplete{
  // callback called when cursor position has been changed
  virtual void onCursorPositionChange(int fileID, int cursorPosition);

  // returns ID of token at currnet position
  virtual TokenID getCurrnetTokenID();

  // returns all useful informations about current token
  virtual TokenInfo getTokenInfo();

  // returns all useful informations about given token
  virtual TokenInfo getTokenInfo(TokenID);

  // returns all useful informations about given type (class, function, enum)
  virtual TypeInfo getTypeInfo(TypeID);

  // returs all occurrences of current token in open file
  virtual std::vector<int> getOccurrences();

  enum AccessBit{
    abAll = 0,
    abAccessible = 1,
    abPrivate = 2,
    abProtected = 4,
    //etc
  };
  // returns IDs of members that have specified acces from current token
  virtual std::vector<TokenID> getMembers(int accesBit);
  

  int currentPosition;
  int currentFileID;
  TokenID currentTokenID;
};
And possible autocomplete implementation:
Code
// object.mem|ber - '|' as always means caret position, "ber" is a rest of matching member name
TokenInfo ti = cc->getTokenInfo();
cc->setCurrentToken(ti.parentID);
std::vector<TokenID> members = cc->getMembers(abAccessible);
std::string autocomplete;
for(auto mem : members){
  TokenInfo memInfo = cc->getTokenInfo(mem);
  if(memInfo.name.startsWith(ti.name)) // ti.name == "mem" in this case
    autocomplete += memInfo.fullName + "\n";
}
//send autocomplete string to scintilla here
cc->setCurrnetToken(ti.ID);
Most imporntant thing is that CC doesnt touch UI code at all.  
Title: Re: CC plugin interface redesign
Post by: Alpha on July 15, 2013, 03:10:41 am
https://github.com/alpha0010/codeblocks_sf/tree/cc_interface (https://github.com/alpha0010/codeblocks_sf/tree/cc_interface)
Title: Re: CC plugin interface redesign
Post by: Alpha on July 15, 2013, 05:17:07 am
Tooltips are mostly migrated to the new CCManager; it functions almost exactly the same, except for a few of my improvements that should make CC tooltips try to be the optimal size/shape when squished into small monitors.  (To compile, you may have to patch the appropriate build system; I only added the files to the unix cbp currently.)
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 15, 2013, 10:39:38 am
1. Please don't use wxArrayXXX it is bloody dangerous, because there is only one operator[], which is const and which returns a reference to the element. This allows the user to change the content of a const wxArrayXXX. This has led to very mysterious bugs in the past. At least this is the situation in wx2.8. They might have fixed it in 2.9, but I doubt. Use std::vector<wxString> instead.
2. Do you still display the current parameter bold?
Title: Re: CC plugin interface redesign
Post by: Alpha on July 15, 2013, 02:50:42 pm
1.  Sorry, I forgot you had asked that; I will switch it.
2.  Only the tooltips (show full name of symbol under mouse) have been migrated.  The calltips (which show bold parameters) are currently still provided by the CC plugin.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 15, 2013, 02:54:10 pm
Hm, I'm a bit confused here...
What is the difference between a calltip and a tooltip?
I don't think there is any and the code should be combined somehow :)
Title: Re: CC plugin interface redesign
Post by: Alpha on July 15, 2013, 10:41:52 pm
What is the difference between a calltip and a tooltip?
Not much, really.  They used to be more different: calltips would only show when you either type '(' after a function, or press Ctrl-Shift-Space; and tooltips by a mouse dwell event.  However some time ago, I changed the mouse dwell to also show calltips if no tooltips are available.  The main difference at the source level is that calltips have an extra section of code specifying what needs to be highlighted.
I will look to see how easily these could be merged.
Title: Re: CC plugin interface redesign
Post by: Alpha on July 15, 2013, 11:01:44 pm
What is the difference between a calltip and a tooltip?
Not much, really. [...]
Correction: inside CC, calltips take multiple more steps to calculate, where as tooltips only display the full symbol below the mouse.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 15, 2013, 11:47:21 pm
So, if I see it right, the CC plugins should only implement the GetCalltipsAt() and GetFullSymbolAt() virtual functions and then the CCManager should decide what to show.
I don't think you need 3 functions, when you can get with 2. GetFullSymbolAt could return a struct probably containing a type and visibility flags.
Title: Re: CC plugin interface redesign
Post by: dmoore on July 16, 2013, 03:54:34 pm
Nice to see this is finally happening. Keep up the good work, guys! I am keeping an eye on this thread. When this is done, will I be able use my python CC plugin (after I port it to the new API) alongside the C/C++ CC plugin sharing the same UI?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 16, 2013, 03:57:33 pm
dmoore:
Hopefully you'll be able to do it.
I suppose it will be good if we can try to port your plugin after some works is done in order to provide some real feedback to the usefulness of the API.
Title: Re: CC plugin interface redesign
Post by: MortenMacFly on July 16, 2013, 06:31:56 pm
Interesting to see this discussion.

From my point of view: Make sure its also interface-compatible to Erans CC works. We shouldn't re-invent the wheel another time.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 16, 2013, 06:42:19 pm
Someone should spend some time to write a plugin that utilizes his code.
Title: Re: CC plugin interface redesign
Post by: Alpha on July 17, 2013, 03:04:08 pm
I don't think you need 3 functions, when you can get with 2.
Yes.
When this is done, will I be able use my python CC plugin (after I port it to the new API) alongside the C/C++ CC plugin sharing the same UI?
That is the plan; to allow multiple CC plugins to easily coexist.
Make sure its also interface-compatible to Erans CC works.
Has anyone yet read into the source of his CC, and could give me some pointers?  My priority is still mainly on converting the interface/CC plugin within Code::Blocks, but if time allows, I will look into that as well.

Also, people that have used alternate code completion plugins for Code::Blocks (Python, Fortran), do you have any tips as to what components I should be aware of?

Slightly less related: if I have code such as:
Code
int foo = 1;
int MyClass::MyFunction()
{
    int bar = 4;
    MyInheritedFunction(bar);
}
If I click on foo so my cursor is outside of MyFunction(), hovering the mouse over MyInheritedFunction() usually fails to bring up a tooltip (due to failed lookup of the symbol).  However, it (almost) always works if my cursor is, for example, on bar.
Ideas why?

(By the way, I have committed step one of calltips to my repo; if nothing else, it at least eliminates the annoying multiple blank lines in a row problem.)
Title: Re: CC plugin interface redesign
Post by: MortenMacFly on July 17, 2013, 03:42:45 pm
Has anyone yet read into the source of his CC, and could give me some pointers?  My priority is still mainly on converting the interface/CC plugin within Code::Blocks, but if time allows, I will look into that as well.
Last time I was in touch with Eran he pointed me to a test utility that covered all features you need to drive the CC engine. That was a good starting point for me, hence I need to look up the current sources to tell you the name again. Since Eran switched to GIT I did not follow the CC part any longer.

I could provide you with the core of the plugin I had started several times ago - not much, really, but it may point you into the direction. It doesn't compile anymore, since Eran also changed the dependencies and required libs since then... Send me an email address as PM and I can give you the sources.
Title: Re: CC plugin interface redesign
Post by: Alpha on July 21, 2013, 06:10:55 pm
I have replaced GetToolTips() with:
Code
struct CCToken
{
    CCToken(int _id, const wxString& dispNm) : id(_id), displayName(dispNm) {}
    int id;
    wxString displayName;
};
virtual std::vector<CCToken> GetTokenAt(int pos, cbEditor* ed) = 0;
I will add more fields to CCToken as they become needed.

If I click on foo so my cursor is outside of MyFunction(), hovering the mouse over MyInheritedFunction() usually fails to bring up a tooltip (due to failed lookup of the symbol).  However, it (almost) always works if my cursor is, for example, on bar.
Fixed.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 21, 2013, 08:02:16 pm
Fixed.
I'm a bit worried that you're mixing CC interface changes with CC parser changes. Could you commit the parser fixes to the master branch/svn?
Title: Re: CC plugin interface redesign
Post by: Alpha on July 22, 2013, 02:07:40 am
Could you commit the parser fixes to the master branch/svn?
Okay.
Unrelated changes will remain isolated in the future.
Title: Re: CC plugin interface redesign
Post by: ollydbg on July 25, 2013, 06:32:20 pm
Also every method that is using active editor need boilerplate code something like this;
Code
    cbEditor* editor = Manager::Get()->GetEditorManager()->GetBuiltinActiveEditor();
    if (!editor)
        return;

    if (IsProviderFor(editor))
    {
          //Do stuff probably involving further indirection to cbStyledTextCtrl.
    }
Pass a viable and healthy editor and save the plugin developer from some suffering. :)

Currently, CC just register normal CodeblocksEvent (like project loaded, editor opened) and wxScintilla Events(SCI editor change, SCI editor update...), adding a layer of CCManager is a good idea, thus:

The old way is:

Code
            +-----------------------+                +-----------+
            |                       +--------------->| CC1       |
            |                       +                +-----------+
            |    CodeBlocks Core    |                +-----------+
            |                       +--------------->| CC2       |
            |    Scintilla Core     |                +-----------+
            |                       |                +-----------+
            |                       +--------------->| CC3       |
            +-----------------------+                +-----------+

Becomes
Code
    +-----------------------+        +---------+     +-----------+
    |                       +------->|         +---->| CC1       |
    |                       +        |         |     +-----------+
    |    CodeBlocks Core    |        |CCManager|     +-----------+
    |                       +------->|         +---->| CC2       |
    |    Scintilla Core     |        |         |     +-----------+
    |                       |        |         |     +-----------+
    |                       +------->|         +---->| CC3       |
    +-----------------------+        +---------+     +-----------+

There, CCManager can send valid build-in editor messages.

Mostly, the Parser/Parserthread/Tokenizer class should only have non-GUI things, but if you look at currently trunk code, you will see some GUI stuffs, like:
Code
BrowserOptions       m_BrowserOptions;

BTW: the logic of the NativeParser and Parser is quite complex, and I can't understand all. (one step is what I wrote here: The parsing flow explanation by a simple example (http://forums.codeblocks.org/index.php/topic,18188.0.html), but far from complete)
Title: Re: CC plugin interface redesign
Post by: ToApolytoXaos on July 29, 2013, 03:17:31 pm
I was always curious about how Code::Block autocompletion mechanism really works and it seems this thread is the most appropriate. I have tried to read the codecompletion code and could not find where exactly caching is happening.

Is is it in the form of on an object saved in memory, in a Squirrel table, or a flat file which gets deleted upon exiting project? I have seen Pelles C using SQLite and loads remarkably fast during typing; i was really impressed by it.

Also, what is necessary to be done in order to show class / function description along with keyword auto-completion, like what other IDEs are offering?

Another thing that comes to mind. How often does it reparse a project or a file? Is it an application that runs as a background service or is the actual CC plugin that is doing so? Why not let Squirrel do so at the background and save it in a file accordingly and keep it loaded as bytecode in its VM?
Title: Re: CC plugin interface redesign
Post by: ollydbg on July 29, 2013, 04:13:55 pm
I was always curious about how Code::Block autocompletion mechanism really works and it seems this thread is the most appropriate. I have tried to read the codecompletion code and could not find where exactly caching is happening.

Is is it in the form of on an object saved in memory, in a Squirrel table, or a flat file which gets deleted upon exiting project? I have seen Pelles C using SQLite and loads remarkably fast during typing; i was really impressed by it.
TokenTree object in memory, deleted on project closing. No cache on harddisk is used.


Quote
Also, what is necessary to be done in order to show class / function description along with keyword auto-completion, like what other IDEs are offering?
Mouse hover on the symbol, then there will be a tip window showing those info.

Quote
Another thing that comes to mind. How often does it reparse a project or a file? Is it an application that runs as a background service or is the actual CC plugin that is doing so? Why not let Squirrel do so at the background and save it in a file accordingly and keep it loaded as bytecode in its VM?
reparse triggered depend on user setting, e.g. user is editing on the source file/after saving the source code. Currently there is a thread pool in the CC, so parsing is done in worker thread.

I'm not sure why CC is related to Squirrel?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on July 29, 2013, 04:17:51 pm
Why not let Squirrel do so at the background and save it in a file accordingly and keep it loaded as bytecode in its VM?
Wat? CC is done entirely in C++ because it is a part of the application that is performance critical. Why do you think it needs to be written in Squirrel?
There is no database, everything is stored in objects in memory. Nothing is cached on disk.

p.s. Please stay on topic. This one is related only to the API interface of a CC plugin.
       If you want to know something about CC or want to improve the parser only then start another one.
       Feature requests for CC not related to this API interface should go in another topic, too.
Title: Re: CC plugin interface redesign
Post by: ToApolytoXaos on July 29, 2013, 04:28:39 pm
Wat? CC is done entirely in C++ because it is a part of the application that is performance critical. Why do you think it needs to be written in Squirrel?
There is no database, everything is stored in objects in memory. Nothing is cached on disk.
OK, it's nice to know.

p.s. Please stay on topic. This one is related only to the API interface of a CC plugin.
       If you want to know something about CC or want to improve the parser only then start another one.
       Feature requests for CC not related to this API interface should go in another topic, too.

It was just a question, that's all. I did not suggest anything and there's no need to get irritated for such thing.
Title: Re: CC plugin interface redesign
Post by: Alpha on July 30, 2013, 04:29:57 am
Apologies for the recent silence; my coding is progressing slowly (hopefully I shall have some more to show soon).

After I finish hooking the autocomp box into CCManager, I want to extract the fallback procedures from the main CC plugin to create a dedicated fallback CC (core plugin).  Comments?
Title: Re: CC plugin interface redesign
Post by: ollydbg on July 30, 2013, 07:43:03 am
One patch to add files to codeblocks.cbp(not tested, by I just synchronized with unix.cbp)

Code
 src/CodeBlocks.cbp | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/src/CodeBlocks.cbp b/src/CodeBlocks.cbp
index 9967ad3..8e25d83 100644
--- a/src/CodeBlocks.cbp
+++ b/src/CodeBlocks.cbp
@@ -817,6 +817,9 @@
  <Unit filename="include/cbworkspace.h">
  <Option target="sdk" />
  </Unit>
+ <Unit filename="include/ccmanager.h">
+ <Option target="sdk" />
+ </Unit>
  <Unit filename="include/compileoptionsbase.h">
  <Option target="sdk" />
  </Unit>
@@ -2502,6 +2505,9 @@
  <Unit filename="sdk/cbworkspace.cpp">
  <Option target="sdk" />
  </Unit>
+ <Unit filename="sdk/ccmanager.cpp">
+ <Option target="sdk" />
+ </Unit>
  <Unit filename="sdk/compileoptionsbase.cpp">
  <Option target="sdk" />
  </Unit>

Question:
What does the int id member of the CCToken used for? It is an index to a specific TokenTree (TokenTree only exists in C++ Parser, and in a specified Parser instance)

Some changes are not related to CC interface changes, so I suggest you can use the "git rebase --interactive, then rebase on git-svn branch", or other similar tools like "git cherry pick tool" to extract those patches, and apply on trunk.

I still need more time to read/understand all your changes. :)
Title: Re: CC plugin interface redesign
Post by: Alpha on September 05, 2013, 12:16:46 am
[...] hopefully I shall have some more to show soon.
Well... maybe soon.  Health issues have prevented my coding.  I do have enough completed now that my next commit should be possible this sometime weekend.

What does the int id member of the CCToken used for?
It is non-functional currently, however, I put it there so that a CC callback or documentation popup can pass a plugin specific piece of information back to the plugin.  My next commit should clarify that.
Title: Re: CC plugin interface redesign
Post by: Alpha on September 08, 2013, 01:54:01 am
In reading through what I have done so far, it appears I may have created somewhat redundant functions.
Code
virtual std::vector<CCToken> GetAutocompList(int& tknStart, int& tknEnd, cbEditor* ed) = 0;
virtual std::vector<CCToken> GetTokenAt(int pos, cbEditor* ed) = 0;
Would it be preferred for me to combine them into a single function?
Code
#define CC_PREFIX true
#define CC_FULL_TOKEN false
virtual std::vector<CCToken> GetTokenAt(int pos, cbEditor* ed, bool isPrefix = CC_FULL_TOKEN) = 0;

(...taking a break from code makes it all look so foreign :( .)
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on September 08, 2013, 02:02:02 am
Probably you've lost direction in your changes, because I saw you've added another edit->complete code item (shift-space hotkey). Why? What is it purpose?

The two defines are extremely ugly. Try to no use them and use an enum instead or even enum in a struct (to simulate the strong enum c++11 feature).

What is supposed to do  GetTokenAt and what is supposed to do GetAutocompList?
If they do different things it doesn't matter that they have similar signatures/return types - you should keep them separate.
Title: Re: CC plugin interface redesign
Post by: Alpha on September 08, 2013, 05:42:12 pm
[...] I saw you've added another edit->complete code item (shift-space hotkey). Why? What is it purpose?
The entry used to be supplied by the main CC plugin.  I moved it to the application, so that CCManager can make the same shortcut work for any/all CC plugins.

GetTokenAt() currently only supplies information for showing tooltips, however, I intended it to be something that another plugin, ex. debugger, could call for more accurate results when doing some other operation.  GetAutocompList() supplies tokens for the code completion popup by looking at the prefix.

Probably you've lost direction in your changes [...]
Probably.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on September 08, 2013, 10:17:06 pm
From you explanation I conclude that:
1. GetTokenAt could return multiple tokens depicting full tokens (type, name, namespace, parameteres, etc), for example multiple functions.
2. GetAutocompList returns also multiple tokens, but they are different from the ones returned in GetTokenAt.
    They are just member function names, type info, parameters.
3. The debugger needs only the full name or better name it the expression. No need for function parameters, type, namespace, etc.
    The debugger needs the name of a member expanded to the full expression used to access it.
    At the moment if you hover over a member you'll get just the member and if you try to evaluate it, you'll get "variable not found in scope", because you tried to evaluate variable "member1" instead of "a.b->c->member1". The debugger needs a way to expand a particular string word to the full expression at the cursor. So we need a different method here again.

Also I don't think you need to worry too much if you add too many pure methods.
If the plugin developer thinks it is best to handle them the same, he can do so no problem (he can add a common method and convert parameters).
Providing clear and separate names is better for source readability. An API where you have a few functions that do lots of stuff is most of the times worse than an API with many functions doing simple things each...

Also don't worry, just iterate on the API until it feels good to you...
Title: Re: CC plugin interface redesign
Post by: Alpha on September 15, 2013, 06:57:42 pm
If I need to modify the return of, say:
Code
virtual std::vector<CCToken> GetAutocompList(int& tknStart, int& tknEnd, cbEditor* ed) = 0;
is it the preferred style to use:
Code
virtual void GetAutocompList(std::vector<CCToken>& tokens, int& tknStart, int& tknEnd, cbEditor* ed) = 0;
to avoid a (potentially) expensive copy on the vector?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on September 16, 2013, 10:01:02 am
It depends...

c++11: The first one is almost always better.
c++98: It depends on some things:
   - Is the copy really expensive (many elements in the vector)?
   - Will it be called many times and so the copy will show in the profiler?

If the answers to both questions is no, then I'll use the first style otherwise the second.
Title: Re: CC plugin interface redesign
Post by: jarod42 on September 16, 2013, 11:36:04 am
In this example, it seem that there are several "returned" values (int&).
So, I think it is better to have something similar for all these items (so the return void solution).

Order may also be important, I prefer generally ([in], [in/out], [out]) order.
Title: Re: CC plugin interface redesign
Post by: Alpha on September 28, 2013, 02:13:20 am
c++11: The first one is almost always better.
I do not believe it is yet time to restrict Code::Blocks to the new standard, so:
   - Is the copy really expensive (many elements in the vector)?
   - Will it be called many times and so the copy will show in the profiler?
It occurs each time the auto-complete box is populated.  I do not have exact numbers, however sometimes the number of tokens is large; as this can be a user requested operation, I think it would be best to minimize any potential sense of lag.  (Unfortunately, I currently do not have access to average/slower hardware, so I cannot tell if it is yet noticeable.  8) Only having access to fast computers is a problem I like having.)

I think it might be best to go with the second style.  Which brings up:
Order may also be important, I prefer generally ([in], [in/out], [out]) order.
I normally try to order parameters such that the ones that are most important/vital to the main purpose of function come first, and less useful (possibly omit-able) parameters are last.  Does Code::Blocks already have a design style in reference to this?

To anyone who might be currently trying (or considering trying) my branch, it currently should have all the functionality of the trunk except for automatic adding of parentheses after functions, and documentation popups.  Probably the only visible change is that tooltips should be a bit more stable.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on September 28, 2013, 11:25:55 am
It occurs each time the auto-complete box is populated.  I do not have exact numbers, however sometimes the number of tokens is large; as this can be a user requested operation, I think it would be best to minimize any potential sense of lag.  (Unfortunately, I currently do not have access to average/slower hardware, so I cannot tell if it is yet noticeable.  8) Only having access to fast computers is a problem I like having.)
1. There is a limit to the number of entries this should return in the UI.
2. You can always downclock your CPU and you can disable cores to test. Turn you CPU in a single core 1.5ghz and test again :)

Order may also be important, I prefer generally ([in], [in/out], [out]) order.
I'd follow this one, because it is clear what is in and out parameter.
Title: Re: CC plugin interface redesign
Post by: Alpha on September 28, 2013, 03:54:25 pm
1. There is a limit to the number of entries this should return in the UI.
Yes, but the default limit is currently so high, I do not believe I have ever hit it.
2. You can always downclock your CPU and you can disable cores to test. Turn you CPU in a single core 1.5ghz and test again :)
Downclocked, disabled turbo, and disabled hyperthreading.  For some reason, the BIOS failed to disable extra cores.  When asking for a long auto-complete list (for example, code complete after the letter 's'), I did notice an additional lag.

Order may also be important, I prefer generally ([in], [in/out], [out]) order.
I'd follow this one, because it is clear what is in and out parameter.
Okay.  Will do.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on September 28, 2013, 04:13:59 pm
I did notice an additional lag.
Have you measured how long more does it take?
Title: Re: CC plugin interface redesign
Post by: Alpha on September 29, 2013, 05:36:33 pm
My data seems to conflict with itself.  Using wxStopWatch, pass by reference is on average 1366.13 ms, and return value is 1423.27 ms.  However, the when I measured just the cost of creating a copy, wxStopWatch mostly reported 1 ms, implying the two techniques have no appreciable difference.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on September 29, 2013, 05:46:52 pm
Probably RVO or NVRO optimizations are kicking in...
Title: Re: CC plugin interface redesign
Post by: Alpha on October 05, 2013, 03:20:56 am
Moving
Code
void CodeCompletion::OnAutocompleteSelect(wxListEvent& event)
from CC to CCManager causes a crash whenever timer.Start() is called from within it.  I have tried for several hours to find a cause and am out of ideas to try; thoughts?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on October 05, 2013, 11:02:58 am
Have you tried running C::B with valgrind?  ::)
Title: Re: CC plugin interface redesign
Post by: Jenna on October 05, 2013, 11:11:56 am
Moving
Code
void CodeCompletion::OnAutocompleteSelect(wxListEvent& event)
from CC to CCManager causes a crash whenever timer.Start() is called from within it.  I have tried for several hours to find a cause and am out of ideas to try; thoughts?
Can you provide a patch, where the crash occurs, so I can try to debug it.
Title: Re: CC plugin interface redesign
Post by: Alpha on October 05, 2013, 04:26:10 pm
Have you tried running C::B with valgrind?  ::)
Not tried valgrind yet, but I will now.

Can you provide a patch, where the crash occurs, so I can try to debug it.
Attached is the current state of my code; I could try to minimize the other changes if you would like (however, the crash happens before control reaches any other new code, so I did not bother currently).
Title: Re: CC plugin interface redesign
Post by: Jenna on October 05, 2013, 05:53:31 pm
Against which revision, branch or fork is the patch ?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on October 05, 2013, 06:26:51 pm
Probably this one: https://github.com/alpha0010/codeblocks_sf/commit/0d6b9d72867ae90bea66d7be6acde7604dfc1e73
Title: Re: CC plugin interface redesign
Post by: Jenna on October 05, 2013, 06:41:38 pm
I already cloned alphas github repo, but there is no ccmanager.{h|cpp} inside it.
Title: Re: CC plugin interface redesign
Post by: Alpha on October 05, 2013, 07:14:05 pm
Did you switch to the cc_interface branch?  The files should be there (https://github.com/alpha0010/codeblocks_sf/blob/cc_interface/src/sdk/ccmanager.cpp).
Title: Re: CC plugin interface redesign
Post by: Alpha on October 05, 2013, 07:16:16 pm
Have you tried running C::B with valgrind?  ::)
Not tried valgrind yet, but I will now.
Tested, and got something around 2700 messages.  I could not find any that had an obvious relation to this problem, though.
Title: Re: CC plugin interface redesign
Post by: Jenna on October 05, 2013, 07:35:15 pm
Did you switch to the cc_interface branch?  The files should be there (https://github.com/alpha0010/codeblocks_sf/blob/cc_interface/src/sdk/ccmanager.cpp).
I have it now, thanks.

I will see if I can find anything this evening.
Title: Re: CC plugin interface redesign
Post by: Jenna on October 06, 2013, 12:58:57 am
You need to specify the eventsink in the Connect-function, otherwise the editor-control is used as eventsink, because you connect the event to it.
The correct function is called, but with an incorrect this-pointer and therefore the member variable that holds the timer also points to an incorrect location.
Instead of using the timer some undefined memory that belongs to the editor-control is used => crash.

Just add NULL, this to the parameter list of Connect (and Disconnect) in CCManager::OnEditorOpen ( and CCManager::OnEditorClose).
I'm not sure if it is really needed for Disconnect, but it does not hurd.
Title: Re: CC plugin interface redesign
Post by: Alpha on October 06, 2013, 02:02:39 am
Thank you!  Greatly appreciated.  (I cannot believe I missed that... in fact, I was so certain I had tried it, and just about everything else.  Maybe I was just coding too late at night :-\.)

Auto parenthesis after functions is now reimplemented.  The only major visible component left to complete is the documentation popup.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on October 06, 2013, 02:30:20 am
What about the symbol browser? Have it been separated from the CC plugin?
Title: Re: CC plugin interface redesign
Post by: Alpha on October 06, 2013, 05:48:56 pm
I misspoke; the documentation popup is the only remaining feature that my refactoring has broken, and not yet fixed.  That will be the last (major) piece (I believe) of the autocompletion subset of functions.  The symbol browser is the next section I will deal with, but I currently do not have a clear plan in mind for it.  How much of the symbol browser should belong to CCManager?  Just the tab, and then various plugins supply content panels within it?  Or the entire interface, and then plugins supply the tokens for CCManager to populate it with?  Something else?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on October 06, 2013, 06:26:47 pm
If I were you I'd try to have a single tree that CC plugins has to fill when CCManager tells them to do.
Title: Re: CC plugin interface redesign
Post by: Alpha on October 23, 2013, 10:53:08 pm
For a single cbEditor instance, do we want multiple CC plugins to be able to provide content simultaneously (for tooltips, autocompletion, etc.)?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on October 23, 2013, 11:07:34 pm
Hm, tough question.
Probably the answer is mostly no, but it would be good if the auto completion list could have a list of matching abbreviations in it (this is a user request).
So, we'll need a way for all kind of plugins to be able to fill the list, not only plugins of CC-type.


Title: Re: CC plugin interface redesign
Post by: Alpha on October 24, 2013, 05:38:43 am
Two different ways I can think of implementing that are: create a CCAutocompHook in the style of EditorHook,
or define a new CodeBlocksEvent, and have a CCManager::AddAutocompTokens() function that plugins subscribing to this event can call.

I think I prefer the second, because it reuses existing architecture.  However, it might end up looking uglier in the long term...
Title: Re: CC plugin interface redesign
Post by: dmoore on October 30, 2013, 04:51:53 am
Seriously don't know why you guys love git/github so much. Just needlessly built the master branch :-[ Thought I had "git clone"d the cc branch by switching from master to cc branch in the github interface and copying the clone link, only to learn that I needed to "git checkout cc_interface" because clone grabs everything.  :'( Anyway, building it now and will figure out how to get python stuff working in the next week or so.
Title: Re: CC plugin interface redesign
Post by: dmoore on October 30, 2013, 05:30:08 am
Qs:
1. Am I understanding correctly that the framework now handles the completion, calltip and doc popups, and the plugin just needs to provide data?
2. It looks like the framework expects the plugin to decide whether the caret is in the appropriate place (e.g. don't want to show popups in a comment line) and then immediately return the tokens. Is that right? The current version of the python plugin does the caret check then makes an asynchronous request to a remote python process for the CC tokens because we don't want to cause a potentially long freeze in the editor. How do I handle this case? EDIT: Maybe initially return nothing, but make the async call and then when it returns send a cbEVT_COMPLETE_CODE event to re-call the handler? Can the same be done for Doc strings?
3. What is the plugin supposed to do in the DoAutoComplete functions and why are there two?  (CC plugin appears to place text for the selected token into the editor and do any needed reparsing/formatting/showing additional calls tips? Shouldn't there be a default implementation that just does what Scintilla does by default??)
4. Is there any delay before showing a popup or is this up to the plugin to implement? (e.g. if user is typing fast don't show a popup until they slow down.)
EDIT: 5. What is GetTokenAt for?
Title: Re: CC plugin interface redesign
Post by: Alpha on October 30, 2013, 03:07:40 pm
Seriously don't know why you guys love git/github so much. Just needlessly built the master branch :-[ Thought I had "git clone"d the cc branch by switching from master to cc branch in the github interface and copying the clone link, only to learn that I needed to "git checkout cc_interface" because clone grabs everything.  :'(
Sorry, that might be partially my fault (this is my first time with git).  I think I may have unnecessarily created a master branch in my fork.

1. Am I understanding correctly that the framework now handles the completion, calltip and doc popups, and the plugin just needs to provide data?
That is how I am intending it to work.  Your plugin can also send cbEVT_SHOW_CALL_TIP, cbEVT_EDITOR_TOOLTIP and cbEVT_COMPLETE_CODE events for specialized situations (that the (hopefully more generic) CCManager does not handle).

2. It looks like the framework expects the plugin to decide whether the caret is in the appropriate place (e.g. don't want to show popups in a comment line) and then immediately return the tokens. Is that right? The current version of the python plugin does the caret check then makes an asynchronous request to a remote python process for the CC tokens because we don't want to cause a potentially long freeze in the editor. How do I handle this case? EDIT: Maybe initially return nothing, but make the async call and then when it returns send a cbEVT_COMPLETE_CODE event to re-call the handler?
Yes.  Returning an empty vector will cancel code completion, and sending the event (once your plugin is prepared) will start it up again.  (Although, you may consider first testing to see if it creates a noticeable lag when done synchronously.)
Can the same be done for Doc strings?
Not with the current API.  However, CCManager tries to only request documentation be generated when the user pauses in scrolling through options, so a short freeze here is unlikely to be noticed (at least, that is the theory).

3. What is the plugin supposed to do in the DoAutoComplete functions and why are there two?  (CC plugin appears to place text for the selected token into the editor and do any needed reparsing/formatting/showing additional calls tips?
The function DoAutocomplete(const wxString& token, cbEditor* ed) technically never will be called, however I added it to the code because I cannot guarantee that.  It will only be called if, for some unknown reason, CCManager is unable to return the CCToken equivalent that was selected.
Its purpose is to give the plugin a chance to insert/replace the token into the editor in a more advanced way than simply inserting (for example, also adding argument lists).
Shouldn't there be a default implementation that just does what Scintilla does by default??)
Probably.  The default implementation would be:
Code
void cbCodeCompletionPlugin::DoAutocomplete(const CCToken& /*token*/, cbEditor* /*ed*/)
{
    return;
}
As long as you do not call ed->GetControl()->AutoCompCancel() in this function, default autocomplete action will occur.

4. Is there any delay before showing a popup or is this up to the plugin to implement? (e.g. if user is typing fast don't show a popup until they slow down.)
I intend CCManager to handle that (however, it is currently hardcoded at 10ms, and synchronous if the user types three characters in a row).

EDIT: 5. What is GetTokenAt for?
I think I will need to rename that; it is currently called to supply tooltips when the user hovers the mouse over a token.
Title: Re: CC plugin interface redesign
Post by: dmoore on October 31, 2013, 07:22:28 pm
Sorry, that might be partially my fault (this is my first time with git).  I think I may have unnecessarily created a master branch in my fork.

No I blame them not you. Makes sense to be able to have multiple branches. But when you git clone something it would be friendlier to noobs like me that git should give a hint that you are getting multiple branches and what the default one will be. Maybe it did and I just didn't see it, but git and its output is pretty hard for mere mortals to parse in general. I was more surprised that github's interface doesn't make it clearer how to clone a specific branch.

Quote
That is how I am intending it to work.  Your plugin can also send cbEVT_SHOW_CALL_TIP, cbEVT_EDITOR_TOOLTIP and cbEVT_COMPLETE_CODE events for specialized situations (that the (hopefully more generic) CCManager does not handle).

So when a user types:
* alphanumerics (e.g. say, "im"), the framework is going to call the plugin's GetAutocompList?
* "(" or ",", the framework will call GetCallTips?
Or is it really up to the plugin to hook into the editor events and post the messages you listed above. I'm wondering how one is supposed to handle the calltip with highlighting of the relevant function arg.

I think I saw some places where there is code that looks like "stc->CallTipShow" / "stc->CallTipHide". I would have expected members to handle these sorts of actions to be part of the plugin or CCManager class.

Quote
Yes.  Returning an empty vector will cancel code completion, and sending the event (once your plugin is prepared) will start it up again.  (Although, you may consider first testing to see if it creates a noticeable lag when done synchronously.)

There will definitely be situations where there is noticeable lag. In one case it took >30s to get gtk's symbols :o. That's kind of an extreme case but I expect up to 1s delays to be pretty common, which rules out blocking approaches.

Quote
Not with the current API.  However, CCManager tries to only request documentation be generated when the user pauses in scrolling through options, so a short freeze here is unlikely to be noticed (at least, that is the theory).

This is probably a case where blocking until the asynch call returns would probably have acceptable delays, since the token list should (usually??) have already been generated.

Quote
I intend CCManager to handle that (however, it is currently hardcoded at 10ms, and synchronous if the user types three characters in a row).

That works for me.

Quote
EDIT: 5. What is GetTokenAt for?
I think I will need to rename that; it is currently called to supply tooltips when the user hovers the mouse over a token.

GetTooltipForTokenAt?


One more Q... How does one set the icon used in the CC popup. Is that the "id"?
Title: Re: CC plugin interface redesign
Post by: Alpha on November 01, 2013, 02:31:12 am
So when a user types:
* alphanumerics (e.g. say, "im"), the framework is going to call the plugin's GetAutocompList?
* "(" or ",", the framework will call GetCallTips?
Yes and yes.  (Details on word length and key characters are still hardcoded.)
Or is it really up to the plugin to hook into the editor events and post the messages you listed above. I'm wondering how one is supposed to handle the calltip with highlighting of the relevant function arg.
CCManager calls GetCallTips() whenever one of m_CallTipChars (currently hardcoded) is typed and when the user presses navigation (up/down/left/right) keys (buffered delay), so the relevant argument *should* always be highlighted.
The only time I found it necessary for the CC plugin to send cbEVT_SHOW_CALL_TIP was in DoAutocomplete() when the token is a function.

I think I saw some places where there is code that looks like "stc->CallTipShow" / "stc->CallTipHide". I would have expected members to handle these sorts of actions to be part of the plugin or CCManager class.
I do not think I understand... example please?

There will definitely be situations where there is noticeable lag. In one case it took >30s to get gtk's symbols :o. That's kind of an extreme case but I expect up to 1s delays to be pretty common, which rules out blocking approaches.
You are right; do not block here.  (I think it should be the plugin's job, and not CCManager, to send an event when it is ready.)
(+1 for Code::Blocks CC devs, the search tree is quite fast.)

One more Q... How does one set the icon used in the CC popup. Is that the "id"?
Code
    const wxString idxStr = F(wxT("\n%d"), PARSER_IMG_PREPROCESSOR);
    for (size_t i = 0; i < macros.size(); ++i)
    {
        if (text.IsEmpty() || macros[i][0] == text[0]) // ignore tokens that start with a different letter
            tokens.push_back(CCToken(wxNOT_FOUND, macros[i] + idxStr));
    }
    stc->ClearRegisteredImages();
    stc->RegisterImage(PARSER_IMG_PREPROCESSOR,
                       m_NativeParser.GetImageList()->GetBitmap(PARSER_IMG_PREPROCESSOR));
Currently poor API for this (change is planned).  You must append the string "\n" + xyzNumber to the token's name.
id - unused by CCManager, passed back to the plugin so that the plugin can use it internally to know which token it represents
displayName - verbose text representation of token
name - minimal text representation of token (not currently used, but in place so that CCManager can change between verbose and minimal tooltip/autocomplete modes)
Title: Re: CC plugin interface redesign
Post by: dmoore on November 02, 2013, 03:10:45 pm
That is how I am intending it to work.  Your plugin can also send cbEVT_SHOW_CALL_TIP, cbEVT_EDITOR_TOOLTIP and cbEVT_COMPLETE_CODE events for specialized situations (that the (hopefully more generic) CCManager does not handle).

Note sure how to do this. Should it be...

Code
            CodeBlocksEvent evt(cbEVT_COMPLETE_CODE);
            Manager::Get()->GetCCManager()->ProcessEvent(evt);

Doesn't work because ProcessEvent member isn't exposed. And I suspect this won't work (and even if it does it's totally unintuitive):

Code
            CodeBlocksEvent evt(cbEVT_COMPLETE_CODE);
            Manager::Get()->GetPluginManager()->NotifyPlugins(evt);


Also, how does one format the html for the doc string?
Title: Re: CC plugin interface redesign
Post by: dmoore on November 02, 2013, 05:35:13 pm
Code
            CodeBlocksEvent evt(cbEVT_COMPLETE_CODE);
            Manager::Get()->GetPluginManager()->NotifyPlugins(evt);

So this works, but it is a little confusing for me to say "NotifyPlugins" when I actually mean "Notify CC Manager (to notify me)"
Title: Re: CC plugin interface redesign
Post by: dmoore on November 02, 2013, 05:49:35 pm
Currently poor API for this (change is planned).  You must append the string "\n" + xyzNumber to the token's name.
id - unused by CCManager, passed back to the plugin so that the plugin can use it internally to know which token it represents
displayName - verbose text representation of token
name - minimal text representation of token (not currently used, but in place so that CCManager can change between verbose and minimal tooltip/autocomplete modes)

So it turns out that the old default STC behavior of appending a ? and a number e.g. "print?1" also works.

btw, is it expected that a plugin will need to register its own images:

Code
    m_pImageList = new wxImageList(16, 16);
    wxBitmap bmp;
    bmp = cbLoadBitmap(prefix + _T("class_folder.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // Module
    bmp = cbLoadBitmap(prefix + _T("class.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // Class
    bmp = cbLoadBitmap(prefix + _T("class_public.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // Class Object
    bmp = cbLoadBitmap(prefix + _T("typedef.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // Type
    bmp = cbLoadBitmap(prefix + _T("var_public.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // Type Instance
    bmp = cbLoadBitmap(prefix + _T("method_public.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // BuiltinFunctionType
    bmp = cbLoadBitmap(prefix + _T("method_protected.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // BuiltinMethodType
    bmp = cbLoadBitmap(prefix + _T("method_protected.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // Method
    bmp = cbLoadBitmap(prefix + _T("method_private.png"), wxBITMAP_TYPE_PNG);
    m_pImageList->Add(bmp); // Function

in OnAttach and this in GetAutocompList:

Code
        control->ClearRegisteredImages();
        for (int i = 0; i < m_pImageList->GetImageCount(); i++)
            control->RegisterImage(i+1,m_pImageList->GetBitmap(i));
        m_state=STATE_NONE;

Or is there some globally defined list of bitmaps that should be used? Or do you have some API planned to handle registering plugin defined images?
Title: Re: CC plugin interface redesign
Post by: dmoore on November 02, 2013, 06:33:20 pm
Ok, now calltips ...

1. what do I put in hlStart, hlEnd and argsPos?

Code
wxStringVec CodeCompletion::GetCallTips(int pos, int style, cbEditor* ed, int& hlStart, int& hlEnd, int& argsPos)

For now I am leaving them untouched which means the active arg isn't highlighted

2. The active calltip is getting automatically hidden after less than second. Any idea why this might be happening?
Title: Re: CC plugin interface redesign
Post by: dmoore on November 02, 2013, 07:44:41 pm
Enough coding for today... screenshot attached. Still need to figure out:
* the issue with call tips hiding
* formatting the doc string
* add links to docs and handlers for them.

If you want to test the code is in svn here: http://developer.berlios.de/svn/?group_id=7745

PythonCodeComletion-cc contains the code that's compatible with your branch. I've hard-coded the links in the unix project file. I haven't tested on windows at all, so the windows project file may compile without tweaks, but no guarantee.

[attachment deleted by admin]
Title: Re: CC plugin interface redesign
Post by: Folco on November 02, 2013, 08:30:52 pm
Oh my god, a documentation calltip like what I always wanted to have ! \o/ Good job :)
Title: Re: CC plugin interface redesign
Post by: dmoore on November 02, 2013, 09:11:40 pm
Oh my god, a documentation calltip like what I always wanted to have ! \o/ Good job :)

Yes. Big props to the good work by p2rkw and alpha!
Title: Re: CC plugin interface redesign
Post by: Alpha on November 02, 2013, 09:17:42 pm
Code
            CodeBlocksEvent evt(cbEVT_COMPLETE_CODE);
            Manager::Get()->GetPluginManager()->NotifyPlugins(evt);
So this works, but it is a little confusing for me to say "NotifyPlugins" when I actually mean "Notify CC Manager (to notify me)"
I would recommend:
Code
        CodeBlocksEvent evt(cbEVT_COMPLETE_CODE);
        Manager::Get()->ProcessEvent(evt);

So it turns out that the old default STC behavior of appending a ? and a number e.g. "print?1" also works.
If that works, then the revision you are compiling against is out of date.  Also, it is unnecessary now; instead assign the number to the (new) category member of CCToken.

btw, is it expected that a plugin will need to register its own images:
Currently you will have to.  I am still thinking about if there is a good way to allow plugins to register (or even share) images through CCManager.

1. what do I put in hlStart, hlEnd and argsPos?
hlStart and hlEnd are: if you appended all of the strings that GetCallTips() returns, what would be the starting and ending indicies of the desired highlighted range.

2. The active calltip is getting automatically hidden after less than second. Any idea why this might be happening?
You must put a location in argsPos which represents the starting position (in the stc) of the arguments, for example the argsPos for when the cursor is anywhere inside GetCallTips() - but not inside GetStyleAt() - would be:
Code
    const wxStringVec& tips = ccPlugin->GetCallTips(pos, stc->GetStyleAt(pos), ed, hlStart, hlEnd, argsPos);
                                                   ^
It is being cancelled immediately because CCManager uses the argsPos variable to determine what calltips are the same, or in the case of wxSCI_INVALID_POSITION (-1) , invalid.

If you want to test the code is in svn here: http://developer.berlios.de/svn/?group_id=7745

PythonCodeComletion-cc contains the code that's compatible with your branch. I've hard-coded the links in the unix project file. I haven't tested on windows at all, so the windows project file may compile without tweaks, but no guarantee.
My apologies, but it will not compile just yet, it seems I have already broken API with you.  I will try to post some patches for yours, when I have time.

Oh my god, a documentation calltip like what I always wanted to have ! \o/ Good job :)
Yes. Big props to the good work by p2rkw and alpha!
My branch has mostly just migrated the code p2rkw wrote.  (Display logic in my branch is slightly different, and *hopefully* acts a little more stable.)
Title: Re: CC plugin interface redesign
Post by: dmoore on November 03, 2013, 01:59:25 am
So it turns out that the old default STC behavior of appending a ? and a number e.g. "print?1" also works.
If that works, then the revision you are compiling against is out of date.  Also, it is unnecessary now; instead assign the number to the (new) category member of CCToken.

But is there any good reason not to support the stc method? The stc control wants the symbol?category syntax so why not just support that?

Quote
btw, is it expected that a plugin will need to register its own images:
Currently you will have to.  I am still thinking about if there is a good way to allow plugins to register (or even share) images through CCManager.

It is probably fine as is, but it won't be obvious for someone using the API for the first time, so include some docs somewhere.

Quote
1. what do I put in hlStart, hlEnd and argsPos?
hlStart and hlEnd are: if you appended all of the strings that GetCallTips() returns, what would be the starting and ending indicies of the desired highlighted range.

Can you give an example of what you mean? The array represents the multiple matching calls or the args?

Quote
2. The active calltip is getting automatically hidden after less than second. Any idea why this might be happening?
You must put a location in argsPos which represents the starting position (in the stc) of the arguments, for example the argsPos for when the cursor is anywhere inside GetCallTips() - but not inside GetStyleAt() - would be:
Code
    const wxStringVec& tips = ccPlugin->GetCallTips(pos, stc->GetStyleAt(pos), ed, hlStart, hlEnd, argsPos);
                                                   ^

So argsPos is always the stc position of the opening brace?

What if the user presses "," but the cursor is not in a function call (e.g. python tuple or list)? Will GetCallTip be called and should the plugin return nothing?

Quote
If you want to test the code is in svn here: http://developer.berlios.de/svn/?group_id=7745

PythonCodeComletion-cc contains the code that's compatible with your branch. I've hard-coded the links in the unix project file. I haven't tested on windows at all, so the windows project file may compile without tweaks, but no guarantee.
My apologies, but it will not compile just yet, it seems I have already broken API with you.  I will try to post some patches for yours, when I have time.

No need to - use the time to create doc strings  :). I should be able to keep up with your changes.
Title: Re: CC plugin interface redesign
Post by: dmoore on November 03, 2013, 02:49:16 am
There was only a small change needed to compile against your changes, but now there are other problems...

Code
PythonCodeCompletion::CCProviderStatus PythonCodeCompletion::GetProviderStatusFor(cbEditor* ed)
{
    Manager::Get()->GetLogManager()->Log(_("PYCC: Provider status check"));
    if (ed->GetLanguage() == ed->GetColourSet()->GetHighlightLanguage(wxSCI_LEX_PYTHON))
        return ccpsActive;
    if (ed->GetControl()->GetLexer()==wxSCI_LEX_PYTHON)
        return ccpsActive;
    Manager::Get()->GetLogManager()->Log(_("PYCC: Provider status check INACTIVE"));
    return ccpsInactive;
}

Neither of those checks return true because it seems that the call occurs before the lexer is set. Also, how will you handle the case where the user changes the highlight language to something else? It looks like your code only checks once for each editor, no? What was wrong with the old mechanism? Surely it wasn't that inefficient to ping a few plugins before each call? (I don't mind you changing as long as it works!)
Title: Re: CC plugin interface redesign
Post by: dmoore on November 03, 2013, 06:29:43 am
alpha: my latest svn rev is now in sync with your branch and now more or less correctly displays call tips with highlighting.
Title: Re: CC plugin interface redesign
Post by: Alpha on November 03, 2013, 05:40:14 pm
So it turns out that the old default STC behavior of appending a ? and a number e.g. "print?1" also works.
If that works, then the revision you are compiling against is out of date.  Also, it is unnecessary now; instead assign the number to the (new) category member of CCToken.
But is there any good reason not to support the stc method? The stc control wants the symbol?category syntax so why not just support that?
The stc method is still supported (except it must be '\n'); if you do not set the category member, CCManager will not attempt to append it, so "symbol\ncategory" will work.  I changed both the separator and type separator from defaults because I was becoming frustrated with strange bugs when a returned token contained one of these.  I selected two characters that should never exist in a token: '\n' and '\r'.
Having category as a member, though, will allow CCManager to more easily do things like provide a standard set of icons, and add an offset to the values (planned ideas, no code yet written).

So argsPos is always the stc position of the opening brace?
That is my intention.

What if the user presses "," but the cursor is not in a function call (e.g. python tuple or list)? Will GetCallTip be called and should the plugin return nothing?
Yes, my thoughts were that: rather than having a ccPlugin check every change to an editor to see if it should take an action, filter it so only the 'interesting' changes notify the plugin(s), and then the plugin can decide if it was a false positive.

No need to - use the time to create doc strings  :). I should be able to keep up with your changes.
::) I was writing just for myself for a while... I guess I need to start on some more documentation though.

Neither of those checks return true because it seems that the call occurs before the lexer is set. Also, how will you handle the case where the user changes the highlight language to something else? It looks like your code only checks once for each editor, no? What was wrong with the old mechanism? Surely it wasn't that inefficient to ping a few plugins before each call? (I don't mind you changing as long as it works!)
Strange... it seems to work with the main CC plugin.  I will look into it.
It will check editors multiple times; it only skips the check if it is on the same editor that was last called.  I did this because GetProviderStatusFor() could potentially do multiple string comparisons, and I did not want to worry about avoiding calling this multiple times in quick succession.  (Also, the structure is there so I can add some more planned logic into each time a check is called on a new editor.)
Title: Re: CC plugin interface redesign
Post by: dmoore on November 03, 2013, 06:25:01 pm
If you want to test the provider stuff in my plugin you need to remove the first line of the provider method which immediately returns "active".

Btw, most of this is working really well! When you are happy with this part of the interface and it is stable, I suggest you merge it back to trunk before you move on to symbol browsing.
Title: Re: CC plugin interface redesign
Post by: dmoore on November 03, 2013, 06:29:56 pm
Re docs, the main thing is to put doc strings in cbplugin.h for the virtual methods that explain the things you have explained to me in this thread. I should be able to help with this.
Title: Re: CC plugin interface redesign
Post by: Alpha on November 07, 2013, 05:27:31 am
If you want to test the provider stuff in my plugin you need to remove the first line of the provider method which immediately returns "active".
Should work now.
Some comments on your plugin; I did add default implementations for DoAutocomplete(), so these can be removed (until such a time as you decide to override them).  Avoid calls to control->GetCurrentPos() on any functions where CCManager passes you a position (although the passed position is sometimes the current position, it is not always).  The main CC plugin (@devs: we might need to rename this plugin at some point, to reduce confusion) checks against GetHighlightLanguage() because wxSCI_LEX_CPP lexer is being multipurposed (http://wiki.codeblocks.org/index.php?title=Languages_supported_by_Code::Blocks_lexers) for about seven other languages, but with Python, a simple ID comparison should be fine.
Code
Index: PythonCodeCompletion.cpp
===================================================================
--- PythonCodeCompletion.cpp (revision 434)
+++ PythonCodeCompletion.cpp (working copy)
@@ -7,7 +7,6 @@
 #include <cbeditor.h>
 #include <cbstyledtextctrl.h>
 #include <editor_hooks.h>
-#include <editorcolourset.h>
 #include <configmanager.h>
 #include <logmanager.h>
 //
@@ -399,10 +398,7 @@
 
 PythonCodeCompletion::CCProviderStatus PythonCodeCompletion::GetProviderStatusFor(cbEditor* ed)
 {
-    return ccpsActive; // WORKAROUND UNTIL ALPHA FIXES THIS
     Manager::Get()->GetLogManager()->Log(_("PYCC: Provider status check"));
-    if (ed->GetLanguage() == ed->GetColourSet()->GetHighlightLanguage(wxSCI_LEX_PYTHON))
-        return ccpsActive;
     if (ed->GetControl()->GetLexer()==wxSCI_LEX_PYTHON)
         return ccpsActive;
     Manager::Get()->GetLogManager()->Log(_("PYCC: Provider status check INACTIVE"));
@@ -529,18 +525,6 @@
     return doc;
 }
 
-/// callbacks for actually autocompleting/writing the token to the editor
-void PythonCodeCompletion::DoAutocomplete(const CCToken& token, cbEditor* ed)
-{
-    return;
-}
-
-void PythonCodeCompletion::DoAutocomplete(const wxString& token, cbEditor* ed)
-{
-    DoAutocomplete(CCToken(-1, token), ed);
-}
-
-
 void PythonCodeCompletion::RequestCompletion(cbStyledTextCtrl *control, int pos, const wxString &filename)
 {
     int line = control->LineFromPosition(pos);
Index: PythonCodeCompletion.h
===================================================================
--- PythonCodeCompletion.h (revision 434)
+++ PythonCodeCompletion.h (working copy)
@@ -121,9 +121,6 @@
         virtual std::vector<CCToken> GetTokenAt(int pos, cbEditor* ed);
         /// dismissPopup is false by default
         virtual wxString OnDocumentationLink(wxHtmlLinkEvent& event, bool& dismissPopup);
-        /// callbacks for actually autocompleting/writing the token to the editor
-        virtual void DoAutocomplete(const CCToken& token, cbEditor* ed);
-        virtual void DoAutocomplete(const wxString& token, cbEditor* ed);
 
     protected:
         /** Any descendent plugin should override this virtual method and
Index: PythonCodeCompletion-unix.cbp
===================================================================
--- PythonCodeCompletion-unix.cbp (revision 434)
+++ PythonCodeCompletion-unix.cbp (working copy)
@@ -23,7 +23,7 @@
  <Option type="3" />
  <Option compiler="gcc" />
  <Option parameters="--debug-log --multiple-instance -ni -v -p debug" />
- <Option host_application="/home/damien/src/codeblocks-cc/src/devel/codeblocks" />
+ <Option host_application="../../codeblocks-cc/src/devel/codeblocks" />
  <Option run_host_application_in_terminal="1" />
  <Compiler>
  <Add option="-ansi" />
Title: Re: CC plugin interface redesign
Post by: dmoore on November 07, 2013, 05:55:08 am
Very nice, thanks. And ooh that abs path  :o feel like such a smurf.

I still need to test out the html links and then give some thought to how to handle symbol browser. (Note that the current categories "Global functions", "Global typedefs", Global variables", Preprocessor symbols", "Global macros" don't fit that well with python)
Title: Re: CC plugin interface redesign
Post by: Alpha on November 07, 2013, 06:13:46 am
By the way, you can 'cheat' a bit to prevent the calltip from flashing as you press left/right arrow keys.  (Edit: Just make sure to process cbEVT_SHOW_CALL_TIP afterwards with either the actual results, or an empty string vector so CCManager knows to cancel it.)
Code
Index: PythonCodeCompletion.cpp
===================================================================
--- PythonCodeCompletion.cpp (revision 434)
+++ PythonCodeCompletion.cpp (working copy)
@@ -511,6 +507,11 @@
         cbStyledTextCtrl *control=ed->GetControl();
         m_state = STATE_CALLTIP_REQUEST;
         RequestCallTip(control,argsStartPos,ed->GetFilename());
+        if (control->CallTipActive()) // Probably ours, so return the last know definition so it does not cancel while we recalculate
+        {
+            sv.push_back(m_ActiveCalltipDef);
+            argsPos = m_argsPos;
+        }
         return sv;
     }
     return sv;

(I think your plugin sometimes returns " " as a calltip; probably it is the plugin's job, not CCManager, to filter that.)
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on November 07, 2013, 10:44:31 am
Alpha: Can I ask you to try to describe your commits in more detail (what and why you're doing in the commit is important), especially commits changing core sdk stuff like cbEditor.
Title: Re: CC plugin interface redesign
Post by: Alpha on November 07, 2013, 02:11:25 pm
Okay, will do.  (That specific commit was to prevent editor hooks from calling plugins before the editor was completely initialized.)  Would you like me to try to edit past commit messages?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on November 07, 2013, 04:08:01 pm
Would you like me to try to edit past commit messages?
Yes, please :)
Title: Re: CC plugin interface redesign
Post by: dmoore on November 07, 2013, 04:40:58 pm
By the way, you can 'cheat' a bit to prevent the calltip from flashing as you press left/right arrow keys.  (Edit: Just make sure to process cbEVT_SHOW_CALL_TIP afterwards with either the actual results, or an empty string vector so CCManager knows to cancel it.)
Code
Index: PythonCodeCompletion.cpp
===================================================================
--- PythonCodeCompletion.cpp (revision 434)
+++ PythonCodeCompletion.cpp (working copy)
@@ -511,6 +507,11 @@
         cbStyledTextCtrl *control=ed->GetControl();
         m_state = STATE_CALLTIP_REQUEST;
         RequestCallTip(control,argsStartPos,ed->GetFilename());
+        if (control->CallTipActive()) // Probably ours, so return the last know definition so it does not cancel while we recalculate
+        {
+            sv.push_back(m_ActiveCalltipDef);
+            argsPos = m_argsPos;
+        }
         return sv;
     }
     return sv;

I will incorporate that. thx.

Quote
(I think your plugin sometimes returns " " as a calltip; probably it is the plugin's job, not CCManager, to filter that.)

Yeah, I noticed the empty calltip would pop up occasionally. Maybe that's what is coming from the jedi completion lib because I don't think my code does this. I will try stripping it and see if that fixes the issue.
Title: Re: CC plugin interface redesign
Post by: frithjofh on November 09, 2013, 03:17:47 pm
hi everybody,

I was following this thread for some time now. and I keep stumbling on the same question:

shouldn't all this code completion stuff be separated in two things?

if I understand right one big block of this is the actual analysis of the code and parsing it into some kind of searchable structure, lets call it the TREE. this is dependent on the language. it is focused on analysis and is somewhat abstract. by all means a very basic and low level work and part of c::b.

the other part is putting this information to use for the developer. here in this special case for code completion. this is rather like using only a tiny part of the underlying information and potential of the TREE.

this separation is illustrated too by the  way there exists the symbols tab inside the manager panel. this has functionally nothing to do with the code completion pop up really...

and there are other plugins that could leverage the information of the TREE: the code refactoring plugin to edit individual symbols, the search plugins, which could use it to search for entities by type (variables, members, functions, classes) instead of only by name, the statistic plugin... and I bet a bunch more of functionalities inside of c::b, plugin or not.

so, should this not better be separated? the TREE as a base part of c::b, parsing code for everybody free to use the information, maybe not even as a plugin at all, and then the actual code completion as a plugin?

regards and sorry in advance if this post is only boring/annoying the devs...

frithjofh
Title: Re: CC plugin interface redesign
Post by: Alpha on November 10, 2013, 04:58:22 pm
shouldn't all this code completion stuff be separated in two things?
On the longest term, that would have the greatest flexibility.  However I currently do not have the skill to create a language independent AST.  Maybe that is something for the next iteration of the CC interface... it is quite a bit more complicated than I can easily visualize.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on November 10, 2013, 08:44:06 pm
Interested people could look at kdevelop's source code, I think they have something like this already implemented.
Title: Re: CC plugin interface redesign
Post by: darmar on January 24, 2014, 06:57:18 pm
Hi,

I am playing with this cc_interface branch currently, and found a feature or a bug, but I am not sure.

Code
 int main ()
{
  int i = my_function(undefined_variable);
}

If I keep the mouse over "undefined_variable", a call-tip for "my_function" will be showed. I think,  the function CodeCompletion::GetTokenAt is called at first, but because this function returns an empty vector, the function CodeCompletion::GetCallTips is called.

Is it a feature? If yes, I should find a workaround for FortranProject plugin.

Title: Re: CC plugin interface redesign
Post by: Alpha on January 25, 2014, 01:25:44 am
Is it a feature? If yes, I should find a workaround for FortranProject plugin.
The intent was for it to be a feature.  Would it work better for you if CCManager only used the calltip fallback if it is whitespace under the mouse?  Or do you want to be able to completely disable it?
Title: Re: CC plugin interface redesign
Post by: darmar on January 25, 2014, 07:03:14 am
May be, it would be better to define GetTokenAt with default parameter something like:

std::vector<CCToken> GetTokenAt(int pos, cbEditor* ed, bool fallbackCalltip = true);


Title: Re: CC plugin interface redesign
Post by: Alpha on January 26, 2014, 01:54:17 am
Control parameter committed.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on February 07, 2014, 01:18:30 am
Alpha: Can you start preparation of the branch for a push in svn?
To do so:
0. add and push a tag for the current pointer of your branch. This will help if someone has commits on top of your branch.
1. pull the master from my repo
2. checkout your branch
3. git rebase master (or the remote/blabla/master)
4. fix all conflicts
5. git push -f yourbranch

Probably it would be good to try to setup your repo for svn dcommit, so you can push the changes yourself, because if I push them, then I'll be the author in the history.
Title: Re: CC plugin interface redesign
Post by: Alpha on February 09, 2014, 06:03:20 am
Alpha: Can you start preparation of the branch for a push in svn? [...]
Okay (might have a few delays; sorry, life can get busy :-\ ).
Do you have some details for svn dcommit?  (So far, I have just been using plain git with my branch.)
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on February 09, 2014, 09:47:54 am
Read this topic: http://forums.codeblocks.org/index.php/topic,16096.45.html somewhere in it you can find how to add your svn info to the repo. I'm not sure you need to rewrite your history in order to make git svn dcommit work.
Title: Re: CC plugin interface redesign
Post by: Alpha on February 13, 2014, 05:54:00 pm
0. add and push a tag for the current pointer of your branch. This will help if someone has commits on top of your branch.
1. pull the master from my repo
2. checkout your branch
3. git rebase master (or the remote/blabla/master)
4. fix all conflicts
I have done up to this step, however, now my local branch has a few pieces of code that are different than before (and I am fairly certain I had correctly resolved all conflicts).  Is it okay to commit a fix/revert of these (incorrect) changes on top of the rebase?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on February 14, 2014, 12:22:52 am
Yes, it is. If you don't this dummy commit you can squash it back again in the commit that has introduced the changes you don't like. This way your history will be clean. :)
Title: Re: CC plugin interface redesign
Post by: Alpha on February 14, 2014, 01:39:16 am
Done resolving conflicts, and committed to my git branch.

The problem of setting up git svn, though, I am unsure how to solve.  Trying to use a git svn command resulted in:
Code
Migrating from a git-svn v1 layout...
Data from a previous version of git-svn exists, but
.git/svn
(required for this version (1.8.3.2) of git-svn) does not exist.
Done migrating from a git-svn v1 layout
and then indefinite hang.  From my understanding of reading http://forums.codeblocks.org/index.php/topic,16096.45.html (http://forums.codeblocks.org/index.php/topic,16096.45.html) (and searching the web), the expected process of using git svn is starting from an svn repository.  Because I started originally from standard git, I have not been able to figure out how to hook into the svn repository.  Suggestions?
Title: Re: CC plugin interface redesign
Post by: ollydbg on February 14, 2014, 04:48:44 am
Done resolving conflicts, and committed to my git branch.

The problem of setting up git svn, though, I am unsure how to solve.  Trying to use a git svn command resulted in:
Code
Migrating from a git-svn v1 layout...
Data from a previous version of git-svn exists, but
.git/svn
(required for this version (1.8.3.2) of git-svn) does not exist.
Done migrating from a git-svn v1 layout
and then indefinite hang.  From my understanding of reading http://forums.codeblocks.org/index.php/topic,16096.45.html (http://forums.codeblocks.org/index.php/topic,16096.45.html) (and searching the web), the expected process of using git svn is starting from an svn repository.  Because I started originally from standard git, I have not been able to figure out how to hook into the svn repository.  Suggestions?
Hi, alpha, I just clone your git https://github.com/alpha0010/codeblocks_sf/tree/cc_interface
I see that all the change is based on the latest C::B svn/git head, great!

I guess that the git you use does not contains any svn information to allow you to do git svn dcommit, so this is what I suggest:
1, simply clone a new git repo(we call it local-git-new-repo) from official C::B SVN, note you should use C::B SVN address you have write access. Since a git svn clone of the whole C::B svn history takes long time, you can have either two ways
a, you can clone a C::B svn starting from a specified svn revision, for example, from rev 9640, so in your local git, you can only have a history about 10 commits. That should be a very fast clone, because git svn command only fetch nearly 10 commits. The important thing is with this git clone, you can use git svn dcommit to commit to C::B svn repo.
b, you can copy a raw git achieve from other one, e.g. jens supply one, see this page Re: Read-only Git, SVN Repo for Code::Blocks (http://forums.codeblocks.org/index.php/topic,16096.msg128123.html#msg128123), when downloaded, you need to some further steps to change the svn information, so that you have a git repo which can commit back to svn, also you have a full svn history.

2, you can use the git format-patch command to generate a serial patches from your https://github.com/alpha0010/codeblocks_sf/tree/cc_interface (we call it local-git-old-repo).

3, you can use git am command to apply the patches to local-git-new-repo.

4, use local-git-new-repo to commit(git svn dcommit) back the changes to C::B official svn repo.

Now, you can see all the changes in local-git-old-repo is now in local-git-new-repo, and further in C::B svn repo.

BTW: you can also arrange your changes by "git rebase -i" (interactive git rebase), you can divide commits, merge commits, edit commit log messages, drop commits, all what you know.  :)


EDIT: look at your previous message, I guess some git svn tool is missing? I don't know, because under Windows, I have no problems using msysgit + tortoisegit, many operations can be mouse click on tortoiisegit, but some special command like "git rebase -i" should be run in msysgit's bash shell to get more functionality due to tortoisegit limitation.



Title: Re: CC plugin interface redesign
Post by: Alpha on February 14, 2014, 06:16:12 am
I guess that the git you use does not contains any svn information to allow you to do git svn dcommit, so this is what I suggest: [...]
Thank you, that sounds like it should work.  I shall test soon.

@all devs: Are we ready for this merge to proceed?

EDIT: look at your previous message, I guess some git svn tool is missing? I don't know, because under Windows, I have no problems using msysgit + tortoisegit, many operations can be mouse click on tortoiisegit, but some special command like "git rebase -i" should be run in msysgit's bash shell to get more functionality due to tortoisegit limitation.
I did install the git-svn package, so I believe my problem had been that my local repository just was not set up to handle svn.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on February 14, 2014, 09:24:37 am
@all devs: Are we ready for this merge to proceed?
I've not tried it yet, so I can't tell if it is stable enough.
Probably we should do one round of night builds from this branch.
Killerbot could you do it?
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on February 14, 2014, 10:22:47 am
http://cmpt.benbmp.org/codeblocks/patches/cc/0001-CC-Add-CCManager-to-the-autotools-build-system.patch
http://cmpt.benbmp.org/codeblocks/patches/cc/0002-CC-noPCH-build-fixes-for-CCManager.patch

Two patches to apply save in a folder and then execute git am path/000*
Title: Re: CC plugin interface redesign
Post by: Alpha on February 16, 2014, 03:25:43 am
I guess that the git you use does not contains any svn information to allow you to do git svn dcommit, so this is what I suggest: [...]
Thank you, that sounds like it should work.  I shall test soon.
It works, thanks.

Probably we should do one round of night builds from this branch.
Okay.

http://cmpt.benbmp.org/codeblocks/patches/cc/0001-CC-Add-CCManager-to-the-autotools-build-system.patch
http://cmpt.benbmp.org/codeblocks/patches/cc/0002-CC-noPCH-build-fixes-for-CCManager.patch

Two patches to apply save in a folder and then execute git am path/000*
Done.
Title: Re: CC plugin interface redesign
Post by: Alpha on February 27, 2014, 03:55:20 am
@dmoore:  I added API that should allow asynchronous documentation popups.  (Sorry I forgot about this for so long.)
Title: Re: CC plugin interface redesign
Post by: dmoore on February 27, 2014, 05:21:52 am
@dmoore:  I added API that should allow asynchronous documentation popups.  (Sorry I forgot about this for so long.)

Thanks. Will try it out soon (maybe this weekend).
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on March 28, 2014, 06:27:36 pm
I have big problem with the autocompletion in rev9744.

The steps are something as:
1. type variable ( myvar. )
2. then press ctrl+space to show the completion list with the members of my class
3. start typing to refine the items
4. choose the non-first in the list
5. C::B always completes with the first one.

Unfortunately I couldn't reproduce this problem in a simple one file project :(

edit: After I've restarted C::B the problem disappeared! It is very annoying and this is not the first time it has happened.
Title: Re: CC plugin interface redesign
Post by: Jenna on March 28, 2014, 06:38:31 pm
Same here, this do not happen if you move to a value with the arrow keys.
So press up and down or vice versa can be used as workaround (until it is fixed).

wx2.8 on Fedora 64 bit.
Title: Re: CC plugin interface redesign
Post by: oBFusCATed on March 28, 2014, 06:45:26 pm
Jens: do you know how to reproduce it in a simple project?
Title: Re: CC plugin interface redesign
Post by: Alpha on March 29, 2014, 04:15:33 pm
I have been trying (unsuccessfully) to reproduce; have you been able to isolate a minimal example yet?
Title: Re: CC plugin interface redesign
Post by: Jenna on March 29, 2014, 04:44:59 pm
I have been trying (unsuccessfully) to reproduce; have you been able to isolate a minimal example yet?
I have been trying (unsuccessfully) to reproduce; have you been able to isolate a minimal example yet?

No, unfortunately not, at the moment it seems to work correctly.
But I found another issue.
Simple to reproduce:
create a console-project with the wizard (c++),
in main type std:: and chose any of the suggestions and I get the function with all parameter definitions as completion.
std::abs expands to (e.g.) std::abs(float __x)
Title: Re: CC plugin interface redesign
Post by: Alpha on March 31, 2014, 12:25:43 am
[...]
create a console-project with the wizard (c++),
in main type std:: and chose any of the suggestions and I get the function with all parameter definitions as completion.
std::abs expands to (e.g.) std::abs(float __x)
That is odd.  For me, it just completes to std::abs(|) (where "|" is the caret) when I am within the body of main().  I only get that behaviour if I am in the global scope and have "Add function arguments' types and names when autocompleted outside function" checked.
What does it look like in your autocomplete listing?  (For me it is: [icon]abs(): _GLIBCXX_CONSTEXPR float .)
Title: Re: CC plugin interface redesign
Post by: Jenna on March 31, 2014, 06:22:29 am
[...]
create a console-project with the wizard (c++),
in main type std:: and chose any of the suggestions and I get the function with all parameter definitions as completion.
std::abs expands to (e.g.) std::abs(float __x)
That is odd.  For me, it just completes to std::abs(|) (where "|" is the caret) when I am within the body of main().  I only get that behaviour if I am in the global scope and have "Add function arguments' types and names when autocompleted outside function" checked.
What does it look like in your autocomplete listing?  (For me it is: [icon]abs(): _GLIBCXX_CONSTEXPR float .)
It's the same here.
But I'm again not able to reproduce it.
Unfortunately I had created my test project in my temp-folder (like most of the time) and it does not longer exist.
I do not recall which header I included yesterday, but it was definitey not cmath directly.
I try to investigate and post here again.

Maybe I had some incomplete lines, so CC thought I was outside a function.
Title: Re: CC plugin interface redesign
Post by: dmoore on April 01, 2014, 10:24:37 pm
hmmm... On my windows machine I seem to be missing right click, Find Declaration/Find Implemention options. Just me?
Title: Re: CC plugin interface redesign
Post by: Alpha on April 02, 2014, 12:12:17 am
It works for me (and I believe I had left that part of the CC plugin untouched).
Maybe test some breakpoints in CodeCompletion::BuildModuleMenu() ?
Title: Re: CC plugin interface redesign
Post by: ollydbg on April 02, 2014, 03:10:45 pm
hmmm... On my windows machine I seem to be missing right click, Find Declaration/Find Implemention options. Just me?
You mean you don't have those entries in the context menu?
I don't have such problem with nightly build rev9744.
Title: Re: CC plugin interface redesign
Post by: ollydbg on April 02, 2014, 04:34:50 pm
[...]
create a console-project with the wizard (c++),
in main type std:: and chose any of the suggestions and I get the function with all parameter definitions as completion.
std::abs expands to (e.g.) std::abs(float __x)
That is odd.  For me, it just completes to std::abs(|) (where "|" is the caret) when I am within the body of main().  I only get that behaviour if I am in the global scope and have "Add function arguments' types and names when autocompleted outside function" checked.
What does it look like in your autocomplete listing?  (For me it is: [icon]abs(): _GLIBCXX_CONSTEXPR float .)
It's the same here.
But I'm again not able to reproduce it.
Unfortunately I had created my test project in my temp-folder (like most of the time) and it does not longer exist.
I do not recall which header I included yesterday, but it was definitey not cmath directly.
I try to investigate and post here again.

Maybe I had some incomplete lines, so CC thought I was outside a function.

I don't have such issue.
I'm using rev 9744 on WinXP. I have the options checked on as Alpha said.
Both auto completion inside a function body or in global namespace works fine.