CC plugin interface redesign

[Alpha]

One of the main blockers against having multiple CC plugins (for example, integrating Clang) is the current state of the sdk.  I would like to (at least begin) work on improving the interface, but first we should brainstorm what changes belong in the sdk, and what belongs in individual CC plugins.

My thoughts so far:
The sdk should:

• query plugins if they are active on a file
• ask the active plugin for updates to a shared CC scope toolbar
• have a unified keyboard shortcut/autolaunch, from which it asks the active plugin for the text to fill the autocomp box
• ask, at the correct time, the plugin for the html content of a documentation popup (if the plugin returns nothing, the popup stays hidden)
• keep track of dwell events, and ask the active plugin for tooltip contents

A CC plugin should:

• manage all parsing
• provide its own specialized refactoring/searching tools
• provide its own symbol explorer

What ideas do others have?
Also, there was some recent effort to remove GUI code from the sdk, however this CC redesign would do the opposite.  Solutions?

[oBFusCATed]

My thoughts so far:
The sdk should:

• query plugins if they are active on a file
• ask the active plugin for updates to a shared CC scope toolbar
• have a unified keyboard shortcut/autolaunch, from which it asks the active plugin for the text to fill the autocomp box
• ask, at the correct time, the plugin for the html content of a documentation popup (if the plugin returns nothing, the popup stays hidden)
• keep track of dwell events, and ask the active plugin for tooltip contents

All these are spot on.

A CC plugin should:

• manage all parsing
• provide its own specialized refactoring/searching tools
• provide its own symbol explorer

1. 100% done in the CC plugin.
2. Some parts might be extracted in the sdk, but this is a future task. First we need to have them
3. The UI should be unified, but it should be filled by the CC plugin. Because if you leave it to plugins
   then will have clang symbol browser, symbol browser, fortran symbol browser, python symbol browser, etc, etc.

One thing you've missed is an API for getting the full symbol at one position in a file.
For example if you select a struct member and want to see it in the debugger tooltip, the debugger plugin needs to
expand the string to instance of the object, so the variable could be evaluated correctly by the debugger.

What ideas do others have?
Also, there was some recent effort to remove GUI code from the sdk, however this CC redesign would do the opposite.  Solutions?

No, they won't. See the cbdebugger_interface.h file. The SDK could be used just as a bridge, between plugins and the main app.

[ollydbg]

...
3. The UI should be unified, but it should be filled by the CC plugin. Because if you leave it to plugins
   then will have clang symbol browser, symbol browser, fortran symbol browser, python symbol browser, etc, etc.
...

Agree.
But if we have only one shared symbol browser tree, it may take times to switch from one tree to another. (building current CC's symbol tree takes several minutes).

One thing you've missed is an API for getting the full symbol at one position in a file.
For example if you select a struct member and want to see it in the debugger tooltip, the debugger plugin needs to
expand the string to instance of the object, so the variable could be evaluated correctly by the debugger.

GDB has the ability to solve/parse the complex statement. Here is the example:

Code

instantA.memberB.memberC();

If you hover on memberB, you can directly pass string "instantA.memberB" to GDB.
If you hover on memberC, you can pass string "instantA.memberB.memberC" to GDB.

I'm OK with all the changes.

[oBFusCATed]

Agree.
But if we have only one shared symbol browser tree, it may take times to switch from one tree to another. (building current CC's symbol tree takes several minutes).

Have I said something about rebuilding?

Code

instantA.memberB.memberC();

If you hover on memberB, you can directly pass string "instantA.memberB" to GDB.
If you hover on memberC, you can pass string "instantA.memberB.memberC" to GDB.

I know gdb can do it, but currently there is no method in SDK to ask CC for the full token.

p.s. Alpha if you're not afraid of using git, we can work on this using my git-svn clone on github

[Alpha]

The SDK could be used just as a bridge, between plugins and the main app.

Okay, I see.

Agree.
But if we have only one shared symbol browser tree, it may take times to switch from one tree to another. (building current CC's symbol tree takes several minutes).

Have I said something about rebuilding?

Having a single tab that various plugins can swap the contents with makes sense.  The question is, when to swap?  Each time the active editor changes?  If there are split editors?  No editor open?  Swap on demand (with a combo box)?

p.s. Alpha if you're not afraid of using git, we can work on this using my git-svn clone on github

I have not used git before, but I am willing to try.  Do you have a quick overview of normal git workflow?

What I want to do is first delete most of the CC plugin, except for the parser.  Then add on/rewrite each feature as the changes to the SDK are finished.

[oBFusCATed]

Having a single tab that various plugins can swap the contents with makes sense.  The question is, when to swap?  Each time the active editor changes?  If there are split editors?  No editor open?  Swap on demand (with a combo box)?

Have I said something about swapping. For me there is no problem to show symbol info from all plugins at once.
You can do some filters and stuff. But I suppose this is a feature that should be done at a later stage.
(btw I don't use symbols browsers, so I don't know the workflows).

I have not used git before, but I am willing to try.  Do you have a quick overview of normal git workflow?

Hm, not that easy.
Probably start from here: http://stackoverflow.com/questions/315911/git-for-beginners-the-definitive-practical-guide
For you the steps would be:
1. clone the master
2. create new branch
3. commit, commit, commit
4. push to your github accout,
5 ask for pull in github, so I can upload your changes to the repo.

I'm not really sure about the 4th step, because I've never done multi-people-workflow in github.

What I want to do is first delete most of the CC plugin, except for the parser.  Then add on/rewrite each feature as the changes to the SDK are finished.

If I were you I won't delete any code, but start extracting parts and making sure everything else still works.
And remember to leave as many as needed // FIXME comments.

[oBFusCATed]

Another task I have in mind is one very simple keyword-like cc plugin that works for unsupported files (php, python, lua, etc).
I have to edit such files from time to time and I feel the need for such CC.

[Folco]

You think to a cc based on the xml files of Scintilla lexers ?

[Josh @ Dreamland]

Hi there; I hope no one minds an "outsider" weighing in.  

From where I'm standing, it looks like you're asking quite a lot of a generic CC plugin. A parser is a very easily isolated data operation, while the symbol explorer sounds pretty UI-heavy, and a lot of the refactoring and search-related functionality is universal regardless of who is doing the parsing. I'm of the opinion that there should be two plugin types, here; a parsing plugin (based on Clang, on GCC-XML, or hand-rolled, or what have you), and then a code-completion plugin on top of that to handle the refactoring and symbol browsing (assuming that should not likewise be broken up).

Clang's API, as well as a thin layer over GCC-XML, would both be able to perform tasks such as enumerating contents of a scope, giving occurrences of an identifier which refer to the same object, enumerating parameters to a given function, resolving an overload, and evaluating/coercing expressions. These are fundamentals of any compiler, and all the tools that a bigger code-completion plugin would need to do code completion, refactoring, method/class extraction, among other helpful features.

If there was a plugin for code completion, and a separate plugin for code parsing, the same refactoring and code completion plugin could be used with any decently capable parser plugin.

One thing you've missed is an API for getting the full symbol at one position in a file.
For example if you select a struct member and want to see it in the debugger tooltip, the debugger plugin needs to
expand the string to instance of the object, so the variable could be evaluated correctly by the debugger.

This is also needed for refactor:rename. If it's even slightly useful in the debugger plugin, that's all the more reason to include this in either API.

Just my thoughts.
Cheers

[oBFusCATed]

From where I'm standing, it looks like you're asking quite a lot of a generic CC plugin. A parser is a very easily isolated data operation, while the symbol explorer sounds pretty UI-heavy, and a lot of the refactoring and search-related functionality is universal regardless of who is doing the parsing. I'm of the opinion that there should be two plugin types, here; a parsing plugin (based on Clang, on GCC-XML, or hand-rolled, or what have you), and then a code-completion plugin on top of that to handle the refactoring and symbol browsing (assuming that should not likewise be broken up).

......

If there was a plugin for code completion, and a separate plugin for code parsing, the same refactoring and code completion plugin could be used with any decently capable parser plugin.

But then what happens if the CC plugin is unloaded and the parser plugin tries to use/access it. The common parts should be in the SDK or in the Main application. It is just simpler to design/implement it this way.
If we have to have plugins for the plugins, then something is wrong.

[Alpha]

Have I said something about swapping.

Of course you did.

3. The UI should be unified, but it should be filled by the CC plugin. Because if you leave it to plugins
   then will have clang symbol browser, symbol browser, fortran symbol browser, python symbol browser, etc, etc,
   and we should swap everything.

Probably start from here: http://stackoverflow.com/questions/315911/git-for-beginners-the-definitive-practical-guide

Thanks; I have read through most of that now.
@devs: I can work with either git or on a branch in the main svn repository; what would be most useful to you?

Hi there; I hope no one minds an "outsider" weighing in. 

In the design stage, my opinion is: the more thoughts, the better.

Another task I have in mind is one very simple keyword-like cc plugin that works for unsupported files (php, python, lua, etc).

You think to a cc based on the xml files of Scintilla lexers ?

Currently CC has a fallback of providing suggestions from the keyword sets in the lexers for non-C/C++.  After the sdk is modified, I would like to extract this code into a simple keyword-CC plugin.  Keyword-CC could also tokenize the active file, and store terms for autocompletion when the word length is longer than, for example, 5 characters.

If we have to have plugins for the plugins, then something is wrong.

Agreed, but some day when another attempt is made a semantic highlighting, we will have to revisit this problem.  (Semantic highlight would need the knowledge of the parser, but should be a separate plugin from CC.)

I will try to post some code later today or tomorrow with a candidate new plugin interface.  (Apologies; my programming is going slowly due to personal life issues.)

[oBFusCATed]

Of course you did.

I meant something totally different here, I think.

Agreed, but some day when another attempt is made a semantic highlighting, we will have to revisit this problem.  (Semantic highlight would need the knowledge of the parser, but should be a separate plugin from CC.)

I'd implement it in the SDK or in the Main app, again.

[Alpha]

Of course you did.

I meant something totally different here, I think.

*facepalm* ... I was joking; look back at your original post, I modified the quote.

[Alpha]

Code

class PLUGIN_EXPORT cbCodeCompletionPlugin : public cbPlugin
{
    public:
        struct FunctionRange
        {
            FunctionRange(const wxString& nm, int startLn, int endLn) : name(nm), startLine(startLn), endLine(endLn) {}

            wxString name;
            int startLine;
            int endLine;
        };

        struct ScopeRange
        {
            ScopeRange(const wxString& nm) : name(nm) {}

            wxString name;
            std::vector<FunctionRange> functions;
        };

        typedef std::vector<ScopeRange> ScopeRangesVec;

        cbCodeCompletionPlugin();

        virtual void CodeComplete(int tokenStartPos, int index, const wxString& token) = 0;

        virtual wxArrayString GetAutocompList(int& tokenStartPos, bool isAutoLaunch) = 0;
        virtual wxArrayString GetAutoLaunchStrings() = 0;
        virtual wxString GetAutocompDoc(int index, const wxString& token) = 0;
        virtual wxString GetDocAt(int pos) = 0;
        virtual wxArrayString GetCallTips(int pos) = 0;
        virtual wxArrayString GetToolTips(int pos) = 0;
        virtual wxString GetFullSymbolAt(int pos) = 0;
        virtual ScopeRangesVec GetScopes() = 0;

        virtual bool IsProviderFor(cbEditor* ed) = 0;
};

Thoughts?

[golgepapaz]

care to elaborate further? what is function range? scope range?
What does GetScopes(), GetFullSymbolAt() do ? What is index?

Some traits classes should be added. Instead of just using simple
string arrays. Don't you want to know what kind of a completion it is
(function, variable,type, macro etc..). What context is it in?  (member access,
function scope, translation unit level,file level  etc.)

Since no filename is mentioned I am thinking one supposed to use
active editor right? This should be more generic and pass the filename
or better yet buffer as well