Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Plugins development => Topic started by: dmoore on October 06, 2012, 06:13:10 pm

Title: Python Code Completion
Post by: dmoore on October 06, 2012, 06:13:10 pm
I've started working on a Code Completion plugin for Python. Sources are here:

Code
svn checkout svn://svn.berlios.de/cbilplugin/trunk/PythonCodeCompletion

Build Instructions for Linux and Windows

The project files (respectively PythonCodeCompletion-unix.cbp and PythonCodeCompletion.cbp) will create a packaged cbPlugin, that you can install from Plugins->Manage Plugins

1. Get the sources, using the command above.
2. Make sure you have codeblocks-dev package installed and jedi (https://jedi.readthedocs.org/en/latest/index.html) (and python, of course)
3. In Code::Blocks, open the project file and build target All
4. You should see PythonCodeCompletion.cbplugin in the root of the project folder, install it from Plugins -> Manage Plugins

Features
The current version supports the features of jedi (https://github.com/davidhalter/jedi) for code assist, providing completion popups and calltips.

Todo
* Better integration with C::B (depends on Alpha's progress with CodeCompletion overhaul)
* Symbol browser
* Configuration options

Screenshots

(http://s17.postimage.org/pr2rqbeaj/cc_1.png) (http://postimage.org/image/pr2rqbeaj/) (http://s17.postimage.org/cbfr0v5sr/cc_2.png) (http://postimage.org/image/cbfr0v5sr/)
How it's done

This plugin uses jedi, a python library, as the engine for the code completion running in a separate process and communicates with it via XMLRPC. Python provides a lot of nice introspection features making it easy to parse python source and extract symbols (even from the builtin libs) that jedi takes advantage of. Jedi improves on the limitations of rope, the library that was being used in an earlier version of this plugin.
Title: Re: Python Code Completion
Post by: dmoore on October 11, 2012, 09:17:24 pm
New:
1. Windows project file (which means you can now build on linux, the linux project file is now named PythonCodeCompletion-unix.cbp)
2. Call tips with call sig and docstrings for standard library functions
3. Now can do the XMLRPC communication with the python completion process over pipe as an alternative to a socket. (Pipe is currently hardcoded as default.)
4. Parsing files outside of the standard lib
5. Understanding symbol from context (i.e. should know module sys only after it is imported, or function f when it is defined)

Coming:
4. Parsing files outside of the standard lib
5. Understanding symbol from context (i.e. should know module sys only after it is imported, or function f when it is defined)

6. Symbol/module browsing
7. Use python specific icons for graphic representation of symbol types (module, class, function, attribute etc.)
8. Configuration options
9. User-defined rules for determining return types that can't be determined using introspection (e.g. return type of numpy.array -> ndarray)
Title: Re: Python Code Completion
Post by: dmoore on October 13, 2012, 05:13:51 am
Update: Now using Rope (http://rope.sourceforge.net) for code completion. I've updated the original post with new build instructions. WARNING: Rope litters your project folders with hidden .ropeproject folders.
Title: Re: Python Code Completion
Post by: ollydbg on October 13, 2012, 01:51:35 pm
Pretty nice!
I see we have C/C++ code completion plugin(in trunk), Fortran code completion plugin, python code completion plugin. Maybe, we need to a common interface for all those plugins. ;)
Title: Re: Python Code Completion
Post by: dmoore on October 13, 2012, 03:54:13 pm
I see we have C/C++ code completion plugin(in trunk), Fortran code completion plugin, python code completion plugin. Maybe, we need to a common interface for all those plugins. ;)

Indeed. :)

For calltips and the code completion lists, not too much needs to change, I think. Code completion for python doesn't appear to get in the way of the C/C++ one and, mostly, vice versa. One thing that does get in the way occasionally are the keyword completions. Does anyone know how I can disable those?

Related to sharing GUI, it would be nice if:

* the shortcut keys (e.g. ctrl + space, ctrl+shift+space) worked for all CC plugins.

* For symbol browsing (which I haven't even begun to implement for python yet), share the symbol tab in the manager pane across plugins, though not sure how feasible that is. How would one separate the symbols for different languages?

*  the goto declaration/implementation stuff had a shared GUI (or at least didn't show inappropriately).
Title: Re: Python Code Completion
Post by: MortenMacFly on October 14, 2012, 07:44:34 am
* the shortcut keys (e.g. ctrl + space, ctrl+shift+space) worked for all CC plugins.
Isn't this the case when all CC plugins call event.Ski()  in their handler accordingly?

* For symbol browsing (which I haven't even begun to implement for python yet), share the symbol tab in the manager pane across plugins, though not sure how feasible that is. How would one separate the symbols for different languages?
I would prefer having a separate UI for the languages, like done with Fortran. Because for mixed language projects (i.e. Fortran is often mixed with C/C++) this is the only way to go.

*  the goto declaration/implementation stuff had a shared GUI (or at least didn't show inappropriately).
I don't get it: Did it have a shared UI (when/where is it it gone?!) or do you wish it had one?!
Title: Re: Python Code Completion
Post by: dmoore on October 14, 2012, 04:31:45 pm
* the shortcut keys (e.g. ctrl + space, ctrl+shift+space) worked for all CC plugins.
Isn't this the case when all CC plugins call event.Ski()  in their handler accordingly?

No because the menu items are provided by the Code Completion plugin and not the SDK

see plugins/codecompletion/codecompletion.cpp(635)
Code
            m_EditMenu->Append(idMenuCodeComplete, _("Complete code\tCtrl-Space"));

        m_EditMenu->Append(idMenuShowCallTip, _("Show call tip\tCtrl-Shift-Space"));

Quote
I would prefer having a separate UI for the languages, like done with Fortran. Because for mixed language projects (i.e. Fortran is often mixed with C/C++) this is the only way to go.

Well I think it would be useful to at least have them share the symbols tab.

Quote
*  the goto declaration/implementation stuff had a shared GUI (or at least didn't show inappropriately).
I don't get it: Did it have a shared UI (when/where is it it gone?!) or do you wish it had one?!

You missed the "it would be nice if..." that prefaced those bullets. So no it doesn't have a shared UI and never did because the CC plugin currently provides those menu options. It would be better if the SDK did and called each plugins' handler. Also, can we switch off the goto declaration/implementation options in CC if the highlight language is not in some predefined set (e.g. C/C++/D/Fortran?).

see plugins/codecompletion/codecompletion.cpp(644)
Code
    pos = menuBar->FindMenu(_("Sea&rch"));
    if (pos != wxNOT_FOUND)
    {
        m_SearchMenu = menuBar->GetMenu(pos);
        m_SearchMenu->Append(idMenuGotoFunction,       _("Goto function...\tCtrl-Alt-G"));
        m_SearchMenu->Append(idMenuGotoPrevFunction,   _("Goto previous function\tCtrl-PgUp"));
        m_SearchMenu->Append(idMenuGotoNextFunction,   _("Goto next function\tCtrl-PgDn"));
        m_SearchMenu->Append(idMenuGotoDeclaration,    _("Goto declaration\tCtrl-Shift-."));
        m_SearchMenu->Append(idMenuGotoImplementation, _("Goto implementation\tCtrl-."));
        m_SearchMenu->Append(idMenuFindReferences,     _("Find references\tAlt-."));
        m_SearchMenu->Append(idMenuOpenIncludeFile,    _("Open include file\tCtrl-Alt-."));
    }
Title: Re: Python Code Completion
Post by: MortenMacFly on October 14, 2012, 04:39:28 pm
No because the menu items are provided by the Code Completion plugin and not the SDK
Understood and makes sense.

Well I think it would be useful to at least have them share the symbols tab.
But how would you differ symbols with the same name coming from different CC's? And also, syncing the symbols tab with the user's selection and re-freshing it becomes an issues, especially if it is thread based (as it is now). I think having a symbol tab for each language is not too much and makes it way easier to handle. Maybe a CC plugin can "request" such a tab from the SDK.

So no it doesn't have a shared UI and never did because the CC plugin currently provides those menu options. [...]
Well - so altogether this sounds like a similar concept as for the debugger, isn't it? Sharing the dialogs via the SDK and having the plugins to specialise it. It would be unique (I've never heard of an IDE having such) but sounds good.
Title: Re: Python Code Completion
Post by: oBFusCATed on October 15, 2012, 05:31:23 pm
But how would you differ symbols with the same name coming from different CC's? And also, syncing the symbols tab with the user's selection and re-freshing it becomes an issues, especially if it is thread based (as it is now). I think having a symbol tab for each language is not too much and makes it way easier to handle. Maybe a CC plugin can "request" such a tab from the SDK.
You'll have a combobox with the current language showed by the symbol browser. Changing the value in the combo could change the UI - ask the plugin to build the correct UI.
Title: Re: Python Code Completion
Post by: dmoore on October 16, 2012, 04:29:58 am
You'll have a combobox with the current language showed by the symbol browser. Changing the value in the combo could change the UI - ask the plugin to build the correct UI.

Yes, I think this would work well. Perhaps the combo could be switched automatically to whatever type of source the user has open in the active editor.
Title: Re: Python Code Completion
Post by: MortenMacFly on October 16, 2012, 07:10:46 am
You'll have a combobox with the current language showed by the symbol browser. Changing the value in the combo could change the UI - ask the plugin to build the correct UI.
I am still not really convinced. If you have large projects with a lot of CC members, building the UI every time you switch really takes time and can be avoided if you have just two symbol browser that both would need no update when switching between them. You can see this now already with the C++ tree, if you enable "workspace level".

Have you ever worked with the Fortran project? There, you have another symbol browser which works just fine besides the C/C++ one. This usage is really convenient. And in fact if you have multiple language projects (which is very common at least for C/C++ and Fortran) you would switch pretty often. For Fortran, I am pretty happy with how it is.

...just my 2 cents...
Title: Re: Python Code Completion
Post by: oBFusCATed on October 16, 2012, 09:20:35 am
...If you have large projects with a lot of CC members, building the UI every time you switch really takes time...
When switching the language the only operations that will happen will be oldPanel->Hide() and newPanel->Show(). The plugins will keep all panels up to date and full with the correct data.
Title: Re: Python Code Completion
Post by: MortenMacFly on October 16, 2012, 10:23:16 am
When switching the language the only operations that will happen will be oldPanel->Hide() and newPanel->Show().
True, but it wouldn't allow you to have i.e. both symbol trees under each other, which may be desired. If that's possible, too I am fine with it. Remember, that in mixed-code project you switch rather often from one to another language. So seeing i.e. all relevant methods of your project together is helpful for navigation.
Title: Re: Python Code Completion
Post by: oBFusCATed on October 16, 2012, 10:33:18 am
We can provide a way to have any number of symbol browser tabs, so the user can have some control.
Title: Re: Python Code Completion
Post by: dmoore on October 18, 2012, 02:52:12 pm
It would be easy enough to develop prototypes with different approaches to handling the UI (all in one tab, across multiple tabs etc)

What's bothering me more at the moment is that CC still tries to offer suggestions for python if the python completion returns nothing. This is quite annoying as is the find declaration/implementation stuff. Would anyone mind if I did some lexer checks in the  CC plugin to not show tips/options for python files? What happens with Fortran files?
Title: Re: Python Code Completion
Post by: oBFusCATed on October 18, 2012, 03:38:40 pm
Yes, this is probably the first step.
Title: Re: Python Code Completion
Post by: MortenMacFly on October 18, 2012, 04:33:41 pm
What happens with Fortran files?
I recall a chat I had with the Fortran plugin maintainer. Maybe you should ask him first, before implementing such. IIRC he solved it somehow and provided a patch. Sore there might be an API already in place for that.
Title: Re: Python Code Completion
Post by: dmoore on October 18, 2012, 05:58:50 pm
What happens with Fortran files?
I recall a chat I had with the Fortran plugin maintainer. Maybe you should ask him first, before implementing such. IIRC he solved it somehow and provided a patch. Sore there might be an API already in place for that.

Did we accept a patch for this or does he just maintain patches for the repackaged C::B he releases?
Title: Re: Python Code Completion
Post by: MortenMacFly on October 18, 2012, 08:11:38 pm
Did we accept a patch for this or does he just maintain patches for the repackaged C::B he releases?
IIRC he sent me something. But honestly I don't recall what I did with it because it conflicted with the re-factoring of the SmartIdent feature. Its best to ask him again. I cannot have a closer look atm...
Title: Re: Python Code Completion
Post by: dmoore on October 18, 2012, 10:06:51 pm
That's fine I'll follow up with him later. I see he registers handlers for cbEVT_COMPLETE_CODE and cbEVT_SHOW_CALL_TIP which the CC plugin (and not the SDK!) post messages to. I have now added the same handlers in the python code completion plugin.

Attached is a patch to get rid of some of the noise from the CC plugin on python and fortran files, which is really just a sequence of lexer checks scattered throughout the code like this:

Code
            //WORKAROUND FOR PYTHON/FORTRAN
            cbStyledTextCtrl *control = ed->GetControl();
            if (control->GetLexer() == wxSCI_LEX_PYTHON || control->GetLexer() == wxSCI_LEX_F77 || control->GetLexer() == wxSCI_LEX_FORTRAN)
                return;

Realize this is a bit of a hack, but it will do the job until we refactor parts of CC to the SDK.
Title: Re: Python Code Completion
Post by: MortenMacFly on October 19, 2012, 06:48:20 am
Realize this is a bit of a hack, but it will do the job until we refactor parts of CC to the SDK.
No problem - I'll try. What makes me wonder is that this is needed in so many places?! :o
Title: Re: Python Code Completion
Post by: MortenMacFly on October 19, 2012, 06:59:10 am
...going through all my email again, I found the patch of the Fortran guy again. NOtice that its indeed better to query filegroups and masks, because of customisation of the file extensions that are rather common in the Fortran world.

Code
Index: codecompletion.cpp
===================================================================
--- codecompletion.cpp (revision 7604)
+++ codecompletion.cpp (working copy)
@@ -57,6 +57,7 @@
 #include "parser/tokenizer.h"
 #include "selectincludefile.h"
 #include "ccdebuginfo.h"
+#include "filegroupsandmasks.h"
 
 #define CC_CODECOMPLETION_DEBUG_OUTPUT 0
 
@@ -809,6 +810,12 @@
 
     if (type == mtEditorManager)
     {
+        if (cbEditor* ed = Manager::Get()->GetEditorManager()->GetBuiltinActiveEditor())
+        {
+            if (IsFortranFile(ed->GetShortName()))
+                return;
+        }
+
         wxString NameUnderCursor;
         bool IsInclude = false;
         const bool nameUnderCursor = EditorHasNameUnderCursor(NameUnderCursor, IsInclude);
@@ -930,6 +937,21 @@
 
     m_NativeParser.CreateClassBrowser();
 
+    // determine Fortran Source file group index
+    m_FortranGroupIdx = -1;
+    const FilesGroupsAndMasks* fgam = Manager::Get()->GetProjectManager()->GetFilesGroupsAndMasks();
+    if (fgam)
+    {
+        for (unsigned int i = 0; i < fgam->GetGroupsCount(); i++)
+        {
+            if (fgam->GetGroupName(i).Lower().Contains(_T("fortran")))
+            {
+                m_FortranGroupIdx = i;
+                break;
+            }
+        }
+    }
+
     // hook to editors
     EditorHooks::HookFunctorBase* myhook = new EditorHooks::HookFunctor<CodeCompletion>(this, &CodeCompletion::EditorEventHook);
     m_EditorHookId = EditorHooks::RegisterHook(myhook);
@@ -1505,6 +1527,9 @@
     if (!ed)
         return;
 
+    if (IsFortranFile(ed->GetShortName()))
+        return;
+
     // calculate the size of the calltips window
     int pos = ed->GetControl()->GetCurrentPos();
     wxPoint p = ed->GetControl()->PointFromPosition(pos); // relative point
@@ -1745,6 +1770,9 @@
     if (!ed)
         return;
 
+    if (IsFortranFile(ed->GetShortName()))
+        return;
+
     cbStyledTextCtrl* control = ed->GetControl();
     const int pos = control->GetCurrentPos();
     const int style = control->GetStyleAt(pos);
@@ -2593,6 +2621,12 @@
             return;
         }
 
+        if (IsFortranFile(ed->GetShortName()))
+        {
+            event.Skip();
+            return;
+        }
+
         if (ed->GetControl()->CallTipActive())
             ed->GetControl()->CallTipCancel();
 //        CCLogger::Get()->DebugLog(F(_T("CodeCompletion::OnValueTooltip: %p"), ed));
@@ -3174,6 +3208,12 @@
         return;
     }
 
+    if (IsFortranFile(editor->GetShortName()))
+    {
+        event.Skip();
+        return;
+    }
+
     cbStyledTextCtrl* control = editor->GetControl();
 
 //    if (event.GetEventType() == wxEVT_SCI_CHARADDED)
@@ -3493,3 +3533,19 @@
     if (m_NativeParser.ReparseFile(project, m_LastFile))
         CCLogger::Get()->DebugLog(_T("Reparsing when typing for editor ") + m_LastFile);
 }
+
+bool CodeCompletion::IsFortranFile(wxString fileName)
+{
+    bool isFort = false;
+    if (m_FortranGroupIdx != -1)
+    {
+        const FilesGroupsAndMasks* fgam = Manager::Get()->GetProjectManager()->GetFilesGroupsAndMasks();
+        if (fgam)
+        {
+            if ( fileName.Find('.',true) != wxNOT_FOUND &&
+                fgam->MatchesMask(fileName.AfterLast('.').Prepend(_T(".")), m_FortranGroupIdx) )
+                isFort = true;
+        }
+    }
+    return isFort;
+}
Index: codecompletion.h
===================================================================
--- codecompletion.h (revision 7604)
+++ codecompletion.h (working copy)
@@ -235,6 +235,9 @@
     /** delayed running of editor activated event, only the last activated editor should be considered*/
     void OnEditorActivatedTimer(wxTimerEvent& event);
 
+    /** determines if file is Fortran file */
+    bool IsFortranFile(wxString fileName);
+
     /** Not used*/
     int                     m_PageIndex;
     /** Indicates CC's initialization is done*/
@@ -329,6 +332,9 @@
     typedef std::map<cbProject*, wxArrayString> ReparsingMap;
     ReparsingMap m_ReparsingMap;
 
+    /** index of Fortran Source file group */
+    int m_FortranGroupIdx;
+
     DECLARE_EVENT_TABLE()
 };
 
Title: Re: Python Code Completion
Post by: oBFusCATed on October 19, 2012, 09:16:20 am
This patch looks rather ugly. C++ CC must not care about other file types.
The check should be something like If file is C/C++ process it otherwise do nothing.
Title: Re: Python Code Completion
Post by: MortenMacFly on October 19, 2012, 10:04:21 am
The check should be something like If file is C/C++ process it otherwise do nothing.
I didn't mean to apply it, I was showing it for reference. In the end in your patch, don't you check for other "filetypes" than C/C+, too? What I meant is not only to trust the editor style - this might be wrong.
Title: Re: Python Code Completion
Post by: oBFusCATed on October 19, 2012, 10:11:30 am
What I meant is not only to trust the editor style - this might be wrong.
Do we have a function we could trust?
Title: Re: Python Code Completion
Post by: MortenMacFly on October 19, 2012, 11:34:14 am
What I meant is not only to trust the editor style - this might be wrong.
Do we have a function we could trust?
Not atm.

But better than either only checking for the editor style or only checking for the file types and mask is probably doing both in a proper way.
Title: Re: Python Code Completion
Post by: dmoore on October 19, 2012, 02:24:59 pm
For the UI I thought it would be more reliable to trust the Lexer:

* C++ std headers have no extension
* Python scripts on unix are often missing an extension (they do usually have shebang and so could be inferred)
* As Rick has mentioned to me in the past, CC can be used for C like languages, hence we dont want to just restrict CC to C/C++
* The user can switch the highlight mode of a file. When they do i think their intent should be respected by CC.
* Note that i dont think either patch affects what the parser parses, just the files it offers a UI for. CC has a UI to control the parser i think.

So how about a more refined version. Each plugin has public member bool IsProviderFor(cbEditor *ed). That can do whatever checks that plugin prefers. For CC itself, since it is fallback, its method would be

Code
For each CC plugin instance except me:
    if Plugin.IsProviderFor(ed):
        return false
return true
Title: Re: Python Code Completion
Post by: MortenMacFly on October 19, 2012, 02:48:25 pm
For CC itself, since it is fallback, its method would be

Code
For each CC plugin instance except me:
    if Plugin.IsProviderFor(ed):
        return false
return true

Yes, I think it should be something like that. Similar to what we actually do with plugins:
if Plugin.OffersFor(CC, Debugger...) [Do something.]
It can / has to be handled with CC plugins, too.

BTW: We should also consider several CC plugins for one language, like Clang and Ctags - if one fails, the other can be queried as fallback. While working on the integration of Erans CC into C::B I was facing such. So in the code above, not only return "false", but do some more intelligent stuff, like "if generally responsible, do CC and return "true", but if the not enough information to complete, return false and let others try"... You get the idea.
Title: Re: Python Code Completion
Post by: dmoore on October 19, 2012, 03:04:53 pm
BTW: We should also consider several CC plugins for one language, like Clang and Ctags - if one fails, the other can be queried as fallback. While working on the integration of Erans CC into C::B I was facing such. So in the code above, not only return "false", but do some more intelligent stuff, like "if generally responsible, do CC and return "true", but if the not enough information to complete, return false and let others try"... You get the idea.

I think that will need to wait until we refactor stuff into the SDK. Right now CC provides a couple of UI bits that all plugins use (call tip and completion) but plugins must provide their own symbol tree, right click menu options etc. Defining more complicated fallback behavior as you are proposing above will require more extensive reworking.
Title: Re: Python Code Completion
Post by: dmoore on October 24, 2012, 03:36:41 am
Here's a revised patch to try. I've made the needed change to the fortran plugin as well.

(EDIT: Patch revised again, thx OllyDbg)

Code
Index: src/plugins/codecompletion/codecompletion.cpp
===================================================================
--- src/plugins/codecompletion/codecompletion.cpp (revision 8464)
+++ src/plugins/codecompletion/codecompletion.cpp (working copy)
@@ -727,8 +727,12 @@
             if (   ParserCommon::FileType(filename) == ParserCommon::ftOther
                 && Manager::Get()->GetPluginManager()->IsFileExtRegistered(filename) )
                 return;
+
+            if(!IsProviderFor(ed))
+                return;
         }
 
+
         wxString NameUnderCursor;
         bool IsInclude = false;
         const bool nameUnderCursor = CodeCompletionHelper::EditorHasNameUnderCursor(NameUnderCursor, IsInclude);
@@ -825,6 +829,16 @@
     return true;
 }
 
+bool CodeCompletion::IsProviderFor(cbEditor *ed)
+{
+    PluginsArray pa=Manager::Get()->GetPluginManager()->GetCodeCompletionOffers();
+    for(unsigned int i = 0; i < pa.Count(); ++i)
+        if(pa[i]!=this && static_cast<cbCodeCompletionPlugin*>(pa[i])->IsProviderFor(ed))
+            return false;
+    return true;
+}
+
+
 int CodeCompletion::CodeComplete()
 {
     if (!IsAttached() || !m_InitDone)
@@ -835,6 +849,9 @@
     if (!ed)
         return -3;
 
+    if(!IsProviderFor(ed))
+        return -3;
+
     FileType ft = FileTypeOf(ed->GetShortName());
     const bool caseSens = m_NativeParser.GetParser().Options().caseSensitive;
 
@@ -1015,6 +1032,9 @@
     if (!ed)
         return;
 
+    if(!IsProviderFor(ed))
+        return;
+
     wxString filename = ed->GetShortName();
     if (   ParserCommon::FileType(filename) == ParserCommon::ftOther
         && Manager::Get()->GetPluginManager()->IsFileExtRegistered(filename) )
@@ -1342,6 +1362,9 @@
         return;
     }
 
+    if(!IsProviderFor(editor))
+        return;
+
     cbStyledTextCtrl* control = editor->GetControl();
 
     if      (event.GetEventType() == wxEVT_SCI_CHARADDED)
Index: src/plugins/codecompletion/codecompletion.h
===================================================================
--- src/plugins/codecompletion/codecompletion.h (revision 8464)
+++ src/plugins/codecompletion/codecompletion.h (working copy)
@@ -89,6 +89,7 @@
     virtual wxArrayString GetCallTips() { return wxArrayString(); }
     virtual int CodeComplete();
     virtual void ShowCallTip();
+    virtual bool IsProviderFor(cbEditor *ed);
 
     /** give auto suggestions on preprocessor directives*/
     void CodeCompletePreprocessor();
Index: src/sdk/cbplugin.cpp
===================================================================
--- src/sdk/cbplugin.cpp (revision 8464)
+++ src/sdk/cbplugin.cpp (working copy)
@@ -1216,6 +1216,7 @@
     return -1;
 }
 
+
 //ToDo: Is this c++ only?
 int cbSmartIndentPlugin::GetFirstBraceInLine(cbStyledTextCtrl* stc, int string_style)const
 {
Index: src/include/cbplugin.h
===================================================================
--- src/include/cbplugin.h (revision 8464)
+++ src/include/cbplugin.h (working copy)
@@ -743,6 +743,24 @@
         virtual wxArrayString GetCallTips() = 0;
         virtual int CodeComplete() = 0;
         virtual void ShowCallTip() = 0;
+        /** @brief Does this plugin handle code completion for the editor cb?
+          *
+          * A plugin should override this function to indicate whether it will
+          * provide completion and call tips for the editor. The plugin should
+          * then prepare to handle codecomplete and calltip menu messages if
+          * it returns true. To implement this function, plugins will usually
+          * check the mimetype of the file or the current lexer (highlight
+          * language).
+          *
+          * Note: Currently the core CC plugin provides a default CodeCompletion
+          * implementation for any file type that is not provided for by any
+          * other CC plugins. The calltip and main menu options that can be handled
+          * by any CC plugin is also supplied by the core CC plugin.
+          *
+          * @param cb The editor for which code completion
+          * @return return true if the plugin handles completion for this editor,
+          * false otherwise*/
+        virtual bool IsProviderFor(cbEditor* /*cb*/) { return false; }
 };
 
 /** @brief Base class for wizard plugins
@@ -857,6 +875,7 @@
 
         /** Get the last non-whitespace character from position in line */
         wxChar GetNextNonWhitespaceCharOfLine(cbStyledTextCtrl* stc, int position = -1, int *pos = 0)const;
+
         bool AutoIndentEnabled()const;
         bool SmartIndentEnabled()const;
         bool BraceSmartIndentEnabled()const;
Index: src/plugins/contrib/FortranProject/fortranproject.cpp
===================================================================
--- src/plugins/contrib/FortranProject/fortranproject.cpp (revision 27)
+++ src/plugins/contrib/FortranProject/fortranproject.cpp (working copy)
@@ -1226,6 +1226,16 @@
     return -5;
 }
 
+bool FortranProject::IsProviderFor(cbEditor *cb)
+{
+    if (!ed
+        || !m_pNativeParser->IsFileFortran(ed->GetShortName())
+        || !Manager::Get()->GetConfigManager(_T("fortran_project"))->ReadBool(_T("/use_code_completion"), true))
+    {
+        return false;
+    }
+    return true;
+}
+
 void FortranProject::ShowCallTip()
 {
     if (!IsAttached() || !m_InitDone)
Index: src/plugins/contrib/FortranProject/fortranproject.h
===================================================================
--- src/plugins/contrib/FortranProject/fortranproject.h (revision 27)
+++ src/plugins/contrib/FortranProject/fortranproject.h (working copy)
@@ -46,6 +46,7 @@
         virtual wxArrayString GetCallTips();
         virtual int CodeComplete();
         virtual void ShowCallTip();
+        virtual bool IsProviderFor(cbEditor *cb);
         void OnCodeComplete(wxCommandEvent& event);
         void OnShowCallTip(wxCommandEvent& event);
         void CompleteCodeEvt(CodeBlocksEvent& event);
Title: Re: Python Code Completion
Post by: ollydbg on October 24, 2012, 04:45:00 am
Code
+bool FortranProject::IsProviderFor(cbEditor *cb)
+{
+    if (!ed
+        || !m_pNativeParser->IsFileFortran(ed->GetShortName())
+        || !Manager::Get()->GetConfigManager(_T("fortran_project"))->ReadBool(_T("/use_code_completion"), true))
+    {
+        return;
+    }
+}
+
return true or false?
Title: Re: Python Code Completion
Post by: dmoore on October 24, 2012, 05:31:53 am
oops  ;D

false

and return true outside the block.
Title: Re: Python Code Completion
Post by: MortenMacFly on October 29, 2012, 11:33:55 am
Here's a revised patch to try. I've made the needed change to the fortran plugin as well.
This patch does not apply for me. :-\ Furthermore, it won't work with Fortran as you provide a cbEditor* cb via the interface and then use "ed->" internally.

However, after applying this patch manually and adjusting the tiny issues I am now testing...
Title: Re: Python Code Completion
Post by: dmoore on October 29, 2012, 01:41:53 pm
However, after applying this patch manually and adjusting the tiny issues I am now testing...

As should now be clear I didn't test the FORTRAN bits, because I have never used it and didn't have a FORTRAN project handy. (I did test analogous patch for the python code completer which seemed to be doing the right thing)
Title: Re: Python Code Completion
Post by: MortenMacFly on October 29, 2012, 03:59:00 pm
As should now be clear I didn't test the FORTRAN bits
Maybe darmar can do some trials, too. He might have something to contribute... I'll send him a PM.
Title: Re: Python Code Completion
Post by: darmar on October 30, 2012, 09:50:21 am
Here's a revised patch to try. I've made the needed change to the fortran plugin as well.

I would like to explain how the same thing is functioning now (without this patch).

Code
Index: src/plugins/codecompletion/codecompletion.cpp
===================================================================
--- src/plugins/codecompletion/codecompletion.cpp (revision 8464)
@@ -727,8 +727,12 @@
             if (   ParserCommon::FileType(filename) == ParserCommon::ftOther
                 && Manager::Get()->GetPluginManager()->IsFileExtRegistered(filename) )
                 return;

The idea is the same as in dmoore's patch. The function "IsFileExtRegistered" is doing similar thing as "IsProviderFor(ed)" in this patch.
Every CC plugin should register file extensions it like to handle. In FortranProject plugin the corresponding code is:

Code
void FortranProject::RegisterFileExtensions()
{
    PluginManager* plugman = Manager::Get()->GetPluginManager();
    StringSet fileExts;
    m_pNativeParser->GetFortranFileExts(fileExts);
    plugman->RegisterCCFileExts(_T("FortranProject"), fileExts);
}


So, Python CC should register file extensions it like to handle with "plugman->RegisterCCFileExts(_T("PythonCC"), fileExts);".

If developers prefer to apply this patch, it is OK from FortranProject plugin's point of view. You just need to delete Rev 7922 patch (in this patch the required changes were made in C::B API and CC plugin).

Title: Re: Python Code Completion
Post by: danselmi on October 30, 2012, 11:41:52 am
Aren't these the same extensions as used for Syntax highlighting? (configured in Settings->Editor->SyntaxHighlighting->Filemasks)
Title: Re: Python Code Completion
Post by: MortenMacFly on October 30, 2012, 01:12:57 pm
Aren't these the same extensions as used for Syntax highlighting? (configured in Settings->Editor->SyntaxHighlighting->Filemasks)
Yes, ...so what? Id that bad? ???

The only issue I see here is that oe can change the nbame of the file group. Like I did for example with "Sources" which I renamed to "C/C++ Sources" and "Fortran Sources" to group these files. As the categories can change this may break functionality, see for example:
FileType FileTypeOf(const wxString& filename) in globals.cpp
Here I had to change:
Code
                    if (fgm->GetGroupName(i) == _T("Sources") && fgm->MatchesMask(ext, i))
                        return ftSource;
                    if (fgm->GetGroupName(i) == _T("Headers") && fgm->MatchesMask(ext, i))
                        return ftHeader;
...to:
Code
                    if (fgm->GetGroupName(i).Contains(_T("Source")) && fgm->MatchesMask(ext, i))
                        return ftSource;
                    if (fgm->GetGroupName(i).Contains(_T("Header")) && fgm->MatchesMask(ext, i))
                        return ftHeader;
(Notice the "Contains").

To use this properly we should introduce constant identifiers for file groups and readable names only as an alias, which may be extendible by plugins (using a GetNewFileGroupID() method or alike).
Title: Re: Python Code Completion
Post by: danselmi on October 30, 2012, 01:51:42 pm
Aren't these the same extensions as used for Syntax highlighting? (configured in Settings->Editor->SyntaxHighlighting->Filemasks)
Yes, ...so what? Id that bad? ???
So I have to configure the same thing twice? Once for syntax highlighting and a second time for CC?
Title: Re: Python Code Completion
Post by: darmar on October 30, 2012, 02:02:10 pm
Aren't these the same extensions as used for Syntax highlighting? (configured in Settings->Editor->SyntaxHighlighting->Filemasks)

Not exactly.
It is because CodeCompletion plugin wants to make code-completion not only for C/C++, but for other languages similar to C++ too.

The logic in CC is:
if (file is C/C++ or other language but without specific code-completion plugin):
   make code-completion.

Maybe it would be possible to register language instead of file extensions.
Title: Re: Python Code Completion
Post by: dmoore on October 30, 2012, 05:22:20 pm
Code
Index: src/plugins/codecompletion/codecompletion.cpp
===================================================================
--- src/plugins/codecompletion/codecompletion.cpp (revision 8464)
@@ -727,8 +727,12 @@
             if (   ParserCommon::FileType(filename) == ParserCommon::ftOther
                 && Manager::Get()->GetPluginManager()->IsFileExtRegistered(filename) )
                 return;

The idea is the same as in dmoore's patch. The function "IsFileExtRegistered" is doing similar thing as "IsProviderFor(ed)" in this patch.

I am a little confused by this. Is it only the plugins other than CC that register files? So only IsFileExtRegistered returns true only for Fortran files if only CC and fortran plugins are installed?

For python, I thought it made the most sense to rely on the lexer to decide what to offer python code completion for, but basing it on file extensions might make the most sense in other cases. The patch I proposed leaves it to the plugin to decide what it wants to do.
Title: Re: Python Code Completion
Post by: darmar on October 31, 2012, 12:13:36 pm
Is it only the plugins other than CC that register files? So only IsFileExtRegistered returns true only for Fortran files if only CC and fortran plugins are installed?

I think, currently only FortranProject plugin registers the file extensions for  IsFileExtRegistered and, consequently, it returns 'true' only for Fortran files (if FortranProject plugin is active).

Quote
For python, I thought it made the most sense to rely on the lexer to decide what to offer python code completion for, but basing it on file extensions might make the most sense in other cases.

The editor's highlighting (lexer) is based on file extensions anyway.

Quote
The patch I proposed leaves it to the plugin to decide what it wants to do.

The same situation is in the current implementation: every CC plugin (FortranProject, PythonCC etc.) decides which files (recognized by file extension) it wants to handle and registers the file extensions accordingly.

I don't argue, that the current C::B implementation is better. If developers think that the new patch is better (simpler, shorter, easier to understand etc.), it is OK.
Title: Re: Python Code Completion
Post by: MortenMacFly on October 31, 2012, 12:57:20 pm
I think in all cases, we can summarise:
- CC can be based on either file extension or lexer type (cbEditor)
- for file extensions we have a file extension registry, that might be mis-leading
- for CC, we have an own file extension registry, that must not be compatible with the global file extension registry
- the file extension registry is not error-free, as it does not offer constant ID's for groups of files

- solution 1 (dmoore) provides an cbEditor to the plugin to have the pugin decide what to do with it
- solution 2 (darmar) provides a file extension to the plugin to have the pugin decide what to do with it and plugins can/have to register file extemsions they care for

- solution 1 is implemented via patch, not yet fully tested
- solution 2 works already

Drawbacks I see for both are (for example): STL header files. Those files do not have no file extension and no lexer style.

However, a CC plugin might know by the file's name it if can be interpreted. cbEditor provides both: The (Scintilla) editor and the file name as backup. So unless we have use cases were we have no cbEditor (do we???) this seems like a more generic interface. (One case I can think of is querying for a file's function w/o having it opened.)

Maybe a combination of both approaches is the best?
Title: Re: Python Code Completion
Post by: dmoore on October 31, 2012, 01:38:10 pm
Morten: agree with what you wrote above. Note that my patch is only attempting to solve the problem of what plugin should offer it's GUI when the user is editing a file. A separate issue is what files the plugin should parse to extract symbol information. E.g. for CC it is all project files and everything #included. In that case you can't rely on having an editor available. But I still think it is useful to have the option to use the language info of the editor for offering the UI if for no other reason than that the user can manually change it (this comes up a lot when editing python scripts that don't have an extension, but that could be solved with sniffing I guess)
Title: Re: Python Code Completion
Post by: darmar on November 03, 2012, 05:17:05 pm
I have tested dmoore's patch. It works fine with some changes.
My comments:
1) If plugin cares for code-completion list and call-tips, it should care for tool-tips too. Therefore the call to 'IsProviderFor(ed)' should be added in 'OnEditorTooltip' function too.
2) The call to 'IsProviderFor(ed)' was added in 'CodeComplete()' function. I think, this call should be moved near to the start of function 'DoCodeComplete()', because 'DoCodeComplete()' cares for making the code-completion list for the C/C++ preprocessor ('#...') and then calls 'CodeComplete()' (no other function calls 'CodeComplete()'). In my opinion, CodeCompletion plugin should suggest CC list for preprocessor for C/C++ only (e.g. '#' is start of comment in Python).

I have attached a patch for codecompletion.cpp. I have deleted calls to 'IsFileExtRegistered' in this patch also.
Title: Re: Python Code Completion
Post by: MortenMacFly on November 07, 2012, 11:55:25 am
I have attached a patch for codecompletion.cpp. I have deleted calls to 'IsFileExtRegistered' in this patch also.
So - this seems pretty much complete and does what I should do. @dmoore: are you happy with his, too? It looks fine here, after a few hours of testing... Looks like the "least common denominator" we were looking for... ;-)
Title: Re: Python Code Completion
Post by: dmoore on November 07, 2012, 01:11:03 pm
I haven't had time to look, but everything darmar said sounds good to me, so commit away unless you want me to test.
Title: Re: Python Code Completion
Post by: MortenMacFly on November 07, 2012, 01:32:01 pm
I haven't had time to look, but everything darmar said sounds good to me, so commit away unless you want me to test.
Committed with credits. Whats missing are the adjustments to the Fortran project plugin - darmar will take case I guess... ;D
Title: Re: Python Code Completion
Post by: dmoore on November 09, 2012, 10:47:21 pm
Now working nicely with PythonCodeCompletion plugin.
Title: Re: Python Code Completion
Post by: MortenMacFly on November 10, 2012, 10:09:12 am
Now working nicely with PythonCodeCompletion plugin.
Well done then. :-)
Title: Re: Python Code Completion
Post by: darmar on November 24, 2012, 10:00:27 am
The functions "RegisterCCFileExts" and "IsFileExtRegistered" are left still in "src/sdk/pluginmanager.cpp". They are unused any more and can be deleted, I think.
Title: Re: Python Code Completion
Post by: MortenMacFly on November 24, 2012, 12:11:09 pm
They are unused any more and can be deleted, I think.
Done in trunk. Thank you!
Title: Re: Python Code Completion
Post by: dmoore on October 29, 2013, 04:54:11 pm
Update: Now using the jedi (https://jedi.readthedocs.org/en/latest/index.html) library to complete python code. Works a little better than rope and is more actively developed.
Title: Re: Python Code Completion
Post by: oBFusCATed on October 29, 2013, 05:56:49 pm
dmoore: Do you follow Alpha's changes to the CC in this branch https://github.com/alpha0010/codeblocks_sf/compare/cc_interface?
Title: Re: Python Code Completion
Post by: dmoore on October 29, 2013, 06:14:30 pm
dmoore: Do you follow Alpha's changes to the CC in this branch https://github.com/alpha0010/codeblocks_sf/compare/cc_interface?

Not closely (i.e. I read the occasional post in the related thread), but I might soon. Is there enough ready to be able to do anything with it yet? Testing out his work with Python would obviously be a good test.
Title: Re: Python Code Completion
Post by: Alpha on October 30, 2013, 12:52:38 am
Is there enough ready to be able to do anything with it yet? Testing out his work with Python would obviously be a good test.
I think so.  API for auto-complete popups, documentation popups, tooltips, and calltips are present and (mostly) functional.  These subsystems still contain some hardcoded defaults, but other than that, I believe it is ready for a test with Python.  (Be prepared for uncaught bugs and frequent API breaks though ::).)
Title: Re: Python Code Completion
Post by: dmoore on October 30, 2013, 03:33:35 am
I think so.  API for auto-complete popups, documentation popups, tooltips, and calltips are present and (mostly) functional.  These subsystems still contain some hardcoded defaults, but other than that, I believe it is ready for a test with Python.  (Be prepared for uncaught bugs and frequent API breaks though ::).)

I will take a look soon (if for no other reason than I want to use the docstring UI rather than roll my own). Some hints (e.g. doc strings) on how to use the plugin virtuals (i.e. what the plugin is supposed to do with each one) and what events are available and need to be used would be extremely helpful! Looks like a fairly extensive overhaul of the existing interface. (Well done!)