Author Topic: Allowing Plugin Interdependency and Improving Plugin Management  (Read 27366 times)

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Allowing Plugin Interdependency and Improving Plugin Management
« on: November 10, 2013, 06:53:32 pm »
Making a new thread based on the discussion started here.

The main ideas:
1. Only load plugin shared library objects for plugins that are enabled
2. Allow plugins to interact with each other. e.g. a compiler add-on plugin might use methods provided by the compiler plugin.

Attached is a patch for #1, which is a work in progress. PluginManager does some stuff with member variables being shared across functions that make it prone to bugs when making changes and, ideally, that should be fixed as part of any overhaul.

I am working on part 2.

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #1 on: November 10, 2013, 07:33:14 pm »
Patch updated to handle safemode

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #2 on: November 10, 2013, 08:08:45 pm »
Some additional thoughts on plugin dependencies

At least on Linux It looks like it is possible to have a plugin compile against another plugin and use its plugin instance without too much trouble if the SOs/DLLs will be in the same place. See the attached plugin that is made to depend on the Compiler plugin (you will need to change some of the hardcoded paths in "Build Options" -> "Search Directories").

Just need to include the relevant plugin headers
Code
#include <compilergcc.h>

and trivial usage:
Code
void Dep_Plugin::OnAttach()
{
    cbPlugin *plug = Manager::Get()->GetPluginManager()->FindPluginByName(_T("Compiler"));
    if (plug)
    {
        cbCompilerPlugin *cplug = dynamic_cast<cbCompilerPlugin*>(plug);
        bool attached = cplug->IsAttached();
        Manager::Get()->GetLogManager()->Log(wxString::Format(_T("Dep plugin found compiler plugin instance %i"),attached));
    }
}

So I think that the only changes to the SDK that are needed:
1. make sure that our dependent plugins are loaded after the plugins they depend on
2. don't load plugins if the plugins they depend on aren't loaded
3. unload plugins if the plugins they depend on are disabled/removed


[attachment deleted by admin]

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #3 on: November 10, 2013, 09:26:52 pm »
At least on Linux It looks like it is possible to have a plugin compile against another plugin and use its plugin instance....
This is because on linux we use -fvisibility=default and this will change in the future, when I have some free time to play with this.
On windows the default is -fvisibility=hidden, so this won't work anyway.

Just need to include the relevant plugin headers
Code
#include <compilergcc.h>
This is bloody dangerous. Nothing in this header is intended for external plugin access.
If you want to access code from other plugins, you'll have to define a clear interface which can be guaranteed to be relatively stable.

I'm not sure also that it is a good idea to allow plugins to depend on other plugins
and I think if something should be accessed by more plugins it is better to provide a SDK API for it.
This way we have a bit clearer interface. But this is just me.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #4 on: November 10, 2013, 09:28:18 pm »
On a separate topic:
Have you measured the time needed to load C::B with and without your patch.
Currently C::B loads pretty slow if all contrib plugins are installed, but disabled.
Does you patch improve the load time?
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #5 on: November 10, 2013, 09:52:32 pm »
On a separate topic:
Have you measured the time needed to load C::B with and without your patch.
Currently C::B loads pretty slow if all contrib plugins are installed, but disabled.
Does you patch improve the load time?

Some, C::B still has to read all of the manifests, but no DLLs for disabled plugins get loaded. I will do some timings when I have time.

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #6 on: November 10, 2013, 11:55:11 pm »
At least on Linux It looks like it is possible to have a plugin compile against another plugin and use its plugin instance....
This is because on linux we use -fvisibility=default and this will change in the future, when I have some free time to play with this.
On windows the default is -fvisibility=hidden, so this won't work anyway.

That's obviously a choice you can make per plugin. But what are you proposing as the alternative? The way wxSmith does it? i.e. A separate lib for common stuff that all plugins import. That sucks for deployment because what I found was that the common lib needs to be in the same place as libcodeblocks (the way we do it on windows) or in one of the system lib paths (unix). So much for manage plugins -> install new...

Quote
Just need to include the relevant plugin headers
Code
#include <compilergcc.h>
This is bloody dangerous. Nothing in this header is intended for external plugin access.
If you want to access code from other plugins, you'll have to define a clear interface which can be guaranteed to be relatively stable.

Sure, my intention was only to figure out what's possible now NOT how it should work. I don't think it would be terribly complicated expose the relevant parts of the compiler plugin to another plugin.

Quote
I'm not sure also that it is a good idea to allow plugins to depend on other plugins

Not what you said in the other thread...  ???

Quote
and I think if something should be accessed by more plugins it is better to provide a SDK API for it.

If you mean adding to the C::B SDK, that just bloats C::B unnecessarily e.g Most of the devs couldn't give a shit about python support, but there are a bunch of common things in my python plugins (e.g. XMLRPC stuff) that I would like all of my plugins to be able to use. No reason to have this in the C::B SDK, but I would still like some method of deploying this shared library using our existing plugin management mechanisms

Another example is Tools+. Initially I had the part that provides that notebook tabs built as a separate library so that more than one plugin could define windows that would be displayed there (e.g. output from shell commands, a python interpreter). There is no reason this necessarily needs to be part of the SDK if it isn't being used.

Curious to hear what others think. I won't spend any more time on this if enough of you think this is a dead end.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #7 on: November 11, 2013, 12:15:54 am »
Not what you said in the other thread...  ???
I'm not sure what are you referring here. I've just said that if we need to have one plugin depend on another, then doing this as part of the plugin manager is the right way to do it.


... but I would still like some method of deploying this shared library using our existing plugin management mechanisms
One way is using static libraries. With -fvisibility=hidden plus some other useful options the linker is able to remove the unused code.
Another option is using -rpath, but I'm not sure if it is available on windows or osx.

Another example is Tools+.
I've always felt that the tools+ should have been written over the original code and thus the old code removed. And this is the reason I'm not using the Tools+.
C::B has one very serious problem - it has tons of features that look like each other: tools/tools+, build-in search/threadsearch/incremental search.
We should try to do a better job in this regard - copy the good parts from all implementation and remove the bad once or at least extracting built-in tools and built-in search in plugins.

Curious to hear what others think. I won't spend any more time on this if enough of you think this is a dead end.
As I've said in the other topic: if you need this feature go on and implement it. Handling dependencies automatically is the only proper way to do this if you need it. Playing with the weights and so on is just a hack.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline beqroson

  • Multiple posting newcomer
  • *
  • Posts: 63
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #8 on: November 11, 2013, 12:17:09 am »
Curious to hear what others think. I won't spend any more time on this if enough of you think this is a dead end.

I dont know much about the inner workings of Codeblocks. But it sounds useful to me. If one plugin does a few things on disc for example, like scanning a lot of header files. Instead of another plugin doing the same thing, it could get the result from the first plugin. If two plugins need the same mechanism that would not use amounts of resources, however, it would be better if each plugin manages on its own. Also, it would be necessary to select such behavior that does not really change over longer periods of time. If the interface of the plugin would change or the format of the data, then the interdependency could break too often for it to be usable. The best thing would be that the interdependency actually never breaks, which would demand that the plugin kept backwards compatible of its inner workings. Compare with DLL-hell.

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #9 on: November 11, 2013, 12:31:40 am »
On a separate topic:
Have you measured the time needed to load C::B with and without your patch.
Currently C::B loads pretty slow if all contrib plugins are installed, but disabled.
Does you patch improve the load time?

Some, C::B still has to read all of the manifests, but no DLLs for disabled plugins get loaded. I will do some timings when I have time.

Some timings on an i386 laptop

Results for safemode
Code
time codeblocks --safe-mode -ns -ni --multiple-instance (quitting immediately after)
before: ~10.0s first time*, ~5.8s subsequently
after: ~2.8s first time*, ~2.6s subsequently

* NOT from cold start (so don't trust these numbers)

Results with only core plugins enabled
Code
time codeblocks -ns -ni --multiple-instance (quitting immediately after)
before: ~8.1s**
after: ~5.3s**

** based on second and subsequent runs

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #10 on: November 11, 2013, 12:50:50 am »
Not what you said in the other thread...  ???
I'm not sure what are you referring here. I've just said that if we need to have one plugin depend on another, then doing this as part of the plugin manager is the right way to do it.

But how do you avoid linking one plugin against some subset of another?

Quote
... but I would still like some method of deploying this shared library using our existing plugin management mechanisms
One way is using static libraries. With -fvisibility=hidden plus some other useful options the linker is able to remove the unused code.
Another option is using -rpath, but I'm not sure if it is available on windows or osx.

Not sure what you mean, but probably because I don't sufficiently understand shared object linking. Suppose we have plugin A, plugin B and a library C that they both depend on. If I statically link A and C, how do I make sure B can find the stuff in C?

Quote
Another example is Tools+.
I've always felt that the tools+ should have been written over the original code and thus the old code removed. And this is the reason I'm not using the Tools+.
C::B has one very serious problem - it has tons of features that look like each other: tools/tools+, build-in search/threadsearch/incremental search.
We should try to do a better job in this regard - copy the good parts from all implementation and remove the bad once or at least extracting built-in tools and built-in search in plugins.

Agree, it's just a bit more work to make sure Tools+ properly duplicates the old features (and clean up the UI) then remove the old stuff from C::B core. (There are historical reasons why it is the way it is, but they don't matter here.) But even if we do that, to allow other plugins to use Tools+ features means putting some of that stuff in the SDK or allowing for some way for plugins to depend on one another.

Quote
Curious to hear what others think. I won't spend any more time on this if enough of you think this is a dead end.
As I've said in the other topic: if you need this feature go on and implement it. Handling dependencies automatically is the only proper way to do this if you need it. Playing with the weights and so on is just a hack.

I wasn't proposing weights, I was proposing an explicit dependency chain be specified in the manifest (i.e. a "dependency" attribute with a comma separated list of the unique names of the plugins that it depends on). I don't think you can get by with just trying to load the lib and praying it works.
« Last Edit: November 11, 2013, 12:54:04 am by dmoore »

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #11 on: November 11, 2013, 02:40:25 am »
Not sure what you mean, but probably because I don't sufficiently understand shared object linking. Suppose we have plugin A, plugin B and a library C that they both depend on. If I statically link A and C, how do I make sure B can find the stuff in C?
A static lib is just an archive with object files, so if you link both A and B to C (which is static lib), then both A and B will have all the code in C.
A and B will have their own copy of the code, nothing will be shared and they will have no external dependency to C. They will be self-sufficient.

In this case the data structures for A and B will be separate/duplicated in memory.

p.s. about the weights: I've mentioned it because it is another simpler approach to handle plugin deps.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #12 on: November 11, 2013, 05:52:13 am »
In this case the data structures for A and B will be separate/duplicated in memory.

That's a non-solution.

p.s. about the weights: I've mentioned it because it is another simpler approach to handle plugin deps.

But fundamentally broken. if you disable a plugin with weight >X then you would have to unload all plugins with weight < X to be sure you haven't just crippled (and made unstable) a dependent plugin.

Your philosophical objection to linking plugins together doesn't make a lot of sense to me. It sort of sounds like you are trying to sandbox plugins, but when plugins can link against any lib in the system it isn't really much of a sandbox. And it's not like our plugins expose the user to any more risk than C::B itself -- even scripts have access to wxExecute. So what am I missing? That it's just cleaner to link against a common lib (a la wxSmith) than linking plugins together?

A real problem is the technical one i.e. the need to use -rpath (or, on windows, AddDllDirectory). The former only lets you set a link directory relative to $ORIGIN, which would make it impossible to put objects that a plugin depends on in users home.

I guess I can live with using the approach we use for wxSmith and putting all of the shared stuff in a separate lib in the same dir as libcodeblocks (or some other dir that we can -rpath to). It makes a deployment a bit more of a pain unless the core plugin is part of contrib because we don't have a mechanism for installing the shared libs from a cbplugin, which, at least on linux, requires admin rights. But maybe that's what I/we should work on?

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #13 on: November 11, 2013, 09:09:43 am »
Hm, I think I'm being misunderstood. I don't like the idea of having plugins for plugins, I'm relatively ok with the idea of plugin communications.
So, lets stop arguing and just do it if you need it.

Have you though of putting all of your python subplugins in a single binary? I think this is possible already in C::B and probably will be easier for the user. :)
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline frithjofh

  • Regular
  • ***
  • Posts: 376
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #14 on: November 11, 2013, 02:46:36 pm »
maybe not plugins for plugins.

maybe some base functionality inside c::b like services. these provide interface and API. the work internally with plugins that implement and parametrize the service.

then, outside, plugins that use the services through their public interface.

that way the plugins wont be depending on each other. only outside plugins on services.

maybe some functionalities of c::b are already so general, that actually no one ever unloads the plugin representing that functionality. maybe some things have been plugins for so long that nobody questions if they really should be plugins... or at least plugins directly exposed to the user. defintely if one looks at the plugins today, there are already two groups of them: those that one virtually HAS to use in order to do anything with an IDE, and then all the others...

for instance an IDE would likely always offer some compiler. well, that compiler would be a service, the service to be able to compile something. inside the service there would be plugins to specialize the service, for instance being of type gcc or intel compiler, local or distant, etc.

AST would be a general service specialized by plugin for each language. it would be used outside by plugins like CC or refactoring or searching or statistics or symbol browser or whatever.

just thinking. most probably this would be so great a change it is not worth while. not to repeat the netscape experience... by all means.
architect with some spare time  -  c::b compiled from last svn  -   openSuSE leap x86_64  -  AMD FX-4100

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #15 on: November 11, 2013, 06:38:46 pm »
Hm, I think I'm being misunderstood. I don't like the idea of having plugins for plugins, I'm relatively ok with the idea of plugin communications.
So, lets stop arguing and just do it if you need it.

Yes, clearly we are talking past each other. :) The reason I haven't just "done it" is I am still not sure of the best way to proceed.

The bottom line is that I want to scratch two immediate itches:
1. I had at one time developed a Python Interpreter "console" that would appear in the tools plus dockable and I want to get it working again, but now that Tools+ is part of contrib this is a pain. If I moved the relevant bits of Tools+ into the core, then I just need to expose these bits in the SDK API. Alternatively, I need to figure out some other way for the python interpreter to use that notepage (either (a) link against Tools+ itself or (b) have Tools+ and the Python Interpreter links against a common shared lib a la wxSmith).
2. The rest of my python programs uses some XMLRPC stuff that I want to share across plugins. The common bits take care of all the messy process management and threaded communication using a manager class that there really should only be one of. So again, I either need a shared library approach, a link against plugins approach, or I could do the multiple plugins in a single binary as you propose below.

Quote
Have you though of putting all of your python subplugins in a single binary? I think this is possible already in C::B and probably will be easier for the user. :)

This isn't a horrible idea. BUT... Do any of our current plugins do this already? I ask because as far as I can tell the PluginManager code doesn't really handle all of the nuances involved in this. e.g. has anyone tried to uninstall or export one of these plugins?
My patch will need to be modified to handle this case as well. e.g. I unload after detaching a plugin, but I should only unload after the last plugin in the bundle has been disabled.

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #16 on: November 11, 2013, 06:56:25 pm »
(note: I've only read the first 5 posts)

I find the idea great in general, but it would need to be done with thought and care. Even if it turns out to be really easy, please spend another month or so thinking about it. The reason is that we can't change the way plugins work every few weeks (or we'll get abuse both from plugin writers and from users who have broken setups), so whatever we do, it will be "The Way Plugins Are" for the next couple of months/years. Think twice, then think again. Other than that, great plan.

As for linking one DLL (a plugin is just a DLL after all) against another DLL under Windows, this is something I've done many times in the past, without doing anything special (I heard of the visibility flag for the fist time too  :P). Are you sure it's problematic? I think it might just work fine.
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #17 on: November 11, 2013, 07:25:15 pm »
As for linking one DLL (a plugin is just a DLL after all) against another DLL under Windows, this is something I've done many times in the past, without doing anything special (I heard of the visibility flag for the fist time too  :P). Are you sure it's problematic? I think it might just work fine.
dmoore's problem is that he doesn't want to place his so/dll file next to libcodeblocks.so/dll, but this is a requirement at the moment (I've not checked if this is really the case, but I suppose it is).

The visibility flag is something unrelated to this problem. What it will do is to change the defaults to be a bit more sane, aka windows like. This is something that will happen in the future, when I find the time to do it.

dmoore: I've never tried to place two plugins in a single binary, but I've thought it is supported... but I might be wrong.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #18 on: November 12, 2013, 03:05:25 pm »
please spend another month or so thinking about it.

Roger that.

Quote
As for linking one DLL (a plugin is just a DLL after all) against another DLL under Windows, this is something I've done many times in the past, without doing anything special (I heard of the visibility flag for the fist time too  :P). Are you sure it's problematic? I think it might just work fine.

On windows it should be straightforward because we can add the plugin dirs (user and builtin) to the library path at runtime. On linux,  We can use -rpath to add say /usr/lib/codeblocks/plugins, but I haven't figured out how to add the plugin dir in the users home. It's almost as if the GCC devs think that's a security risk  ;D


Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #19 on: November 12, 2013, 04:22:18 pm »
It's almost as if the GCC devs think that's a security risk  ;D
GCC devs have nothing to do here. Its the dynamic linker devs (glibc) that think this way.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #20 on: November 12, 2013, 04:38:00 pm »
The visibility flag is something unrelated to this problem. What it will do is to change the defaults to be a bit more sane, aka windows like. This is something that will happen in the future, when I find the time to do it.

Maybe this makes my first patch mostly redundant. If using visibility radically improves link time then it doesn't really matter that all libs get loaded.

Quote
dmoore: I've never tried to place two plugins in a single binary, but I've thought it is supported... but I might be wrong.

I think it "kinda" works, but I am not sure about how the manifest naming works and I'm pretty sure that export and uninstall won't do the right thing (i.e. export or uninstall everything in the same dll/so)
« Last Edit: November 12, 2013, 04:44:02 pm by dmoore »

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #21 on: November 12, 2013, 04:50:30 pm »
Maybe this makes my first patch mostly redundant. If using visibility radically improves link time then it doesn't really matter that all libs get loaded.
I don't think it is related to the load time. The only benefit might be smaller size plugins and of course no more symbol collisions (not really serious problem in c::b).
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #22 on: November 12, 2013, 05:20:48 pm »
As for linking one DLL (a plugin is just a DLL after all) against another DLL under Windows, this is something I've done many times in the past, without doing anything special (I heard of the visibility flag for the fist time too  :P). Are you sure it's problematic? I think it might just work fine.
dmoore's problem is that he doesn't want to place his so/dll file next to libcodeblocks.so/dll, but this is a requirement at the moment (I've not checked if this is really the case, but I suppose it is).

Strictly speaking, the so/dll can be placed anywhere that's in the loader's search path. On windows we can change this path at run time, so no big deal. On Linux, -rpath/-rpath-link adds some flexibility, but it means we can never have plugins that are depended on in users home (becasue we don't know where home is at compile time). Maybe that's a limitation we can live with?

Offline Alpha

  • Developer
  • Lives here!
  • *****
  • Posts: 1513
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #23 on: November 12, 2013, 05:36:51 pm »
On Linux, -rpath/-rpath-link adds some flexibility, but it means we can never have plugins that are depended on in users home (becasue we don't know where home is at compile time). Maybe that's a limitation we can live with?
I found the following here.  It is not elegant, but it might work if needed.
Quote
On UNIXish sytem there is a notion of TMPDIR - if needed, you can first find all the needed libraries, then symlink them to a directory under TMPDIR, then use the the full path with that directory.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #24 on: November 12, 2013, 06:39:48 pm »
I found the following here.  It is not elegant, but it might work if needed.
Hm, are you really proposing this solution? :)
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #25 on: January 23, 2014, 10:43:11 pm »
Small update on this. Apparently on some linuxes (e.g. I think YES Fedora, but I think NO for Ubuntu) ~/.local/lib is in the library search path for each user. So this would make it possible to use a cbplugin package deploy a lib shared by multiple plugins into the user space (i.e. no root/admin permissions required). So plugins could still be installed to ~/.codeblocks/share/... but common libs would need to go in ~/.local/lib in order to be found.

As a side note, according to the free desktop standard, C::B should really be using ~/.config and ~/.local for settings and plugin files, so maybe plugins should go somewhere like ~/.local/lib/codeblocks (see http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html)

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #26 on: January 30, 2016, 07:26:03 pm »
I've just re-read the topic.
And I really think we should get at least the first part of the patch in trunk.
Does it change anything in the plugins (manifest/api/abi)? If no then it should be relatively safe to apply it. Hopefully it will improve the load time of cb, which is pretty bad at the moment.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #27 on: January 30, 2016, 08:31:47 pm »
And I really think we should get at least the first part of the patch in trunk.
Probably, but please not the comment in this line:
Code: diff
+    cbPlugin* plugin = FindPluginByName(basename); // THIS DOESN'T WORK RIGHT...
So its not ready yet... Furthermore it should be carefully tested. E.g. is it still possible to enable/disable/enable a plugin.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #28 on: February 01, 2016, 02:13:45 pm »
It needs to be changed to handle enabling and disabling multiplugin bundles. Off the top of my head, I cannot recall what the issue was with the DOESN'T WORK RIGHT comment.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5918
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #29 on: February 16, 2016, 02:34:39 am »
I've just re-read the topic.
And I really think we should get at least the first part of the patch in trunk.
Does it change anything in the plugins (manifest/api/abi)? If no then it should be relatively safe to apply it. Hopefully it will improve the load time of cb, which is pretty bad at the moment.
I agree.

It needs to be changed to handle enabling and disabling multiplugin bundles. Off the top of my head, I cannot recall what the issue was with the DOESN'T WORK RIGHT comment.
I just apply the patch, and I can enable or disable multiply plugins without any issue. :)
EDIT: I just forget to rebuild the SDK target, sorry, so this is not tested yet, I'm just rebuilding the whole codeblocks_w30.cbp.
EDIT2: After the rebuild, and just tested again, and see no issue here when I enable or disable multiply plugins.
« Last Edit: February 16, 2016, 04:11:10 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5918
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #30 on: February 16, 2016, 06:26:56 am »
On my test, install a bundle(such as a file named ProjectsImporter-1.1.cbplugin which I exported from C::B) failed in function bool PluginManager::InstallPlugin(const wxString& pluginName, bool forAllUsers, bool askForConfirmation)
Code
    // extract plugin from bundle
    if (!ExtractFile(actualName,
                    localName,
                    pluginFilename))
        return false;
Not sure the reason.

EDIT: uninstall an unenabled plugin also failed without any dialog or message box.
« Last Edit: February 16, 2016, 06:31:28 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5918
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #31 on: February 16, 2016, 06:37:53 am »
On my test, install a bundle(such as a file named ProjectsImporter-1.1.cbplugin which I exported from C::B) failed in function bool PluginManager::InstallPlugin(const wxString& pluginName, bool forAllUsers, bool askForConfirmation)
Code
    // extract plugin from bundle
    if (!ExtractFile(actualName,
                    localName,
                    pluginFilename))
        return false;
Not sure the reason.

EDIT: uninstall an unenabled plugin also failed without any dialog or message box.

For an disabled plugin.
Code
void PluginsConfigurationDlg::OnUninstall(cb_unused wxCommandEvent& event)
{
    wxListCtrl* list = XRCCTRL(*this, "lstPlugins", wxListCtrl);
    if (list->GetSelectedItemCount() == 0)
        return;

    wxBusyCursor busy;

    long sel = -1;
    wxString failure;
    while (true)
    {
        sel = list->GetNextItem(sel, wxLIST_NEXT_ALL, wxLIST_STATE_SELECTED);
        if (sel == -1)
            break;

        PluginElement* elem = (PluginElement*)list->GetItemData(sel);
        if (elem && elem->plugin)
        {
            if (!Manager::Get()->GetPluginManager()->UninstallPlugin(elem))
                failure << elem->info.title << _T('\n');
        }
    }

    FillList();
    if (!failure.IsEmpty())
        cbMessageBox(_("One or more plugins were not un-installed successfully:\n\n") + failure, _("Warning"), wxICON_WARNING, this);
}

elem->plugin is NULL, but we still need to uninstall it, right?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5918
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #32 on: February 16, 2016, 06:40:24 am »
Patch updated to handle safemode
:o, you have version 2 of your patch here, but I'm testing your version 1 patch...
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5918
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #33 on: February 16, 2016, 07:51:15 am »
My new patch is out, which is based on dmoore's version 2 patch, it can uninstall an unloaded plugin.
Code::Blocks / Tickets / #300 Only load plugin shared library objects for plugins that are enabled

But I still have an issue that I can't install an exported plugin bundle, reported in my previous posts.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5918
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #34 on: February 16, 2016, 08:53:39 am »
But I still have an issue that I can't install an exported plugin bundle, reported in my previous posts.
OK, I found the reason, I see that in my exported plugin bundle, the dll and zip file name are all in lowercase, but the bundle name is "ProjectsImporter-1.1.cbplugin", so that the name are not matched. I'm bulding C::B against wx trunk(wx31)

This does not happens in our 16.01 release(it will generated ProjectsImporter.dll and ProjectsImporter.zip file)
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5918
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #35 on: February 16, 2016, 09:11:15 am »
Return to the original issue when install a new plugin
Code
    ScanForPlugins(pluginDir);
    LoadAllPlugins();
    cbPlugin* plugin = FindPluginByName(basename); //THIS DOESN'T WORK RIGHT...
    const PluginInfo* info = GetPluginInfo(plugin);

Here, I see the plugin is NULL. Because I think LoadAllPlugins() only load the manifest(xml), but the dll is not loaded yet.

EDIT:
Maybe, we need to just Load the plugin as in the code:
Code
void PluginsConfigurationDlg::OnToggle(wxCommandEvent& event)
{
    wxListCtrl* list = XRCCTRL(*this, "lstPlugins", wxListCtrl);
    if (list->GetSelectedItemCount() == 0)
        return;
    bool isEnable = event.GetId() == XRCID("btnEnable");

    wxBusyCursor busy;

    wxProgressDialog pd(wxString::Format(_("%s plugin(s)"), isEnable ? _("Enabling") : _("Disabling")),
                        _T("A description wide enough for the dialog ;)"),
                        list->GetSelectedItemCount(),
                        this,
                        wxPD_AUTO_HIDE | wxPD_APP_MODAL | wxPD_CAN_ABORT);

    int count = 0;
    long sel = -1;
    bool skip = false;
    while (true)
    {
        sel = list->GetNextItem(sel, wxLIST_NEXT_ALL, wxLIST_STATE_SELECTED);
        if (sel == -1)
            break;

        PluginElement* elem = (PluginElement*)list->GetItemData(sel);
        if (elem)
        {
            pd.Update(++count,
                        wxString::Format(_("%s \"%s\"..."), isEnable ? _("Enabling") : _("Disabling"), elem->info.title.c_str()),
                        &skip);
            if (skip)
                break;

            if (!isEnable && elem->plugin && elem->plugin->IsAttached())
            {
                Manager::Get()->GetPluginManager()->DetachPlugin(elem->plugin);
                Manager::Get()->GetPluginManager()->UnloadPlugin(elem->plugin);
            }
            else if (isEnable && !(elem->plugin && elem->plugin->IsAttached()))
            {
                if (!elem->plugin)
                    Manager::Get()->GetPluginManager()->LoadPlugin(elem->fileName, elem);
                Manager::Get()->GetPluginManager()->AttachPlugin(elem->plugin, true); // ignore safe-mode here
            }
            else
                continue;

See the above code:
Code
                if (!elem->plugin)
                    Manager::Get()->GetPluginManager()->LoadPlugin(elem->fileName, elem);
« Last Edit: February 16, 2016, 09:17:33 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5918
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #36 on: February 17, 2016, 02:32:11 am »
I have find the solution to solve the "THIS DOESN'T WORK RIGHT" problem in dmoore's original v2 patch.

Please have a look and test the v4.patch in Code::Blocks / Tickets / #300 Only load plugin shared library objects for plugins that are enabled.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline blauzahn

  • Almost regular
  • **
  • Posts: 170
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #37 on: February 17, 2016, 08:00:29 am »

just glanced over the patch.

Code
void PluginManager::AddUnloadedPluginElem(const wxString &path, const PluginInfo &info)
{
    PluginElement *plugElem  = new PluginElement;
    plugElem->fileName = path;
    plugElem->info = info;
    plugElem->library = 0;
    plugElem->freeProc = 0;
    plugElem->plugin = 0;
    m_Plugins.Add(plugElem);
}

I would prefer nullptr instead of 0. At other places in the patch as well.

The method name AddUnloadedPluginElement reveals, that it is doing actually 2 things:
Creating a PluginElement and adding it. To me, the creation details belong to PluginElement.
I'd convert it into a ctor or maybe into a static method to give it a name like:

Code
static PluginElement PluginElement::unloaded(const wxString &path, const PluginInfo &info);

That way, the void method wil evaporate into a mere call :

Code
m_Plugins.Add( new PluginElement::unloaded(path, info));

where at least the creation is more exception safe and the newly created element is given
directly to the array. I always shiver when I see naked new and delete.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #38 on: February 17, 2016, 09:36:09 am »
I always shiver when I see naked new and delete.
Welcome to the real world outside of the committee...
Except for the nullptrs everything else is not that bad in this function.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline frithjofh

  • Regular
  • ***
  • Posts: 376
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #39 on: February 17, 2016, 10:29:21 am »
@obfuscated

is there a special reason, you don't implement and use a constructor for PluginElement taking a String and a PluginInfo and setting the other members to nullptr in the initializer list?
architect with some spare time  -  c::b compiled from last svn  -   openSuSE leap x86_64  -  AMD FX-4100

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Allowing Plugin Interdependency and Improving Plugin Management
« Reply #40 on: February 17, 2016, 10:37:41 am »
Because it doesn't matter that much.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]