Author Topic: issue about Memory VFS already contains file xxx when re-enable a plugin  (Read 8368 times)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5913
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
When I debug the code we discussed in Re: Allowing Plugin Interdependency and Improving Plugin Management, I notice an error message box from wx, here is the call stack:
Code
[debug]> bt 30
[debug]#0  wxMemoryFSHandlerBase::CheckDoesntExist (filename=L"codecompletion.zip") at F:\wx\wxWidgets_trunk\src\common\fs_mem.cpp:175
[debug]#1  0x01970787 in wxMemoryFSHandlerBase::AddFileWithMimeType (filename=L"codecompletion.zip", binarydata=0x11671dc0, size=7580, mimetype=L"") at F:\wx\wxWidgets_trunk\src\common\fs_mem.cpp:199
[debug]#2  0x01970889 in wxMemoryFSHandlerBase::AddFile (filename=L"codecompletion.zip", binarydata=0x11671dc0, size=7580) at F:\wx\wxWidgets_trunk\src\common\fs_mem.cpp:217
[debug]#3  0x694ec051 in wxMemoryFSHandler::AddFile (filename=L"codecompletion.zip", binarydata=0x11671dc0, size=7580) at F:\wx\wxWidgets_trunk\include\wx\fs_mem.h:97
[debug]#4  0x6933518e in Manager::LoadResource (file=L"codecompletion.zip") at F:\jens-codeblocks-mirror\codeblocks.git\src\sdk\manager.cpp:533
[debug]#5  0x619586df in CodeCompletion::CodeCompletion (this=0x11699070) at F:\jens-codeblocks-mirror\codeblocks.git\src\plugins\codecompletion\codecompletion.cpp:498
[debug]#6  0x619d6325 in PluginRegistrant<CodeCompletion>::CreatePlugin () at F:\jens-codeblocks-mirror\codeblocks.git\src\include\cbplugin.h:1115
[debug]#7  0x69346aa0 in PluginManager::LoadPlugin (this=0x115b74d8, pluginName=L"F:\\jens-codeblocks-mirror\\codeblocks.git\\src\\devel30\\share\\codeblocks\\plugins\\codecompletion.dll", plugElem0=0x115bc9d8) at F:\jens-codeblocks-mirror\codeblocks.git\src\sdk\pluginmanager.cpp:1098
[debug]#8  0x6934aa9b in PluginsConfigurationDlg::OnToggle (this=0x22f170, event=...) at F:\jens-codeblocks-mirror\codeblocks.git\src\sdk\pluginsconfigurationdlg.cpp:217
...

It sounds like in our code, we need to release the in memory files?

Code
bool Manager::LoadResource(const wxString& file)
{
    wxString resourceFile = ConfigManager::LocateDataFile(file, sdDataGlobal | sdDataUser);
    wxString memoryFile = _T("memory:") + file;

    if (wxFile::Access(resourceFile, wxFile::read) == false)
    {
        Get()->GetLogManager()->LogError(_("Manager failed to access XRC resource '") + resourceFile + _("'."));
        return false;
    }

    // The code below forces a reload of the resource
    // Currently unused...

//    {
//        wxMemoryFSHandler::RemoveFile(file);
//    }
//#if wxABI_VERSION > 20601
//    // unload old resources with the same name
//    wxXmlResource::Get()->Unload(memoryFile);
//#endif

    wxFile f(resourceFile, wxFile::read);
    char *buf = nullptr;

    try
    {
        size_t len = f.Length();
        buf = new char[len];
        f.Read(buf, len);
        {
            wxMemoryFSHandler::AddFile(file, buf, len);
        }
        if ( !wxXmlResource::Get()->Load(memoryFile) )
            Get()->GetLogManager()->LogError(_("Manager failed to load XRC resource '") + resourceFile + _("'."));
        delete[] buf;
        return true;
    }
    catch (...)
    {
        delete[] buf;
        Get()->GetLogManager()->LogError(_("Manager hardly failed to load XRC resource '") + resourceFile + _("'."));
        return false;
    }
}


Any ideas?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5913
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: issue about Memory VFS already contains file xxx when re-enable a plugin
« Reply #1 on: February 17, 2016, 01:57:48 am »
Enable the commented code snippets, I see that
Quote
08:58:25: Trying to remove file 'resources.zip' from memory VFS, but it is not loaded!
08:58:25: Trying to remove file 'manager_resources.zip' from memory VFS, but it is not loaded!
08:58:27: Trying to remove file 'autosave.zip' from memory VFS, but it is not loaded!
08:58:28: Trying to remove file 'codecompletion.zip' from memory VFS, but it is not loaded!
08:58:29: Trying to remove file 'occurrenceshighlighting.zip' from memory VFS, but it is not loaded!
...
like messages happens.

So, we need to check whether those zip files were exists in VFS system, but I don't see which method I can use...
wxWidgets: wxFileSystemHandler Class Reference
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5913
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: issue about Memory VFS already contains file xxx when re-enable a plugin
« Reply #2 on: February 17, 2016, 02:18:32 am »
This is wx's source:
Code
// ----------------------------------------------------------------------------
// wxMemoryFSHandlerBase
// ----------------------------------------------------------------------------

class WXDLLIMPEXP_BASE wxMemoryFSHandlerBase : public wxFileSystemHandler
{
public:
    wxMemoryFSHandlerBase();
    virtual ~wxMemoryFSHandlerBase();

    // Add file to list of files stored in memory. Stored data (bitmap, text or
    // raw data) will be copied into private memory stream and available under
    // name "memory:" + filename
    static void AddFile(const wxString& filename, const wxString& textdata);
    static void AddFile(const wxString& filename, const void *binarydata, size_t size);
    static void AddFileWithMimeType(const wxString& filename,
                                    const wxString& textdata,
                                    const wxString& mimetype);
    static void AddFileWithMimeType(const wxString& filename,
                                    const void *binarydata, size_t size,
                                    const wxString& mimetype);

    // Remove file from memory FS and free occupied memory
    static void RemoveFile(const wxString& filename);

    virtual bool CanOpen(const wxString& location) wxOVERRIDE;
    virtual wxFSFile* OpenFile(wxFileSystem& fs, const wxString& location) wxOVERRIDE;
    virtual wxString FindFirst(const wxString& spec, int flags = 0) wxOVERRIDE;
    virtual wxString FindNext() wxOVERRIDE;

protected:
    // check that the given file is not already present in m_Hash; logs an
    // error and returns false if it does exist
    static bool CheckDoesntExist(const wxString& filename);

    // the hash map indexed by the names of the files stored in the memory FS
    static wxMemoryFSHash m_Hash;

    // the file name currently being searched for, i.e. the argument of the
    // last FindFirst() call or empty string if FindFirst() hasn't been called
    // yet or FindNext() didn't find anything
    wxString m_findArgument;

    // iterator into m_Hash used by FindFirst/Next(), possibly m_Hash.end() or
    // even invalid (can only be used when m_findArgument is not empty)
    wxMemoryFSHash::const_iterator m_findIter;
};

// ----------------------------------------------------------------------------
// wxMemoryFSHandler
// ----------------------------------------------------------------------------

#if wxUSE_GUI

// add GUI-only operations to the base class
class WXDLLIMPEXP_CORE wxMemoryFSHandler : public wxMemoryFSHandlerBase
{
public:
    // bring the base class versions into the scope, otherwise they would be
    // inaccessible in wxMemoryFSHandler
    // (unfortunately "using" can't be used as gcc 2.95 doesn't have it...)
    static void AddFile(const wxString& filename, const wxString& textdata)
    {
        wxMemoryFSHandlerBase::AddFile(filename, textdata);
    }

    static void AddFile(const wxString& filename,
                        const void *binarydata,
                        size_t size)
    {
        wxMemoryFSHandlerBase::AddFile(filename, binarydata, size);
    }
    static void AddFileWithMimeType(const wxString& filename,
                                    const wxString& textdata,
                                    const wxString& mimetype)
    {
        wxMemoryFSHandlerBase::AddFileWithMimeType(filename,
                                                   textdata,
                                                   mimetype);
    }
    static void AddFileWithMimeType(const wxString& filename,
                                    const void *binarydata, size_t size,
                                    const wxString& mimetype)
    {
        wxMemoryFSHandlerBase::AddFileWithMimeType(filename,
                                                   binarydata, size,
                                                   mimetype);
    }

#if wxUSE_IMAGE
    static void AddFile(const wxString& filename,
                        const wxImage& image,
                        wxBitmapType type);

    static void AddFile(const wxString& filename,
                        const wxBitmap& bitmap,
                        wxBitmapType type);
#endif // wxUSE_IMAGE

};

#else // !wxUSE_GUI

// just the same thing as the base class in wxBase
class WXDLLIMPEXP_BASE wxMemoryFSHandler : public wxMemoryFSHandlerBase
{
};

#endif // wxUSE_GUI/!wxUSE_GUI

Note that the function
Code
static bool CheckDoesntExist(const wxString& filename);

But this function is protected, so we can't access it, this gives the dilemma:

1, if we try to remove a file which does not exists in the system, we get a log message from wx.
2, if we try to add a file which the same file already exists in the system, we also get a log message from wx.

So, how can we handle this? Do we need to always remove the file when a plugin is unloaded?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5913
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: issue about Memory VFS already contains file xxx when re-enable a plugin
« Reply #3 on: February 17, 2016, 02:49:54 am »
We have a bool Manager::LoadResource(const wxString& file) function, but do we need to have a function UnloadResource() which we need to call from the destructor of a plugin?

Code
ProjectsImporter::ProjectsImporter()
{
    //ctor
    if (!Manager::LoadResource(_T("projectsimporter.zip")))
        NotifyMissingFile(_T("projectsimporter.zip"));
}

ProjectsImporter::~ProjectsImporter()
{
    //dtor
    Manager::UnloadResource(...);
}
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: issue about Memory VFS already contains file xxx when re-enable a plugin
« Reply #4 on: February 17, 2016, 02:05:51 pm »
But this function is protected, so we can't access it, this gives the dilemma:
Except if we make our own class cbMemoryFSHandlerBase and derive from it... if that's feasible in this context.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5913
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: issue about Memory VFS already contains file xxx when re-enable a plugin
« Reply #5 on: February 17, 2016, 02:46:57 pm »
But this function is protected, so we can't access it, this gives the dilemma:
Except if we make our own class cbMemoryFSHandlerBase and derive from it... if that's feasible in this context.
Why do we need a cbMemoryFSHandlerBase?

I think we can directly derive from the wxMemoryFSHandler, and make a static function, like:

Code
class WXDLLIMPEXP_BASE cbMemoryFSHandler : public wxMemoryFSHandler
{
    static void AddFile(const wxString& filename,
                        const void *binarydata,
                        size_t size)
    {
        // add our code here
        if ( !wxMemoryFSHandlerBase::CheckDoesntExist(filename) )
            wxMemoryFSHandlerBase::RemoveFile(filename);

        // do what the base class do
        wxMemoryFSHandler::AddFile(filename, binarydata, size);
    }
}

Is the above code prototype correct?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5913
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: issue about Memory VFS already contains file xxx when re-enable a plugin
« Reply #6 on: February 17, 2016, 04:01:42 pm »
Hi, Morten, here is the patch to solve the error message from wx

Code
commit edb7615a930263862941e94c4326e834ab1fcb1d
Author: asmwarrior <asmwarrior@gmail.com>
Date:   Wed Feb 17 23:12:52 2016 +0800

    test patch to avoid the error message from wx that when a resoure file
    is already in the memory file system, and a new one try to load again.
   
    See discussion here:
    http://forums.codeblocks.org/index.php/topic,20952.0.html

diff --git a/src/sdk/manager.cpp b/src/sdk/manager.cpp
index 1122b0e..68663a5 100644
--- a/src/sdk/manager.cpp
+++ b/src/sdk/manager.cpp
@@ -48,6 +48,35 @@
 #include "ccmanager.h"
 #include "debuggermanager.h"
 
+
+class DLLIMPORT cbMemoryFSHandler : public wxMemoryFSHandler
+{
+public:
+    static void AddFile(const wxString& filename,
+                        const void *binarydata,
+                        size_t size);
+    static void RemoveFile(const wxString& filename);
+};
+/* static */
+void cbMemoryFSHandler::AddFile(const wxString& filename,
+                    const void *binarydata,
+                    size_t size)
+{
+    // if the old file exists, remove the old one, and add it later
+    if ( m_Hash.count(filename) )
+        wxMemoryFSHandler::RemoveFile(filename);
+
+    // do what the base class do
+    wxMemoryFSHandler::AddFile(filename, binarydata, size);
+}
+
+/* static */
+void cbMemoryFSHandler::RemoveFile(const wxString& filename)
+{
+    // if the old file exists, remove it
+    if ( m_Hash.count(filename) )
+        wxMemoryFSHandler::RemoveFile(filename);
+}
 static Manager* s_ManagerInstance = nullptr;
 
 
@@ -511,15 +540,12 @@ bool Manager::LoadResource(const wxString& file)
     }
 
     // The code below forces a reload of the resource
-    // Currently unused...
 
-//    {
-//        wxMemoryFSHandler::RemoveFile(file);
-//    }
-//#if wxABI_VERSION > 20601
-//    // unload old resources with the same name
-//    wxXmlResource::Get()->Unload(memoryFile);
-//#endif
+    // this will remove the file if it is already loaded
+    cbMemoryFSHandler::RemoveFile(file);
+
+    // unload old resources with the same name
+    wxXmlResource::Get()->Unload(memoryFile);
 
     wxFile f(resourceFile, wxFile::read);
     char *buf = nullptr;
@@ -530,7 +556,7 @@ bool Manager::LoadResource(const wxString& file)
         buf = new char[len];
         f.Read(buf, len);
         {
-            wxMemoryFSHandler::AddFile(file, buf, len);
+            cbMemoryFSHandler::AddFile(file, buf, len);
         }
         if ( !wxXmlResource::Get()->Load(memoryFile) )
             Get()->GetLogManager()->LogError(_("Manager failed to load XRC resource '") + resourceFile + _("'."));
diff --git a/src/src/app.cpp b/src/src/app.cpp
index 837d974..169e360 100644
--- a/src/src/app.cpp
+++ b/src/src/app.cpp
@@ -76,6 +76,15 @@
 inline wxString GetResourcesDir(){ return wxEmptyString; }
 #endif
 
+class DLLIMPORT cbMemoryFSHandler : public wxMemoryFSHandler
+{
+public:
+    static void AddFile(const wxString& filename,
+                        const void *binarydata,
+                        size_t size);
+    static void RemoveFile(const wxString& filename);
+};
+
 namespace
 {
 bool s_Loading = false;
@@ -588,7 +597,7 @@ bool CodeBlocksApp::OnInit()
 
     // we'll do this once and for all at startup
     wxFileSystem::AddHandler(new wxZipFSHandler);
-    wxFileSystem::AddHandler(new wxMemoryFSHandler);
+    wxFileSystem::AddHandler(new cbMemoryFSHandler);
     wxXmlResource::Get()->InsertHandler(new wxToolBarAddOnXmlHandler);
     wxXmlResource::Get()->InsertHandler(new wxScrollingDialogXmlHandler);
     wxInitAllImageHandlers();


It works fine.

But I believe there are some better solutions.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: issue about Memory VFS already contains file xxx when re-enable a plugin
« Reply #7 on: February 17, 2016, 04:05:30 pm »
Is the above code prototype correct?
Yes, should be fine. I just copied&pasted from your code snippet.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5913
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: issue about Memory VFS already contains file xxx when re-enable a plugin
« Reply #8 on: February 17, 2016, 04:08:08 pm »
Is the above code prototype correct?
Yes, should be fine. I just copied&pasted from your code snippet.
I have just make a patch file to implement this feature, you can have a loot at my previous post in this thread. :)
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.