Non-DDE IPC?

[TheNullinator]

I'm putting finishing touches on the WM_COPYDATA IPC system now.  I'll finish them tomorrow hopefully.  It works fine under Windows ME (ANSI) and Windows XP (ANSI/unicode).  It works good over a network aswell.  It is fully supportive of multiple instances of Code::Blocks and sends files to the last active window so you can choose which one you want to receive a file.

There are two problems I've come across so far.  They are:

• Sometimes a message is displayed saying the config file couldn't be saved.  This happens only when wildly double clicking on C::B registered files (in particular non-projects/workspaces) in Windows Explorer.
• If the first instance is still loading and another codeblocks.exe is executed with a file argument and the first instance crashes while loading a plugin, the codeblocks.exe with the file argument hangs.

The first issue seems to have been fixed with some timed retry mods to CfgMgrBldr::Close().  I'm going to find some other way around that one though.  I've got an idea about the second one, too.

I'll upload a diff file when those issues are fixed (and any others that may arise) so if you guys want to use it, then that's cool.  Otherwise, I had fun doing it anyway.   

Cheers

[thomas]

The first issue seems to have been fixed with some timed retry mods to CfgMgrBldr::Close().

Eeeeeeeeh... please do not ever tamper with that function. This is absolutely no go.

First of all, it is not recommended to use CfgMgrBldr for anything at all. The actual reason why CfgMgrBldr has such a stupid name is to give you a hint that this is not thought to be used in normal operation. Treat it as if it was a private internal class of Manager (although it is not, for simplicity).

Also, calling Close() will result in undefined behaviour, and will leak memory - use Free() instead. However, it is best if you don't use Free() either, since that, too, may result in undefined behaviour.

The configuration manager and all pointer references to it are, by definition, valid from its first invokation to the destruction of the Manager singleton (which happens immediately before the application is terminated). Parts of the SDK rely on this fact and may crash if you destroy ConfigManager.

The same or similar is true for other managers. The shutdown procedure follows a well-defined order of destruction which ensures that certain things are reliably availabler at given times.

For example, a plugin developer can rely blindly on the availability and validity of ScriptingManager, ProjectManager, EditorManager, PersonalityManager, MacrosManager, UserVariableManager, MessageManager, and ConfigManager (no null-pointer checks required!) for the entire lifetime of a plugin.
Also, this ensures that for example EditorManager is valid for the entire lifetime of ProjectManager, because the former is used by the latter.
If you free managers in an uncontrolled manner, these contraints are no longer effective and very, very nasty things (which are impossible to debug) may happen.

What can be done is, you can simply do all your IPC before ConfigManager is instantiated for the first time. In that case, nothing is saved to the config.

[TheNullinator]

The first issue seems to have been fixed with some timed retry mods to CfgMgrBldr::Close().

Eeeeeeeeh... please do not ever tamper with that function. This is absolutely no go.

Yeah, I thought it was a bit of an out-of-the-way function.  That's why I'm looking for a different way.  The original version of the IPC code needed the config file to check /environment/single_instance but I rewrote a lot of it yesterday to support mutliple instances so I can move everything to before ConfigManager is instantiated.

Also, calling Close() will result in undefined behaviour, and will leak memory - use Free() instead. However, it is best if you don't use Free() either, since that, too, may result in undefined behaviour.

I never called Close(), it got called naturally through CfgMgrBldr::~CfgMgrBldr() when the applicated terminated.  It just happened to happen at the same time in two different instances of Code::Blocks that were sending IPC messages.

[thomas]

I never called Close(), it got called naturally through CfgMgrBldr::~CfgMgrBldr() when the applicated terminated.  It just happened to happen at the same time in two different instances of Code::Blocks that were sending IPC messages.

Ah ok, that is something we'll have to live with then, because I am not using a Windows mutex for portability (only way we could avoid that). wxWidgets mutexes don't work between processes...

[TheNullinator]

Ah ok, that is something we'll have to live with then, because I am not using a Windows mutex for portability (only way we could avoid that). wxWidgets mutexes don't work between processes...

Yeah, bummer about that.    I got a bit excited when I saw wxMutex in the class list of the wx docs, but then I found out they were local to the process.

I've fixed those two issues.  I did some really frantic clicking and opening of multiple instances all at the same time without any issue.  Also tested the plugin crash issue and it worked properly.

I'm building the ANSI version now to test on Windows ME and if all goes well, I'll post the diff.

[TheNullinator]

I've attached a 7z archive of the diff file for the DDE replacement.  I've tested it quite a bit on both Windows ME and XP (both ANSI and unicode) and haven't come across any more problems.

I had to modify associations.cpp to detect and remove DDE information from the association keys in the registry or Windows thinks Code::Blocks still handles DDE messages.  The only other modified files are app.cpp and app.h.

[attachment deleted by admin]

[thomas]

Tested. First, it tossed a "cannot find file" error at me every time. After manually removing all the stale DDE keys from the registry (the automatic uninstall does not seem to catch them all), that error was gone.

Although writing the window ID to the registry gives me a bit of a pain in the stomach, it seems to work perfectly for some files.
For some? Yes, unluckily... While all source files go into one instance just fine, XRC files and headers are opened in a separate instance. Worse yet, if I select 10 XRC files and select "open" from Explorer's context menu, 10 instances of Code::Blocks launch in parallel, each opening one file...

So... too early for thumbs up yet, but it looks like a start

[TheNullinator]

Tested. First, it tossed a "cannot find file" error at me every time. After manually removing all the stale DDE keys from the registry (the automatic uninstall does not seem to catch them all), that error was gone.

Hmm, I'll look into that a bit more.

Although writing the window ID to the registry gives me a bit of a pain in the stomach, it seems to work perfectly for some files.

Yeah, that is a rather dodgy thing to be doing    but it did the job at the time.  I'm looking into other ways of achieving the same result.

For some? Yes, unluckily... While all source files go into one instance just fine, XRC files and headers are opened in a separate instance.

I don't have any issues with header files but do experience some with XRCs now you mention it.  I'll look into that one more, too.

Worse yet, if I select 10 XRC files and select "open" from Explorer's context menu, 10 instances of Code::Blocks launch in parallel, each opening one file...

Oh man, I didn't even think of testing that.   :roll: 

So... too early for thumbs up yet, but it looks like a start

A start it is.    And thanks for the pointers to issues you encountered I hadn't come across.    I'll go have a look at them and see what I can do.

[thomas]

I still think that using a memory mapped file should work fine (could get rid of the registry stuff, and makes it cross-platform-able, too).
It is possible to open a named MMF and query the error message if it was already created (works the same way as a named mutex!).

There is even a complete wrapper class available to handle memory mapped files and shared memory areas: http://www.naughter.com/memmap.html
It may not be 100% what you need and might need some tweaking, but I guess it is quite good for a start.

One would also need a kind of linked list or queue that does not allocate memory on the heap, but is happy to live inside a dedicated memory area, as well as a mutex to protect access to it. That's probably the hardest part.

Then, you could start up and open the file mapping (or a named shared memory section, doesn't matter). If it does *not* exist, then you are the first instance. Write your window handle *here* (not into the registry).

If it *does* exist, you are (obviously) not the first one to use it, so you lock the mutex, append your data to the linked list, and release the mutex.
Then fire off a notification to the window ID which you can find in the shared memory and quit.

Instead of using window IDs and sending messages, one could also use a sepmaphore to signal that new data has arrived, this is somewhat nicer.
The main instance of Code::Blocks could either launch a separate thread that blocks on the semaphore, or simply query the semaphore's state (non-blocking) from inside OnIdle. The latter solution would delay opening files slightly if the PC is under heavy load, but that is not necessarily a bad thing. Actually, it is even a very desirable feature, since this avoids adding more and more load when the machine is already thrashing anyway.

If using things like mmap, mutexes, and semaphores, we could implement the very same functionality for Windows and Linux and only need to change a few lines in each.

[TheNullinator]

Hmm, sounds pretty good.    I'll have a look into it now.

[thomas]

Actually this code snippet is almost exactly what we need (forget the DLL stuff):
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/using_shared_memory_in_a_dynamic_link_library.asp

Darn, lol... should have looked at MSDN earlier. 

[TheNullinator]

The memory mapped file IPC system is going pretty well.  About up to the WM_COPYDATA system's usefulness.  It is sending the files to the last active window quite well.  The custum queue that resides in the shared memory (64K) seems to be holding up, too.

There are some issues when the currently active window is closed and you open a file from Explorer before activating one of the other windows.  The file gets stuck in the queue because noone can claim it.  I'm adding a check now to see if the window a file wanted to be opened in is destroyed and if so it will be adopted by the next active window.

Code::Blocks also likes to hang after being closed (only instances that had the main window).  It needs a bit of help from the Task Manager to be on it's merry way.  It doesn't use any CPU time, it just never terminates.

[TheNullinator]

Actually this code snippet is almost exactly what we need (forget the DLL stuff):
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/using_shared_memory_in_a_dynamic_link_library.asp

Darn, lol... should have looked at MSDN earlier. 

Pretty good example, that one.    Quite a good idea, too.

[thomas]

There are some issues when the currently active window is closed and you open a file from Explorer before activating one of the other windows.  The file gets stuck in the queue because noone can claim it.

This is no problem at all if you use the semaphore approach instead of messaging a window. The worst thing to happen is you might lose one "open file" event, but for this to happen, you would have to hit the window's close box and open the next file within, say 0.1 seconds or so... :lol:

• When the application starts, the shared memory area either exists, or it does not.
• If it does not exist, you start as usual. Some time later, during your application's OnIdle function, the semaphore is queried regularly (non-blocking). It costs very few CPU cycles to do that. At some point, its status will be signalled, so you know that you have data waiting for you.
• If the SHM does already exist, you know at least one other process must be running. Thus, you lock the mutex, append to the list, release the mutex, and signal the semaphore. Then you exit the application (closing the file mapping, of course). You don't bother to create a window or initialise the application in any way. This will probably take only a fraction of a second.
• When the last instance has terminated, all file mappings are closed, so there is no SHM any more. If you used a memory mapped file (not just system VM), then the OS will save the data to disk, otherwise it will discard the data. The next instance that starts will know that it is the primary instance.
• The only possible pitfall is that the primary instance could be closed just at the very same time as another instance is still busy writing to the SHM. You have to click very fast to achieve this.
  Anyway, what will happen? Nothing:
  
  • The worst thing to happen is that the application does not open these files. So what, you just closed it! Of course you would not expect the application to open more files if you close it, so that is perfectly good behaviour.
  • Still, if this is a problem, you can simply post the semaphore once during startup for good. There is no harm in doing so (except wasting ~200 CPU cycles once).
  • After starting up, the application will eventually get into OnIdle, and will see that the semaphore is signalled (because you just posted it). It will therefore look at the list and find any data that may be there (or find nothing, which is fine, too).

[TheNullinator]

This is no problem at all if you use the semaphore approach instead of messaging a window.

I'm not using any window messages or WindowProcs any more nor the registry for storing the current window handle (now stored in the shared memory as you suggested).    The problem was fixed when I fixed the hang on exit for an instance with a window.

This is the basic flow of things in OnInit:

• Create 64K named shared memory.
• Map the memory.  If this is the first instance, initialize some data in the memory
• Create the semaphore initially unsignalled.
• Create the mutex which is initially owned.
• If this is not the first instance and files were passed on the command line, wait for a window to exist (handle stored at the start of the shared memory) and make it the foreground window (if done just before loading the file in OnIdle, it sometimes doesn't work on XP and never works on ME) using mutex protection, add the files to the queue using mutex protection increasing the semaphore count for each file, and then terminate.
• Create the main frame and register it as the current window to receive new files.
• Release the mutex so files can be added to the queue.

In OnIdle the state of the semaphore is checked and if signalled and a file exists in the queue for this instance's window or can be taken from a no longer existing window (can only be done by activating this instance's window), remove it from the queue and open it.  If no file existed for this window, the semaphore count is kept correct.

It (ANSI build) seems to work quite well under both XP and ME at the moment.  I'm yet to do a unicode build for testing on XP.  More tests and code tidy-ups planned for tomorrow morning.  I'll post a diff when satisfied that all issues are sorted out on my two Windows boxes.