Author Topic: PipedProcess usage?  (Read 10000 times)

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
PipedProcess usage?
« on: October 30, 2006, 04:45:32 am »
Hi CBers. I want to use the PipedProcess class (defined in the SDK) to execute some console commands and capture the input/output in the Interpreted Languages plugin that I've been working on. I can't figure out what I'm supposed to pass to as the first argument to the constructor. From the SDK:

Code
PipedProcess::PipedProcess(void** pvThis, wxEvtHandler* parent, int id, bool pipe, const wxString& dir)
    : wxProcess(parent, id),
m_Parent(parent),
m_Id(id),
m_Pid(0),
m_pvThis(pvThis)
{
wxSetWorkingDirectory(UnixFilename(dir));
if (pipe)
Redirect();
}

so what should I be passing as pvThis? Just an uninitialized pointer?

Offline Game_Ender

  • Lives here!
  • ****
  • Posts: 551
Re: PipedProcess usage?
« Reply #1 on: October 30, 2006, 05:29:45 am »
A better question is why is there a void pointer in the API?  You should probably do a search on the CB source code and see how Mandrav uses it.  Then possibly submit a patch that adds the proper doxygen comments to the headers.

EDIT- Improved my bad grammar.
« Last Edit: October 30, 2006, 07:31:43 am by Game_Ender »

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: PipedProcess usage?
« Reply #2 on: October 30, 2006, 05:42:37 am »
yes, I agree it is very ugly. It looks like the PipedProcess is just a bit of a hack around the limitations of wxProcess to get the debugger to work. I would need to study the code much more carefully to get a good handle on it, which would have to wait till a later date (I suspect Mandrav would have a far better understanding of this issue :D ). In any case, after inspecting the DebuggerGDB plugin code, I think I've figured out that I just pass a pointer to the newly instantiated piped process object to get the process instantiated correctly.

But now I'm not sure about how to read from stdout at progressive intervals. I saw that DebuggerGDB handles EVT_PIPEDPROCESS_STDOUT messages, but I also noticed EVT_IDLE and EVT_TIMER message processing as well. Do I need all three of these message declarations (and assocaited handlers) to receive output correctly?


Offline Game_Ender

  • Lives here!
  • ****
  • Posts: 551
Re: PipedProcess usage?
« Reply #3 on: October 30, 2006, 07:45:52 am »
I have not looked at the code, but my guess that is that there might be a timer in PipedProcess that checks to see if the process has written anything to stdout and if so generates an event.  But if you want things quicker you can poll the PipedProcess your self and gather the input.  My guess is Mandrav started out with just listening to the events, but then to improve response time in the debugger added processing in the idle event and another timer loop.

Buts that just a guess, more digging through the class will be needed, but I have to finish my CS project. (Linked lists in C are boring).

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: PipedProcess usage?
« Reply #4 on: October 30, 2006, 08:48:12 am »
Quote
I can't figure out what I'm supposed to pass to as the first argument to the constructor.

The first parameter takes a pointer to the PipedProcess* variable you are just creating. You can pass NULL if you want, but I wouldn't advise it. The reason it's there is because when the piped process closes its handles, your variable (in your code) gets NULLed automatically. Perhaps some pseudocode will help explain better:

Code
wxProcess* myProcess = 0;
myProcess = new PipedProcess(&myProcess, blah, blah);
...
(process executes, you get data, etc)
...
(now you want to send a command through the process)
if (myProcess)
    myProcess->SendString(_T("break main.cpp:8"));
...
(if the process had died, myProcess would have been nulled automatically)
(else the call proceeds)

As for it being void**, well, that's because you could actually put any pointer there you would like nulled when the process dies. You might put there a pointer to char* if you like: PipedProcess doesn't care. It will still null it.
Be patient!
This bug will be fixed soon...

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: PipedProcess usage?
« Reply #5 on: October 30, 2006, 08:57:59 am »
But now I'm not sure about how to read from stdout at progressive intervals. [...]
EVT_PIPEDPROCESS_STDOUT messages
Although reading at regular intervals is possible, it is not advisable. If you did not get a message, then there is nothing to read, so you'll needlessly block.
Granted, passing all data around as messages is not the most efficient way, but that's how wxExecute works internally, too. To get to your data, simply implement a handler for EVT_PIPEDPROCESS_STDOUT (and optionally EVT_PIPEDPROCESS_STDERR, if you want to listen what might arrive there). To know when the child process has exited, listen to EVT_PIPEDPROCESS_TERMINATED.
Note that due to the generally dismal performance of wxExecute, a replacement is in work (actually should be finished long ago). However, while creating a process may be slightly different, the method of getting output via messages will stay available (though not the only option).
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: PipedProcess usage?
« Reply #6 on: October 30, 2006, 05:07:04 pm »
thanks for the responses from all of you. I now have a slightly better understanding.
My next stumbling block: It doesn't look like EVT_PIPEDPROCESS_STDOUT messages get passed to standard CB plugins? (unless i've done something silly - quite likely)

Code snipped:

Code
BEGIN_EVENT_TABLE(InterpretedLangs, cbPlugin)
    EVT_MENU(ID_LangMenu_RunPiped,InterpretedLangs::OnRunPiped)
    EVT_PIPEDPROCESS_STDOUT(ID_PipedProcess, InterpretedLangs::OnPipedOutput)
END_EVENT_TABLE()

void InterpretedLangs::OnPipedOutput(wxCommandEvent& event)
{
    wxMessageBox(_T("Piped output"));
    wxString msg = event.GetString();
    if (!msg.IsEmpty())
    {
        wxMessageBox(msg);
    }
}

void InterpretedLangs::OnRunPiped(wxCommandEvent &event)
{
    m_pp=new PipedProcess((void **)&m_pp,this,ID_PipedProcess);
    m_pp->Launch(_T("C:/Python25/python.exe"),100);
}

As you can see, I have a menu item that calls OnRunPiped, which opens the piped process with unique identifier ID_PipedProcess. This works, but I don't seem to receive any EVT_PIPEDPROCESS_STDOUT messages. On the other hand, if I just call m_pp->Read(...) (not shown) there is stuff coming in through standard out.

Unrelated Question: Why is wxString manipulation so poor? It doesn't seem to be able convert char *, int, double etc to wxStrings (or back). Is this just incompleteness of unicode wxWidgets?

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: PipedProcess usage?
« Reply #7 on: October 30, 2006, 06:52:18 pm »
Quote
m_pp->Launch(_T("C:/Python25/python.exe"),100);

That function doesn't work (is incomplete). You should do the polling yourself. Check debugger/compiler plugins, mainly EVT_TIMER and EVT_IDLE.

Quote
Unrelated Question: Why is wxString manipulation so poor? It doesn't seem to be able convert char *, int, double etc to wxStrings (or back). Is this just incompleteness of unicode wxWidgets?

It's not poor, it's just the hassle of supporting unicode. In an ANSI wx build, you wouldn't have any of those problems...
Be patient!
This bug will be fixed soon...

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: PipedProcess usage?
« Reply #8 on: October 30, 2006, 09:41:02 pm »
That function doesn't work (is incomplete). You should do the polling yourself. Check debugger/compiler plugins, mainly EVT_TIMER and EVT_IDLE.

This clears things up somewhat. I was looking at the GDB plugin and it seemed to be hadling EVT_TIMER and EVT_IDLE, hence my question about these above. I'm still a little uncertain about how the GDB plugin is actually parsing stdout. It seems to handle a EVT_PIPEDPROCESS_STDOUT event, but since this is incomplete I wonder how this works. EVT_IDLE just seems to pass/block the event, so not sure where the output is finally parsed. (I could of course trace through the code in the debugger, but thought i'd ask the experts first).

Quote
It's not poor, it's just the hassle of supporting unicode. In an ANSI wx build, you wouldn't have any of those problems...

so what is the standard way of converting int/float/double to a unicode wxString and back? operator<<, operator>> and the relevant constructor appear to simply fail without warning.

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: PipedProcess usage?
« Reply #9 on: October 30, 2006, 10:29:46 pm »
Quote
I'm still a little uncertain about how the GDB plugin is actually parsing stdout. It seems to handle a EVT_PIPEDPROCESS_STDOUT event, but since this is incomplete I wonder how this works.

No, no...
EVT_PIPEDPROCESS_STDOUT works fine (proven).
PipedProcess::Launch() doesn't work and it's not used.

Now that I remember, I think ToolsManager uses PipedProcess too so you might want to check that out. It's a much smaller unit than the compiler and debugger units so it might be easier to follow.

Quote
so what is the standard way of converting int/float/double to a unicode wxString and back? operator<<, operator>> and the relevant constructor appear to simply fail without warning.

wxString::Format(_T("%d, %5.2f"), i, f);
Be patient!
This bug will be fixed soon...

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2750
Re: PipedProcess usage?
« Reply #10 on: October 31, 2006, 12:11:20 am »
so what is the standard way of converting int/float/double to a unicode wxString and back? operator<<, operator>> and the relevant constructor appear to simply fail without warning.
"...simply fail without warning."  So true! Unicode string streaming is terrible.

The following code snippets from others may be of interest to you.
Code
#include <sstream>
// Read in and set window position --------------------------------
    wxFileConfig cfgFile(wxTheApp->GetAppName(),     // appname
                    wxEmptyString,      // vendor
                    wxEmptyString,      // local filename
                    wxEmptyString,      // global file
                    wxCONFIG_USE_LOCAL_FILE);
    wxString winPos;
cfgFile.Read(_T("WindowPosition"),  &winPos) ;

if ( not winPos.IsEmpty() )
{ long windowXpos, windowYpos, windowWidth, windowHeight;
wxWX2MBbuf buf = cbU2C(winPos);
std::string cstring( buf );
std::stringstream istream(cstring);
istream >> windowXpos ;
istream >> windowYpos ;
istream >> windowWidth ;
istream >> windowHeight ;
this->SetSize(windowXpos, windowYpos, windowWidth, windowHeight);
}

// Record position ------------------------------------------------------
    wxFileConfig cfgFile(g_AppName,     // appname
                        wxEmptyString,      // vendor
                        wxEmptyString,      // local filename
                        wxEmptyString,      // global file
                        wxCONFIG_USE_LOCAL_FILE);

wxWindow* pwin = wxTheApp->GetTopWindow();
    int winXposn, winYposn, winWidth, winHeight;
    pwin->GetPosition( &winXposn, &winYposn );
    pwin->GetSize( &winWidth, &winHeight );

    // Using std::stringstream; wxStringStream dont work in unicode
    std::string cwinPos;
    std::ostringstream ostream;
    ostream << winXposn <<" " << winYposn <<" " << winWidth <<" " <<winHeight;
    cwinPos = ostream.str();
    wxString winPos = cbC2U( cwinPos.c_str());
cfgFile.Write(_T("WindowPosition"),  winPos) ;

// This could also be done as:
winPos = winPos.Format(wxT("%d %d %d %d"),
winXposn, winYposn, winWidth, winHeight);


Conversions:

Code
#include <wx/string.h>

    // Return @c str as a proper unicode-compatible string
    wxString cbC2U(const char* str)
    {
        #if wxUSE_UNICODE
            return wxString(str, wxConvUTF8);
        #else
            return wxString(str);
        #endif
    }

    // Return multibyte (C string) representation of the string
    wxWX2MBbuf cbU2C(const wxString& str)
    {
        #if wxUSE_UNICODE
            return str.mb_str(wxConvUTF8);
        #else
            return (wxChar*)str.mb_str();
        #endif
    }


Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: PipedProcess usage?
« Reply #11 on: November 01, 2006, 12:19:40 am »
No, no...
EVT_PIPEDPROCESS_STDOUT works fine (proven).
PipedProcess::Launch() doesn't work and it's not used.

whoops. anyway, I went back to basics and did some exploration with wxProcess to pipe a simple python debug session and send something down the output stream and retrieve from the input stream - this works - see code snippet below. Perhaps someone can tell me how it could be done more efficiently with a PipedProcess and the role of the Idle message handler (I tried to implement the latter but it just crashed C::B). I'm still confused by PipedProcess - do i need to setup a wxTimer to poll stdout buffer, or does processing EVT_PIPEDPROCESS_STDOUT handle all this for me. The latter doesn't seem to be the case, and if not how do I tell EVT_PIPEDPROCESS_STDOUT to hijack the timer that I create?? Sacrificing a goat to Bjarne + going through the source code, macro definitions, wxWidgets docs and C::B wikis trying to figure this out is killing me - I had a hard enough time with the python interpreter until i realized it doesn't update stdin and stdout if you don't specify the right set of options

Code
BEGIN_EVENT_TABLE(InterpretedLangs, cbPlugin)
    EVT_MENU(ID_LangMenu_RunPiped,InterpretedLangs::OnRunPiped)
    EVT_TIMER(ID_TimerPollDebugger, InterpretedLangs::OnTimer)
    ...
END_EVENT_TABLE()

void InterpretedLangs::OnTimer(wxTimerEvent& event)
{
    if (m_pp && m_pp->IsInputAvailable())
    {
        char buf0[1001]; //Sloppy i know - was in a hurry
        for(int i=0;i<1001;++i)
            buf0[i]=0;
        m_istream->Read(buf0,1000);
        wxString sbuf=wxString::FromAscii(buf0);
        wxMessageBox(sbuf);
    }
}

void InterpretedLangs::OnRunPiped(wxCommandEvent &event)
{
    m_TimerPollDebugger=new wxTimer(this, ID_TimerPollDebugger);
    m_TimerPollDebugger->Start(100);
    m_pp=new wxProcess(this,ID_PipedProcess);
    m_pp->Redirect();
    wxExecute(_T("c:/python25/python.exe -u -m pdb c:/python25/test.py"),wxEXEC_ASYNC,m_pp);
    m_ostream=m_pp->GetOutputStream();
    m_ostream->Write("w\n",2);
    m_istream=m_pp->GetInputStream();
}

Quote
wxString::Format(_T("%d, %5.2f"), i, f);

color me pleasantly surprised. I gave up on the wxstring stream style commands after wxstring::printf didn't work.

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: PipedProcess usage?
« Reply #12 on: November 01, 2006, 11:52:16 pm »
No, no...
EVT_PIPEDPROCESS_STDOUT works fine (proven).

ok, so maybe i was a little bit lazy before writing my previous post, so I went back and studied various bits of CB source a little more closely and read up some more on wxWidgets (and slaughtered a few goats along the way). Having done that, I'm not sure I would say that EVT_PIPEDPROCESS_STDOUT works fine since you have to define your own timer to force idle events in order to ever see EVT_PIPEDPROCESS_STDOUT messages. While this is the fault of wxWidgets, it still introduces a bunch of redundancy to have to declare a timer every time I want to use PipedProcess and will cause problems down the road if wxWidgets ever sorts out this problem with idle events: Every plugin that uses the PipedProcess will need to be modified to remove the redundant timers. Shouldn't PipedProcess create its owner timer and idle events, which it can use to generate EVT_PIPEDPROCESS_STDOUT messages? (Presumably that's what PipedProcess::Launch was for?) At least then when wxWidgets gets its act together on this, it's only one module that needs fixing.

Finally, it is also unclear to me that idle time processing is necessarily going to be more efficient for piping stdout/stderr than just starting/stopping a wxTimer as needed by any given plugin. I can imagine a dozen or more plugins receiving idle events when only one of them is actually active (although they should all just return quickly). Each programmer would need to take care to declare their plugins in such a way that they only receive idle messages while in their activated state.