Attention massive speed up of the debugger

[oBFusCATed]

Maybe this code causes the problem:

Code

void GDB_driver::ParseOutput(const wxString& output)
{
    m_Cursor.changed = false;

    // Watch for initial debug info and grab the child PID
    // this is put here because we need this info even if
    // we don't get a prompt back.
    // It's "cheap" anyway because the line we 're after is
    // the very first line printed by gdb when running our
    // program. It then sets the child PID and never enters here
    // again because the "want_debug_events" condition below
    // is not satisfied anymore...
    if (platform::windows && want_debug_events)
    {
        wxRegEx* re = 0;
        if ((m_GDBVersionMajor > 6 || (m_GDBVersionMajor == 6 && m_GDBVersionMinor >= 7)) &&
            output.Contains(_T("CREATE_PROCESS_DEBUG_EVENT")))
        {
            re = &reChildPid2;
        }
        else if (m_GDBVersionMajor <= 6 && output.Contains(_T("do_initial_child_stuff")))
        {
            re = &reChildPid;
        }

        if (re)
.
.
.

Maybe we should do the splitting here

Code

void DebuggerGDB::ParseOutput(const wxString& output)
{
    if (!output.IsEmpty() && m_State.HasDriver())
    {
        m_State.GetDriver()->ParseOutput(output);
    }
}

Can you test?

[Jenna]

Maybe this code causes the problem:

[...]

Can you test?

I will test it, if I am at work.

[Jenna]

The problem we have, is that ParseOutput (in gdb_driver.cpp) relies on strings that belongs together and begin with special sequences, like gdb: or Warning:,  or do not contain anything but the response to one command (everything beginning with the GDB_PROMPT is removed inside the if(cmd)-clause).

But with your patch we can get output-strings that contain multiple lines of gdb's output.
This behaviour can break parsing (might depend on system load and whatever).

So the whole ParseOutput-function has to be overworked to work reliable with your patch.
The question is, whether the splitting and extra-parsing will eat up the time we save with the patch.

[oBFusCATed]

Do you have sample output from gdb? On linux I've no problems with it.

There is an easy fix you could try (pseudo code below):

Code

void DebuggerGDB::ParseOutput(const wxString& output)
{
    if (!output.IsEmpty() && m_State.HasDriver())
    {
        wxStringArray lines = split(output, '\n');

        for(int ii = 0; ii < lines.count(); ++ii)
             m_State.GetDriver()->ParseOutput(lines[ii]);
    }
}

This one should simulate the old behaviour...

[Jenna]

Your code does not work, here is a corrected version (EDIT: I overread that it is pseudo-code, sorry):

Code

void DebuggerGDB::ParseOutput(const wxString& output)
{
    if (!output.IsEmpty() && m_State.HasDriver())
    {
        wxArrayString lines = GetArrayFromString(output, _T('\n'));

        for(int i = 0; i < lines.GetCount(); ++i)
             m_State.GetDriver()->ParseOutput(lines[i]);
    }
}

I'm not sure if it can work reliably.
Can we reach OnGDBOutput, OnGDBError or the OnTimer-function while we are still parsing splitted output ?
If yes we would most likely mess up the order we send the data to the drivers ParseOutput.

Nevertheless, I attach a file containing the output of debugger's debug-log.
I just wrote the output-parameter from ParseOutput into the log.

[attachment deleted by admin]

[MortenMacFly]

There is an easy fix you could try (pseudo code below):

I am not sure if this would work. Can you be sure that the lines are complete? What if the first buffer contains 1.5 lines and the second buffer the second half of the second line (0.5 lines)? In that case the parser will not understand the second line of the first buffer and the first line of the second buffer. Thus one response is skipped.

To make it more clear... what I am talking about is a scenario like this:
Pseudo buffer1:

Code

gdb: whatever_message1 what_ever_param1
gdb: whatever_message2

Pseudo buffer2:

Code

 what_ever_param2

[oBFusCATed]

Hm, yes...
What about buffering if there is non terminated (\n) output?

[oBFusCATed]

OK, I've tried the buffering approach and it doesn't work  . It doesn't work because there messages that doesn't end with \n (the output from set prompt for example).

Morten: If somehow the message is split in two the ParseOutput code will break, with or without the patch! This is a problem of the used protocol. (you could prove me wrong though)

Jens: I've missed your message before: why the debug log has double '\n'?

[Jenna]

The additional crlf's should not be there. I'm quite sure I did not log additional line-endings, and the logs did not contain them.
I deleted the original files, so I can not compare them, to see where this strange thing happens.

If I insert a patch in the textbox while posting the same things happens sometimes( additional line-endings), so it might be an issue of the board software, even if I do not know how this should happen with attached files (but I know that MS Outlook does something similar with mail attachements).

Nevertheless you can ignore the additional newlines.


I don't think an output line can be split in the middle, and if it can, this might happen in your code and in the original code.

But as I wrote before:

Can we reach OnGDBOutput, OnGDBError or the OnTimer-function while we are still parsing splitted output ?
If yes we would most likely mess up the order we send the data to the drivers ParseOutput.

[oBFusCATed]

Can we reach OnGDBOutput, OnGDBError or the OnTimer-function while we are still parsing splitted output ?
If yes we would most likely mess up the order we send the data to the drivers ParseOutput.

What exactly are you talking about?
Entering OnGDBOutput while we are in ParseOutput or getting OnGDBError, OnGDBOutput, OnTimer in the middle of a result stream?

I think the first is not possible because the messages are in single thread.
And here is an example of the second situation:

Code

>input line1
>input line 2
>error line 1
>input line 3
>error line 2

[Jenna]

With your change we can get multiple lines of output, that have to be split into an array to work correctly in any cases as discussed before.
This will take some time (not really much, but that depeon the system's load and speed).

Then we loop through the array and send it to ParseOutput.

While this happens it might be that we reach another EVT_PIPEDPROCESS_STDOUT, EVT_PIPEDPROCESS_STDERR or EVT_TIMER, because the debugger process runs asynchronously, the timer should do this also of course.

So it might happen, that we are just parsing a large amount of output, while we get another event with content to parse, or am I missing something.

[oBFusCATed]

Jens can you provide some link documenting this kind of behaviour (I'm not an wx expert (not even wx advanced guy), most of the GUI code I've done is in MFC/win32 api),
I think we can't get a EVT_PIPEDPROCESS_STDOUT, EVT_PIPEDPROCESS_STDERR or EVT_TIMER,
While we are in a handler of some of them (ParseOutput is called only from handlers of these three messages).

Also: Can you test this patch: http://smrt.is-a-geek.org/codeblocks/dbg_speedup_test.patch

It does the splitting of the output,
logging to file ( change /home/obfuscated/cb.log to some path on your machine),
sends the non '\n' terminated output in the OnTimer handler.

If you have problems can you send me the result log file?
Also you could revert the changes in the DebuggerGDB::ParseOutput, rerun the debug session and send me the output file.

[Jenna]

No, I do not have any documentation about that (except for the wxwidgets docu).

It's just a question.

If it is sure, that we cannot get one of the events while we are processing one of them, there is no problem using this (splitting the messages in debuggergdb's ParseOutput and call the drivers ParseOutput for each line).

No problems here (until now) but I do not have much time for testing on windows at the moment.

[oBFusCATed]

No, I do not have any documentation about that (except for the wxwidgets docu).

It's just a question.

If it is sure, that we cannot get one of the events while we are processing one of them, there is no problem using this (splitting the messages in debuggergdb's ParseOutput and call the drivers ParseOutput for each line).

It can happen if someone does wxMesssageBox() or shows another modal dialog.
I've tested it with two timers. I don't know if there aren't any other cases where you can send/receive events from the queue inside a handler of another event? In win32 API you could do:

Code

	MSG	msg;
	while(PeekMessage(&msg, windowHandle, 0, 0, PM_REMOVE /*| PM_QS_PAINT*/) == TRUE)
	{
		TranslateMessage (&msg);
		DispatchMessage(&msg);
	}

to pop all messages from the queue.
Is there wx equivalent for this piece of code?

I'm thinking of a strategy how to fix this problem, but I don't have much time/energy at the moment

No problems here (until now) but I do not have much time for testing on windows at the moment.

Are you talking about the last patch?

[oBFusCATed]

What about this kind of solution:

Code

/// in the class declaration
    private:
        std::deque<wxString> m_lines;
        wxString m_timer1, m_timer2;

        bool m_in_handler;


/// in the cpp file
void wx_timerDialog::OnTimer(wxTimerEvent& event)
{
    m_timer1 = Common(m_timer1 + "t1_line1\nt1_line2\nt1_line3\n");
}

void wx_timerDialog::OnTimer2(wxTimerEvent& event)
{
    m_timer2 = Common(m_timer2 + "t2_line1\nt2_line2\n=t2_line3\n");
}

wxString wx_timerDialog::CommonFunc(wxString const &output)
{
    if(m_in_handler)
    {
        int start = 0;

        while(1)
        {
            wxString::size_type pos = output.find('\n', start);
            if(pos != wxString::npos)
            {
                m_lines.push_back(output.substr(start, pos - start));
                start = pos + 1;
            }
            else
                break;
        }

        if(start < output.length())
        {
            return output.substr(start, output.length() - start);
        }
    }
    else
    {
        m_in_handler = true;

        while(!m_lines.empty())
        {
            wxString const &s = m_lines.front();

            /// do some processing

            m_lines.pop_front();
        }

        m_in_handler = false;
    }
}

Any comments and suggestions are welcome.