C::B with DirectWrite enabled will crash when in Windows remote desktop

[ollydbg]

I see this crash, if I enabled DirectWrite, and I first close the remote desktop, and keep C::B running in the remote PC. And later if I open the remote desktop, the C::B will crash.

Here is the crash call stack I got from the remotePC's gdb.

Code

[debug]#0  0x000007fee5120f2d in ID2D1RenderTarget::DrawBitmap (this=0x0, bitmap=0xa5a15f0, destinationRectangle=..., opacity=1, interpolationMode=D2D1_BITMAP_INTERPOLATION_MODE_NEAREST_NEIGHBOR, sourceRectangle=...) at D:\code\msys2\mingw64\include\d2d1.h:904
[debug]#1  0x000007fee4f5dc75 in SurfaceD2D::Copy (this=0xa771040, rc=..., from=..., surfaceSource=...) at D:\code\cbsource\cb_svn_git\src\sdk\wxscintilla\src\PlatWX.cpp:1523
[debug]#2  0x000007fee5039b18 in EditView::PaintText (this=0xa4d5ec0, surfaceWindow=0xa771040, model=..., rcArea=..., rcClient=..., vsDraw=...) at D:\code\cbsource\cb_svn_git\src\sdk\wxscintilla\src\scintilla\src\EditView.cxx:2096
[debug]#3  0x000007fee5011c0b in Editor::Paint (this=0xa4d5910, surfaceWindow=0xa771040, rcArea=...) at D:\code\cbsource\cb_svn_git\src\sdk\wxscintilla\src\scintilla\src\Editor.cxx:1762
[debug]#4  0x000007fee4f64f37 in ScintillaWX::DoPaint (this=0xa4d5910, dc=0x23eb90, rect=...) at D:\code\cbsource\cb_svn_git\src\sdk\wxscintilla\src\ScintillaWX.cpp:918
[debug]#5  0x000007fee4f54dad in wxScintilla::OnPaint (this=0xa4d2910) at D:\code\cbsource\cb_svn_git\src\sdk\wxscintilla\src\wxscintilla.cpp:5234
[debug]#6  0x000007feea2d2e77 in wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#7  0x000007feea422aa5 in wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#8  0x000007feea422c8b in wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#9  0x000007feea4232cd in wxEvtHandler::TryHereOnly(wxEvent&) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#10 0x000007feea423343 in wxEvtHandler::ProcessEventLocally(wxEvent&) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#11 0x000007feea423421 in wxEvtHandler::ProcessEvent(wxEvent&) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#12 0x000007feea424edb in wxEvtHandler::SafelyProcessEvent(wxEvent&) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#13 0x000007feea5289d8 in wxWindow::HandlePaint() () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#14 0x000007feea52aa68 in wxWindow::MSWHandleMessage(long long*, unsigned int, unsigned long long, long long) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#15 0x000007feea515efc in wxWindow::MSWWindowProc(unsigned int, unsigned long long, long long) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#16 0x0000000077008975 in USER32!GetWindowDC () from C:\Windows\system32\user32.dll
[debug]#17 0x000000007700729b in USER32!SetWindowTextW () from C:\Windows\system32\user32.dll
[debug]#18 0x00000000770067e9 in USER32!IsDialogMessageW () from C:\Windows\system32\user32.dll
[debug]#19 0x000000007729b5cf in ntdll!KiUserCallbackDispatcher () from C:\Windows\SYSTEM32\ntdll.dll
[debug]#20 0x0000000077006e2a in USER32!BeginPaint () from C:\Windows\system32\user32.dll
[debug]#21 0x0000000077006e3c in USER32!BeginPaint () from C:\Windows\system32\user32.dll
[debug]#22 0x0000000077006782 in USER32!IsDialogMessageW () from C:\Windows\system32\user32.dll
[debug]#23 0x000007feea519330 in wxWindow::MSWSafeIsDialogMessage(tagMSG*) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#24 0x000007feea5195d9 in wxWindow::MSWProcessMessage(tagMSG*) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#25 0x000007feea54a4a0 in wxGUIEventLoop::PreProcessMessage(tagMSG*) () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#26 0x000007feea54a7ef in wxGUIEventLoop::Dispatch() () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#27 0x000007feea3047e3 in wxEventLoopManual::ProcessEvents() () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#28 0x000007feea3048f8 in wxEventLoopManual::DoRun() () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug]#29 0x000007feea3045c8 in wxEventLoopBase::Run() () from D:\code\wxWidgets-3.2.1\lib\gcc_dll\wxmsw32u_gcc_cb.dll
[debug](More stack frames follow...)
[debug]>>>>>>cb_gdb:

The problem is that the "this" pointer is 0.

when calling the function:

Code

    void DrawBitmap(ID2D1Bitmap *bitmap, const D2D1_RECT_F &destinationRectangle, FLOAT opacity, D2D1_BITMAP_INTERPOLATION_MODE interpolationMode, const D2D1_RECT_F &sourceRectangle) {
        DrawBitmap(bitmap, &destinationRectangle, opacity, interpolationMode, &sourceRectangle);
    }

Which means the "m_pRenderTarget" will be 0.

In the function:

Code

void SurfaceD2D::Copy(PRectangle rc, Point from, Surface &surfaceSource) {
    SurfaceD2D &surfOther = static_cast<SurfaceD2D &>(surfaceSource);
    surfOther.FlushDrawing();

    wxCOMPtr<ID2D1Bitmap> pBitmap;
    wxCOMPtr<ID2D1BitmapRenderTarget> pCompatibleRenderTarget(
        reinterpret_cast<ID2D1BitmapRenderTarget*>(surfOther.m_pRenderTarget.get()));

    HRESULT hr = pCompatibleRenderTarget->GetBitmap(&pBitmap);

    if ( SUCCEEDED(hr) )
    {
        D2D1_RECT_F rcDestination = {rc.left, rc.top, rc.right, rc.bottom};
        D2D1_RECT_F rcSource =
            {from.x, from.y, from.x + rc.Width(), from.y + rc.Height()};
        m_pRenderTarget->DrawBitmap(pBitmap, rcDestination, 1.0f,
            D2D1_BITMAP_INTERPOLATION_MODE_NEAREST_NEIGHBOR, rcSource);

        hr = m_pRenderTarget->Flush();
        if ( FAILED(hr) )
        {
            Platform::DebugPrintf("Failed Flush 0x%x\n", hr);
        }
    }
}

I have reported this kind of crash before, but I have never try to get the call stack of the crash.

[BlueHazzard]

So scintilla or wxWidgets bug?

[ollydbg]

So scintilla or wxWidgets bug?

I think it is in our wxscintilla in cb code base.

I don't see the crash issue when using Notepad++. Please note that Notepad++ has direct2d enabled.

[BlueHazzard]

I just checked... Are we really using version 3.7.5. of scintilla?
I think we should update....
scintilla in wxWidgets seems to be 3.7.2 and official scintilla version is 5.3.1

there could be tons of improvements...

> Added change history which can display document changes (modified, saved, ...) in the margin or in the text.
we probably could also remove some of our own code in favour of this... As far as i know the change bar is our own implementation...

[Miguel Gimenez]

Following this tutorial it says:

Your application should create render targets once and hold on to them for the life of the application or until the D2DERR_RECREATE_TARGET error is received. When you receive this error, you need to recreate the render target (and any resources it created).

[AndrewCot]

I just checked... Are we really using version 3.7.5. of scintilla?
I think we should update....
scintilla in wxWidgets seems to be 3.7.2 and official scintilla version is 5.3.1

there could be tons of improvements...

> Added change history which can display document changes (modified, saved, ...) in the margin or in the text.
we probably could also remove some of our own code in favour of this... As far as i know the change bar is our own implementation...

Yes C::B is using an ancient version of scintilla. See https://sourceforge.net/p/codeblocks/tickets/1107/ ticket for upgrading scintilla. A few weeks ago I downloaded the scintilla source and spent a few hours trying to build/link it so it could replace the existing one, but failed. It will take a bit of work, unless someone with scintilla code experience does the upgrade. I do not have enough time to do the work.

I agree that C::B should upgrade and use the un-modified source if possible, but if not then try to get the changes incorporated into the truck so it is there for next time.

[BlueHazzard]

Ok, so we hit the c++17 and even c++20 problems again...
i have read the discussion in https://github.com/wxWidgets/wxWidgets/pull/1331 and they want to support tow versions one pre c++17 and one post c++17 based on the used compiler...
What is our requirement for c++ standard?

[ollydbg]

Following this tutorial it says:

Your application should create render targets once and hold on to them for the life of the application or until the D2DERR_RECREATE_TARGET error is received. When you receive this error, you need to recreate the render target (and any resources it created).

Hi, Miguel Gimenez. Yes, I agree with this quoted text. That's the direction to fix the crash issue. The reason I would like to try the DirectWrite is that some TTF font renders(the font has hinting feature) extremely slow in the traditional default GDI code .

About how to use a modern scintilla, my idea is that our own wxScintilla should be updated to some modern ones. I believe our friend CodeLite already do that.
I only thing we need to keep is the parsing of the wxSmith generated code, which is something like:

Code

    //(*EventTable(SomeFrame)
    //*)

This is how the folding support about the wxSmith generated code.

Do we need to support other C::B special options?

[BlueHazzard]

>  CodeLite already do that.
have you some link? i searched the repo, but did not find any related code. The only thing i could find was that he is using the wxStyledTextCtrl, but i may be wrong here...

[ollydbg]

>  CodeLite already do that.
have you some link? i searched the repo, but did not find any related code. The only thing i could find was that he is using the wxStyledTextCtrl, but i may be wrong here...

Oh, sorry, you are correct.

I think Codelite use the wxStyledTextCtrl shipped with the wx release.

[ollydbg]

I'm just testing the stc sample from official wx 3.2.1. (by this project: codeblocks cbp projects for wx samples）

In it's menu, there is an option named "Technology", and I change the value to DirectWrite.

Now, I just close the remote desktop client, and logged again, I see the same crash happens in the stc sample application. So, the crash issue also exists in the wxStyledTextCtrl. 

[ollydbg]

I wrote an issue in wx's github, see here:

wxStyledTextCtrl with DirectWrite enabled will crash when user logs in by Windows remote desktop  Issue #22872 wxWidgets/wxWidgets

[ollydbg]

I just download the scintilla 3.21.1(I think it is the long term branch, see here: https://sourceforge.net/projects/scintilla/files/scintilla/3.21.1/ ) source code, and search the keyword: D2DERR_RECREATE_TARGET.

I got this:

Code

bool ScintillaWin::PaintDC(HDC hdc) {
	if (technology == SC_TECHNOLOGY_DEFAULT) {
		AutoSurface surfaceWindow(hdc, this);
		if (surfaceWindow) {
			Paint(surfaceWindow, rcPaint);
			surfaceWindow->Release();
		}
	} else {
#if defined(USE_D2D)
		EnsureRenderTarget(hdc);
		if (pRenderTarget) {
		AutoSurface surfaceWindow(pRenderTarget, this);
		if (surfaceWindow) {
			pRenderTarget->BeginDraw();
			Paint(surfaceWindow, rcPaint);
			surfaceWindow->Release();
			const HRESULT hr = pRenderTarget->EndDraw();
			if (hr == static_cast<HRESULT>(D2DERR_RECREATE_TARGET)) {
				DropRenderTarget();
					return false;
				}
			}
		}
#endif
	}

	return true;
}

And this is our C::B's implementation of the Technology in sdk\wxscintilla\src\ScintillaWX.cpp:

Code

#if defined(__WXMSW__) && wxUSE_GRAPHICS_DIRECT2D
        case SCI_SETTECHNOLOGY:
            if ((wParam == SC_TECHNOLOGY_DEFAULT) || (wParam == SC_TECHNOLOGY_DIRECTWRITE)) {
                if (technology != static_cast<int>(wParam)) {
                    SurfaceDataD2D* newSurfaceData(NULL);

                    if (static_cast<int>(wParam) > SC_TECHNOLOGY_DEFAULT) {
                        newSurfaceData =  new SurfaceDataD2D(this);

                        if (!newSurfaceData->Initialised()) {
                            // Failed to load Direct2D or DirectWrite so no effect
                            delete newSurfaceData;
                            return 0;
                        }
                    }

                    technology = static_cast<int>(wParam);
                    if ( m_surfaceData ) {
                        delete m_surfaceData;
                    }
                    m_surfaceData = newSurfaceData;

                    // Invalidate all cached information including layout.
                    DropGraphics(true);
                    InvalidateStyleRedraw();
                }
            }
            break;
#endif

And this is the scintilla 3.21.1's SCI_SETTECHNOLOGY handling in ScintillaWin.cxx:

Code

	case SCI_SETTECHNOLOGY:
		if ((wParam == SC_TECHNOLOGY_DEFAULT) ||
			(wParam == SC_TECHNOLOGY_DIRECTWRITERETAIN) ||
			(wParam == SC_TECHNOLOGY_DIRECTWRITEDC) ||
			(wParam == SC_TECHNOLOGY_DIRECTWRITE)) {
			const int technologyNew = static_cast<int>(wParam);
			if (technology != technologyNew) {
				if (technologyNew > SC_TECHNOLOGY_DEFAULT) {
#if defined(USE_D2D)
					if (!LoadD2D())
						// Failed to load Direct2D or DirectWrite so no effect
						return 0;
#else
					return 0;
#endif
				}
#if defined(USE_D2D)
				DropRenderTarget();
#endif
				technology = technologyNew;
				// Invalidate all cached information including layout.
				DropGraphics(true);
				InvalidateStyleRedraw();
			}
		}
		break;

It looks like we need a function call like DropRenderTarget() in some places. But it is a bit complex for me to understand the logic. 

[Miguel Gimenez]

As I see this problem, the Direct2D driver can nullify m_pRenderTarget (because it knows the address) whenever the target is not valid (looks like Remote Desktop tends to do this). The driver reports this situation returning D2DERR_RECREATE_TARGET when calling the driver functions, and the expected actions are:
  1- Recreate the target, probably using SurfaceDataD2D::CreateGraphicsResources()
  2- Calli the function again.

Sadly, this must be done on every call to the driver, so a lot of editing is expected.

[ollydbg]

As I see this problem, the Direct2D driver can nullify m_pRenderTarget (because it knows the address) whenever the target is not valid (looks like Remote Desktop tends to do this). The driver reports this situation returning D2DERR_RECREATE_TARGET when calling the driver functions, and the expected actions are:
  1- Recreate the target, probably using SurfaceDataD2D::CreateGraphicsResources()
  2- Calli the function again.

Sadly, this must be done on every call to the driver, so a lot of editing is expected.

Hi, Miguel Gimenez, I think you are correct. To reproduce the crash, we don't need to use the remote desktop, we just need to click the "switch user", and re-log in again.

I see in the Windows native implementation of scintilla, there is a function named EnsureRenderTarget(), this function has the ability to create a Direct2D target.

Code

void ScintillaWin::EnsureRenderTarget(HDC hdc) {
	if (!renderTargetValid) {
		DropRenderTarget();
		renderTargetValid = true;
	}
	if (pD2DFactory && !pRenderTarget) {
		HWND hw = MainHWND();

I see that this function has been called in the OnPaint event handler:

Code

bool ScintillaWin::PaintDC(HDC hdc) {
	if (technology == SC_TECHNOLOGY_DEFAULT) {
		AutoSurface surfaceWindow(hdc, this);
		if (surfaceWindow) {
			Paint(surfaceWindow, rcPaint);
			surfaceWindow->Release();
		}
	} else {
#if defined(USE_D2D)
		EnsureRenderTarget(hdc);
		if (pRenderTarget) {
		AutoSurface surfaceWindow(pRenderTarget, this);
		if (surfaceWindow) {
			pRenderTarget->BeginDraw();
			Paint(surfaceWindow, rcPaint);
			surfaceWindow->Release();
			const HRESULT hr = pRenderTarget->EndDraw();
			if (hr == static_cast<HRESULT>(D2DERR_RECREATE_TARGET)) {
				DropRenderTarget();
					return false;
				}
			}
		}
#endif
	}

	return true;
}