I add some comments in the function:
bool Tokenizer::ReplaceBufferForReparse(const wxString& target, bool updatePeekToken)
// TODO ollydbg, a raw memory copy on wxString is not saft, if we will migrate to wx 2.9.x
// This is because different wxString under Linux internally use UTF8 encoding, which have
// variable length of code unit, wxChar is always wchar_t in any OS if Unicode is defined.
// Replacement back
wxChar* p = const_cast<wxChar*>((const wxChar*)m_Buffer) + m_TokenIndex - bufferLen;
TRACE(_T("ReplaceBufferForReparse() : <FROM>%s<TO>%s"), wxString(p, bufferLen).wx_str(), buffer.wx_str());
memcpy(p, (const wxChar*)target, bufferLen * sizeof(wxChar));
For some record, I add some comments about how ReplaceBufferForReparse work
/** Backward buffer replacement for re-parsing
* http://forums.codeblocks.org/index.php/topic,13384.msg90391.html#msg90391
*
* Macro expansion is just replace some characters in the m_Buffer.
*
* xxxxxxxxxAAAA(u,v)yyyyyyyyy
* ^---m_TokenIndex, m_Token = "AAAA"
* For example, the above is a wxChar Array m_Buffer, then "AAAA(u,v)" need to do a Macro
* expansion to some other text. So, we just do a "backward" text replace, so that, after
* replacement, The last replacement char was ")" in "AAAA(u,v)" (We say it as an entry point),
* so the text becomes:
*
* xxxNNNNNNNNNNNNNNNyyyyyyyyy
* ^---m_TokenIndex
* Note that "NNNNNNNNNNNN" is some macro expansion text. then the m_TokenIndex was moved
* backward to the beginning of the text.
* if the macro expansion result text is small enough, then m_Buffer's length do not need to
* change.
* The situation when our m_Buffer's length need to be change is that the macro expansion text
* is too long, so the buffer before "entry point" can not hold the new text, this way,
* m_Buffer's length will adjusted. like below:
* NNNNNNNNNNNNNNNNNNNNNNyyyyyyyyy
* ^---m_TokenIndex
*/
bool ReplaceBufferForReparse(const wxString& target, bool updatePeekToken = true);