hello ,devs:
the codes in Parser::Parse in parser file:
if(!opts.loader) //this should always be true (memory will leak if a loader has already been initialized before this point)
opts.loader=Manager::Get()->GetFileManager()->Load(bufferOrFilename, true);
if I change the function Load() args from false(before) to true(the black one above). then the cc can not parse the codes correctly.
I think it is something relatived to these codes:
NullLoader *nl = new NullLoader(file, (char*) s.c_str(), s.length() * sizeof(wxChar));
I have a question here.
here are the class LoaderBase's implementation.
// ***** class: LoaderBase *****
class LoaderBase : public AbstractJob
{
wxSemaphore sem;
bool wait;
void WaitReady()
{
if(wait)
{
wait = false;
sem.Wait();
}
};
protected:
wxString fileName;
char *data;
size_t len;
void Ready()
{
sem.Post();
};
public:
LoaderBase() : wait(true), data(0), len(0) {};
~LoaderBase();
void operator()() {};
wxString FileName() const { return fileName; };
bool Sync();
char* GetData();
size_t GetLength();
};
here the member variable data's type is char* ,why not wxChar*
when we do this
NullLoader *nl = new NullLoader(file, (char*) s.c_str(), s.length() * sizeof(wxChar));
and in the body of bool Tokenizer::ReadFile() in tokenizer.cpp
these codes:
if (m_pLoader)
{
fileName = m_pLoader->FileName();
char* data = m_pLoader->GetData();
m_BufferLen = m_pLoader->GetLength();
// the following code is faster than DetectEncodingAndConvert()
// DetectEncodingAndConvert(data, m_Buffer);
// same code as in cbC2U() but with the addition of the string length (3rd param in unicode version)
// and the fallback encoding conversion
#if wxUSE_UNICODE
m_Buffer = wxString(data, wxConvUTF8, m_BufferLen + 1); // + 1 => sentinel
if (m_Buffer.Length() == 0)
{
// could not read as utf-8 encoding, try iso8859-1
m_Buffer = wxString(data, wxConvISO8859_1, m_BufferLen + 1); // + 1 => sentinel
}
#else
m_Buffer = wxString(data, m_BufferLen + 1); // + 1 => sentinel
#endif
success = (data != 0);
}
char* data = m_pLoader->GetData(); some error happened and make the cc cannot parse the codes correctly.
Note:
if want to reproduce the issue,just change the false to true in the funciton opts.loader=Manager::Get()->GetFileManager()->Load(bufferOrFilename, true);
and these codes may cause some problem.
the variable data point to the args buffer directly.but what will happened if the buffer is destroyed?
class NullLoader : public LoaderBase
{
public:
NullLoader(const wxString& name, char* buffer, size_t size) { fileName = name; data = buffer; len = size; Ready(); };
void operator()(){};
};
and then check out these codes.
if(reuseEditors)
{
EditorManager* em = Manager::Get()->GetEditorManager();
if(em)
{
wxFileName fileName(file);
for(int i = 0; i < em->GetEditorsCount(); ++i)
{
cbEditor* ed = em->GetBuiltinEditor(em->GetEditor(i));
if(ed && fileName == ed->GetFilename())
{
wxString s(ed->GetControl()->GetText());
#if wxCHECK_VERSION(2, 9, 0)
NullLoader *nl = new NullLoader(file, (char*) s.wx_str(), s.length() * sizeof(wxChar));
#else
NullLoader *nl = new NullLoader(file, (char*) s.c_str(), s.length() * sizeof(wxChar));
#endif
return nl;
}
}
}
}
the wxString type varialbe s will be destroyed after s leave the if scope.and at this time,the data in class NullLoader will point to something which had been destroyed.
So, since you're saying that the data isn't read correctly, the probable reason is unicode text being copied from the editor and multibyte expected in the parser, or something similar.
hello,Thomas
what is your way to solve this issue.
my friend VisualFC give me such a way to solve it.
codes:
before :
NullLoader(const wxString& name, char* buffer, size_t size) { fileName = name; data = buffer; len = size; Ready(); };
VisualFC's advice:
NullLoader(const wxString& name, const wxString& str)
{
//fileName = name; data = buffer; len = size; Ready();
data = new char[str.Len()*2 + 1];
strcpy(data, (const char*)str.mb_str());
fileName = name; len = strlen(data);Ready();
};
and change the associate calling.
if(ed && fileName == ed->GetFilename())
{
//wxString s(ed->GetControl()->GetText());
//#if wxCHECK_VERSION(2, 9, 0)
//NullLoader *nl = new NullLoader(file, (char*) s.wx_str(), s.length() * sizeof(wxChar));
//#else
//NullLoader *nl = new NullLoader(file, (char*) s.c_str(), s.length() * sizeof(wxChar));
//#endif
NullLoader *nl = new NullLoader(file, ed->GetControl()->GetText());
return nl;
}
yes , it work
so thomas,what is yuor opinion on these changes? thanks
Edit: Does strlen work here in case the buffer contains \00??? What about sizeof((const char*)str.mb_str()) (pseudo-code)?
hi,morten
no idea about this.can have a try.
Edit:
have a thought on "interface compatible".maybe we can overload the construct functioin NullLoader(const wxString& name, char* buffer, size_t size),I meant adding another function NullLoader(const wxString& name, const wxString& str)
NullLoader(const wxString& name, char* buffer, size_t size)
{
fileName = name; data = buffer; len = size; Ready();
};
NullLoader(const wxString& name, const wxString& str)
{
//fileName = name; data = buffer; len = size; Ready();
data = new char[str.Len()*2 + 1];
strcpy(data, (const char*)str.mb_str());
fileName = name; len = strlen(data);Ready();
};
what is your idea here?
This snippet (FileManager::Load) copies the bytes, m_pchData points to, into NullLoader's data, so no crash anymore (I'm not sure if everything is correct, as written before I have no time, to work on it seriously).
if(ed && fileName == ed->GetFilename())
{
wxString s(ed->GetControl()->GetText());
int len = s.length() + 1;
NullLoader *nl = new NullLoader(file, (char*)wxStrcpy(new wxChar[len], s.c_str()), len * sizeof(wxChar));
return nl;
}
But you get a problem here:
If I see it correctly, FileLoader loads the content of the files into data (byte by byte from disk), but the editors hold a widechar-string (at least in unicode-builds), so loading a file from disk or via fileloader does not lead to the same content automatically and parsing might fail (happens here for example).
hi,devs:
I think we can learn something from file.h of wxWidget.
code snippet:
bool Write(const wxString& s, const wxMBConv& conv = wxConvUTF8)
{
const wxWX2MBbuf buf = s.mb_str(conv);
if (!buf)
return false;
size_t size = strlen(buf);
return Write((const char *) buf, size) == size;
}