parser test rev 7157 error?

Developer forums (C::B DEVELOPMENT STRICTLY!) > CodeCompletion redesign

<< < (3/7) > >>

Loaden:

--- Quote from: ollydbg on May 26, 2011, 10:47:38 am ---@loaden, In the parsertest.cbp, we do NOT use any kind of threadpool for batch parsing, do we? The parsertest project does not use multithread, does it?

--- End quote ---
In ParserTest, we are not use threadpool.
But in CB, we use threadpool, see here.

--- Quote ---bool Parser::Parse(const wxString& bufferOrFilename, bool isLocal, ParserThreadOptions& opts)
{
bool result = false;
do
{
if (!opts.useBuffer)
{
wxCriticalSectionLocker locker(s_TokensTreeCritical);

bool canparse = !m_TokensTree->IsFileParsed(bufferOrFilename);
if (canparse)
canparse = m_TokensTree->ReserveFileForParsing(bufferOrFilename, true) != 0;

if (!canparse)
{
if (opts.loader) // if a loader is already open at this point, the caller must clean it up
Manager::Get()->GetLogManager()->DebugLog(_T("Parse() : CodeCompletion Plugin: FileLoader memory leak likely while loading file ")+bufferOrFilename);
break;
}

if (!opts.loader) // this should always be true (memory will leak if a loader has already been initialized before this point)
opts.loader = Manager::Get()->GetFileManager()->Load(bufferOrFilename, m_NeedsReparse);
}

TRACE(_T("Parse() : Creating task for: %s"), bufferOrFilename.wx_str());
ParserThread* thread = new ParserThread(this, bufferOrFilename, isLocal, opts, m_TokensTree);
bool doParseNow = opts.useBuffer;
#if CC_PARSER_PROFILE_TEST
doParseNow = true;
#endif
//if we are parsing a memory buffer or Parser is under Profile
// CC_PARSER_PROFILE_TEST is defined as 1), then just call Parse()
// directly and thread pool is NOT used.
// Otherwise, we are parsing a local file, so the thread pool is used.
// The ParserThread generated was pushed to the memory pool.

if (doParseNow)
{
result = thread->Parse();
LinkInheritance(true);
delete thread;
break;
}

TRACE(_T("Parse() : Parsing %s"), bufferOrFilename.wx_str());

if (!m_IsPriority)
{
TRACE(_T("Parse() : Parallel Parsing %s"), bufferOrFilename.wx_str());

// Add a task for all project files
if (m_IsFirstBatch)
{
m_IsFirstBatch = false;
m_PoolTask.push(PTVector());
}

if (m_IsParsing)
m_Pool.AddTask(thread, true);
else
m_PoolTask.back().push_back(thread);
}
else if (m_IsPriority)
{
if (isLocal) // Parsing priority files
{
TRACE(_T("Parse() : Parsing priority header, %s"), bufferOrFilename.wx_str());
result = thread->Parse();
delete thread;
break;
}
else // Add task when parsing priority files
{
TRACE(_T("Parse() : Add task for priority header, %s"), bufferOrFilename.wx_str());
m_PoolTask.push(PTVector());
m_PoolTask.back().push_back(thread);
}
}

result = true;
}
while (false);

return result;
}
--- End quote ---

One header/source file per one ParserThread.
And operating them one by one.

Loaden:
So, you can create a stand project, and debug again.
This case only happen with parsertest project.

Loaden:

--- Quote from: Loaden on May 27, 2011, 10:34:13 am ---So, you can create a stand project, and debug again.
This case only happen with parsertest project.

--- End quote ---
OMG, Seems still have some problem.

--- Quote ---bool Parser::Parse(const wxString& bufferOrFilename, bool isLocal, ParserThreadOptions& opts)
{
bool result = false;
do
{
if (!opts.useBuffer)
{
wxCriticalSectionLocker locker(s_TokensTreeCritical);

bool canparse = !m_TokensTree->IsFileParsed(bufferOrFilename);
if (canparse)
canparse = m_TokensTree->ReserveFileForParsing(bufferOrFilename, true) != 0;

if (!canparse)
{
if (opts.loader) // if a loader is already open at this point, the caller must clean it up
Manager::Get()->GetLogManager()->DebugLog(_T("Parse() : CodeCompletion Plugin: FileLoader memory leak likely while loading file ")+bufferOrFilename);
break;
}

if (!opts.loader) // this should always be true (memory will leak if a loader has already been initialized before this point)
opts.loader = Manager::Get()->GetFileManager()->Load(bufferOrFilename, m_NeedsReparse);
}

TRACE(_T("Parse() : Creating task for: %s"), bufferOrFilename.wx_str());
ParserThread* thread = new ParserThread(this, bufferOrFilename, isLocal, opts, m_TokensTree);
bool doParseNow = opts.useBuffer;
#if CC_PARSER_PROFILE_TEST
doParseNow = true;
#endif
//if we are parsing a memory buffer or Parser is under Profile
// CC_PARSER_PROFILE_TEST is defined as 1), then just call Parse()
// directly and thread pool is NOT used.
// Otherwise, we are parsing a local file, so the thread pool is used.
// The ParserThread generated was pushed to the memory pool.

if (doParseNow)
{
result = thread->Parse();
LinkInheritance(true);
delete thread;
break;
}

TRACE(_T("Parse() : Parsing %s"), bufferOrFilename.wx_str());

if (!m_IsPriority)
{
TRACE(_T("Parse() : Parallel Parsing %s"), bufferOrFilename.wx_str());

// Add a task for all project files
if (m_IsFirstBatch)
{
m_IsFirstBatch = false;
m_PoolTask.push(PTVector());
}

if (m_IsParsing)
m_Pool.AddTask(thread, true);
else
m_PoolTask.back().push_back(thread);
}
else if (m_IsPriority)
{
if (isLocal) // Parsing priority files
{
TRACE(_T("Parse() : Parsing priority header, %s"), bufferOrFilename.wx_str());
result = thread->Parse(); // ### HERE, some thing is wrong!
delete thread;
break;
}
else // Add task when parsing priority files
{
TRACE(_T("Parse() : Add task for priority header, %s"), bufferOrFilename.wx_str());
m_PoolTask.push(PTVector());
m_PoolTask.back().push_back(thread);
}
}

result = true;
}
while (false);

return result;
}

--- End quote ---
:(

ollydbg:
@loaden, let's focus our discussion on the ParserTest project.
As there is no multi-thread running thread pool, all the parsing is done in the main thread. and there are some recursive called if the parser meets a #include directive. My puzzle is still the same, as it is only one global critical section, why it cam create different critical section lockers? and it seems these sections were entered several times???? :(

Loaden:
I'm removed the locker in my local commit (git-svn).
And commit them on tonight.

Navigation

[0] Message Index

[#] Next page

[*] Previous page

Go to full version