Developer forums (C::B DEVELOPMENT STRICTLY!) > Development
Several improvements to Code Completion plugin
ollydbg:
Today, I have tested the patch: cc_includes_parsing.patch
But I found it cause some GUI hang problem, see my report question about running parser in thread, I guess the reason is:
If you recursively parse the #include directive.
--- Code: --- if (!locked)
CC_LOCKER_TRACK_TT_MTX_LOCK(s_TokenTreeMutex)
//this is a recursive call to Parse() function????
result = thread->Parse();
delete thread;
if (!locked)
CC_LOCKER_TRACK_TT_MTX_UNLOCK(s_TokenTreeMutex)
--- End code ---
So, you hold the s_TokenTreeMutex for a very long time. Some GUI function may try to query information from Tokentree, thus they get hangs.
Folco:
What is approximatively the size of the data that you lock ? If it is not too huge, would it be a solution to lock a mutex, then copy the data that you need, then unlock the mutex ? It could be really faster that what you are currently doing.
I have this code in one of my projects :
--- Code: --- SDL_mutexP (rsd->RenderingMutex);
rgd = rsd->rgd;
SDL_mutexV (rsd->RenderingMutex);
--- End code ---
This code is run in a rendering thread, which grabs only the data that it needs to draw, then unlock data for the main thread to continue. Perhaps something like that would remove some CC hangs and wouldn't freeze the GUI anymore.
ollydbg:
Hi, thanks Folco for the help.
--- Quote ---What is approximatively the size of the data that you lock ?
--- End quote ---
The tokentree, which holds all the tokens(symbols), you can regard it as symbol database.
When we parse a file, you mainly do syntax analysis and add symbols one by one.
In the current implementation, we lock the tokentree when starting a single file, after a file is parsed, the locker is released. Something like this:
--- Code: ---while (file queue is not empty)
{
get one file from the queue
lock the tokentree
while (parsing file not finished)
{
syntax analysis
add symbol to tokentree
if (meet a #include directive)
{
add file to the queue(for later parsing)
}
}
release the locker of tokentree
remove the file from the queue
}
--- End code ---
So, I think it is not good idea to put the locker around "add symbol to tokentree".
--- Code: ---while (file queue is not empty)
{
get one file from the queue
while (parsing file not finished)
{
syntax analysis
lock the tokentree
add symbol to tokentree
release the locker of tokentree
if (meet a #include directive)
{
add file to the queue(for later parsing)
}
}
remove the file from the queue
}
--- End code ---
Because lock and release will be called too frequently.
With Huki's patch, the code becomes:
--- Code: ---while (file queue is not empty)
{
get one file from the queue
lock the tokentree
while (parsing file not finished)
{
syntax analysis
add symbol to tokentree
if (meet a #include directive)
{
recursive parse on the included source file
}
}
release the locker of tokentree
remove the file from the queue
}
--- End code ---
So, you can get the correct token, as the #include directive is correctly expanded(it does the same thing as C preprocessor do), the time of the locker becomes longer.
I have no idea what is the correct idea, I think the GUI should not access to the tokentree when the parsing is not finished.
Folco:
And something like that :
--- Code: ---lock the tokentree
copy the token tree
release the tokentree
while (file queue not empty)
{
analysis / add symbol / fill queue with include directives, modifying copied_tokentree
lock the tokentree, update tokentree with copied_tokentree, then release it
}
--- End code ---
Note that the token tree should contain a var saying "I am (not) in a writable state/copiable state". I don't know what use it.
This should have 2 consequences :
- the token tree is locked during short peroid of time, without risk of infinite loop (recursive inclusion or other funky trick)
- nobody has to wait for the parser to finish its task. Even if a parser would crash, the GUI would keep responsive.
ollydbg:
--- Quote from: Folco on January 04, 2014, 02:14:31 pm ---And something like that :
--- Code: ---lock the tokentree
copy the token tree
release the tokentree
while (file queue not empty)
{
analysis / add symbol / fill queue with include directives, modifying copied_tokentree
lock the tokentree, update tokentree with copied_tokentree, then release it
}
--- End code ---
--- End quote ---
What does the above pseudo code used for? For parser? the copied tokentree are used for GUI?
Note A Tokentree is a very big object, so copy tokentree takes a lot of time. So, it is a shared object, can either write or read at a time.
I think if the parser is running, I mean the tokentree should not be accessed by GUI thread, the GUI should show some words like: parsing is still running. Or delay the operation until tokentree stabilized (parsing finished).
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version