Hi, thanks Folco for the help.
What is approximatively the size of the data that you lock ?
The tokentree, which holds all the tokens(symbols), you can regard it as symbol database.
When we parse a file, you mainly do syntax analysis and add symbols one by one.
In the current implementation, we lock the tokentree when starting a single file, after a file is parsed, the locker is released. Something like this:
while (file queue is not empty)
{
get one file from the queue
lock the tokentree
while (parsing file not finished)
{
syntax analysis
add symbol to tokentree
if (meet a #include directive)
{
add file to the queue(for later parsing)
}
}
release the locker of tokentree
remove the file from the queue
}
So, I think it is not good idea to put the locker around "add symbol to tokentree".
while (file queue is not empty)
{
get one file from the queue
while (parsing file not finished)
{
syntax analysis
lock the tokentree
add symbol to tokentree
release the locker of tokentree
if (meet a #include directive)
{
add file to the queue(for later parsing)
}
}
remove the file from the queue
}
Because lock and release will be called too frequently.
With Huki's patch, the code becomes:
while (file queue is not empty)
{
get one file from the queue
lock the tokentree
while (parsing file not finished)
{
syntax analysis
add symbol to tokentree
if (meet a #include directive)
{
recursive parse on the included source file
}
}
release the locker of tokentree
remove the file from the queue
}
So, you can get the correct token, as the #include directive is correctly expanded(it does the same thing as C preprocessor do), the time of the locker becomes longer.
I have no idea what is the correct idea, I think the GUI should not access to the tokentree when the parsing is not finished.