Slowdown after saving files...

[rickg22]

Thanks to Thomas, I realized why Code::Blocks "freezes" for a couple of seconds when you save files.

Code

bool Parser::Reparse(const wxString& filename, bool isLocal)
{
        if (!Done())
                return false; // if still parsing, exit with error

        wxString file = UnixFilename(filename);
//	Manager::Get()->GetMessageManager()->DebugLog(_("Reparsing %s"), file.c_str());
	RemoveFile(file);
        ClearTemporaries();
        ReCreateTree();
        {
        wxCriticalSectionLocker lock(s_mutexListProtection);
        m_ReparsedFiles.Add(file);
        }

        return Parse(file, isLocal);
}

bool Parser::RemoveFile(const wxString& filename)
{
        wxCriticalSectionLocker lock(s_mutexListProtection);
        wxCriticalSectionLocker lock1(s_mutexProtection);

        wxString file = UnixFilename(filename);
        if (m_ParsedFiles.Index(file) != wxNOT_FOUND)
        {
                // only if it has been parsed before...
		// delete any entries that belong to the file in question
		// FIXME: what happens with entries *linked* to this entry?
		unsigned int i = 0;
                while (i < m_Tokens.GetCount())
                {
                        if (m_Tokens[i]->m_Filename.Matches(file))
                                m_Tokens.RemoveAt(i);
                        else
                                ++i;
                }
                m_ParsedFiles.Remove(file);
        }
        else
                return false;
        return true;
}

void Parser::ClearTemporaries()
{
        if (!Done())
                return;

        unsigned int i = 0;
        while (i < m_Tokens.GetCount())
        {
                Token* token = m_Tokens[i];
                if (token->m_IsTemporary)
                        m_Tokens.RemoveAt(i);
                else
                        ++i;
        }
}

void Parser::ReCreateTree()
{
    m_TokensTree.Clear();
    unsigned int i;
    for(i = 0; i < m_Tokens.GetCount();i++)
    {
        m_TokensTree.AddToken(m_Tokens[i]->m_Name,m_Tokens[i]);
    }
}

The ReCreateTree was added by Moi, as a quick hack to sync the Tree and the TokensArray.

But if all this is done once PER file, ugh

[thomas]

To summarize, three things are evil here:
1. Matches() is used where IsSameAs() would do -- we don't have wildcards here
2. m_Tokens.RemoveAt(i) requires a memcpy(), this is done many thousand times
3. It all runs in the GUI thread (and there is a mutex/critical section which synchronises the time-consuming operation in addition)

[rickg22]

I'll rewrite it.

The ParserThread will be now passed a flag whether it must reparse the file in question. If so, delete the corresponding tokens.

The TokensTree class (a wrapper around my SearchTree) will have various arrays (each per filename + 1 for temporaries) of token numbers, which will be indexes to the m_Tokens array. When deleting a token, its index will be added to a "free tokens" stack, so the m_Tokens array will not have to be resized *AT ALL*. (Additionally, it'll remove the token number from the SearchTree).

That'll fix it.

[tiwag]

...That'll fix it.

doit asap plz 

[thomas]

Moved by popular demand:

I'll try to figure out how to fix it.
Edit: I'll post about this on another thread.

I have played with it, and found out that even allocating a second array, copying the remaining tokens, and then re-assigning the temporary (which is a copy operation, again) is about 20% faster than the current algorithm.
It would still be a lot faster if TokensArray* was used instead of TokensArray (swap pointers instead of assignment copy), but that would require a lot of code to be rewritten.

But another question is... do we need the array at all? If we have the tree, what do we need the array for? It seems the only thing the array is really used for is building the array... everything else can be done with the tree alone (unless I miss something).

My main point is that it does not really matter how long it takes, as long as the main thread is not stalled. If a separate thread which has sub-GUI priority takes 2-3 seconds to do whatever stuff, nobody cares. This just MUST not happen in the main thread (and not be locked with a mutex...)

[rickg22]

But another question is... do we need the array at all? If we have the tree, what do we need the array for? It seems the only thing the array is really used for is building the array... everything else can be done with the tree alone (unless I miss something).

You know, you're so right about that! I'll get rid of that array and replace it with a vector *inside* my TokensTree class.

But then again, I have to ask Yiannis how the temporary tokens work. I can't get it fixed without that info.