Author Topic: Slowdown after saving files...  (Read 9798 times)

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Slowdown after saving files...
« on: December 27, 2005, 06:01:50 pm »
Thanks to Thomas, I realized why Code::Blocks "freezes" for a couple of seconds when you save files.

Code: [Select]
bool Parser::Reparse(const wxString& filename, bool isLocal)
{
        if (!Done())
                return false; // if still parsing, exit with error

        wxString file = UnixFilename(filename);
// Manager::Get()->GetMessageManager()->DebugLog(_("Reparsing %s"), file.c_str());
RemoveFile(file);
        ClearTemporaries();
        ReCreateTree();
        {
        wxCriticalSectionLocker lock(s_mutexListProtection);
        m_ReparsedFiles.Add(file);
        }

        return Parse(file, isLocal);
}

bool Parser::RemoveFile(const wxString& filename)
{
        wxCriticalSectionLocker lock(s_mutexListProtection);
        wxCriticalSectionLocker lock1(s_mutexProtection);

        wxString file = UnixFilename(filename);
        if (m_ParsedFiles.Index(file) != wxNOT_FOUND)
        {
                // only if it has been parsed before...
// delete any entries that belong to the file in question
// FIXME: what happens with entries *linked* to this entry?
unsigned int i = 0;
                while (i < m_Tokens.GetCount())
                {
                        if (m_Tokens[i]->m_Filename.Matches(file))
                                m_Tokens.RemoveAt(i);
                        else
                                ++i;
                }
                m_ParsedFiles.Remove(file);
        }
        else
                return false;
        return true;
}

void Parser::ClearTemporaries()
{
        if (!Done())
                return;

        unsigned int i = 0;
        while (i < m_Tokens.GetCount())
        {
                Token* token = m_Tokens[i];
                if (token->m_IsTemporary)
                        m_Tokens.RemoveAt(i);
                else
                        ++i;
        }
}

void Parser::ReCreateTree()
{
    m_TokensTree.Clear();
    unsigned int i;
    for(i = 0; i < m_Tokens.GetCount();i++)
    {
        m_TokensTree.AddToken(m_Tokens[i]->m_Name,m_Tokens[i]);
    }
}
The ReCreateTree was added by Moi, as a quick hack to sync the Tree and the TokensArray.

But if all this is done once PER file, ugh :(

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Slowdown after saving files...
« Reply #1 on: December 27, 2005, 06:19:44 pm »
To summarize, three things are evil here:
1. Matches() is used where IsSameAs() would do -- we don't have wildcards here
2. m_Tokens.RemoveAt(i) requires a memcpy(), this is done many thousand times
3. It all runs in the GUI thread (and there is a mutex/critical section which synchronises the time-consuming operation in addition)
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: Slowdown after saving files...
« Reply #2 on: December 27, 2005, 07:32:39 pm »
I'll rewrite it.

The ParserThread will be now passed a flag whether it must reparse the file in question. If so, delete the corresponding tokens.

The TokensTree class (a wrapper around my SearchTree) will have various arrays (each per filename + 1 for temporaries) of token numbers, which will be indexes to the m_Tokens array. When deleting a token, its index will be added to a "free tokens" stack, so the m_Tokens array will not have to be resized *AT ALL*. (Additionally, it'll remove the token number from the SearchTree).

That'll fix it.
« Last Edit: December 27, 2005, 07:34:39 pm by rickg22 »

Offline tiwag

  • Developer
  • Lives here!
  • *****
  • Posts: 1196
  • sailing away ...
    • tiwag.cb
Re: Slowdown after saving files...
« Reply #3 on: December 27, 2005, 08:30:30 pm »

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Slowdown after saving files...
« Reply #4 on: December 27, 2005, 08:50:50 pm »
Moved by popular demand:

I'll try to figure out how to fix it.
Edit: I'll post about this on another thread.
I have played with it, and found out that even allocating a second array, copying the remaining tokens, and then re-assigning the temporary (which is a copy operation, again) is about 20% faster than the current algorithm.
It would still be a lot faster if TokensArray* was used instead of TokensArray (swap pointers instead of assignment copy), but that would require a lot of code to be rewritten.

But another question is... do we need the array at all? If we have the tree, what do we need the array for? It seems the only thing the array is really used for is building the array... everything else can be done with the tree alone (unless I miss something).

My main point is that it does not really matter how long it takes, as long as the main thread is not stalled. If a separate thread which has sub-GUI priority takes 2-3 seconds to do whatever stuff, nobody cares. This just MUST not happen in the main thread (and not be locked with a mutex...)
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: Slowdown after saving files...
« Reply #5 on: December 27, 2005, 09:05:36 pm »
But another question is... do we need the array at all? If we have the tree, what do we need the array for? It seems the only thing the array is really used for is building the array... everything else can be done with the tree alone (unless I miss something).

You know, you're so right about that! I'll get rid of that array and replace it with a vector *inside* my TokensTree class.

But then again, I have to ask Yiannis how the temporary tokens work. I can't get it fixed without that info.