Developer forums (C::B DEVELOPMENT STRICTLY!) > CodeCompletion redesign
using memory pool for the SearchTree Node
ollydbg:
Hi, here is the patch, I just make a simple test, it can reduce 25% parsing time.
--- Code: ---Index: searchtree.h
===================================================================
--- searchtree.h (revision 5679)
+++ searchtree.h (working copy)
@@ -5,7 +5,8 @@
#ifndef SEARCHTREE_H
#define SEARCHTREE_H
-#include <wx/string.h>
+#include <wx/string.h>
+#include "blockallocated.h"
#include <vector>
#include <map>
@@ -74,7 +75,7 @@
SearchTreePoint (nSearchTreeNode nn, size_t dd) { n = nn; depth = dd; }
};
-class SearchTreeNode
+class SearchTreeNode:public BlockAllocated<SearchTreeNode, 10000>
{
friend class BasicSearchTree;
friend class BasicSearchTreeIterator;
--- End code ---
Can any one tested for me?
By the way, can any developer would like to use the way I stated in this post?
http://forums.codeblocks.org/index.php/topic,10598.msg72616.html#msg72616
Thanks! :D
thomas:
This is exactly what I wrote block allocator for. :-)
However, the code completion parser was later made multithreaded in an attempt to speed it up, and the block allocator is (obviously) not thread-safe. Well, in fact it is thread safe, but not concurrency safe. Two threads allocating at the exact same time may get the same block of valid memory, and two blocks freeing at the exact same time may leak a block of memory.
So, your patch will likely work, but not 100% reliably. To make the block allocator thread-safe, one would have to use an atomic compare/exchange for the single-linked list of free nodes, but then again, this would make it slower.
ollydbg:
Thanks for the reply.
I noticed that the current Token is already allocated from memory pool.
--- Code: ---class Token : public BlockAllocated<Token, 10000>
--- End code ---
seems it always works fine :D
ollydbg:
Ok, I find some code that can confirm it is quite "SAFE", When adding a Token, it use a s_MutexProtection. In the parsing thread.
see the code below:
--- Code: ---Token* ParserThread::DoAddToken(TokenKind kind, const wxString& name, int line, int implLineStart, int implLineEnd, const wxString& args, bool isOperator, bool isImpl)
{
if(TestDestroy())
return 0;
if (name.IsEmpty())
return 0; // oops!
// if (m_Options.useBuffer && !m_Options.isTemp && TokenExists(name, m_pLastParent, kind))
// return 0;
s_MutexProtection.Enter();
Token* newToken = 0;
wxString newname(name);
m_Str.Trim();
//if (name == _T("AlsoDoSomething"))
// asm("int $3;");
if (kind == tkDestructor)
{
// special class destructors case
newname.Prepend(ParserConsts::tilde);
m_Str.Clear();
}
Token* localParent = 0;
// preserve m_EncounteredTypeNamespaces; needed further down this function
std::queue<wxString> q = m_EncounteredTypeNamespaces;
if ((kind == tkDestructor || kind == tkConstructor) && !q.empty())
{
// look in m_EncounteredTypeNamespaces
localParent = FindTokenFromQueue(q, 0, true, m_pLastParent);
if (localParent)
newToken = TokenExists(newname, localParent);
}
// check for implementation member function
if (!newToken && !m_EncounteredNamespaces.empty())
{
localParent = FindTokenFromQueue(m_EncounteredNamespaces, 0, true, m_pLastParent);
if (localParent)
newToken = TokenExists(newname, localParent);
}
// none of the above; check for token under parent (but not if we 're parsing a temp buffer)
if (!newToken && !m_Options.isTemp)
newToken = TokenExists(name, m_pLastParent, kind);
if (newToken && newToken->m_TokenKind == kind && newToken->m_Args == args)
{
m_pTokens->m_modified = true;
}
else
{
Token* finalParent = localParent ? localParent : m_pLastParent;
newToken = new Token(newname,m_File,line);
newToken->m_ParentIndex = finalParent ? finalParent->GetSelf() : -1;
newToken->m_TokenKind = kind;
newToken->m_Scope = m_LastScope;
newToken->m_Args = args;
int newidx = m_pTokens->insert(newToken);
if (finalParent)
finalParent->AddChild(newidx);
}
if (!(kind & (tkConstructor | tkDestructor)))
{
wxString readType = m_Str;
wxString actualType = GetActualTokenType();
if (actualType.Find(_T(' ')) == wxNOT_FOUND)
{
// token type must contain all namespaces
actualType.Prepend(GetQueueAsNamespaceString(m_EncounteredTypeNamespaces));
}
newToken->m_Type = readType;
newToken->m_ActualType = actualType;
}
newToken->m_IsLocal = m_IsLocal;
newToken->m_IsTemp = m_Options.isTemp;
newToken->m_IsOperator = isOperator;
if (!isImpl)
{
newToken->m_File = m_File;
newToken->m_Line = line;
// Manager::Get()->GetLogManager()->DebugLog(F(_T("Added/updated token '%s' (%d), type '%s', actual '%s'. Parent is %s (%d)"), name.c_str(), newToken->GetSelf(), newToken->m_Type.c_str(), newToken->m_ActualType.c_str(), newToken->GetParentName().c_str(), newToken->m_ParentIndex));
}
else
{
newToken->m_ImplFile = m_File;
newToken->m_ImplLine = line;
newToken->m_ImplLineStart = implLineStart;
newToken->m_ImplLineEnd = implLineEnd;
m_pTokens->m_FilesMap[newToken->m_ImplFile].insert(newToken->GetSelf());
}
while (!m_EncounteredNamespaces.empty())
m_EncounteredNamespaces.pop();
while (!m_EncounteredTypeNamespaces.empty())
m_EncounteredTypeNamespaces.pop();
s_MutexProtection.Leave();
// wxMilliSleep(0);
return newToken;
}
--- End code ---
Jenna:
Parsing a large test-project (linux-kernel 2.6.29):
C::B from wxAui test-branch without your patch:
--- Quote ---Parsing stage done (22382 total parsed files, 914536 tokens in 2 minute(s), 21.896 seconds).
--- End quote ---
and with your patch applied:
--- Quote ---Parsing stage done (22382 total parsed files, 914536 tokens in 2 minute(s), 22.432 seconds).
--- End quote ---
Exact the same code except for your patch, clean compile in both cases, same system-load.
I don't see a time improvement here (debian 64-bit Core2Duo 2x2GB, 2x4MB cache).
But as thomas stated it might be dangerous in some (most likely very rare) cases.
Did not (yet) check on any other system.
Navigation
[0] Message Index
[#] Next page
Go to full version