Can someone help me in identifying the hot spots in the plugin ? Where should I start to look for possible candidates ?
Can you first try what happens if you disable the ToDo list plugin (yes: ToDo)?
I have tried that, and unfortunately it did not help.
I have tried to measure the code speed on my laptop at work : I have found a few solutions, although it is not perfect.
BUG 1 : loading a workspace is 10 times slower on Win XP than on Win 7 (laptop on WinXP is an Intel Core I7, laptop on Win7 64 is an Intel Core I5)
=> most of the time is spent in the method NativeParser::AddCompilerDirs.
More specifically : it seems to be in these following pieces of code:
// get targets include dirs
for (int i = 0; i < project->GetBuildTargetsCount(); ++i)
{
ProjectBuildTarget* target = project->GetBuildTarget(i);
if (target && target->SupportsCurrentPlatform())
{
if (compiler)
{
// post-processed search dirs (from build scripts)
for (unsigned int ti = 0; ti < compiler->GetCompilerSearchDirs(target).GetCount(); ++ti)
{
wxString out = compiler->GetCompilerSearchDirs(target)[ti];
wxFileName dir(out);
if (NormalizePath(dir, base))
{
parser->AddIncludeDir(dir.GetFullPath());
TRACE(_T("AddCompilerDirs() : Adding compiler target dir to parser: ") + dir.GetFullPath());
}
else
CCLogger::Get()->DebugLog(F(_T("Error normalizing path: '%s' from '%s'"), out.wx_str(), base.wx_str()));
}
}
// apply target vars
// target->GetCustomVars().ApplyVarsToEnvironment();
for (unsigned int ti = 0; ti < target->GetIncludeDirs().GetCount(); ++ti)
{
wxString out = target->GetIncludeDirs()[ti];
// Manager::Get()->GetMacrosManager()->ReplaceMacros(out);
wxFileName dir(out);
if (NormalizePath(dir, base))
{
parser->AddIncludeDir(dir.GetFullPath());
TRACE(_T("AddCompilerDirs() : Adding target dir to parser: ") + dir.GetFullPath());
}
else
CCLogger::Get()->DebugLog(F(_T("Error normalizing path: '%s' from '%s'"), out.wx_str(), base.wx_str()));
}
// get the compiler
wxString CompilerIndex = target->GetCompilerID();
Compiler* tgtCompiler = CompilerFactory::GetCompiler(CompilerIndex);
if (tgtCompiler)
{
Compilers[nCompilers] = tgtCompiler;
++nCompilers;
}
} // if (target)
} // end loop over the targets
// add compiler include dirs
for (int idxCompiler = 0; idxCompiler < nCompilers; ++idxCompiler)
{
const wxArrayString& dirs = (Compilers[idxCompiler])->GetIncludeDirs();
for (unsigned int i = 0; i < dirs.GetCount(); ++i)
{
// CCLogger::Get()->Log(mltDevDebug, "Adding %s", dirs[i].c_str());
wxString out = dirs[i];
Manager::Get()->GetMacrosManager()->ReplaceMacros(out);
wxFileName dir(out);
if (NormalizePath(dir,base))
{
parser->AddIncludeDir(dir.GetFullPath());
TRACE(_T("AddCompilerDirs() : Adding compiler dir to parser: ") + dir.GetFullPath());
}
else
CCLogger::Get()->DebugLog(F(_T("Error normalizing path: '%s' from '%s'"), out.wx_str(), base.wx_str()));
}
// find out which compiler, if gnu, do the special trick
// to find it's internal include paths
// but do only once per C::B session, thus cache for later calls
wxString CompilerID = (Compilers[idxCompiler])->GetID();
if (CompilerID.Contains(_T("gcc")))
AddGCCCompilerDirs(Compilers[idxCompiler], parser);
} // end of while loop over the found compilers
(code for finding the compiler and target include directories, and compute full-path from that).
So I have optimized a bit this method :
1 - cache loop boundaries (no need to call each loop GetCount()). I know it is probably not significant, but just for testing
2 - Optimize ParserBase::AddIncludeDir : there is a call here to wxArrayString::Index, which is linear (O(n)). I have added a HashSet for string (available in wxWidgets)
3 - clean up the code : there is 3 times the same code to do: fetch include dir, do macro expansion, normalize the path, add to include dir list. I have moved this code inside a new method ParserBase::AddIncludeDirFast, and commented out the previous code
4 - there is probably some other optimisations to perform : there is a lot of wxString copy overall.
I attach a patch with :
- the modifications discussed above
- some additional debugging code in NativeParser.cpp and codecompletion.cpp (can be deactivated).
Applying the patch above helped a lot... at first. After 1 hour, the slow behaviour resumed, so there must be another bottleneck somewhere.
BUG 2 : same freezing behaviour, but I can reproduce it at will now. Use case is as follow:
- large workspace, with several projects
- project A is active
- open a file from another project (not from the active project) => freeze for 2 to 6 minutes observed.
Debug logs pointed out to NativeParser::AddCompilerDirs again. The patch above solved this problem as well - for 1 hour...
I am sorry if I could not find out more - after a while, I had to resume what I should have been doing at work : working. I will try again when possible
Regards,
Seb
I see you add some profile timers, that's nice, I will test it. I sometimes experience that lag problem I'm editing, about 2-3 seconds, at that time, the whole editor locks.
BTW:
There are many linear searches in function like:
cbProject* NativeParser::GetProjectByFilename(const wxString& filename)
- when I omit to normalize the path of the include directories (no call to wxFileName::Normalize in NativeParser::AddCompilerDirs), I do not get the blocking behaviour. (this can perhaps help to diagnose the cause of the problem).
- all the time is spent in the method NativeParser::AddCompilerDirs (140 seconds on 141). Skipping this method resolves the blocking behaviour (but less includes directories are parsed, so there are less risk of blocking)
Do you mean this one in globals.cpp
bool NormalizePath(wxFileName& f,const wxString& base)
{
bool result = true;
// if (!f.IsAbsolute())
{
f.Normalize(wxPATH_NORM_ALL & ~wxPATH_NORM_CASE, base);
result = f.IsOk();
}
return result;
}
Truth to be said, I am a bit lost currently, because I do not know how the plugin works. I can read the source, of course, but it takes time. Currently, I work with the last debugger nightly.
The CodeCompletion plugin wiki page may give you some hints:
http://wiki.codeblocks.org/index.php?title=Code_Completion_Design
I believe that you code have some encoding problems, see the log when I set the debug log output:
Start : OnEditorClosed
End : 湏摅瑩牯汃獯摥. Done in 0 ms.
Start : OnEditorActivated
End : 湏摅瑩牯捁楴慶整d. Done in 0 ms.
Start : OnEditorOpen
End : 湏摅瑩牯灏湥. Done in 0 ms.
Start : OnEditorActivated
End : 湏摅瑩牯捁楴慶整d. Done in 0 ms.
Start : OnEditorActivated
End : 湏摅瑩牯捁楴慶整d. Done in 0 ms.
Start : OnWorkspaceChanged
End : 湏潗歲灳捡䍥慨杮摥. Done in 0 ms.
Start : OnProjectActivated
湏牐橯捥䅴瑣癩瑡摥㨠匠整⁰1. Done in 0 ms.
Start : NativeParser::CreateParser
慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰1Full parsing failed!. Done in 0 ms.
Start : NativeParser::DoFullParsing
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥ㄠ. Done in 0 ms.
Start : NativeParser::AddCompilerDirs
慎楴敶慐獲牥㨺摁䍤浯楰敬䑲物›瑳灥ㄠ. Done in 0 ms.
慎楴敶慐獲牥㨺摁䍤浯楰敬䑲物›瑳灥㈠. Done in 219 ms.
慎楴敶慐獲牥㨺摁䍤浯楰敬䑲物›瑳灥㌠. Done in 219 ms.
慎楴敶慐獲牥㨺摁䍤浯楰敬䑲物›瑳灥㐠. Done in 219 ms.
慎楴敶慐獲牥㨺摁䍤浯楰敬䑲物›瑳灥㔠. Done in 375 ms.
慎楴敶慐獲牥㨺摁䍤浯楰敬䑲物›瑳灥㘠. Done in 375 ms.
Start : ParseFunctionsAndFillToolbar
End : 慐獲䙥湵瑣潩獮湁䙤汩呬潯扬牡. Done in 0 ms.
Caching GCC dir: E:\code\gcc\mypcxmingw463\lib\gcc\i686-w64-mingw32\4.6.3\include\c++
Caching GCC dir: E:\code\gcc\mypcxmingw463\lib\gcc\i686-w64-mingw32\4.6.3\include\c++\i686-w64-mingw32
Caching GCC dir: E:\code\gcc\mypcxmingw463\lib\gcc\i686-w64-mingw32\4.6.3\include\c++\backward
Caching GCC dir: E:\code\gcc\mypcxmingw463\lib\gcc\i686-w64-mingw32\4.6.3\include
Caching GCC dir: E:\code\gcc\mypcxmingw463\include
Caching GCC dir: E:\code\gcc\mypcxmingw463\lib\gcc\i686-w64-mingw32\4.6.3\include-fixed
Caching GCC dir: E:\code\gcc\mypcxmingw463\i686-w64-mingw32\include
慎楴敶慐獲牥㨺摁䍤浯楰敬䑲物›瑳灥㜠. Done in 1734 ms.
End : 慎楴敶慐獲牥㨺摁䍤浯楰敬䑲物s. Done in 1734 ms.
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㈠. Done in 1734 ms.
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㌠. Done in 1859 ms.
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㐠. Done in 1859 ms.
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㔠. Done in 1859 ms.
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㘠. Done in 1859 ms.
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㜠. Done in 2359 ms.
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㜠. Done in 2391 ms.
Passing list of files to batch-parser.
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㠠. Done in 2391 ms.
Header to parse with priority: 'E:\code\gcc\mypcxmingw463\lib\gcc\i686-w64-mingw32\4.6.3\include\c++\cstddef'
Header to parse with priority: 'E:\code\gcc\mypcxmingw463\i686-w64-mingw32\include\w32api.h'
Header to parse with priority: 'E:\code\cb\cb_debugger_branch\src\include\sdk.h'
Add 3 priority parsing file(s) for project 'Code::Blocks wx2.8.x'...
慎楴敶慐獲牥㨺潄畆汬慐獲湩›瑳灥㤠. Done in 2391 ms.
Added 816 file(s) for project 'Code::Blocks wx2.8.x' to batch-parser...
End : 慎楴敶慐獲牥㨺潄畆汬慐獲湩g慎楴敶慐獲牥㨺潄畆汬慐獲湩›潮瀠牡敳rNativeParser::DoFullParsing(). Done in 2391 ms.
慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰2慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰3慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰4*NONE*. Done in 2391 ms.
慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰3慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰4*NONE*. Done in 2422 ms.
慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰4*NONE*. Done in 2422 ms.
Create new parser for project 'Code::Blocks wx2.8.x'
慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰5慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰6Parser does not exist for delete '%s'!. Done in 2422 ms.
慎楴敶慐獲牥㨺牃慥整慐獲牥㨠猠整⁰6Parser does not exist for delete '%s'!. Done in 2422 ms.
湏牐橯捥䅴瑣癩瑡摥㨠匠整⁰2湏牐橯捥䅴瑣癩瑡摥㨠匠整⁰3. Done in 2422 ms.
湏牐橯捥䅴瑣癩瑡摥㨠匠整⁰3. Done in 2422 ms.
End : 湏牐橯捥䅴瑣癩瑡摥伀偮潲敪瑣捁楴慶整›瑓灥ㄠ. Done in 2422 ms.
Updating class browser...
Class browser updated.
Starting batch parsing for project 'Code::Blocks wx2.8.x'...
Start : OnParserStart
End : 湏慐獲牥瑓牡tOnParserEnd. Done in 0 ms.
Project 'Code::Blocks wx2.8.x' parsing stage done!
Project 'Code::Blocks wx2.8.x' parsing stage done (1079 total parsed files, 84540 tokens in 0 minute(s), 58.641 seconds).
Start : OnParserEnd
End : 湏慐獲牥湅d. Done in 0 ms.
Updating class browser...
Start : ParseFunctionsAndFillToolbar
End : 慐獲䙥湵瑣潩獮湁䙤汩呬潯扬牡. Done in 15 ms.
Class browser updated.
Start : OnEditorActivated
End : 湏摅瑩牯捁楴慶整d. Done in 0 ms.
Start : OnEditorActivatedTimer
湏摅瑩牯捁楴慶整呤浩牥搠潮桴湩g. Done in 0 ms.
I'm using WinXP (my local language is Chinese), So I think the _T() is need like:
CC_TRACE_FILE_END(_T("CodeComplete"));
right?
Look:
#define CC_TRACE_LOG_START(x) \
wxString sStart(_T("Start : ")); \
sStart += _T(x); \
CCLogger::Get()->DebugLog(sStart); \
std::ofstream os("debug_cc.txt", \
std::ios_base::app | std::ios_base::out); \
os << "Start : " << x << std::endl;
So, _T(x) is here.
#define CC_TRACE_LOG_END(x, y) \
wxString sEnd; \
sEnd.Printf(_T("End : %s. Done in %d ms."),x, y); \
CCLogger::Get()->DebugLog(sEnd); \
os << "End : " << x << " done in " << y << " ms." << std::endl;
It should be:
sEnd.Printf(_T("End : %s. Done in %d ms."),_T(x), y);