User forums > Using Code::Blocks
Bug Report: [#18755] C::B hangs for 20 seconds while opening large project...
dmoore:
Fine with me. I won't have time to test anything new for at least several days.
rickg22:
GOOD NEWS EVERYONE!
I'm very close to fixing the bug!
I made a relative path cache: If a file is in an already-tested directory, its relative path is calculated based on its directory's relative path. When a file is not found in the cache, its paths are calculated using MortenMacFly's slow-but-sure path calculation via wxFileName.
Here are the preliminary results:
WITHOUT THE PATH CACHE:
--- Code: ---C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():394 took : 50 ms
Project's common toplevel path: C:\htdocs\SIG\SGPG\
C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():460 took : 13496 ms
C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():461: Num Files: 1927; Num Cache Misses: 1927
--- End code ---
WITH THE PATH CACHE:
--- Code: ---C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():404 took : 55 ms
Project's common toplevel path: C:\htdocs\SIG\SGPG\
C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():464 took : 2437 ms
C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():465: Num Files: 1927; Num Cache Misses: 318
--- End code ---
Now, since the files in the project are not in order (because we're using a HashMap and not an ordered map), many directories still trigger cache misses. Even so, execution time has disminished by 80%.
I was able to diminish the load time by a further 2% by aditionally checking the parent directory in the cache; Then again, the randomized file order doesn't help. So I'm rewriting the algorithm by using a cleaned-up recursive version of the cache so that we'll have to use wxFileName's functions only twice: Once for the working directory, and once for the project's common top level path; the rest will be mere string comparisons and concatenations. I'll keep you updated when I get the algorithm working.
rickg22:
UPDATE:
I finished polishing the Path Cache. There are some edge cases I haven't tested, mainly at root directories or the top directories of other drives, both for filenames and for the project file.
For standard projects with all files under the project file's directory, things work Okay.
So... ladies and gentlemen... drum rolls, please...
--- Code: ---Loading project files...
1927 files loaded
Done loading project in 2042ms
Project's base path: C:\htdocs\SIG\SGPG\
C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():530 took : 50 ms
Project's common toplevel path: C:\htdocs\SIG\SGPG\
C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():570 took : 120 ms
C:\projects\cb\src\sdk\cbproject.cpp::void cbProject::CalculateCommonTopLevelPath():571: Num Files: 1927; Num Cache Misses: 0
--- End code ---
Ta-da! 8)
I'm attaching the patch. It needs some styling (you know, braces, tabs, etc), but I hope you like it.
dmoore:
Attached is an untested refinement to Morten's patch that uses the old fast method in the more normal case where files are below the project dir, and uses MakeRelativeTo otherwise. This could be combined with Rick's patch to reduce the amount of cache usage.
Here's the changed code:
--- Code: --- if ( (prjHasUNCName && fileHasUNCName)
|| ( !prjHasUNCName
&& !fileHasUNCName
&& vol.IsSameAs(f->file.GetVolume()) ) )
{
if (!fileName.StartsWith(m_CommonTopLevelPath))
{
wxFileName relFileCTLP(f->file);
relFileCTLP.MakeRelativeTo( m_CommonTopLevelPath );
f->relativeToCommonTopLevelPath = relFileCTLP.GetFullPath();
}
else
f->relativeToCommonTopLevelPath = fileName.Right(fileName.Length() - m_CommonTopLevelPath.Length());
if (!fileName.StartsWith(GetBasePath()))
{
wxFileName relFileBase(f->file);
relFileBase.MakeRelativeTo( GetBasePath() );
f->relativeFilename = relFileBase.GetFullPath();
}
else
f->relativeFilename = fileName.Right(fileName.Length() - GetBasePath().Length());
}
--- End code ---
I am a little bit confused by this:
--- Code: ---+ // The commented (old) method to obtain the relativeToCommonTopLevelPath is fast, but does *not* work, if you save
+ // the project on a different drive in a sub-folder of an existing source file on that (different) drive:
+ // I.e.: Project on C:\Folder\Project.cbp has file C:\Folder\SubFolder\foo.cpp and D:\Folder\bar.cpp
+ // Saved the project under D:\Folder\SubFolder\ProjectNew.cbp would cause a wrong computation of bar.cpp otherwise!!
--- End code ---
In this case should D:\Folder be the new common top level path? I'm not sure why the old method doesn't work unless the common top level path isn't being calculated correctly. Maybe the problem is really that f->relativeFilename isn't being calculated correctly?
rickg22:
--- Quote from: dmoore on November 03, 2012, 02:29:35 pm ---Attached is an untested refinement to Morten's patch that uses the old fast method in the more normal case where files are below the project dir, and uses MakeRelativeTo otherwise. This could be combined with Rick's patch to reduce the amount of cache usage.
Here's the changed code:
--- Code: ---...
--- End code ---
--- End quote ---
Hmm... maybe that code will make my patch redundant, but I like having a separate cache for relative paths. It could become useful later, and it does separate all the path checking.
--- Quote ---In this case should D:\Folder be the new common top level path? I'm not sure why the old method doesn't work unless the common top level path isn't being calculated correctly. Maybe the problem is really that f->relativeFilename isn't being calculated correctly?
--- End quote ---
Perhaps. Anyway I got another idea to speed up project loading, I'll post it in the next reply.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version