User forums > Using Code::Blocks
Bug Report: [#18755] C::B hangs for 20 seconds while opening large project...
rickg22:
--- Quote from: dmoore on October 17, 2012, 03:29:59 pm ---Using wxStopWatch I can confirm that almost all of the opening time is spent on wxFileName::MakeRelativeTo. Rebuilding the tree control isn't all that expensive.
--- End quote ---
Mind showing us the cumulative stats?
Anyway, I got an idea that might just work. Here's the pseudocode.
--- Code: ---1. wxString lastparentpath = "", curparentpath = "", lastrelativepath = "", currelativepath = "", currelativefilename = "", DS = "/"; // DS is the directory separator, adapt to OS needs.
2. for each of the filenames to be processed as curfilename: {
2.1. curparentpath = obtain_path_only(curfilename); // get the full path without the filename
2.2. basefilename = obtain_filename_only(curfilename); // filename only, plus extension
2.3. if(curparentpath == lastparentpath && lastparentpath !== "") {
currelativepath = lastrelativepath;
currelativefilename = currelativepath + DS + basefilename;
}
2.4. else {
currelativefilename = MakeRelative(filename,commontopprojectdirectory);
lastparentpath = curparentpath;
lastrelativepath = obtain_path_only(currelativefilename);
}
2.5. Do the rest of the tree node adding here.
}
--- End code ---
This way, MakeRelative is only called for the first file of a given directory. The rest of the relative filenames are just calculated using a concatenation.
As for the obtain_path_only and obtain_filename_only functions, they are easily implemented using a reverse searching for the "/" and "\" characters on each filename, and splitting the string in two at that position. In fact, we could use a single function that splits the filename into path and basename using only one search.
dmoore:
@rick: The MakeRelativeTo calls were taking up >90% of the ProjectManager::RebuildTree (i.e. at least 1800 of about 2000 ms)
EDIT: Here's an example:
--- Code: ---Done loading project in 233ms
BuildTree
MakeRelativeTo time 1807 ms
Total time 1905 ms
BuildTree
MakeRelativeTo time 1793 ms
Total time 1890 ms
--- End code ---
Not sure the more complicated patch is needed because ProjectFile already has a member relativeToCommonTopLevelPath. This is initialized when the project is first opened, but it does not use MakeRelativeTo. Instead it does:
--- Code: --- pf->relativeToCommonTopLevelPath = fullFilename.Right(fullFilename.Length() - m_CommonTopLevelPath.Length());
--- End code ---
Which I'm sure breaks in the corner case where files are split across volumes.
But anyway, if we use that member we have a simple patch:
--- Code: ---Index: C:/Users/damienm/Documents/damien/Source/codeblockssrc/trunk/src/sdk/cbproject.cpp
===================================================================
--- C:/Users/damienm/Documents/damien/Source/codeblockssrc/trunk/src/sdk/cbproject.cpp (revision 8456)
+++ C:/Users/damienm/Documents/damien/Source/codeblockssrc/trunk/src/sdk/cbproject.cpp (working copy)
@@ -907,9 +907,7 @@
ftd->SetProjectFile(f);
ftd->SetFolder(f->file.GetFullPath());
- wxFileName nodefile = f->file;
- nodefile.MakeRelativeTo(m_CommonTopLevelPath);
- wxString nodetext = nodefile.GetFullPath();
+ wxString nodetext = f->relativeToCommonTopLevelPath;
FileTreeData::FileTreeDataKind folders_kind = FileTreeData::ftdkFolder;
// by default, the parent node is the project node (in case of no grouping, no virtual folders)
--- End code ---
If we want a patch that is robust to files split across volumes it should be a fix for the way relativeToCommonTopLevelPath is initialized.
There is still a small delay from the project loader (200ms), but that may just be parsing the xml and checking that files exist and, thus, not much can be done about.
Jenna:
The commonTopLevelPath might hav echanged since initializing relativeToCommonTopLevelPath.
This can happen every time a new file is added.
One way to handle this would be to reset this variable every time the commonTopLevelPath has changed.
Another would be to use a function instead, which does exactly what happens when it is initialized.
This should not be too expensive (just a wxString.Right() ).
A different volume on windows could be handled here also, most likely.
Should be doable without the use of wxFileName in case fullFileName also keeps the volume of a file, if not we might introduce another member, that keeps it.
dmoore:
we've been here before :P
http://forums.codeblocks.org/index.php/topic,6288.0.html
svn
But yes, I agree with you Jens that probably the best way to go is to replace that member with a function that does the calculation on the fly.
Jenna:
@dmoore:
I attach a patch, that combines your patch and a slightly enhanced patch of MortenMacFly (see http://forums.codeblocks.org/index.php/topic,16947.msg115392.html#msg115392).
Soe measurements on linux with wxWidgets trunk (> 10.000 files):
without the patch:
--- Quote ---/home/jens/codeblocks-build/codeblocks.git/src/sdk/cbproject.cpp::void cbProject::BuildTree(cbTreeCtrl*, const wxTreeItemId&, int, FilesGroupsAndMasks*):1008 took : 1406 ms
/home/jens/codeblocks-build/codeblocks.git/src/sdk/cbproject.cpp::void cbProject::BuildTree(cbTreeCtrl*, const wxTreeItemId&, int, FilesGroupsAndMasks*):1008 took : 1432 ms
CalculateCommonTopLevelPath() took 275 ms
iterating through all 10677 files took 1030 ms
iterating through all 10677 files took 1066 ms
--- End quote ---
with the patch:
--- Quote ---/home/jens/codeblocks-build/codeblocks.git/src/sdk/cbproject.cpp::void cbProject::BuildTree(cbTreeCtrl*, const wxTreeItemId&, int, FilesGroupsAndMasks*):1002 took : 1123 ms
/home/jens/codeblocks-build/codeblocks.git/src/sdk/cbproject.cpp::void cbProject::BuildTree(cbTreeCtrl*, const wxTreeItemId&, int, FilesGroupsAndMasks*):1002 took : 1132 ms
CalculateCommonTopLevelPath() took 275 ms
iterating through all 10677 files took 753 ms
iterating through all 10677 files took 751 ms
--- End quote ---
Not much in absolute time, but about 25 to 30 %.
I will test it on windows soon, also with mixed volume projects.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version