Time used to open a file

[Jenna]

Because it also litters the sources.

To find a call that needs more time than others, I have debug statements before and after nearly any call.
There are of couse some general debug-statements, but they are in the sources already.

If I remmebr correctly they slipped in accidently (by MortenMacFly as svn blame [unreliable at the moment] tells me).
They come from one of my patches and use "fileload_measuring" (all lowercase).

[Jenna]

I found one more reason, that can slow down load speed for large projects (if multiple files get opened):
We use SetPageText to set the the tab-title in SetProjectFile (we can notreliable set the title earlier, because we need the path relative to the projects basepath, depending on the tab-title options).
I looked into the wxWidgets sources, and found that every call does a Refresh() and an Update() of the notebook.
I copied the function into our cbAuiNotebook class and added a parameter to allow (or forbid) the calls to Refresh() and Update(), because they don't make sense while loading the project.
With this change and the movement from SetLanguage int SetEditorStylesAfterFileOpen the load time decreased about 40 to 50 % for my two test projects (200 files web-project and C::B core) with all files opened on startup.

It's not (yet) tested on windows.

Please test and give feedback.

With fileload_measuring defined, you get the overall load time (the time used by cbProject::LoadLayout() ) printed into C::B's debug log.

[mandrav]

Please test and give feedback.

Works good for me Jens, nice work

[MortenMacFly]

It's not (yet) tested on windows.

I can't try for the moment (I am on holiday ;-)). Did Yiannis try on Windows?!

[Jenna]

It's not (yet) tested on windows.

I can't try for the moment (I am on holiday ;-)).

I'm at home but the children are in holiday (except my youngest daughter), nearly as good.

Did Yiannis try on Windows?!

I tried in the meantime and it also works, but without this great improvement.

I did not dig into it deeper, but I guess wxWidgets on windows (or windows itself ?) does not do a real refresh/update for hidden notebooks, but wxWidgets on linux seems to do so.

[mandrav]

It's not (yet) tested on windows.

Did Yiannis try on Windows?!

Nope.

[MortenMacFly]

OK, unfortunately this does not work for me on Windows. What happens is that the option to use #defines is ALWAYS set to true whenever I open a file. When I go to editor settings, enable and again disable the option the colouring is correct again. However, opening the next file the setting is again ignored. :-(

[thomas]

(slightly off-topic)

A similarly surprising thing is that while Code::Blocks takes a minimum of 5-6 seconds to start up in Ubuntu 11.04 (or any other version I've used it under, for that matter), though under Windows it launches in under a second. Now, yesterday, much to my surprise, I found out that it also launches "almost instantly" (less than a second) under Fedora-14 KDE.

Not like that's likely to be something we can improve, but it's stunning nevertheless how the same software behaves so differently on identical hardware with kind-of-identical operating systems (Linux vs. Linux).

[Jenna]

OK, unfortunately this does not work for me on Windows. What happens is that the option to use #defines is ALWAYS set to true whenever I open a file. When I go to editor settings, enable and again disable the option the colouring is correct again. However, opening the next file the setting is again ignored. :-(

Folding is also broken, but I'm working on another patch, that seems to speed loading of very large files much more, but respects folding.
And it doesn't need the cbAuiNotebook-hack hack to speed up loading of many files on linux, it just (really) hides the notebooks while loading.
The current implementation in editormanager {Hide|Show}Notebook does nothing.

I will attach the patch during the day.

[Jenna]

Here comes the new patch (attached) and some mesurements I have made:


Opening all files, if project is loaded
1. = default (actual trunk)
2. = HideNotebook (notebook hidden while loading the project)
3. = HideNotebook and move Colourise ( 2. + call to Colourise() moved, so it's called if the editor is empty)
4. = HideNotebook and move Colourise with blocked updates of title and statusbar ( 3. and calls to update title and statusbar for every editor blocked while loading a project)
5. = HideNotebook and move Colourise and ProgressDialog (4. and ProgressDialog while loading the files)

C::B core
1. 467626 ms = 7 m 47.626 s = 100.0 % => gain  0.0 % ( 1.0 x faster than trunk )
2. 348728 ms = 5 m 48.728 s =  74.6 % => gain 25.4 % ( 1.3 x faster than trunk )
3. 345099 ms = 5 m 45.099 s =  73.8 % => gain 26.2 % ( 1.4 x faster than trunk )
4. 342316 ms = 5 m 42.316 s =  73.2 % => gain 26.8 % ( 1.4 x faster than trunk )
5. 423451 ms = 7 m  3.451 s =  90.6 % => gain  9.4 % ( 1.1 x faster than trunk )

test project with five very large (3.5 MB) files
1.  22955 ms =     22.955 s = 100.0 % => gain  0.0 % ( 1.0 x faster than trunk )
2.  18516 ms =     18.516 s =  80.7 % => gain 19.3 % ( 1.2 x faster than trunk )
3.   3624 ms =      3.624 s =  15.8 % => gain 84.2 % ( 6.3 x faster than trunk )
4.   3600 ms =      3.600 s =  15.7 % => gain 84.3 % ( 6.4 x faster than trunk )
5.   3741 ms =      3.741 s =  16.3 % => gain 83.7 % ( 6.1 x faster than trunk )

~200 files web-project (html, php, css, js)
1.  38754 ms =     38.754 s = 100.0 % => gain  0.0 % ( 1.0 x faster than trunk )
2.  24345 ms =     24.345 s =  62.8 % => gain 37.2 % ( 1.6 x faster than trunk )
3.  23446 ms =     23.446 s =  60.5 % => gain 39.5 & ( 1.7 x faster than trunk )
4.  22557 ms =     22.557 s =  58.2 % => gain 41.8 % ( 1.7 x faster than trunk )
5.  24840 ms =     24.840 s =  64.1 % => gain 35.9 % ( 1.6 x faster than trunk )


Changes to enable 4. and 5. are commented out in the patch.
The progress dialog in 5. only exists if all files are opened.
5. eats up nearly all speed-gain on large projects, but on the other hand it shows the user, that something happens and what's more it adds the ability to cancel the loading into the editor (project loading is not affected by cancelling). It can easily be implemented for loading last opened files, too.

A good compromise is 3. loading can not be cancelled, but title and statusbar are updated for each file, so the user sees, that C::B is still working.
The speedup for loading very large files (5 cpp-files with ~ 3.5 MB) is really good and should be visible on windows also.
I will test it tomorrow or this weekend (on win7 64bit).

[oBFusCATed]

Jens you have very strange method for calculation your performance boost....

For example in "C::B core" the gain is something like 6x, not 84% -> 22.9/3.6 = 6.36 times faster

[Jenna]

Jens you have very strange method for calculation your performance boost....

For example in "C::B core" the gain is something like 6x, not 84% -> 22.9/3.6 = 6.36 times faster

I added it to the original post.

It's a matter of taste I think, I personally prefer to see the percentage.

[MortenMacFly]

Here comes the new patch (attached) and some mesurements I have made:

OK, tested on Windows and suddenly I get strange crashes on shutdown (rarely, luckily) that may be related. Here's the trace:

******************************
* Found (another) call stack *
******************************
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DE50C36:
C:\Devel\CodeBlocks\codeblocks.dll[6DE50C36]:
C:/Devel/CodeBlocks/src/include/editormanager.h:78
code:
        cbAuiNotebook* GetNotebook() { return m_pNotebook; }
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DBCDA3A:
C:\Devel\CodeBlocks\codeblocks.dll[6DBCDA3A]:
C:/Devel/CodeBlocks/src/sdk/cbeditor.cpp:721
code:
    // when all editors are closed gain focus to the project tree to key to the next file to edit
    cbAuiNotebook* nb = Manager::Get()->GetEditorManager()->GetNotebook();
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DBCDB7B:
C:\Devel\CodeBlocks\codeblocks.dll[6DBCDB7B]:
C:/Devel/CodeBlocks/src/sdk/cbeditor.cpp:724
code:
  [closing bracket of the destructor cbEditor::~cbEditor() ]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DBCA5FF:
C:\Devel\CodeBlocks\codeblocks.dll[6DBCA5FF]:
C:/Devel/CodeBlocks/src/sdk/cbauibook.cpp:93
code:
  [closing bracket of the destructor cbAuiNotebook::~cbAuiNotebook() ]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DBCA69B:
C:\Devel\CodeBlocks\codeblocks.dll[6DBCA69B]:
C:/Devel/CodeBlocks/src/sdk/cbauibook.cpp:93
code:
  [closing bracket of the destructor cbAuiNotebook::~cbAuiNotebook() ]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00427FCD:
C:\Devel\CodeBlocks\codeblocks.exe[00427FCD]:
C:/Devel/CodeBlocks/src/src/main.cpp:619
code:
  [closing bracket of the destructor MainFrame::~MainFrame() ]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00428177:
C:\Devel\CodeBlocks\codeblocks.exe[00428177]:
C:/Devel/CodeBlocks/src/src/main.cpp:619
code:
  [closing bracket of the destructor MainFrame::~MainFrame() ]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 004058C4:
C:\Devel\CodeBlocks\codeblocks.exe[004058C4]:
C:/Devel/CodeBlocks/src/src/app.cpp:788
code [wxApp loop...)
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00401D71:
C:\Devel\CodeBlocks\codeblocks.exe[00401D71]:
C:/Devel/CodeBlocks/src/src/app.cpp:260
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00469A76:
C:\Devel\CodeBlocks\codeblocks.exe[00469A76]:
main.c:0
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 004010DB:
C:\Devel\CodeBlocks\codeblocks.exe[004010DB]:
crt1.c:0
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00401158:
C:\Devel\CodeBlocks\codeblocks.exe[00401158]:
crt1.c:0
----------------------------------------

[Jenna]

In the actual patch there are no changes to cbAuiNotebook, but while closing a project the notebook is hidden, this might be related.

Will test on windows later.
Did you delete the pch's   :lol: (just to be sure) ?

[vbspam]

Hi
I posted another thread about very similar topic here http://forums.codeblocks.org/index.php/topic,14644.0.html .
I have got there response with link to this thread. On the first view it looks like it is the same problem I experiencing however I think that you are fighting another one.
If I understand correctly, this thread is beating the slow opening of project with already opened files (reopening those previously opened files in the editor). I am fighting the problem of opening  new project and/or creating new project with many files. Please correct me if I am wrong.

By the way I disabled and then removed all plugins (by removing them from the disk) and it had no influence to the loading speed. I think that the reason is that plugins do not come to the scope at the time of creating of the virtual folder's tree.

An on the end I can contribute to fighting your problem. From my experience the big speed up comes when you disable automatic completion update (what is probably done every keypress) and instead of this let it be updated every "file save". This can be set somewhere in editor settings.