Author Topic: Time used to open a file  (Read 19112 times)

Offline Folco

  • Regular
  • ***
  • Posts: 343
    • Folco's blog (68k lover)
Time used to open a file
« on: April 18, 2011, 08:22:47 pm »
Hi,

I use two versions of C::B :
- nightly 7075 under Win XP Pro SP3 (32b)
- current SVN under Kubuntu (64b)

I am currently on a new project which is about ~15 files (this is a C project, but I got same behavior with other projects, like assembly or C++ programs).
When I open this project under Win and nux, it take about 1/10 second under Win, and perhaps 5 seconds under nux. Why is there such a great difference ? I have always noticed that, but didn't asked about that. Any idea ? Is there a way to speed up opening time under Linux ?

(hmmm, the topic title should be "Time used to open a project"...)
Kernel Extremist - PedroM power

Offline stahta01

  • Lives here!
  • ****
  • Posts: 6589
    • My Best Post
Re: Time used to open a file
« Reply #1 on: April 18, 2011, 08:41:05 pm »
The Linux Folks Un-Install all the plugins they do not use; because it takes noticeable time to load the disabled plugins under Linux.
No idea if that is your problem.

Note: I would try to disable/uninstall Code Completion first.

Tim S
C Programmer working to learn more about C++ and Git.
On Windows 7 64 bit and Windows 10 32 bit.
On Debian Stretch, compiling CB Trunk against wxWidgets 3.0.
--
When in doubt, read the CB WiKi FAQ. http://wiki.codeblocks.org

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11899
    • Travis build status
Re: Time used to open a file
« Reply #2 on: April 18, 2011, 08:51:02 pm »
I have a workspace with 5+ project it opens for 1-2 seconds.
The cb project opens for 1-2 seconds, too.

Try disabling some of the plugins. Starting with CC.

p.s. is it slow the second time you open it?
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline Folco

  • Regular
  • ***
  • Posts: 343
    • Folco's blog (68k lover)
Re: Time used to open a file
« Reply #3 on: April 18, 2011, 09:02:10 pm »
Ok, thanks for your answers. Indeed, I compile by hand under Linux, and I perform a ./configure --with-contrib-plugins=all. I'll try only with the plugins I really use, then I will try disabling CC.

I'm currently under Win, but I test soon.
Kernel Extremist - PedroM power

Offline Folco

  • Regular
  • ***
  • Posts: 343
    • Folco's blog (68k lover)
Re: Time used to open a file
« Reply #4 on: April 19, 2011, 06:12:18 pm »
My internet was down yesterday, so I didn't recompile, but what I can say is that :
- the second time I open a project, the time used is the same than the first one
- disabling CC doesn't change anything
Kernel Extremist - PedroM power

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4291
    • Code::Blocks IDE
Re: Time used to open a file
« Reply #5 on: April 19, 2011, 06:47:22 pm »
I 'm experiencing the same issue too and I 'm planning to investigate it more in the next few days. As soon as I find the culprit, I 'll fix it. Please be patient.
Be patient!
This bug will be fixed soon...

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #6 on: April 19, 2011, 06:54:06 pm »
Yiannis: I believe it may be related to the lexers. If you use way less lexers it seems to work faster.

@all: Can you try that, please?
« Last Edit: April 19, 2011, 06:55:39 pm by MortenMacFly »
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #7 on: April 19, 2011, 07:22:31 pm »
Yiannis: I believe it may be related to the lexers. If you use way less lexers it seems to work faster.

@all: Can you try that, please?
There seems to be no difference, even if I remove all lexers except the c++-lexer.
Also all plugins except the compiler-plugin have been removed.

I opened the C::B project with some files (less than 10) open.
If no files are to be opened the project opens almost immediately.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11899
    • Travis build status
Re: Time used to open a file
« Reply #8 on: April 19, 2011, 07:32:13 pm »
If no files are to be opened the project opens almost immediately.
I've opened all files in my debugger_gdbmi plugin, closed the workspace and opened it again.
The second opening took 2-3 seconds, it was relatively fast, but it is slower than no-files-to-be-opened case.
I could see how every file is opened, then redrawn, bla bla...
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4291
    • Code::Blocks IDE
Re: Time used to open a file
« Reply #9 on: April 19, 2011, 08:01:42 pm »
Yiannis: I believe it may be related to the lexers. If you use way less lexers it seems to work faster.

@all: Can you try that, please?

I think not.
The exact problem I 'm seeing is that it takes a *lot* of time (about a second each) to reopen each previously opened file while loading the project. When opening a project that has no open files in the layout, the loading is blazingly fast.
From a very quick glance I had the other day, I feel it's the creation of the editors that's slowing everything down. I will investigate more though.
At first I suspected the encoding detection but that takes a couple of milliseconds so that's not it.
Be patient!
This bug will be fixed soon...

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #10 on: April 19, 2011, 09:44:25 pm »
The exact problem I 'm seeing is that it takes a *lot* of time (about a second each) to reopen each previously opened file while loading the project.
It may be related, just for the record: I experienced a different bad bug: If I open a WS with many projects (~15-20) and files are being opened due to the layout, sometimes CC freaks out and never stops parsing. This only happens if file were opened, but I didn't find a way to reproduce. There seems no logic behind.

However, (on second thoughts) as Jens said CC is disabled for him it may be a another issue... but still: it could be related. I've disabled the option to re-open files for exactly that issue, so I didn't realise the issue you mentioned.

(I've no access to C::B at the moment btw...)
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #11 on: April 20, 2011, 12:41:59 am »
One thing that speeds up loading of large files a lot is moving SetLanguage(HL_AUTO) from SetEditorStyleBeforeFileOpen to SetEditorStyleAfterFileOpen.

All measurements are made with a lot of debug statements in the code, that write to C::B's debug-log, that's one of the causes for the very long overall loading time.

For a 3.5 MB test cpp-file the load-time changes from 6.349 s to 1.654 s (74 % better).
Load-time of a web-project (all files opened) that contains mainly small php,js and css-files decreased from 40.614 ms to 36.501 ms (10 % better).
C::B project's load-time changes from 8 m 56.375 ms to 7 m 37.214 s (15 % better).

This change can decrease the loadtime depending on the filesizes a lot or at least a little bit, but it should not harm in any cases.
Any tests are welcome, to ensure there are no side-effects.

There are at least two other places where a little more time is needed:
Sending the cbEVT_EDITOR_ACTIVATED-event in void EditorManager::SetActiveEditor(EditorBase* ed) and the call to m_pControl->GotoPos(m_pProjectFile->editorPos) (and m_pControl2->GotoPos(m_pProjectFile->editorPos_2) if editor is in split-view) in void cbEditor::SetProjectFile(ProjectFile* project_file, bool preserve_modified) also take a resonable amount of time. The second is a call to a scintilla-function and therefore not so easy to change (without hacking scintilla).
The first only needs to be called if the editor is the topmost editor, but is called for every editor that gets opened via cbEditor* EditorManager::Open(LoaderBase* fileLdr, const wxString& filename, int /*pos*/, ProjectFile* data) at the moment.


Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #12 on: April 20, 2011, 08:17:55 am »
One thing that speeds up loading of large files a lot is moving SetLanguage(HL_AUTO) from SetEditorStyleBeforeFileOpen to SetEditorStyleAfterFileOpen.
Nice catch, Jens.

However, I recall that this call was moved to that place for some reason (fixing a bug) recently... Or am I wrong (I cannot consult SVN blame/log for the moment)?

In addition: Can you commit the instrumentation concerning measuring the times needed to SVN and making it "toggle-able" with a #define? I feel as if we'll need this again in the future... :)
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #13 on: April 20, 2011, 09:08:06 am »
One thing that speeds up loading of large files a lot is moving SetLanguage(HL_AUTO) from SetEditorStyleBeforeFileOpen to SetEditorStyleAfterFileOpen.
Nice catch, Jens.

However, I recall that this call was moved to that place for some reason (fixing a bug) recently... Or am I wrong (I cannot consult SVN blame/log for the moment)?

In addition: Can you commit the instrumentation concerning measuring the times needed to SVN and making it "toggle-able" with a #define? I feel as if we'll need this again in the future... :)
Parts of it are already in svn, the needed define is "fileload_measuring", but it does not cover all parts.
At the moment I use a much more finegrained debug-output, but that can not be committed, because it litters the sources with tons of ifdefs.

svn blame does not really work for some commits, because when remerging cc-branch wit trunk (or aui-branch I do not remember exactly) the sourcees have been copied instead of doing a real merge (that was broken as far as I know).

One major bottleneck was fixed two years ago:
http://forums.codeblocks.org/index.php/topic,10552.msg72360.html

If I remember correctly, I also did something with SetLanguage, but I do not recall exactly what it was, I will search the websvn to find if there is anything related.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11899
    • Travis build status
Re: Time used to open a file
« Reply #14 on: April 20, 2011, 10:31:40 am »
At the moment I use a much more finegrained debug-output, but that can not be committed, because it litters the sources with tons of ifdefs.
Why don't you add something like
Code: [Select]
#ifdef CB_FILELOAD_MEASURING
CB_FILELOAD_PRINT printf
#else
#define CB_FILELOAD_PRINT(...)
#endif
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #15 on: April 20, 2011, 10:57:07 am »
Because it also litters the sources.

To find a call that needs more time than others, I have debug statements before and after nearly any call.
There are of couse some general debug-statements, but they are in the sources already.

If I remmebr correctly they slipped in accidently (by MortenMacFly as svn blame [unreliable at the moment] tells me).
They come from one of my patches and use "fileload_measuring" (all lowercase).

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #16 on: April 22, 2011, 12:08:30 am »
I found one more reason, that can slow down load speed for large projects (if multiple files get opened):
We use SetPageText to set the the tab-title in SetProjectFile (we can notreliable set the title earlier, because we need the path relative to the projects basepath, depending on the tab-title options).
I looked into the wxWidgets sources, and found that every call does a Refresh() and an Update() of the notebook.
I copied the function into our cbAuiNotebook class and added a parameter to allow (or forbid) the calls to Refresh() and Update(), because they don't make sense while loading the project.
With this change and the movement from SetLanguage int SetEditorStylesAfterFileOpen the load time decreased about 40 to 50 % for my two test projects (200 files web-project and C::B core) with all files opened on startup.

It's not (yet) tested on windows.

Please test and give feedback.

With fileload_measuring defined, you get the overall load time (the time used by cbProject::LoadLayout() ) printed into C::B's debug log.

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4291
    • Code::Blocks IDE
Re: Time used to open a file
« Reply #17 on: April 22, 2011, 01:17:38 am »
Please test and give feedback.

Works good for me Jens, nice work :)
Be patient!
This bug will be fixed soon...

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #18 on: April 23, 2011, 06:08:34 pm »
It's not (yet) tested on windows.
I can't try for the moment (I am on holiday ;-)). Did Yiannis try on Windows?!
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #19 on: April 23, 2011, 07:00:12 pm »
It's not (yet) tested on windows.
I can't try for the moment (I am on holiday ;-)).
I'm at home but the children are in holiday (except my youngest daughter), nearly as good.
Did Yiannis try on Windows?!
I tried in the meantime and it also works, but without this great improvement.

I did not dig into it deeper, but I guess wxWidgets on windows (or windows itself ?) does not do a real refresh/update for hidden notebooks, but wxWidgets on linux seems to do so.

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4291
    • Code::Blocks IDE
Re: Time used to open a file
« Reply #20 on: April 25, 2011, 09:57:09 am »
Be patient!
This bug will be fixed soon...

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #21 on: May 04, 2011, 09:26:55 pm »
OK, unfortunately this does not work for me on Windows. What happens is that the option to use #defines is ALWAYS set to true whenever I open a file. When I go to editor settings, enable and again disable the option the colouring is correct again. However, opening the next file the setting is again ignored. :-(
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Time used to open a file
« Reply #22 on: May 05, 2011, 12:43:44 pm »
(slightly off-topic)

A similarly surprising thing is that while Code::Blocks takes a minimum of 5-6 seconds to start up in Ubuntu 11.04 (or any other version I've used it under, for that matter), though under Windows it launches in under a second. Now, yesterday, much to my surprise, I found out that it also launches "almost instantly" (less than a second) under Fedora-14 KDE.

Not like that's likely to be something we can improve, but it's stunning nevertheless how the same software behaves so differently on identical hardware with kind-of-identical operating systems (Linux vs. Linux).
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #23 on: May 05, 2011, 01:05:05 pm »
OK, unfortunately this does not work for me on Windows. What happens is that the option to use #defines is ALWAYS set to true whenever I open a file. When I go to editor settings, enable and again disable the option the colouring is correct again. However, opening the next file the setting is again ignored. :-(
Folding is also broken, but I'm working on another patch, that seems to speed loading of very large files much more, but respects folding.
And it doesn't need the cbAuiNotebook-hack hack to speed up loading of many files on linux, it just (really) hides the notebooks while loading.
The current implementation in editormanager {Hide|Show}Notebook does nothing.

I will attach the patch during the day.

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #24 on: May 06, 2011, 12:55:09 am »
Here comes the new patch (attached) and some mesurements I have made:


Opening all files, if project is loaded
1. = default (actual trunk)
2. = HideNotebook (notebook hidden while loading the project)
3. = HideNotebook and move Colourise ( 2. + call to Colourise() moved, so it's called if the editor is empty)
4. = HideNotebook and move Colourise with blocked updates of title and statusbar ( 3. and calls to update title and statusbar for every editor blocked while loading a project)
5. = HideNotebook and move Colourise and ProgressDialog (4. and ProgressDialog while loading the files)

C::B core
1. 467626 ms = 7 m 47.626 s = 100.0 % => gain  0.0 % ( 1.0 x faster than trunk )
2. 348728 ms = 5 m 48.728 s =  74.6 % => gain 25.4 % ( 1.3 x faster than trunk )
3. 345099 ms = 5 m 45.099 s =  73.8 % => gain 26.2 % ( 1.4 x faster than trunk )
4. 342316 ms = 5 m 42.316 s =  73.2 % => gain 26.8 % ( 1.4 x faster than trunk )
5. 423451 ms = 7 m  3.451 s =  90.6 % => gain  9.4 % ( 1.1 x faster than trunk )

test project with five very large (3.5 MB) files
1.  22955 ms =     22.955 s = 100.0 % => gain  0.0 % ( 1.0 x faster than trunk )
2.  18516 ms =     18.516 s =  80.7 % => gain 19.3 % ( 1.2 x faster than trunk )
3.   3624 ms =      3.624 s =  15.8 % => gain 84.2 % ( 6.3 x faster than trunk )
4.   3600 ms =      3.600 s =  15.7 % => gain 84.3 % ( 6.4 x faster than trunk )
5.   3741 ms =      3.741 s =  16.3 % => gain 83.7 % ( 6.1 x faster than trunk )

~200 files web-project (html, php, css, js)
1.  38754 ms =     38.754 s = 100.0 % => gain  0.0 % ( 1.0 x faster than trunk )
2.  24345 ms =     24.345 s =  62.8 % => gain 37.2 % ( 1.6 x faster than trunk )
3.  23446 ms =     23.446 s =  60.5 % => gain 39.5 & ( 1.7 x faster than trunk )
4.  22557 ms =     22.557 s =  58.2 % => gain 41.8 % ( 1.7 x faster than trunk )
5.  24840 ms =     24.840 s =  64.1 % => gain 35.9 % ( 1.6 x faster than trunk )


Changes to enable 4. and 5. are commented out in the patch.
The progress dialog in 5. only exists if all files are opened.
5. eats up nearly all speed-gain on large projects, but on the other hand it shows the user, that something happens and what's more it adds the ability to cancel the loading into the editor (project loading is not affected by cancelling). It can easily be implemented for loading last opened files, too.

A good compromise is 3. loading can not be cancelled, but title and statusbar are updated for each file, so the user sees, that C::B is still working.
The speedup for loading very large files (5 cpp-files with ~ 3.5 MB) is really good and should be visible on windows also.
I will test it tomorrow or this weekend (on win7 64bit).

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11899
    • Travis build status
Re: Time used to open a file
« Reply #25 on: May 06, 2011, 01:12:43 am »
Jens you have very strange method for calculation your performance boost....

For example in "C::B core" the gain is something like 6x, not 84% -> 22.9/3.6 = 6.36 times faster :)
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #26 on: May 06, 2011, 07:03:13 am »
Jens you have very strange method for calculation your performance boost....

For example in "C::B core" the gain is something like 6x, not 84% -> 22.9/3.6 = 6.36 times faster :)
I added it to the original post.

It's a matter of taste I think, I personally prefer to see the percentage.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #27 on: May 06, 2011, 01:05:12 pm »
Here comes the new patch (attached) and some mesurements I have made:
OK, tested on Windows and suddenly I get strange crashes on shutdown (rarely, luckily) that may be related. Here's the trace:

******************************
* Found (another) call stack *
******************************
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DE50C36:
C:\Devel\CodeBlocks\codeblocks.dll[6DE50C36]:
C:/Devel/CodeBlocks/src/include/editormanager.h:78
code:
        cbAuiNotebook* GetNotebook() { return m_pNotebook; }
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DBCDA3A:
C:\Devel\CodeBlocks\codeblocks.dll[6DBCDA3A]:
C:/Devel/CodeBlocks/src/sdk/cbeditor.cpp:721
code:
    // when all editors are closed gain focus to the project tree to key to the next file to edit
    cbAuiNotebook* nb = Manager::Get()->GetEditorManager()->GetNotebook();
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DBCDB7B:
C:\Devel\CodeBlocks\codeblocks.dll[6DBCDB7B]:
C:/Devel/CodeBlocks/src/sdk/cbeditor.cpp:724
code:
  [closing bracket of the destructor cbEditor::~cbEditor() ???]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DBCA5FF:
C:\Devel\CodeBlocks\codeblocks.dll[6DBCA5FF]:
C:/Devel/CodeBlocks/src/sdk/cbauibook.cpp:93
code:
  [closing bracket of the destructor cbAuiNotebook::~cbAuiNotebook() ???]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.dll 6DBCA69B:
C:\Devel\CodeBlocks\codeblocks.dll[6DBCA69B]:
C:/Devel/CodeBlocks/src/sdk/cbauibook.cpp:93
code:
  [closing bracket of the destructor cbAuiNotebook::~cbAuiNotebook() ???]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00427FCD:
C:\Devel\CodeBlocks\codeblocks.exe[00427FCD]:
C:/Devel/CodeBlocks/src/src/main.cpp:619
code:
  [closing bracket of the destructor MainFrame::~MainFrame() ???]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00428177:
C:\Devel\CodeBlocks\codeblocks.exe[00428177]:
C:/Devel/CodeBlocks/src/src/main.cpp:619
code:
  [closing bracket of the destructor MainFrame::~MainFrame() ???]
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 004058C4:
C:\Devel\CodeBlocks\codeblocks.exe[004058C4]:
C:/Devel/CodeBlocks/src/src/app.cpp:788
code [wxApp loop...)
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00401D71:
C:\Devel\CodeBlocks\codeblocks.exe[00401D71]:
C:/Devel/CodeBlocks/src/src/app.cpp:260
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00469A76:
C:\Devel\CodeBlocks\codeblocks.exe[00469A76]:
main.c:0
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 004010DB:
C:\Devel\CodeBlocks\codeblocks.exe[004010DB]:
crt1.c:0
----------------------------------------
addr2line -e C:\Devel\CodeBlocks\src\devel\codeblocks.exe 00401158:
C:\Devel\CodeBlocks\codeblocks.exe[00401158]:
crt1.c:0
----------------------------------------
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #28 on: May 06, 2011, 06:20:17 pm »
In the actual patch there are no changes to cbAuiNotebook, but while closing a project the notebook is hidden, this might be related.

Will test on windows later.
Did you delete the pch's   :lol: (just to be sure) ?

Offline vbspam

  • Single posting newcomer
  • *
  • Posts: 2
Re: Time used to open a file
« Reply #29 on: May 07, 2011, 08:17:19 am »
Hi
I posted another thread about very similar topic here http://forums.codeblocks.org/index.php/topic,14644.0.html .
I have got there response with link to this thread. On the first view it looks like it is the same problem I experiencing however I think that you are fighting another one.
If I understand correctly, this thread is beating the slow opening of project with already opened files (reopening those previously opened files in the editor). I am fighting the problem of opening  new project and/or creating new project with many files. Please correct me if I am wrong.

By the way I disabled and then removed all plugins (by removing them from the disk) and it had no influence to the loading speed. I think that the reason is that plugins do not come to the scope at the time of creating of the virtual folder's tree.

An on the end I can contribute to fighting your problem. From my experience the big speed up comes when you disable automatic completion update (what is probably done every keypress) and instead of this let it be updated every "file save". This can be set somewhere in editor settings.





Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #30 on: May 07, 2011, 09:00:03 am »
Hi
I posted another thread about very similar topic here http://forums.codeblocks.org/index.php/topic,14644.0.html .
I have got there response with link to this thread. On the first view it looks like it is the same problem I experiencing however I think that you are fighting another one.
If I understand correctly, this thread is beating the slow opening of project with already opened files (reopening those previously opened files in the editor). I am fighting the problem of opening  new project and/or creating new project with many files. Please correct me if I am wrong.

By the way I disabled and then removed all plugins (by removing them from the disk) and it had no influence to the loading speed. I think that the reason is that plugins do not come to the scope at the time of creating of the virtual folder's tree.

An on the end I can contribute to fighting your problem. From my experience the big speed up comes when you disable automatic completion update (what is probably done every keypress) and instead of this let it be updated every "file save". This can be set somewhere in editor settings.

I already have read your other post and will try to look into it.
I use the linux-kernel sources as test project (just to load it, not to compile or anything).

If I have the code-completion plugin removed, it does not change anything when opening a great amount of files.
On linux (debian 64-bit gnome 3) there are two slowdowns: for great files it's the styling of the editor (probably fixed by the patch) and the labeling of the notebook (hopefully also fixed).
But I did not yet test on windows.
Speed-up should be much less there (because at least the labeling is already much faster on windows, than on linux [wxWidgets ?]).


Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #31 on: May 10, 2011, 12:32:25 am »
I already have read your other post and will try to look into it.
I use the linux-kernel sources as test project (just to load it, not to compile or anything).

Here comes a patch, that replaces wxList with a wxHashSet for the ProjectFile*-list, what leads to a more or less great amount of changes.
The patch is not deeply tested and nottested on windows at all.
It's mixed with the other patch, described earlier in this thread.

I used the linux-kernel 2.6.35 as test project, no files opened, just loading the project and filling the treectrl in management pane.
CC-plugin is disabled, because it sometimes eats up all of my memeory and makes the system nearly unusable.

Overall load-time with trunk (from closing the starthere-page until selecting the active build-target):
1 m 30.846 s
with my changes:
0 m 11.398 s

in other words:
about 8 times faster

closing is much faster with trunk (at the moment):
2.255 s
and with my patch:
5.465 s
about 2.4 times slower

exact values (snippets from debug log with DisplayEvents-plugin):

Code: [Select]
wxHashSet (codecompletion disabled)

open linux-kernel 2.6.35


23:50:31,744  =>  cbEVT_EDITOR_CLOSE
Loading project file...
Parsing project file...
23:50:32,141  =>  cbEVT_PROJECT_RENAMED
23:50:32,149  =>  cbEVT_BUILDTARGET_ADDED
23:50:32,149  =>  cbEVT_PROJECT_TARGETS_MODIFIED
Loading target Debug
23:50:32,150  =>  cbEVT_BUILDTARGET_ADDED
23:50:32,150  =>  cbEVT_PROJECT_TARGETS_MODIFIED
Loading target Release
Loading project files...
23:50:32,175  =>  cbEVT_PROJECT_BEGIN_ADD_FILES
23:50:35,732  =>  cbEVT_PROJECT_END_ADD_FILES
33115 files loaded
Done loading project in 3991ms
Project's base path: /home/jens/kernel-tmp.2.6.35/
Project's common toplevel path: /home/jens/kernel-tmp.2.6.35/
23:50:39,498  =>  cbEVT_PROJECT_OPEN
23:50:39,499  =>  cbEVT_WORKSPACE_CHANGED
23:50:43,138  =>  cbEVT_PROJECT_ACTIVATE
23:50:43,142  =>  cbEVT_BUILDTARGET_SELECTED

and close it

Removed test from all deps
23:53:53,051  =>  cbEVT_EDITOR_SWITCHED
23:53:53,138  =>  cbEVT_EDITOR_ACTIVATED
23:53:53,259  =>  cbEVT_PROJECT_CLOSE
23:53:58,248  =>  cbEVT_BUILDTARGET_SELECTED
23:53:58,289  =>  cbEVT_WORKSPACE_CHANGED
23:53:58,431  =>  cbEVT_APP_ACTIVATED
23:53:58,447  =>  cbEVT_EDITOR_SWITCHED
23:53:58,516  =>  cbEVT_EDITOR_ACTIVATED

trunk

open

23:55:21,116  =>  cbEVT_EDITOR_CLOSE
Loading project file...
Parsing project file...
23:55:21,345  =>  cbEVT_PROJECT_RENAMED
23:55:21,354  =>  cbEVT_BUILDTARGET_ADDED
23:55:21,355  =>  cbEVT_PROJECT_TARGETS_MODIFIED
Loading target Debug
23:55:21,355  =>  cbEVT_BUILDTARGET_ADDED
23:55:21,355  =>  cbEVT_PROJECT_TARGETS_MODIFIED
Loading target Release
Loading project files...
23:55:21,379  =>  cbEVT_PROJECT_BEGIN_ADD_FILES
23:56:43,454  =>  cbEVT_PROJECT_END_ADD_FILES
33115 files loaded
Done loading project in 82341ms
Project's base path: /home/jens/kernel-tmp.2.6.35/
Project's common toplevel path: /home/jens/kernel-tmp.2.6.35/
23:56:47,745  =>  cbEVT_PROJECT_OPEN
23:56:47,746  =>  cbEVT_WORKSPACE_CHANGED
23:56:51,936  =>  cbEVT_PROJECT_ACTIVATE
23:56:51,962  =>  cbEVT_BUILDTARGET_SELECTED

and close it

Removed test from all deps
23:58:36,742  =>  cbEVT_EDITOR_SWITCHED
23:58:36,762  =>  cbEVT_EDITOR_ACTIVATED
23:58:36,876  =>  cbEVT_PROJECT_CLOSE
23:58:38,597  =>  cbEVT_BUILDTARGET_SELECTED
23:58:38,671  =>  cbEVT_WORKSPACE_CHANGED
23:58:38,886  =>  cbEVT_APP_ACTIVATED
23:58:38,906  =>  cbEVT_EDITOR_SWITCHED
23:58:38,997  =>  cbEVT_EDITOR_ACTIVATED

As mentioned not deeply tested (might break some stuff), so be careful.
Make a backup of your project to be secure .

EDIT:

patch updated, see: http://forums.codeblocks.org/index.php/topic,14543.msg103572.html#msg103572

Offline Folco

  • Regular
  • ***
  • Posts: 343
    • Folco's blog (68k lover)
Re: Time used to open a file
« Reply #32 on: August 24, 2011, 08:24:01 pm »
Have these patches been applied ?
I think that now, the time spent to open a project is significantly lower, am I right ?
Kernel Extremist - PedroM power

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #33 on: October 28, 2011, 05:38:23 pm »
Updated wxHashSet-patch (against svn r7546).

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #34 on: October 29, 2011, 12:18:23 pm »
...for all those that cannot apply this version of the patch, try the one attached. It is the same (just another format, and the files got re-ordered).
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #35 on: November 09, 2011, 09:44:17 am »
Updated wxHashSet-patch (against svn r7546).
Jens, after testing for a couple of days I think this patch still has serious issues. I am facing the following strange phenomena:
  • The ordering of the files in the project file always change for each save - bad. :-(
  • If I copy a target ion the project settings, the "build files" are empty
  • Clicking in the build files crashes C::B (even without creating as log)
So... something is seriously wrong here, unfortunately.

Could you check, if you see files in the project's build options (lower right part)?
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #36 on: November 09, 2011, 12:39:59 pm »
...ok, one thing I noticed is a bug that gets introduces with this path. In nativeparser.cpp you exchange:
-    for (int i = 0; project && i < project->GetFilesCount(); ++i)
...with:
+    for (FilesList::iterator it = project->GetFilesList().begin(); it != project->GetFilesList().end(); ++it)
Thus, you missed the NULL pointer check (notice project && i < project->GetFilesCount()). Surrounding this for loop with a if (project) clause resolves some crashes at least.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #37 on: November 09, 2011, 01:52:40 pm »
I will check for misiing nullpointer-checks as soon as possible (I never had any crashes).
And of course test all others issues you have described also.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #38 on: November 09, 2011, 02:04:06 pm »
I will check for misiing nullpointer-checks as soon as possible (I never had any crashes).
I think that was the only one. The crash in that case is gone.

And of course test all others issues you have described also.
Besides the project file changes the only major remaining issue is in the project properties dialog: You don't even need to copy a target. The build file list is always empty. And if you click into it C::B crashes w/o backtrace (!). Looks its related to the fact that you set the project file as client data to the checked list...
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #39 on: November 09, 2011, 02:26:15 pm »
I will check for misiing nullpointer-checks as soon as possible (I never had any crashes).
I think that was the only one. The crash in that case is gone.

And of course test all others issues you have described also.
Besides the project file changes the only major remaining issue is in the project properties dialog: You don't even need to copy a target. The build file list is always empty. And if you click into it C::B crashes w/o backtrace (!). Looks its related to the fact that you set the project file as client data to the checked list...

I can reproduce the random fileorder, I hope it can be fixed easily.
I can see all my files in the "Build targets" tab, and there is no crash.
Either it's a windows-special, or you need to do a really clean build, with manually removed pch's etc. .

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #40 on: November 09, 2011, 02:40:56 pm »
Either it's a windows-special, or you need to do a really clean build, with manually removed pch's etc. .
Maybe, the crash is like:
Code: [Select]
#0 BAADF00D ?? () (??:??)
#1 627FB36C wxListBox::MSWOnDraw(void**) () (C:\Devel\wxWidgets\lib\gcc_dll\wxmsw28u_gcc_custom.dll:??)
#2 0028ACA4 ?? () (??:??)
#3 ?? ?? () (??:??)
Unfortunately that's all I get from the debugger session for the moment...

Notice "BAADF00D" and "MSWOnDraw". Looks like an invalid pointer access, so I suppose it's really related to the ClientData stuff... BTW I did a full re-build already.

EDIT: Reminds me: Do you actually clear the ClientData somewhere? Otherwise it's probably a memory leak, in addition...
« Last Edit: November 09, 2011, 02:56:58 pm by MortenMacFly »
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #41 on: November 09, 2011, 03:43:56 pm »
Sometimes...

Look here:
http://docs.wxwidgets.org/2.8/wx_wxchecklistbox.html
It states:
Quote
Please note that wxCheckListBox uses client data in its implementation, and therefore this is not available to the application.

That's probably the actual issue. And it works again, if I don't set the ClientData. Surely this is not an option - looks we need another solution to store the ProjectFile pointer.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #42 on: November 09, 2011, 05:44:07 pm »
Sometimes...

Look here:
http://docs.wxwidgets.org/2.8/wx_wxchecklistbox.html
It states:
Quote
Please note that wxCheckListBox uses client data in its implementation, and therefore this is not available to the application.

That's probably the actual issue. And it works again, if I don't set the ClientData. Surely this is not an option - looks we need another solution to store the ProjectFile pointer.
Unfortunately it works on linux, and so I did not stumble over it.
Thanks for finding this issue, I will look into it.

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #43 on: November 10, 2011, 11:11:27 am »
EDIT: Reminds me: Do you actually clear the ClientData somewhere? Otherwise it's probably a memory leak, in addition...

I don't think there is a need to do so.
The client-data is just a pointer to a projectfile.
The only way to "delete" it would be to set it to 0, but that needs exact the same amount of memory.

The memory the pointer points to is handled in cbproject.cpp in exactly the same way as without my patch.

By the way:
I attach a new one, which does not use the ClientData, sorts the files in the list correctly and also sorts the files in the project-files correctly.

The patch still contains some debug-code ( debug-log output).

And it is usable with TortoisSVN on windows, even if it is created with git.

[attachment deleted by admin]

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #44 on: November 11, 2011, 01:23:57 pm »
The client-data is just a pointer to a projectfile.
Ah - OK. I missed that.

I attach a new one, which does not use the ClientData, sorts the files in the list correctly and also sorts the files in the project-files correctly.
Tested and looks good. From my perspective the patch is ready to go - I clearly see speed improvements. I would commit a couple of very tiny refinements if you did afterwards.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ

Offline jens

  • Administrator
  • Lives here!
  • *****
  • Posts: 7265
    • Jens' unofficial debian-repository for the Code::Blocks - IDE
Re: Time used to open a file
« Reply #45 on: November 13, 2011, 05:22:08 pm »
Tested and looks good. From my perspective the patch is ready to go - I clearly see speed improvements. I would commit a couple of very tiny refinements if you did afterwards.

I just commited it (svn r7588), without the debugging-stuff.
So you can start your commits.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11899
    • Travis build status
Re: Time used to open a file
« Reply #46 on: November 13, 2011, 06:24:54 pm »
Jens: why have you left some commented code in /trunk/src/sdk/cbproject.cpp ?

And you have not increased the SDK version numbers.
« Last Edit: November 13, 2011, 06:26:26 pm by oBFusCATed »
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9496
Re: Time used to open a file
« Reply #47 on: November 13, 2011, 08:12:40 pm »
Jens: why have you left some commented code in /trunk/src/sdk/cbproject.cpp ?
...because that was my duty... ;-)
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ