Author Topic: Time used to open a file  (Read 32219 times)

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Time used to open a file
« Reply #30 on: May 07, 2011, 09:00:03 am »
Hi
I posted another thread about very similar topic here http://forums.codeblocks.org/index.php/topic,14644.0.html .
I have got there response with link to this thread. On the first view it looks like it is the same problem I experiencing however I think that you are fighting another one.
If I understand correctly, this thread is beating the slow opening of project with already opened files (reopening those previously opened files in the editor). I am fighting the problem of opening  new project and/or creating new project with many files. Please correct me if I am wrong.

By the way I disabled and then removed all plugins (by removing them from the disk) and it had no influence to the loading speed. I think that the reason is that plugins do not come to the scope at the time of creating of the virtual folder's tree.

An on the end I can contribute to fighting your problem. From my experience the big speed up comes when you disable automatic completion update (what is probably done every keypress) and instead of this let it be updated every "file save". This can be set somewhere in editor settings.

I already have read your other post and will try to look into it.
I use the linux-kernel sources as test project (just to load it, not to compile or anything).

If I have the code-completion plugin removed, it does not change anything when opening a great amount of files.
On linux (debian 64-bit gnome 3) there are two slowdowns: for great files it's the styling of the editor (probably fixed by the patch) and the labeling of the notebook (hopefully also fixed).
But I did not yet test on windows.
Speed-up should be much less there (because at least the labeling is already much faster on windows, than on linux [wxWidgets ?]).


Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Time used to open a file
« Reply #31 on: May 10, 2011, 12:32:25 am »
I already have read your other post and will try to look into it.
I use the linux-kernel sources as test project (just to load it, not to compile or anything).

Here comes a patch, that replaces wxList with a wxHashSet for the ProjectFile*-list, what leads to a more or less great amount of changes.
The patch is not deeply tested and nottested on windows at all.
It's mixed with the other patch, described earlier in this thread.

I used the linux-kernel 2.6.35 as test project, no files opened, just loading the project and filling the treectrl in management pane.
CC-plugin is disabled, because it sometimes eats up all of my memeory and makes the system nearly unusable.

Overall load-time with trunk (from closing the starthere-page until selecting the active build-target):
1 m 30.846 s
with my changes:
0 m 11.398 s

in other words:
about 8 times faster

closing is much faster with trunk (at the moment):
2.255 s
and with my patch:
5.465 s
about 2.4 times slower

exact values (snippets from debug log with DisplayEvents-plugin):

Code
wxHashSet (codecompletion disabled)

open linux-kernel 2.6.35


23:50:31,744  =>  cbEVT_EDITOR_CLOSE
Loading project file...
Parsing project file...
23:50:32,141  =>  cbEVT_PROJECT_RENAMED
23:50:32,149  =>  cbEVT_BUILDTARGET_ADDED
23:50:32,149  =>  cbEVT_PROJECT_TARGETS_MODIFIED
Loading target Debug
23:50:32,150  =>  cbEVT_BUILDTARGET_ADDED
23:50:32,150  =>  cbEVT_PROJECT_TARGETS_MODIFIED
Loading target Release
Loading project files...
23:50:32,175  =>  cbEVT_PROJECT_BEGIN_ADD_FILES
23:50:35,732  =>  cbEVT_PROJECT_END_ADD_FILES
33115 files loaded
Done loading project in 3991ms
Project's base path: /home/jens/kernel-tmp.2.6.35/
Project's common toplevel path: /home/jens/kernel-tmp.2.6.35/
23:50:39,498  =>  cbEVT_PROJECT_OPEN
23:50:39,499  =>  cbEVT_WORKSPACE_CHANGED
23:50:43,138  =>  cbEVT_PROJECT_ACTIVATE
23:50:43,142  =>  cbEVT_BUILDTARGET_SELECTED

and close it

Removed test from all deps
23:53:53,051  =>  cbEVT_EDITOR_SWITCHED
23:53:53,138  =>  cbEVT_EDITOR_ACTIVATED
23:53:53,259  =>  cbEVT_PROJECT_CLOSE
23:53:58,248  =>  cbEVT_BUILDTARGET_SELECTED
23:53:58,289  =>  cbEVT_WORKSPACE_CHANGED
23:53:58,431  =>  cbEVT_APP_ACTIVATED
23:53:58,447  =>  cbEVT_EDITOR_SWITCHED
23:53:58,516  =>  cbEVT_EDITOR_ACTIVATED

trunk

open

23:55:21,116  =>  cbEVT_EDITOR_CLOSE
Loading project file...
Parsing project file...
23:55:21,345  =>  cbEVT_PROJECT_RENAMED
23:55:21,354  =>  cbEVT_BUILDTARGET_ADDED
23:55:21,355  =>  cbEVT_PROJECT_TARGETS_MODIFIED
Loading target Debug
23:55:21,355  =>  cbEVT_BUILDTARGET_ADDED
23:55:21,355  =>  cbEVT_PROJECT_TARGETS_MODIFIED
Loading target Release
Loading project files...
23:55:21,379  =>  cbEVT_PROJECT_BEGIN_ADD_FILES
23:56:43,454  =>  cbEVT_PROJECT_END_ADD_FILES
33115 files loaded
Done loading project in 82341ms
Project's base path: /home/jens/kernel-tmp.2.6.35/
Project's common toplevel path: /home/jens/kernel-tmp.2.6.35/
23:56:47,745  =>  cbEVT_PROJECT_OPEN
23:56:47,746  =>  cbEVT_WORKSPACE_CHANGED
23:56:51,936  =>  cbEVT_PROJECT_ACTIVATE
23:56:51,962  =>  cbEVT_BUILDTARGET_SELECTED

and close it

Removed test from all deps
23:58:36,742  =>  cbEVT_EDITOR_SWITCHED
23:58:36,762  =>  cbEVT_EDITOR_ACTIVATED
23:58:36,876  =>  cbEVT_PROJECT_CLOSE
23:58:38,597  =>  cbEVT_BUILDTARGET_SELECTED
23:58:38,671  =>  cbEVT_WORKSPACE_CHANGED
23:58:38,886  =>  cbEVT_APP_ACTIVATED
23:58:38,906  =>  cbEVT_EDITOR_SWITCHED
23:58:38,997  =>  cbEVT_EDITOR_ACTIVATED

As mentioned not deeply tested (might break some stuff), so be careful.
Make a backup of your project to be secure .

EDIT:

patch updated, see: http://forums.codeblocks.org/index.php/topic,14543.msg103572.html#msg103572
« Last Edit: October 28, 2011, 05:39:09 pm by jens »

Offline Folco

  • Regular
  • ***
  • Posts: 343
    • Folco's blog (68k lover)
Re: Time used to open a file
« Reply #32 on: August 24, 2011, 08:24:01 pm »
Have these patches been applied ?
I think that now, the time spent to open a project is significantly lower, am I right ?
Kernel Extremist - PedroM power ©

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Time used to open a file
« Reply #33 on: October 28, 2011, 05:38:23 pm »
Updated wxHashSet-patch (against svn r7546).

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Time used to open a file
« Reply #34 on: October 29, 2011, 12:18:23 pm »
...for all those that cannot apply this version of the patch, try the one attached. It is the same (just another format, and the files got re-ordered).
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Time used to open a file
« Reply #35 on: November 09, 2011, 09:44:17 am »
Updated wxHashSet-patch (against svn r7546).
Jens, after testing for a couple of days I think this patch still has serious issues. I am facing the following strange phenomena:
  • The ordering of the files in the project file always change for each save - bad. :-(
  • If I copy a target ion the project settings, the "build files" are empty
  • Clicking in the build files crashes C::B (even without creating as log)
So... something is seriously wrong here, unfortunately.

Could you check, if you see files in the project's build options (lower right part)?
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Time used to open a file
« Reply #36 on: November 09, 2011, 12:39:59 pm »
...ok, one thing I noticed is a bug that gets introduces with this path. In nativeparser.cpp you exchange:
-    for (int i = 0; project && i < project->GetFilesCount(); ++i)
...with:
+    for (FilesList::iterator it = project->GetFilesList().begin(); it != project->GetFilesList().end(); ++it)
Thus, you missed the NULL pointer check (notice project && i < project->GetFilesCount()). Surrounding this for loop with a if (project) clause resolves some crashes at least.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Time used to open a file
« Reply #37 on: November 09, 2011, 01:52:40 pm »
I will check for misiing nullpointer-checks as soon as possible (I never had any crashes).
And of course test all others issues you have described also.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Time used to open a file
« Reply #38 on: November 09, 2011, 02:04:06 pm »
I will check for misiing nullpointer-checks as soon as possible (I never had any crashes).
I think that was the only one. The crash in that case is gone.

And of course test all others issues you have described also.
Besides the project file changes the only major remaining issue is in the project properties dialog: You don't even need to copy a target. The build file list is always empty. And if you click into it C::B crashes w/o backtrace (!). Looks its related to the fact that you set the project file as client data to the checked list...
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Time used to open a file
« Reply #39 on: November 09, 2011, 02:26:15 pm »
I will check for misiing nullpointer-checks as soon as possible (I never had any crashes).
I think that was the only one. The crash in that case is gone.

And of course test all others issues you have described also.
Besides the project file changes the only major remaining issue is in the project properties dialog: You don't even need to copy a target. The build file list is always empty. And if you click into it C::B crashes w/o backtrace (!). Looks its related to the fact that you set the project file as client data to the checked list...

I can reproduce the random fileorder, I hope it can be fixed easily.
I can see all my files in the "Build targets" tab, and there is no crash.
Either it's a windows-special, or you need to do a really clean build, with manually removed pch's etc. .

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Time used to open a file
« Reply #40 on: November 09, 2011, 02:40:56 pm »
Either it's a windows-special, or you need to do a really clean build, with manually removed pch's etc. .
Maybe, the crash is like:
Code
#0 BAADF00D	?? () (??:??)
#1 627FB36C wxListBox::MSWOnDraw(void**) () (C:\Devel\wxWidgets\lib\gcc_dll\wxmsw28u_gcc_custom.dll:??)
#2 0028ACA4 ?? () (??:??)
#3 ?? ?? () (??:??)
Unfortunately that's all I get from the debugger session for the moment...

Notice "BAADF00D" and "MSWOnDraw". Looks like an invalid pointer access, so I suppose it's really related to the ClientData stuff... BTW I did a full re-build already.

EDIT: Reminds me: Do you actually clear the ClientData somewhere? Otherwise it's probably a memory leak, in addition...
« Last Edit: November 09, 2011, 02:56:58 pm by MortenMacFly »
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Time used to open a file
« Reply #41 on: November 09, 2011, 03:43:56 pm »
Sometimes...

Look here:
http://docs.wxwidgets.org/2.8/wx_wxchecklistbox.html
It states:
Quote
Please note that wxCheckListBox uses client data in its implementation, and therefore this is not available to the application.

That's probably the actual issue. And it works again, if I don't set the ClientData. Surely this is not an option - looks we need another solution to store the ProjectFile pointer.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Time used to open a file
« Reply #42 on: November 09, 2011, 05:44:07 pm »
Sometimes...

Look here:
http://docs.wxwidgets.org/2.8/wx_wxchecklistbox.html
It states:
Quote
Please note that wxCheckListBox uses client data in its implementation, and therefore this is not available to the application.

That's probably the actual issue. And it works again, if I don't set the ClientData. Surely this is not an option - looks we need another solution to store the ProjectFile pointer.
Unfortunately it works on linux, and so I did not stumble over it.
Thanks for finding this issue, I will look into it.

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Time used to open a file
« Reply #43 on: November 10, 2011, 11:11:27 am »
EDIT: Reminds me: Do you actually clear the ClientData somewhere? Otherwise it's probably a memory leak, in addition...

I don't think there is a need to do so.
The client-data is just a pointer to a projectfile.
The only way to "delete" it would be to set it to 0, but that needs exact the same amount of memory.

The memory the pointer points to is handled in cbproject.cpp in exactly the same way as without my patch.

By the way:
I attach a new one, which does not use the ClientData, sorts the files in the list correctly and also sorts the files in the project-files correctly.

The patch still contains some debug-code ( debug-log output).

And it is usable with TortoisSVN on windows, even if it is created with git.

[attachment deleted by admin]

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9694
Re: Time used to open a file
« Reply #44 on: November 11, 2011, 01:23:57 pm »
The client-data is just a pointer to a projectfile.
Ah - OK. I missed that.

I attach a new one, which does not use the ClientData, sorts the files in the list correctly and also sorts the files in the project-files correctly.
Tested and looks good. From my perspective the patch is ready to go - I clearly see speed improvements. I would commit a couple of very tiny refinements if you did afterwards.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: https://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: https://wiki.codeblocks.org/index.php?title=FAQ