« previous next »
Pages: [1] 2  All   Go Down

Author Topic: Loading big projects on windows is extreamly slow  (Read 9293 times)

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Loading big projects on windows is extreamly slow
« on: February 04, 2021, 12:43:20 am »
Hi,
loading the attached project on my new windows machine takes
Code
ProjectManager::LoadProject took: 124.372 seconds.
This loading makes codeblocks completely non repsonsive
in a linux VM (considerable less performant) loading the same project takes only
Code
ProjectManager::LoadProject took: 9.684 seconds.

I think the main reason is the wxWidgets function wxFileName::MakeRelativeTo

more information will follow...

attached is the ziped project file in question
Logged

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Loading big projects on windows is extreamly slow
« Reply #1 on: February 04, 2021, 10:52:27 am »
Do you have other timing lines in the log?
This is the output for me on linux (Ryzen 7 1700):
Code
cbProject::Open took: 1.428 seconds.
NativeParser::DoFullParsing took: 0.068 seconds.
ProjectManager::SetProject took: 0.073 seconds.
ProjectManager::LoadProject took: 2.434 seconds.

This is my version info:
Code
Name             : Code::Blocks
Version          : svn-r12278
SDK Version      : 2.6.0
Scintilla Version: 3.7.5
Author           : The Code::Blocks Team
E-mail           : info@codeblocks.org
Website          : http://www.codeblocks.org

wxWidgets Library (wxGTK port)
Version 3.0.4 (Unicode: wchar_t, debug level: 1),
compiled at May 14 2020 22:46:15

Runtime version of toolkit used is 2.24.
Compile-time GTK+ version is 2.24.32.

Are you using a release build of everything? Do you reproduce this if you use minimal set of plugins?
Logged
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Loading big projects on windows is extreamly slow
« Reply #2 on: February 04, 2021, 10:59:27 am »
If you want to really know what is going on, then V-Tune is one of the the easiest tool to use.
And it seems that it supports mingw: https://ib-krajewski.blogspot.com/2019/07/vtune-amplifier-and-profiling-mingw-on.html

Hint: You all you need is "basic hot spots" type of profile and then look at the "callers and callees" tab. It should work on both Intel and Amd processors.
Logged
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Loading big projects on windows is extreamly slow
« Reply #3 on: February 04, 2021, 09:37:38 pm »
Looks cool, but i am not able to download it....
every time i try to download it asks me for user name and password, or for registration... I register but never get the email, i click resend, get the email, click on confirm, try to download, get error message that i am not logged in...
bu***sh**t website...
Logged

Offline stahta01

  • Lives here!
  • ****
  • Posts: 7588
    • My Best Post
Re: Loading big projects on windows is extreamly slow
« Reply #4 on: February 04, 2021, 10:05:05 pm »
Remember to check your junk mail folder online if using GMail it blocks a lot of stuff.

Tim S.
Logged
C Programmer working to learn more about C++ and Git.
On Windows 7 64 bit and Windows 10 64 bit.
--
When in doubt, read the CB WiKi FAQ. http://wiki.codeblocks.org

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Loading big projects on windows is extreamly slow
« Reply #5 on: February 05, 2021, 01:34:41 am »
There seems to be two accounts, one normal intel account and one "intel dev account" in my profile is written "dev account activated" but if i try to login in my dev account i get wrong username/password.... ridiculous complicated...
on the amd page for µProf it was a simple download link, no registration, nothing... Sadly it seems that µProf is not able to use gdb symbols....

Code
cbProject::Open took: 83.271 seconds.
NativeParser::DoFullParsing took: 0.590 seconds.
ProjectManager::SetProject took: 0.686 seconds.
ProjectManager::LoadProject took: 92.235 seconds.
Logged

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Loading big projects on windows is extreamly slow
« Reply #6 on: February 05, 2021, 02:07:25 am »
Why are times so different? The first time you posted was 124.372 seconds, this one is 92.
Logged
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline Miguel Gimenez

  • Developer
  • Lives here!
  • *****
  • Posts: 1557
Re: Loading big projects on windows is extreamly slow
« Reply #7 on: February 05, 2021, 08:12:55 am »
In the Sign In to Get Your Download dialog there is a link in the bottom left corner, Maybe next time. Please take me to my download, it worked for me.
Logged

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Loading big projects on windows is extreamly slow
« Reply #8 on: February 05, 2021, 01:19:14 pm »
Quote from: Miguel Gimenez on February 05, 2021, 08:12:55 am
In the Sign In to Get Your Download dialog there is a link in the bottom left corner, Maybe next time. Please take me to my download, it worked for me.

This did it.... Thank you!
Logged

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Loading big projects on windows is extreamly slow
« Reply #9 on: February 05, 2021, 11:05:55 pm »
Code
Function / Call Stack	CPU Time	Module	Function (Full)	Source File	Start Address
GetLongPathNameW	65.7%	KERNEL32.DLL	GetLongPathNameW	[Unknown]	0x1800068c0
  ↖ wxFileName::GetLongPath ← wxFileName::Normalize	65.7%	wxmsw314u_gcc_custom.dll	wxFileName::GetLongPath(void) const	[Unknown]	0x64c5dd10
    wxFileName::MakeRelativeTo	62.2%	wxmsw314u_gcc_custom.dll	wxFileName::MakeRelativeTo(wxString const&, wxPathFormat)	[Unknown]	0x64c64b50
      ↖ cbProject::CalculateCommonTopLevelPath ← cbProject::Open ← cbProject::cbProject ← ProjectManager::LoadProject ← MainFrame::DoOpenProject ← MainFrame::OpenGeneric ← MainFrame::OnDropFiles ← MainFrame::DoOnFileOpen ← MainFrame::OnFileOpen ← wxAppConsoleBase::CallEventHandler ← wxEvtHandler::ProcessEventIfMatchesId ← wxEventHashTable::HandleEvent ← wxEvtHandler::TryHereOnly ← wxEvtHandler::DoTryChain ← wxEvtHandler::ProcessEvent ← wxWindowBase::TryAfter ← wxEvtHandler::SafelyProcessEvent ← wxMenuBase::DoProcessEvent ← wxMenuBase::SendEvent ← wxFrame::MSWWindowProc ← DispatchMessageW ← wxGUIEventLoop::Dispatch ← wxEventLoopManual::ProcessEvents ← wxEventLoopManual::DoRun ← wxEventLoopBase::Run ← wxAppConsoleBase::MainLoop ← CodeBlocksApp::OnRun ← func@0x64c73c90 ← WinMain ← func@0x4dd760 ← func@0x40114b ← func@0x4010ed ← BaseThreadInitThunk ← RtlUserThreadStart	38.7%	codeblocks.dll	cbProject::CalculateCommonTopLevelPath(void)	cbproject.cpp	0x7096cc5c
      ↖ cbProject::AddFile ← ProjectLoader::DoUnits ← ProjectLoader::Open ← cbProject::Open ← cbProject::cbProject ← ProjectManager::LoadProject ← MainFrame::DoOpenProject ← MainFrame::OpenGeneric ← MainFrame::OnDropFiles ← MainFrame::DoOnFileOpen ← MainFrame::OnFileOpen ← wxAppConsoleBase::CallEventHandler ← wxEvtHandler::ProcessEventIfMatchesId ← wxEventHashTable::HandleEvent ← wxEvtHandler::TryHereOnly ← wxEvtHandler::DoTryChain ← wxEvtHandler::ProcessEvent ← wxWindowBase::TryAfter ← wxEvtHandler::SafelyProcessEvent ← wxMenuBase::DoProcessEvent ← wxMenuBase::SendEvent ← wxFrame::MSWWindowProc ← DispatchMessageW ← wxGUIEventLoop::Dispatch ← wxEventLoopManual::ProcessEvents ← wxEventLoopManual::DoRun ← wxEventLoopBase::Run ← wxAppConsoleBase::MainLoop ← CodeBlocksApp::OnRun ← func@0x64c73c90 ← WinMain ← func@0x4dd760 ← func@0x40114b ← func@0x4010ed ← BaseThreadInitThunk ← RtlUserThreadStart	19.7%	codeblocks.dll	cbProject::AddFile(int, wxString const&, bool, bool, unsigned short)	cbproject.cpp	0x7096f898
      ↖ HeaderDirTraverser::OnFile ← wxDir::Traverse	3.8%	codecompletion.dll	HeaderDirTraverser::OnFile(wxString const&)	systemheadersthread.cpp	0x6775ca54
    ↖ wxPathList::Add ← ConfigManager::LocateDataFile	3.5%	wxmsw314u_gcc_custom.dll	wxPathList::Add(wxString const&)	[Unknown]	0x64c521e0
    ↖ NormalizePath ← ParserBase::FindFileInIncludeDirs ← ParserBase::FindFirstFileInIncludeDirs ← ParserBase::GetFullFileName ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← ParserThread::Execute ...	0.0%	codeblocks.dll	NormalizePath(wxFileName&, wxString const&)	globals.cpp	0x709f339e

Really nice tool i have to say... Installation gives some problems, but it is nice to use...

Quote
Why are times so different? The first time you posted was 124.372 seconds, this one is 92.
Don't know, probably had something running in the background... This machine is quite some working horse for me...
I have tested in now a few times, and came to a middle of 63 seconds with an nearly idle PC, with the timing percentage on the top
Logged

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Loading big projects on windows is extreamly slow
« Reply #10 on: February 06, 2021, 10:10:18 am »
Which tab is this info coming from? Seems to be "bottom something".
Why is this API so slow for you?
Do you see matching times if you switch the "CPU Time" column from percentages to actual time (right click in the column)?
Logged
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Loading big projects on windows is extreamly slow
« Reply #11 on: February 06, 2021, 10:11:07 am »
Can someone else report the time on his/her windows machine?
Logged
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline Miguel Gimenez

  • Developer
  • Lives here!
  • *****
  • Posts: 1557
Re: Loading big projects on windows is extreamly slow
« Reply #12 on: February 06, 2021, 11:48:34 am »
Loading the project with C::B in output31 directory:

Code
ProjectManager::LoadProject took: 38.701 seconds.

The devel31 one:

Code
ProjectManager::LoadProject took: 40.413 seconds.

When using VTune with the devel31 executable:

Bottom-up
Code
Function / Call Stack	CPU Time	Module	Function (Full)	Source File	Start Address
NtOpenFile	9.435s	ntdll.dll	NtOpenFile	[Unknown]	0x4b2f1a20
NtQueryDirectoryFileEx	7.939s	ntdll.dll	NtQueryDirectoryFileEx	[Unknown]	0x4b2f2b10
NtClose	5.195s	ntdll.dll	NtClose	[Unknown]	0x4b2f17c0
NtQueryAttributesFile	4.033s	ntdll.dll	NtQueryAttributesFile	[Unknown]	0x4b2f1ac0
LoadLibraryExW	3.281s	KernelBase.dll	LoadLibraryExW	[Unknown]	0x1010f9f0
func@0x71a3ea30	2.285s	COMCTL32.DLL	func@0x71a3ea30	[Unknown]	0x71a3ea30
NtQueryInformationFile	2.113s	ntdll.dll	NtQueryInformationFile	[Unknown]	0x4b2f17e0
NtCreateFile	1.624s	ntdll.dll	NtCreateFile	[Unknown]	0x4b2f1c40
NtYieldExecution	1.600s	ntdll.dll	NtYieldExecution	[Unknown]	0x4b2f1b50
NtUserMsgWaitForMultipleObjectsEx	1.454s	win32u.dll	NtUserMsgWaitForMultipleObjectsEx	[Unknown]	0x10005860
wxFileName::GetPath	0.894s	wxmsw314u_gcc_custom.dll	wxFileName::GetPath(int, wxPathFormat) const	[Unknown]	0x64e82900

Top-down Tree
Code
Function Stack	CPU Time: Total	CPU Time: Self	Module	Function (Full)	Source File	Start Address
NtOpenFile	17.0%	9.435s	ntdll.dll	NtOpenFile	[Unknown]	0x4b2f1a20
NtQueryDirectoryFileEx	14.3%	7.939s	ntdll.dll	NtQueryDirectoryFileEx	[Unknown]	0x4b2f2b10
NtClose	9.4%	5.195s	ntdll.dll	NtClose	[Unknown]	0x4b2f17c0
NtQueryAttributesFile	7.3%	4.033s	ntdll.dll	NtQueryAttributesFile	[Unknown]	0x4b2f1ac0
LoadLibraryExW	5.9%	3.281s	KernelBase.dll	LoadLibraryExW	[Unknown]	0x1010f9f0
func@0x71a3ea30	4.1%	2.285s	COMCTL32.DLL	func@0x71a3ea30	[Unknown]	0x71a3ea30
NtQueryInformationFile	3.8%	2.113s	ntdll.dll	NtQueryInformationFile	[Unknown]	0x4b2f17e0
NtCreateFile	2.9%	1.624s	ntdll.dll	NtCreateFile	[Unknown]	0x4b2f1c40
NtYieldExecution	2.9%	1.600s	ntdll.dll	NtYieldExecution	[Unknown]	0x4b2f1b50
NtUserMsgWaitForMultipleObjectsEx	2.6%	1.454s	win32u.dll	NtUserMsgWaitForMultipleObjectsEx	[Unknown]	0x10005860
wxFileName::GetPath	1.6%	0.894s	wxmsw314u_gcc_custom.dll	wxFileName::GetPath(int, wxPathFormat) const	[Unknown]	0x64e82900

GetLongPathNameW only took 81 ms

C::B head
wxWidgets 3.1.4 32 bits
Windows 10 20H2 64 bits
i7-8700K @ 3.7 GHz
SSD disk
Logged

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Loading big projects on windows is extreamly slow
« Reply #13 on: February 06, 2021, 04:19:31 pm »
Top-down is mostly useless view. Callers and callees is a lot better. Ideally generating flame-graphs is the most presentable way, but unfortunately there is no native way to do it with vtune.

@Miguel Gimenez: Are you using debug (-Og/-O0) or release (-O2/-O3) build of wx?
Logged
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Loading big projects on windows is extreamly slow
« Reply #14 on: February 06, 2021, 04:22:09 pm »
Also if you want to have meaningful profiles you need to build everything with optimizations (-O2 or higher), symbols (-g), and no stripping (no -s or calls to strip).
Logged
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]
Pages: [1] 2  All   Go Up
« previous next »