Print Page - Codecompletion delays fixed in rev 1826 (update: in rev 1840)

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: rickg22 on January 22, 2006, 02:50:07 am

Title: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 22, 2006, 02:50:07 am

Hi all, I'm glad to announce that the long-time delays introduced in the redesign have been fixed in revision 1826 of Code::Blocks. You can use global includes again :-)

The reparsing with cache issue and the 0.5 second delay still remain, tho.

Title: Re: Codecompletion delays fixed in rev 1826 :)
Post by: sethjackson on January 22, 2006, 02:51:59 am

Bulding rev 1826 right now...... :D Now I have a long-time delay recompiling the SDK. :P

Title: Re: Codecompletion delays fixed in rev 1826 :)
Post by: Michael on January 22, 2006, 03:20:45 pm

Quote from: rickg22 on January 22, 2006, 02:50:07 am

Hi all, I'm glad to announce that the long-time delays introduced in the redesign have been fixed in revision 1826 of Code::Blocks. You can use global includes again :-)

Great, thanks :D.

Michael

[EDIT] After building C::B rev1828 and try opening C::B project and ContribPlugins.workspace, the speed is at least 30% higher as before. Good job :D.

Title: Re: Codecompletion delays fixed in rev 1826 :)
Post by: thomas on January 22, 2006, 06:48:24 pm

Startup when opening the Code::Blocks project takes 7 seconds with code completion on (about 1 second without). Total parse time is now 17 seconds instead of 40 (almost as fast as it used to be). Saving files is instantaneous.

I found two bugs regarding parsing/reparsing:
1. If you change options, you're prompted whether you want to reparse. If you change options again before that reparsing is done, and you answer "yes" another time, Code::Blocks freezes up. Mutex maybe?

2. If you change options and say "yes" to reparse, Code::Blocks will crash on exit if you close the window before the reparse is finished. Funnily, you can otherwise close the window at any time, even if the parse at startup has not finished. The crash only occurs while reparsing.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 23, 2006, 07:24:31 am

I haven't fixed those bugs yet... I think.

I removed a Mutex and replaced it with a critical section, maybe that'd do the trick with the lockup.

I also removed (now confirmed :D ) the delay at startup!
Additionally, thanks to the SearchTree(TM) class (used for global filenames lookup), I could optimize the global files parsing by another 30%! :)

Next week I'll focus on the reparsing bugs.
Uh, now that I think of it... Please submit them to Sourceforge! All the newfound bugs in CC need to be organized in the database.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 23, 2006, 01:02:12 pm

Quote from: rickg22 on January 23, 2006, 07:24:31 am

I removed a Mutex and replaced it with a critical section, maybe that'd do the trick with the lockup.

That's the same thing (well, for our purpose, it is), so that won't be the reason.
It rather looks as if there is no locking at all when entering the parse cycle. Apparently, two threads are concurrently modifying the same tree structure (which eventually freezes up).

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 24, 2006, 04:38:06 am

Thomas: I couldn't replicate the bug. I experience a brief delay, but did as you explained, and no hang / crash was detected.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: tiwag on January 24, 2006, 10:50:47 am

there must still be a bug in the codecompletion plugin ...

how to reproduce / what i've done:

1. create a workspace, which contains CodeBlocks and all Contrib-Plugins
2. open this workspace with CodeCompletion enabled, parse local & global includes,
it lasts about 400sec to load and parse all projects and the memory-consumption is about 600MB !!! (->slow as molasses)
3. when i close the workspace, C::B crashes with a segmentation fault

--- for reference ---

4. open this workspace with CodeCompletion disabled, parse local & global includes,
it lasts about 300sec to load and parse all projects and the memory-consumption is about 270MB
5. when i close the workspace, C::B doesn't crash

6. open this workspace with CodeCompletion disabled, parse local includes only,
it lasts about 22sec to load and parse all projects and the memory-consumption is about 50MB
7. when i close the workspace, C::B doesn't crash

tested on a AMD XP 2800, 512MB RAM,
Windows XP, SP2
C::B SVN rev 1841

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 11:21:02 am

Quote from: rickg22 on January 24, 2006, 04:38:06 am

Thomas: I couldn't replicate the bug.

When closing while reparsing in 1846, I get this from Dr.MinGW:

Code

DEBUG_EVENT:
	dwDebugEventCode = CREATE_PROCESS_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 714
	hProcess = 718
	hThread = 798
	lpBaseOfImage = 400000
	dwDebugInfoFileOffset = 475000
	nDebugInfoSize = 426C
	lpThreadLocalBase = 7FFDD000
	lpStartAddress = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = CREATE_THREAD_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 220
	hThread = 710
	lpThreadLocalBase = 7FFDC000
	lpStartAddress = 7C810856
DEBUG_EVENT:
	dwDebugEventCode = CREATE_THREAD_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 214
	hThread = 70C
	lpThreadLocalBase = 7FFDB000
	lpStartAddress = 7C810856
DEBUG_EVENT:
	dwDebugEventCode = CREATE_THREAD_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 668
	hThread = 708
	lpThreadLocalBase = 7FFD9000
	lpStartAddress = 7C810856
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 704
	lpBaseOfDll = 7C910000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 700
	lpBaseOfDll = 7C800000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6FC
	lpBaseOfDll = 77BE0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6F8
	lpBaseOfDll = 7C9D0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6F4
	lpBaseOfDll = 77DA0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6F0
	lpBaseOfDll = 77E50000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6EC
	lpBaseOfDll = 77EF0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6E8
	lpBaseOfDll = 77D10000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6E4
	lpBaseOfDll = 77F40000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6E0
	lpBaseOfDll = 10000000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6DC
	lpBaseOfDll = 773A0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6D8
	lpBaseOfDll = 76350000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6D4
	lpBaseOfDll = 6FBC0000
	dwDebugInfoFileOffset = 1A00
	nDebugInfoSize = 19A
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6D0
	lpBaseOfDll = 774B0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6CC
	lpBaseOfDll = 770F0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6C8
	lpBaseOfDll = 76AF0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6C4
	lpBaseOfDll = 71A30000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6C0
	lpBaseOfDll = 71A10000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6BC
	lpBaseOfDll = 71A00000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6B8
	lpBaseOfDll = 604C0000
	dwDebugInfoFileOffset = BD4800
	nDebugInfoSize = B0FB
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6B4
	lpBaseOfDll = 8E0000
	dwDebugInfoFileOffset = 3D8000
	nDebugInfoSize = 2C8E
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6B0
	lpBaseOfDll = 5D100000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6AC
	lpBaseOfDll = 5B420000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6A8
	lpBaseOfDll = 5B0F0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6A4
	lpBaseOfDll = 13E0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 6A0
	lpBaseOfDll = 76BB0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 69C
	lpBaseOfDll = 77BD0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 698
	lpBaseOfDll = 77660000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 694
	lpBaseOfDll = 76F20000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 690
	lpBaseOfDll = 71B70000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 68C
	lpBaseOfDll = 1600000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 688
	lpBaseOfDll = 76320000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 684
	lpBaseOfDll = 20000000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 680
	lpBaseOfDll = 4B4D0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 67C
	lpBaseOfDll = 6FFC0000
	dwDebugInfoFileOffset = 3C9800
	nDebugInfoSize = 26C5
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 678
	lpBaseOfDll = 1F90000
	dwDebugInfoFileOffset = 244600
	nDebugInfoSize = 2198
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 674
	lpBaseOfDll = 69C00000
	dwDebugInfoFileOffset = 21B000
	nDebugInfoSize = 1D29
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 670
	lpBaseOfDll = 64B80000
	dwDebugInfoFileOffset = 61F400
	nDebugInfoSize = 4E77
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 66C
	lpBaseOfDll = 25E0000
	dwDebugInfoFileOffset = 237400
	nDebugInfoSize = 1F1E
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 668
	lpBaseOfDll = 2830000
	dwDebugInfoFileOffset = 3EAA00
	nDebugInfoSize = 3C47
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 660
	lpBaseOfDll = 2C30000
	dwDebugInfoFileOffset = 18D000
	nDebugInfoSize = 75B
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 65C
	lpBaseOfDll = 67C40000
	dwDebugInfoFileOffset = 333A00
	nDebugInfoSize = 3635
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 658
	lpBaseOfDll = 636C0000
	dwDebugInfoFileOffset = 22BA00
	nDebugInfoSize = 1E70
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 654
	lpBaseOfDll = 2DD0000
	dwDebugInfoFileOffset = 2B1200
	nDebugInfoSize = 2622
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 650
	lpBaseOfDll = 3090000
	dwDebugInfoFileOffset = 3E5A00
	nDebugInfoSize = 33EE
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 64C
	lpBaseOfDll = 6DF40000
	dwDebugInfoFileOffset = 253A00
	nDebugInfoSize = 1ED7
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 648
	lpBaseOfDll = 3480000
	dwDebugInfoFileOffset = 27EA00
	nDebugInfoSize = 2B8C
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 644
	lpBaseOfDll = 6E500000
	dwDebugInfoFileOffset = 236000
	nDebugInfoSize = 1E06
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 640
	lpBaseOfDll = 6A7C0000
	dwDebugInfoFileOffset = 250A00
	nDebugInfoSize = 2403
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 63C
	lpBaseOfDll = 6C080000
	dwDebugInfoFileOffset = D9E000
	nDebugInfoSize = BE87
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 638
	lpBaseOfDll = 3710000
	dwDebugInfoFileOffset = 20EC00
	nDebugInfoSize = 1683
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 634
	lpBaseOfDll = 3AC0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 630
	lpBaseOfDll = 76C50000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 62C
	lpBaseOfDll = 59DD0000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = LOAD_DLL_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	hFile = 628
	lpBaseOfDll = 77B10000
	dwDebugInfoFileOffset = 0
	nDebugInfoSize = 0
	lpImageName = NULL
	fUnicoded = 1
DEBUG_EVENT:
	dwDebugEventCode = CREATE_THREAD_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 218
	hThread = 624
	lpThreadLocalBase = 7FFDA000
	lpStartAddress = 7C96077B
DEBUG_EVENT:
	dwDebugEventCode = EXCEPTION_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 218
	ExceptionCode = 80000003
	ExceptionFlags = 0
	ExceptionAddress = 7C911230
	dwFirstChance = 1
DEBUG_EVENT:
	dwDebugEventCode = EXCEPTION_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	ExceptionCode = C0000005
	ExceptionFlags = 0
	ExceptionAddress = 65662D20
	dwFirstChance = 0
codeblocks.exe caused an Access Violation at location 65662d20 Reading from location 65662d20.

Registers:
eax=65662d20 ebx=01295f20 ecx=025450e4 edx=00000024 esi=105f3b80 edi=0022dc24
eip=00000000 esp=0022fff8 ebp=00000000 iopl=0         nv up ei pl nz ac po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000216

Call stack:
AddrPC     AddrReturn AddrFrame  AddrStack  Params
00000000   00401240   0022FFF4   0022FFF8   00000000   78746341   00000020   00000001
00000000
DEBUG_EVENT:
	dwDebugEventCode = EXIT_THREAD_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 220
	dwExitCode = C0000005
DEBUG_EVENT:
	dwDebugEventCode = EXIT_THREAD_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 478
	dwExitCode = C0000005
DEBUG_EVENT:
	dwDebugEventCode = EXIT_THREAD_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 214
	dwExitCode = C0000005
DEBUG_EVENT:
	dwDebugEventCode = EXIT_THREAD_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 668
	dwExitCode = C0000005
DEBUG_EVENT:
	dwDebugEventCode = EXIT_PROCESS_DEBUG_EVENT
	dwProcessId = 70C
	dwThreadId = 218
	dwExitCode = C0000005

When running Code::Blocks from the debugger, it points out a segfault in GetNoteBook(). It only ever happens when closing while reparsing after changing settings. Closing the application while it is parsing otherwise works fine.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 24, 2006, 03:18:45 pm

I just tried up to step 2 of what tiwag said and got Code::Blocks to die even before it finished parsing all the projects:

Quote from: Code::Blocks Debug

[08:57:18.281]: Start parsing project Code::Blocks (wx2.6)
[08:57:18.281]: Concurrent threads for pool set to 2
[09:01:20.062]: Done parsing project copystrings (314 total parsed files, 22832 tokens in 239.781 seconds).
[09:01:31.265]: Done parsing project Code Stat (322 total parsed files, 23212 tokens in 11.203 seconds).
[09:01:31.265]: Updating class browser...
[09:01:31.281]: Class browser updated.
[09:01:32.765]: Done parsing project C::B KeyBinder (328 total parsed files, 24011 tokens in 1.500 seconds).
[09:01:33.656]: Done parsing project C::B Profiler (326 total parsed files, 23375 tokens in 0.891 seconds).
[09:01:39.921]: Done parsing project Help Plugin (wx2.6) (352 total parsed files, 25091 tokens in 6.265 seconds).
[09:01:52.203]: Done parsing project Exporter (362 total parsed files, 25283 tokens in 12.282 seconds).
[09:01:54.390]: Done parsing project Dev-C++ DevPak updater/installer (wx2.6) (375 total parsed files, 26298 tokens in 2.187 seconds).
[09:02:31.203]: Done parsing project wxSmith for wx 2.6 (575 total parsed files, 29055 tokens in 36.813 seconds).

Memory usage: 263.392 KB

Something I noticed is it takes 50% CPU while parsing half of the time. It goes like 50% for a few seconds then 0% for some seconds and again.

The reason it takes 50% and not 100% is 'cause my CPU has HT, even though it's supposed to be using 2 threads...

Another funny thing is the way it crashed. The "codeblocks.exe ha detectado un problema..." window is there just reporting it crashed, but I'm still able to "use" Code::Blocks. Really, I can click on menus, the project tree, but I just tried opening the Environment settings and it is finally unresponsive :P

Since I don't compile it with debugging information it's what I get:

Quote from: Dr. MinGW

codeblocks.exe caused an Access Violation at location 7c9206c3 in module ntdll.dll Writing to location 12fffffe.

The call stack is just 00000000.

Tested on an Intel P4 3.0E, 1GiB RAM,
Windows XP, SP2
Code::Blocks SVN revision 1833

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 03:35:23 pm

Quote

Another funny thing is the way it crashed. The "codeblocks.exe ha detectado un problema..." window is there just reporting it crashed, but I'm still able to "use" Code::Blocks.

That must be because Code::Blocks doesn't understand Spanish, so it wouldn't know ;)

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 24, 2006, 04:46:52 pm

Quote from: thomas

That must be because Code::Blocks doesn't understand Spanish, so it wouldn't know :wink:

:lol:

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 24, 2006, 05:31:50 pm

:shock: Agh I'm such a fool! I used static events declaration for a part of the parser, no wonder it crashes! I need to use dynamic event tables! :oops:

Now, regarding the memory consumption, each token takes around 1K of memory usage. By summing the count of all tokens, i get around 247000 tokens, including the codeblocks.cbp (i used 73000 tokens which is what I get for global includes enabled). So the codecompletion tokens alone (without including the extra memory for data structures we end up using 256Megs, and this is because many tokens are duplicated among projects.

I think the whole model needs to be redesigned, it's obviously not fit for so many large projects at once... :(

I need another favor guys. What's the token count *WITHOUT* local or global includes?

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Michael on January 24, 2006, 05:38:14 pm

Hello,

I have checked the memory consumption of C::B (rev1846) with code completion all enabled. When I open C::B project, the memory's requirement increases up to 92-93 MB. Some time later (5-10 secondes, but I do not know exactly. Just the time to re-open this topic and press button "reply") the memory consumption falls to 5-6 MB and the project is opened.

May be storing the search trees in the disk instead of the memory would decrease the memory's requirement...

Michael

[EDIT] And here some more info (from Code::Blocks Debug):

Quote

Done parsing project Code::Blocks (wx2.6) (980 total parsed files, 60891 tokens in 2.394 seconds).

The parsing time seems low, but another try has given me 6.229 seconds.

And without all deselected:

Quote

Done parsing project Code::Blocks (wx2.6) (542 total parsed files, 15551 tokens in 3.495 seconds).

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 24, 2006, 05:54:36 pm

CodeBlocks.cbp for rev1833 with follow local and global includes takes 77328 KB and stays there.

Quote from: Code::Blocks Debug

Done parsing project Code::Blocks (wx2.6) (982 total parsed files, 45558 tokens in 61.109 seconds).

Now without global includes it takes 51272 KB.

Quote from: Code::Blocks Debug

Done parsing project Code::Blocks (wx2.6) (542 total parsed files, 15540 tokens in 1.766 seconds).

With both global and local disabled it takes 51332 KB and has the same results about parsed files and tokens found.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 06:09:19 pm

Hmmm... I have slightly different figures.

Using this workspace:

Code

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<CodeBlocks_workspace_file>
	<Workspace title="All contrib plugins">
		<Project filename="cb\plugins\contrib\codestat\codestat.cbp" active="1" />
		<Project filename="cb\plugins\contrib\copystrings\copystrings.cbp" />
		<Project filename="cb\plugins\contrib\devpak_plugin\DevPakPlugin.cbp" />
		<Project filename="cb\plugins\contrib\help_plugin\help-plugin.cbp" />
		<Project filename="cb\plugins\contrib\keybinder\keybinder.cbp" />
		<Project filename="cb\plugins\contrib\profiler\cbprofiler.cbp" />
		<Project filename="cb\plugins\contrib\source_exporter\Exporter.cbp" />
		<Project filename="cb\plugins\contrib\wxSmith\wxSmith.cbp" />
		<Project filename="cb\CodeBlocks.cbp" />
	</Workspace>
</CodeBlocks_workspace_file>

I get 494.2 MB (with code completion) versus 12.6 MB (no code completion).

[attachment deleted by admin]

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 06:16:30 pm

And here are the token counts.

Everything turned on:

Code

[18:12:07.281]: Start parsing project Code Stat
[18:12:07.281]: Concurrent threads for pool set to 1
[18:12:08.125]: Start parsing project copystrings
[18:12:08.125]: Concurrent threads for pool set to 1
[18:12:08.609]: Start parsing project Dev-C++ DevPak updater/installer (wx2.6)
[18:12:08.609]: Concurrent threads for pool set to 1
[18:12:09.765]: Start parsing project Help Plugin (wx2.6)
[18:12:09.765]: Concurrent threads for pool set to 1
[18:12:10.390]: Start parsing project C::B KeyBinder
[18:12:10.390]: Concurrent threads for pool set to 1
[18:12:11.109]: Start parsing project C::B Profiler
[18:12:11.109]: Concurrent threads for pool set to 1
[18:12:11.656]: Start parsing project Exporter
[18:12:11.656]: Concurrent threads for pool set to 1
[18:12:12.437]: Start parsing project wxSmith for wx 2.6
[18:12:12.437]: Concurrent threads for pool set to 1
[18:12:13.421]: Start parsing project Code::Blocks (wx2.6)
[18:12:13.421]: Concurrent threads for pool set to 1
[18:12:46.687]: Done parsing project Help Plugin (wx2.6) (550 total parsed files, 57254 tokens in 27.953 seconds).
[18:12:52.953]: Done parsing project C::B Profiler (514 total parsed files, 53061 tokens in 6.266 seconds).
[18:12:54.843]: Done parsing project Code Stat (506 total parsed files, 52749 tokens in 1.890 seconds).
[18:12:54.843]: Updating class browser...
[18:12:54.859]: Class browser updated.
[18:12:57.140]: Done parsing project Dev-C++ DevPak updater/installer (wx2.6) (536 total parsed files, 53977 tokens in 2.297 seconds).
[18:12:57.218]: Done parsing project Exporter (566 total parsed files, 57653 tokens in 0.78 seconds).
[18:12:58.531]: Done parsing project C::B KeyBinder (524 total parsed files, 55687 tokens in 1.313 seconds).
[18:13:00.031]: Done parsing project copystrings (573 total parsed files, 57413 tokens in 1.500 seconds).
[18:13:10.171]: Done parsing project wxSmith for wx 2.6 (841 total parsed files, 65030 tokens in 10.140 seconds).
[18:13:16.718]: Done parsing project Code::Blocks (wx2.6) (1210 total parsed files, 79395 tokens in 6.547 seconds).

Everything turned off:

Code

[18:15:16.468]: Start parsing project Code Stat
[18:15:16.468]: Concurrent threads for pool set to 1
[18:15:16.781]: Start parsing project copystrings
[18:15:16.781]: Concurrent threads for pool set to 1
[18:15:16.937]: Done parsing project Code Stat (7 total parsed files, 35 tokens in 0.0 seconds).
[18:15:16.937]: Updating class browser...
[18:15:16.937]: Class browser updated.
[18:15:17.078]: Start parsing project Dev-C++ DevPak updater/installer (wx2.6)
[18:15:17.078]: Concurrent threads for pool set to 1
[18:15:17.234]: Done parsing project copystrings (2 total parsed files, 9 tokens in 0.0 seconds).
[18:15:17.515]: Start parsing project Help Plugin (wx2.6)
[18:15:17.515]: Concurrent threads for pool set to 1
[18:15:17.687]: Done parsing project Dev-C++ DevPak updater/installer (wx2.6) (24 total parsed files, 348 tokens in 0.16 seconds).
[18:15:17.828]: Start parsing project C::B KeyBinder
[18:15:17.828]: Concurrent threads for pool set to 1
[18:15:18.062]: Done parsing project Help Plugin (wx2.6) (6 total parsed files, 59 tokens in 0.78 seconds).
[18:15:18.125]: Start parsing project C::B Profiler
[18:15:18.125]: Concurrent threads for pool set to 1
[18:15:18.296]: Done parsing project C::B KeyBinder (6 total parsed files, 396 tokens in 0.15 seconds).
[18:15:18.437]: Start parsing project Exporter
[18:15:18.437]: Concurrent threads for pool set to 1
[18:15:18.671]: Done parsing project C::B Profiler (6 total parsed files, 47 tokens in 0.78 seconds).
[18:15:18.875]: Start parsing project wxSmith for wx 2.6
[18:15:18.875]: Concurrent threads for pool set to 1
[18:15:19.062]: Done parsing project Exporter (21 total parsed files, 713 tokens in 0.78 seconds).
[18:15:19.234]: Start parsing project Code::Blocks (wx2.6)
[18:15:19.234]: Concurrent threads for pool set to 1
[18:15:19.656]: Done parsing project wxSmith for wx 2.6 (196 total parsed files, 3033 tokens in 0.266 seconds).
[18:15:24.312]: Done parsing project Code::Blocks (wx2.6) (547 total parsed files, 15636 tokens in 1.641 seconds).

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 24, 2006, 06:23:11 pm

When you say "Everything turned on" you mean "Parse preprocessor directives too?".

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 06:24:24 pm

Yep. Everything. :)

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 24, 2006, 06:35:07 pm

Quote from: Rick

I think the whole model needs to be redesigned, it's obviously not fit for so many large projects at once... :(

OK assumming preprocessor directives take 5% of the total token count, and that the wxWidgets project gives us around 55,000 tokens... the sum of distinct tokens for all the workspace gives us a total of less than 90,000 tokens. And the parsing time would be less than 20 seconds.

Argh... I need to consult this with Yiannis.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 24, 2006, 06:58:35 pm

Trying again with revision 1833 and everything on using Thomas' workspace.

Quote from: Code::Blocks Debug

[12:46:51.390]: Start parsing project Code::Blocks (wx2.6)
[12:46:51.390]: Concurrent threads for pool set to 2
[12:46:53.656]: Running startup script
[12:50:57.687]: Done parsing project copystrings (372 total parsed files, 45288 tokens in 244.0 seconds).
[12:51:00.234]: Done parsing project Code Stat (381 total parsed files, 45760 tokens in 2.547 seconds).
[12:51:00.234]: Updating class browser...
[12:51:00.265]: Class browser updated.
[12:51:11.421]: Done parsing project C::B KeyBinder (390 total parsed files, 46579 tokens in 11.187 seconds).
[12:51:12.484]: Done parsing project C::B Profiler (386 total parsed files, 45927 tokens in 1.63 seconds).
[12:51:30.593]: Done parsing project Help Plugin (wx2.6) (414 total parsed files, 47788 tokens in 18.109 seconds).
[12:51:31.578]: Done parsing project Exporter (423 total parsed files, 47872 tokens in 0.985 seconds).
[12:51:33.703]: Done parsing project Dev-C++ DevPak updater/installer (wx2.6) (441 total parsed files, 49220 tokens in 2.125 seconds).
[12:52:10.656]: Done parsing project wxSmith for wx 2.6 (644 total parsed files, 52045 tokens in 36.953 seconds).
[12:52:50.703]: Done parsing project Code::Blocks (wx2.6) (1083 total parsed files, 73764 tokens in 40.47 seconds).

It didn't crash this time!

(http://gda.utp.edu.co/~ceniza/CodeBlocks/.images/cbCPU.png)

Take a look at the CPU usage. Can you see what I meant?

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 06:59:12 pm

Quote

And the parsing time would be less than 20 seconds.

Below 20 seconds? :?
18:12:07 -- 18:13:16 = 69 seconds.
69 seconds * 0,95 = 65 seconds (more or less).

This is a fully optimized build, by the way (-DNDEBUG -O3 -fexpensive-optimizations -march=k8 -ffast-math). Debug build takes about twice as long.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 24, 2006, 07:06:45 pm

What I mean is that it would take 20 seconds if we decouple the tokens / files, so no file would be parsed more than once. This would imply that there would be one global parser, instead of one per project. It would also allow us to make a transition to a database if it was necessary.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 07:08:32 pm

Quote

Take a look at the CPU usage. Can you see what I meant?

Yes, and there is something else (besides the hyperthreading thing) to note. You can very clearly see that max_threads = NUM_CPUS is not a good choice.
Look at the regular "zero CPU" intervals, the CPU is stalled 50% of the time (very likely waiting for an I/O operation to finish). So, overall, you use only 25% of your CPU's capacity. It could very well do using two more threads, so one thread runs while another waits for I/O.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 24, 2006, 09:36:05 pm

So you recommend adding 1 or 2 extra threads?

Hmm interesting :)

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 09:47:33 pm

Well, look at the screenshot. The CPU is completely idle 50% of the time (and runs one thread during the other half, when it could run two).

An even better solution might be to have exactly one dedicated I/O thread which does all reads asynchronously, but sequentially and feeds the results to the blocking NUM_CPU+1 worker threads.
Under Linux, this probably won't bring an awful lot, since the elevator mechanism should already make sure that the hard disk is not trashing when two threads try to read concurrently, but I wouldn't be so sure about that under Windows...

If someone can produce a cross-platform (posix + Windows) memory mapping class, then we might leave that work to the OS, too. It would probably be the most efficient way, anyway.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Michael on January 24, 2006, 10:26:35 pm

Just a suggestion. Would be of some interest the use of Win32 pthreads (http://sourceware.org/pthreads-win32/) library in Windows? It was used to develop e.g., ViTooKi (http://vitooki.sourceforge.net/) (a cross-platform C++ multimedia library).

Michael

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 24, 2006, 10:45:08 pm

Quote from: Michael on January 24, 2006, 10:26:35 pm

Just a suggestion. Would be of some interest the use of Win32 pthreads library in Windows?

Win32 pthreads does not offer memory mapping, unluckily. The functionality which it offers is in wxWidgets too, so I don't think we can gain a lot from using it.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Urxae on January 24, 2006, 11:56:10 pm

Quote from: thomas on January 24, 2006, 09:47:33 pm

If someone can produce a cross-platform (posix + Windows) memory mapping class, then we might leave that work to the OS, too. It would probably be the most efficient way, anyway.

Dare I suggest... Boost (http://boost.org) has such a class, in <boost/spirit/iterator/file_iterator.hpp>. It provides an iterator interface to files on disk.
There are basically 3 implementations (the appropriate one is selected using the preprocessor): One for Win32, one for Posix, and one using only the C++ standard library. The first two implement memory mapping, the other one is provided as a fallback.
If you don't want to include all of Boost though (which is understandable) you'll either have to check exactly which files are needed (and/or edit it to reduce dependencies manually) or perhaps just use it as an example for your own implementation.

The Boost license is GPL compatible, IIRC.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Game_Ender on January 25, 2006, 01:21:07 am

The boost license is about a loose as it gets you just need to basically include the copyright from the author and that's about it. Boost has a good thread library too. If you are going to use an external thread library I would use boost, because then you can use lots of the other wonderful stuff boost has.

As for this I have one question, why are we using threads at all if doesn't run more than one at once? I think you mean it only runs one extra thread, counting the main thread if you have one CPU?, or it only one thread total include the main one? The whole point of threads is to run more than the number of processors you have because the CPU is rarely full on, threads of execution spend quite a bit of time waiting. Also I think the threading model needs a thorough rewrite or going over because it dead locks all the time on linux.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 25, 2006, 01:52:16 am

"One extra thread" the answer is. More refinement Code completion needs, indeed. Hmm.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 25, 2006, 03:05:13 am

I wonder if setting CodeCompletion to use only 1 thread would "help". It's like if instead of using 2 threads it was using 1/2 thread.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 25, 2006, 09:01:43 am

Quote from: Game_Ender on January 25, 2006, 01:21:07 am

As for this I have one question, why are we using threads at all if doesn't run more than one at once?

It does not freeze the GUI for 20 seconds.

Quote

I think you mean it only runs one extra thread, counting the main thread if you have one CPU?, or it only one thread total include the main one?

The former is the case (obviously, nothing else makes sense).
You could indeed argue why the "thread pool" is rather a "thread blocker" than a real thread pool, but that is not the cause of the bottlenecks in parsing. Rick initially wrote that code to be simple, and he is certainly aware of the unnecessary thread creation overhead. That, however, is not the thing which causes all the problems which we are facing in code completion. Since we are only creating a dozen or two unnecessary threads, this is manageable, even on such a bad performer like Windows. He might change that one day, but only after the real issues are solved.

Quote

The whole point of threads is to run more than the number of processors you have because the CPU is rarely full on, threads of execution spend quite a bit of time waiting.

Look at Ceniza's screenshot. It has a higher resolution than mine, so the issue is better visible there.
His CPU is 75% idle on the average (if you count in hyperthreading). 50% of the time, it uses one CPU, and 50% of the time, it is completely stalled. Thus, he could as well run two more concurrent threads without any penalites.

Quote

Also I think the threading model needs a thorough rewrite or going over because it dead locks all the time on linux.

Yes, although the deadlocks do not necessarily come from that, the parser does some heavy locking here and there too, so it might as well be that.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 25, 2006, 11:30:42 am

I tried revision 1854 last night which creates 4 threads and I have the same results :(

BTW, those times (working then stalled) are about 3-5 seconds each (haven't timed them though, but they really count in seconds). Doesn't look like waiting for I/O to me.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: thomas on January 25, 2006, 11:36:21 am

Uh... seconds? What on earth is taking seconds? (I was thinking you had the task monitor running at higher speed... :lol:)

Rick, you don't happen to trigger the next parser with a wxTimer event, do you?

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 25, 2006, 11:42:11 am

By checking the process' threads you can see all of them are executed in the working time (I got 36 threads with Thomas' workspace: 9 projects, 4 threads per project). When the CPU is stalled... well... you know... they do nothing.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 25, 2006, 05:50:26 pm

Quote from: thomas on January 25, 2006, 11:36:21 am

Rick, you don't happen to trigger the next parser with a wxTimer event, do you?

Nope... this is how it works: The wxTimer works as a "countdown for start parsing." When a new file is requested to be parsed (from the main thread, that is), it resets the countdown. When all files are requested, the countdown begins and finally the parsing is triggered. This is to avoid problems with adding files recursively taking forever.

But I think the part that measures the time is broken :P I was going to change it, but i was too tired last night.

But don't worry! Everytime we're getting closer to a bug free plugin :)

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 25, 2006, 06:29:23 pm

Rick: could you add the textbox for the concurrent threads again? I'd like to try it with only 1 thread.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 25, 2006, 07:59:07 pm

Um... :oops: I don't know how... I didn't touch that code :P

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 28, 2006, 12:18:13 am

I'm trying the latest changes in CodeCompletion and the results are very similar: 50% working time and about 60-75% CPU usage (when working).

Even better, Code::Blocks just died with a "Reading from 00000000" error.

Revision 1889.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: rickg22 on January 28, 2006, 12:23:52 am

Even better, Code::Blocks just died with a "Reading from 00000000" error.

If you told me the exact address of the fault (a line # would be even better), i'd gladly appreciiate it.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 28, 2006, 01:28:49 am

Sorry Rick but the report isn't good enough to point you somewhere since I always compile with -s -O3 instead of -g.

I'll rebuild it again with -g and try the same test. If I get something I'll tell you.

Title: Re: Codecompletion delays fixed in rev 1826 (update: in rev 1840)
Post by: Ceniza on January 28, 2006, 01:51:30 am

Quote from: gdb

Program received signal SIGSEGV, Segmentation fault.
[Switching to thread 2772.0x4ac]
0x77c17026 in msvcrt!memcpy () from C:\WINDOWS\system32\msvcrt.dll
(gdb) bt
#0 0x77c17026 in msvcrt!memcpy () from C:\WINDOWS\system32\msvcrt.dll
#1 0x10063346 in wxStringBase::ConcatSelf ()
from C:\WINDOWS\system32\wxmsw26u_gcc_custom.dll
#2 0x65ed1cf8 in wxStringBase::ConcatSelf (this=0x14e0ed8c, nLen=239120052,
src=0x1bd2dac) at C:/MinGW/include/wx/string.h:281
#3 0x65ed1d96 in wxStringBase::append (this=0x14e0ed8c, str=@0x651fc78)
at C:/MinGW/include/wx/string.h:400
#4 0x65ed6438 in wxString::append (this=0x14e0ed8c, str=@0x651fc78)
at C:/MinGW/include/wx/string.h:1157
#5 0x65ed66fc in wxString::operator<< (this=0x14e0ed8c, s=@0x651fc78)
at C:/MinGW/include/wx/string.h:889
#6 0x65eab056 in ParserThread::DoParse (this=0x14e0ed20)
at plugins/codecompletion/parser/parserthread.cpp:654
#7 0x65eacc4f in ParserThread::HandleClass (this=0x14e0ed20, isClass=true)
at plugins/codecompletion/parser/parserthread.cpp:1017
#8 0x65eaa51b in ParserThread::DoParse (this=0x14e0ed20)
at plugins/codecompletion/parser/parserthread.cpp:518
#9 0x65ea9c00 in ParserThread::Parse (this=0x14e0ed20)
at plugins/codecompletion/parser/parserthread.cpp:372
#10 0x65ed1695 in ParserThread::Execute (this=0x14e0ed20)
at plugins/codecompletion/parser/parserthread.h:43
#11 0x61965fa9 in PrivateThread::Entry (this=0x2c051c8)
at sdk/cbthreadpool.cpp:86
...

Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: rickg22 on January 22, 2006, 02:50:07 am