[Linux] Slower performance on opening than under wine

[libfab]

Hi,
I searched forums for the issue, yet did not find anything, so here is my point.
I develop both under linux and windows partly with CB, with a fairly rich building tree
(here is my current package: http://dvd-audio.sourceforge.net/dvda-author-09.03-1.tar.bz2)
Under Windows, opening the whole project (CB_project/dvda-author-09.03.workspace) takes about 15-20 seconds.
On the same computer under linux (which is now my preferred OS), it takes about 130 seconds, that's about a factor 7.
Results were obtained using svn 5532 (./configure --with-contrib-plugins=all)
Results are not markedly changed by using 08.02.

This issue I think has already been mentioned. Yet what is striking is that using wine, running the Windows build of 08.02 under linux/wine is actually MUCH FASTER than running any linux build of 08.02 natively, which is bewildering. It's about twice as fast as the linux build opening.

I'm looking for a plausible explanation to this issue, and this is what I came up with by launching under console. The linux build, not the .exe, repetitively issues warnings for each file opening like this (list was cut):
...
Warning: bypassing C::B's auto-detection!                                                                                           
Encoding requested is: Unicode 8 bit (UTF-8) (ID: 41)                                                                               
Final encoding detected: Unicode 8 bit (UTF-8) (ID: 41)                                                                             
project data set for /home/fab/Dev/dvda-author-09.03/libsoxconvert/lpc10/preemp.c                                                   
Warning: bypassing C::B's auto-detection!                                                                                           
Encoding requested is: Unicode 8 bit (UTF-8) (ID: 41)                                                                               
Final encoding detected: Unicode 8 bit (UTF-8) (ID: 41)                                                                             
project data set for /home/fab/Dev/dvda-author-09.03/libsoxconvert/lpc10/prepro.c                                                   
Warning: bypassing C::B's auto-detection!                                                                                           
Encoding requested is: Unicode 8 bit (UTF-8) (ID: 41)                                                                               
Final encoding detected: Unicode 8 bit (UTF-8) (ID: 41)                                                                             
project data set for /home/fab/Dev/dvda-author-09.03/libsoxconvert/lpc10/random.c                                                   
Warning: bypassing C::B's auto-detection!                                                                                           
Encoding requested is: Unicode 8 bit (UTF-8) (ID: 41)                                                                               
Final encoding detected: Unicode 8 bit (UTF-8) (ID: 41)                                                                             
project data set for /home/fab/Dev/dvda-author-09.03/libsoxconvert/lpc10/rcchk.c 
...

So my wild bet is that by deactivating C::B's auto-detection, or by optimizing it somewhat, launching performance whould be greatly improved under linux.

Any developer suggestion about this?

Best, Fab

[Jenna]

It's most likely a problem with the utf-8 conversion of wxWidgets on linux.
It was already discussed in this thread: http://forums.codeblocks.org/index.php/topic,10159.0.html.

If possible, you can try to checkout the wxaui test-branch.
It does not only use wxAui-notebooks instead of wxFlatnotebook, it also has the patches to encoding-detection and utf-8 loading and saving included.

These are the times I need to open your projects:

on debian 64-bit Core2Duo 2x2.0 GHz with about 3.3 GB usable memory (but some apps running including virtual w2k-machine)

C::B (from trunk) svn r5588:  2:56
C::B (wxaui testbranch) svn r5541: 1:10

w2k on virtual kvm-machine with 768 MB memory
C::B (from trunk) svn r5541: 0:32

That means it's still slower than on windows, but much faster than with trunk.

[libfab]

Thanks Jens, brilliant as usual. Sorry for missing your previous thread on this.
I'm not accustomed to codeblocks branches, as I usually build from trunk, so all that I found in the svn tree was branches/wxfnb_to_wxaui/,
which did not build, perhaps an aging wx2.8? Here is the crash:

filemanager.cpp: In function ‘bool WriteWxStringToFile(wxFile&, const wxString&, wxFontEncoding, bool)’:                                                                                                                                     
filemanager.cpp:213: erreur: no matching function for call to ‘wxMBConvUTF7::cWC2MB(const wxChar*, size_t&, size_t*)’                                                                                                                       
/usr/local/include/wx-2.8/wx/strconv.h:89: note: candidats sont: const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*) const                                                                                                                   
/usr/local/include/wx-2.8/wx/strconv.h:106: note:                 const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*, size_t, size_t*) const                                                                                                 
filemanager.cpp:218: erreur: no matching function for call to ‘wxMBConvUTF8::cWC2MB(const wxChar*, size_t&, size_t*)’
/usr/local/include/wx-2.8/wx/strconv.h:89: note: candidats sont: const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*) const
/usr/local/include/wx-2.8/wx/strconv.h:106: note:                 const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*, size_t, size_t*) const
filemanager.cpp:223: erreur: no matching function for call to ‘wxMBConvUTF16BE::cWC2MB(const wxChar*, size_t&, size_t*)’
/usr/local/include/wx-2.8/wx/strconv.h:89: note: candidats sont: const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*) const
/usr/local/include/wx-2.8/wx/strconv.h:106: note:                 const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*, size_t, size_t*) const
filemanager.cpp:228: erreur: no matching function for call to ‘wxMBConvUTF16LE::cWC2MB(const wxChar*, size_t&, size_t*)’
/usr/local/include/wx-2.8/wx/strconv.h:89: note: candidats sont: const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*) const
/usr/local/include/wx-2.8/wx/strconv.h:106: note:                 const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*, size_t, size_t*) const
filemanager.cpp:233: erreur: no matching function for call to ‘wxMBConvUTF32BE::cWC2MB(const wxChar*, size_t&, size_t*)’
/usr/local/include/wx-2.8/wx/strconv.h:89: note: candidats sont: const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*) const
/usr/local/include/wx-2.8/wx/strconv.h:106: note:                 const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*, size_t, size_t*) const
filemanager.cpp:238: erreur: no matching function for call to ‘wxMBConvUTF32LE::cWC2MB(const wxChar*, size_t&, size_t*)’
/usr/local/include/wx-2.8/wx/strconv.h:89: note: candidats sont: const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*) const
/usr/local/include/wx-2.8/wx/strconv.h:106: note:                 const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*, size_t, size_t*) const
filemanager.cpp:258: erreur: no matching function for call to ‘wxCSConv::cWC2MB(const wxChar*, size_t&, size_t*)’
/usr/local/include/wx-2.8/wx/strconv.h:89: note: candidats sont: const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*) const
/usr/local/include/wx-2.8/wx/strconv.h:106: note:                 const wxCharBuffer wxMBConv::cWC2MB(const wchar_t*, size_t, size_t*) const


Unless I missed the right branch? Do you have a build anywhere downloadable "at home"?
Best, Fab
PS: I expect your branch to be below 1min for opening all projects, which would be fair enough. Actually I could not see how wine overhead + codeblocks.exe could beat it at 1min10

[Jenna]

Unless I missed the right branch? Do you have a build anywhere downloadable "at home"?

No, it's the right branch.
I forgot to post the link. sorry.

The errror looks alittle bit, as if you use an ansi-build of wxWidgets.

Can you post the command that leads to the error ?

EDIT:
no downloadable version of C::B with auibook at the moment, maybe I find the time during the day.

[libfab]

Hi Jens,
simply ./bootstrap; ./configure; make
wx2.8 from links  to wx debian repository linked to on your site.
trunk builds OK with it under all configs tested.
Fab

[Jenna]

What output do you get from

Code

wx-config --list

EDIT

I just compile a 32 version with the same environment, I use for my nightlies, or do you need a 64-bit version ?

I will upload the deb's to a seperate folder on my server, so you can download what you need.
If you have a self-compiled version running on your system, you might need to run "make uninstall" before installing the deb's.

[libfab]

~$ wx-config --list

    Default config is gtk2-ansi-release-2.8

  Default config will be used for output


after I installed all libwx.*2.8.*ansi packages with apt-get.

[Jenna]

You should use unicode-build of wxWidgets.

C::B is normally build against unicode wxWidgets on all platforms except for win98 (what is not officially supported as far as I know).

[Jenna]

I just uploaded an aui-test-build (32- and 64-bit) to my server: http://apt.jenslody.de/aui-test/ .
As usual build for debian and against wxWidgets 2.8.9 from apt.wxwidgets.org.

Due to a change in the debian package-layout, you need to run the following command for a full install (including contrib-plugins, but without C::B and wxSmith development-files):

sudo dpkg -i codeblocks-contrib_8.02svn5543_amd64.deb codeblocks_8.02svn5543_amd64.deb codeblocks-contrib-common_8.02svn5543_all.deb codeblocks-common_8.02svn5543_all.deb libcodeblocks0_8.02svn5543_amd64.deb libwxsmithlib0_8.02svn5543_amd64.deb wxsmith-dev_8.02svn5543_amd64.deb wxsmith-contrib-dev_8.02svn5543_amd64.deb wxsmith-headers_8.02svn5543_all.deb wxsmith-contrib-headers_8.02svn5543_all.deb

For 32-bit systems all occurrences of "amd64" have to be replaced with "i386" of course.

[libfab]

Hi Jens,
thanks a lot, I appreciate.
Your new debian build is markedly faster, it opens the workspace at 88s, yet this remains slightly above wine codeblocks.exe, at 76s, so I suspect there still is room for progress.
I did not find a unicode build of wx under either kubuntu repos or even wx apt repo. D'you have a link somewhere on how to build unicode?
I'll be trying a g++-4.4 (recently released as stable) unicode build in a wild attempt to beat wine...
System was:

CPUs Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz
L2 cache 6144 KB
RAM 2GB

Cheers,
Fab
PS: afterthought: monitoring with gkrellm of the 4 CPUs shows non-optimal usage of CPU power.

[Jenna]

I did not find a unicode build of wx under either kubuntu repos or even wx apt repo.

All linux-builds, that not especially have "ansi" in the name should be unicode-builds.

[libfab]

Jens,
I now have: wx-config --libs

-L/usr/local/lib -pthread   -lwx_gtk2_richtext-2.8 -lwx_gtk2_aui-2.8 -lwx_gtk2_xrc-2.8 -lwx_gtk2_qa-2.8 -lwx_gtk2_html-2.8 -lwx_gtk2_adv-2.8 -lwx_gtk2_core-2.8 -lwx_base_xml-2.8 -lwx_base_net-2.8 -lwx_base-2.8

after uninstalling all debian wx packages and building wx from source.
Same building problem with your testbranch, however.
Attached is config.log
Best, Fab


[attachment deleted by admin]

[Jenna]

I have the following wxwidgets-packages installed (all from apt.wxwidgets.org):
libwxbase2.8-0 , libwxbase2.8-dev , libwxgtk2.8-0 , libwxgtk2.8-dev , python-wxversion (not needed for C::B) , wx2.8-headers , wx2.8-i18n , wx-common

What you need at least to build C::B are the "-dev"-packages and their dependencies.

Your wx-config still gives you the libs and includes for an ansi-release. Either you did not build the unicode-version or it's not your default.

wx-config --list shows all available builds.

If you want to build a unicode-realease you have to run ./configure with the --enable-unicode parameter.

[libfab]

Hi jens,
thanks for your patience, I had missed --enable-unicode on doing ./configure --help, due to eye fatigue...
So I finally made it. I built everything from source OK, and the running times are ...

./configure build : 90 s
./configure CXXFLAGS=-O3 LDFLAGS=-s  build : 86 s

with g++-4.4.

Results are not markedly improved by building from source, yet it's far better than trunk.
I wish you every success with this branch.
Cheers, Fab

[Jenna]

If you find the time, you can try the following change:

in src/sdk/editorcolourset.cpp, function void EditorColourSet::Apply(HighlightLanguage lang, cbStyledTextCtrl* control), line should be about 435 change

Code

    if (defaults)
    {
        for (int i = 0; i < wxSCI_STYLE_MAX; ++i)
        {
            if (i < 33 || i > 39)
                DoApplyStyle(control, i, defaults);
        }
    }

to

Code

     if (defaults)
    {
        int countStyles = 1 << control->GetStyleBits();
        for (int i = 0; i < countStyles; ++i)
        {
            if (i < 33 || i > 39)
                DoApplyStyle(control, i, defaults);
        }
    }

It's not really tested yet, so some more people should have a look at it.

With this patch I need a little more than 40 seconds to load on linux and less than 30 seconds on my virtual w2k-machine.

It should not break anything, but as written, some more eyes should doublecheck it.
After some sleeping, I hope I find the time tomorrow to create a real patch and to phrase a good Subject to start a new topic, that as many users and devs as possible can check it.

[libfab]

Hi Jens,

just brilliant. I tested it this morning and I can confirm your improved loading: 46 s, so that's now well below wine codeblocks.exe, and only about 15 s more than windows (which I find tolerable).
Congratulations, you cut loading time by a factor 3 comparing to trunk!
Fab

[Jenna]

I changed the patch a little bit to be sure (hopefully) not to break any lexer.
I started a new thread to discuss the patch: http://forums.codeblocks.org/index.php/topic,10552.msg72360.html .