Author Topic: Unicode conversion (attention all devs)  (Read 69128 times)

Offline tiwag

  • Developer
  • Lives here!
  • *****
  • Posts: 1196
  • sailing away ...
    • tiwag.cb
Re: UNICODE: ATTENTION ALL DEVS
« Reply #30 on: August 05, 2005, 10:02:10 pm »
One question - if wxStrring is used as a param inside printf-like function should it be converted using c_str() or mb_str() ?

Example:
Code
tmpkey.Printf(_T("%s/editor/keywords/%d"), key.c_str(), i);

hi byo - how did you proceed with this ?

Offline takeshi miya

  • Lives here!
  • ****
  • Posts: 1487
Re: UNICODE: ATTENTION ALL DEVS
« Reply #31 on: August 05, 2005, 10:09:18 pm »
Do not use wxChar when is not a text character, because a wxChar in unicode is an int of 16 bits (not 8 bits):

Example for text:
wxChar im_a_character = _T('f');

Example for not text (not character):
char im_a_byte = 254;
but perhaps better would be to use:
byte im_a_byte = 254;
so it's clear that it's a byte and not a character.

Offline tiwag

  • Developer
  • Lives here!
  • *****
  • Posts: 1196
  • sailing away ...
    • tiwag.cb
Re: UNICODE: ATTENTION ALL DEVS
« Reply #32 on: August 05, 2005, 10:32:03 pm »
what to do here ?
Code
            // indent code accordingly
            wxString code = it->second;
            code.Replace("\n", '\n' + lineIndent);

from the docs:
Quote
wxString::Replace
size_t Replace(const char* szOld, const char* szNew, bool replaceAll = true)

Replace first (or all) occurrences of substring with another one.

replaceAll: global replace (default), or only the first occurrence.

Returns the number of replacements made.


wxString::Replace()  seems to be not fit for unicode ? or what ?

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: UNICODE: ATTENTION ALL DEVS
« Reply #33 on: August 05, 2005, 10:56:39 pm »
Stan: Take the sources from the CVS snapshot. There were 2 or 3 bugs fixed after RC1-1.

Tiwag: If the input is a const char*, use "normal strings". If the input is a wxChar or wxString, use the _T("macros").

So in this case, just use normal strings.
« Last Edit: August 05, 2005, 11:03:08 pm by rickg22 »

Offline darklordsatan

  • Almost regular
  • **
  • Posts: 132
    • http://eviltrend.sf.net
Re: UNICODE: ATTENTION ALL DEVS
« Reply #34 on: August 05, 2005, 11:03:47 pm »
Stan: Take the sources from the CVS snapshot. There were 2 or 3 bugs fixed after RC1-1.

So, the branch is VERSION_1 not HEAD, right?

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: UNICODE: ATTENTION ALL DEVS
« Reply #35 on: August 05, 2005, 11:05:15 pm »
Uh oh... I just noticed from the docs.

wxString& operator <<(const wxString& str)
wxString& operator <<(const char* psz)
wxString& operator <<(char ch)

Apparently the << 's can be used with either normal strings or wxstrings, but not with wxChar's. UGH i have to change one or two sources back... :-/

Offline byo

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 837
Re: UNICODE: ATTENTION ALL DEVS
« Reply #36 on: August 05, 2005, 11:08:34 pm »
what to do here ?
Code
            // indent code accordingly
            wxString code = it->second;
            code.Replace("\n", '\n' + lineIndent);

from the docs:
Quote
wxString::Replace
size_t Replace(const char* szOld, const char* szNew, bool replaceAll = true)

Replace first (or all) occurrences of substring with another one.

replaceAll: global replace (default), or only the first occurrence.

Returns the number of replacements made.


wxString::Replace()  seems to be not fit for unicode ? or what ?


wxWidgets documentation about wxString class seems to be outdated - in header files there's such declaration:

Code
  size_t Replace(const wxChar *szOld,
                 const wxChar *szNew,
                 bool bReplaceAll = true);

I would change the code to:

Code
            // indent code accordingly
            wxString code = it->second;
            code.Replace(_T("\n"), _T('\n') + lineIndent);

Offline byo

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 837
Re: UNICODE: ATTENTION ALL DEVS
« Reply #37 on: August 05, 2005, 11:12:32 pm »
Uh oh... I just noticed from the docs.

wxString& operator <<(const wxString& str)
wxString& operator <<(const char* psz)
wxString& operator <<(char ch)

Apparently the << 's can be used with either normal strings or wxstrings, but not with wxChar's. UGH i have to change one or two sources back... :-/

Same as in my previous post - documentation outdated. In header files for both wx 2.4 and 2.6 is:

Code
      // string += C string
  wxString& operator<<(const wxChar *psz)
    { append(psz); return *this; }
      // string += char

Offline Urxae

  • Regular
  • ***
  • Posts: 376
Re: UNICODE: ATTENTION ALL DEVS
« Reply #38 on: August 05, 2005, 11:15:44 pm »
Uh oh... I just noticed from the docs.

wxString& operator <<(const wxString& str)
wxString& operator <<(const char* psz)
wxString& operator <<(char ch)

Apparently the << 's can be used with either normal strings or wxstrings, but not with wxChar's. UGH i have to change one or two sources back... :-/

I wouldn't if I were you, cause that would be wrong :P. The docs are generated by automated tools that among other things resolve typedefs such as wxChar to whatever they're typedeffed to. I've noticed on several occasions that they seem to be generated in non-unicode mode. If you actually look in <wx/string.h> (2.6.1) you'll see this:
Code
      // string += C string
  wxString& operator<<(const wxChar *psz)
    { append(psz); return *this; }
      // string += char
  wxString& operator<<(wxChar ch) { append(1, ch); return *this; }

Damn. Beaten. Twice even :shock:.

Offline tiwag

  • Developer
  • Lives here!
  • *****
  • Posts: 1196
  • sailing away ...
    • tiwag.cb
Re: UNICODE: ATTENTION ALL DEVS
« Reply #39 on: August 05, 2005, 11:18:36 pm »
what to do here ?
Code
            // indent code accordingly
            wxString code = it->second;
            code.Replace("\n", '\n' + lineIndent);

from the docs:
Quote
wxString::Replace
size_t Replace(const char* szOld, const char* szNew, bool replaceAll = true)

Replace first (or all) occurrences of substring with another one.

replaceAll: global replace (default), or only the first occurrence.

Returns the number of replacements made.


wxString::Replace()  seems to be not fit for unicode ? or what ?


wxWidgets documentation about wxString class seems to be outdated - in header files there's such declaration:

Code
  size_t Replace(const wxChar *szOld,
                 const wxChar *szNew,
                 bool bReplaceAll = true);

I would change the code to:

Code
            // indent code accordingly
            wxString code = it->second;
            code.Replace(_T("\n"), _T('\n') + lineIndent);


thats what i've already done  ... thanks byo ... i really got unsure for a moment ...  :shock:

cbeditor.cpp was really a brainteaser ... puuh

btw
  i really don't like the fact, that we can't rely on the wx.chm docs
  that's a lot of additional work to study the header's all the time
  somewhat boring ...  but what can we do else ?

Offline byo

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 837
Re: UNICODE: ATTENTION ALL DEVS
« Reply #40 on: August 05, 2005, 11:26:00 pm »
There's one thing important when working with concatenated strings:

_() is macro which calls one of wxWidget's internal function so concatenating should look like this:

Code
_("string 1" "string2" ... )

_T() macro simply adds 'L' before string given as a param (in Unicode of course, in normal mode it do nothing with the string) so concatenation should be:

Code
_T("string1") _T("string2") ...

BTW. Answer to my previous post about Printf-like functions is - use c_str() (in examples in wxwidgets.org there are used different arguments for unicode and non-unicode versions where formating string was both "%s")


Offline byo

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 837
Re: UNICODE: ATTENTION ALL DEVS
« Reply #41 on: August 05, 2005, 11:29:57 pm »
btw
  i really don't like the fact, that we can't rely on the wx.chm docs
  that's a lot of additional work to study the header's all the time
  somewhat boring ...  but what can we do else ?

I try to compile changed sources in both Unicode and NonUnicode versions of wxWidgets (and after that in wx2.4 and wx2.6 to make sure it will work everywhere). Unfortunately all project can't be compiled in Unicode but separate files can :)

Offline Ceniza

  • Developer
  • Lives here!
  • *****
  • Posts: 1441
    • CenizaSOFT
Re: UNICODE: ATTENTION ALL DEVS
« Reply #42 on: August 05, 2005, 11:32:51 pm »
Ok, I got the AStyle plugin to compile with wxWidgets UNICODE, but not to link ('cause it needs libcodeblocks.a which I didn't compile, and I think the reasons are obvious).

Anyway, it wasn't that bad. There was only an evil line and this was my workaround:

Code
#ifdef wxUSE_UNICODE
        formattedText << wxString(formatter.nextLine().c_str(), wxConvLocal);
#else
        formattedText << formatter.nextLine().c_str();
#endif

I couldn't find any other way to get it work. If you've got a working trick which looks better let me know, otherwise that could work in the meanwhile if someone else need it.

rickg22: I'll send it to your e-mail when a decision be made, ok?

[edit]
Hmmm, I should add that formatter.nextLine() returns std::string :)
[/edit]
« Last Edit: August 05, 2005, 11:35:03 pm by Ceniza »

Offline byo

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 837
Re: UNICODE: ATTENTION ALL DEVS
« Reply #43 on: August 05, 2005, 11:34:39 pm »
I will take group 15 (previous groups done, sent to rickg22)

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: UNICODE: ATTENTION ALL DEVS
« Reply #44 on: August 05, 2005, 11:36:51 pm »

thats what i've already done  ... thanks byo ... i really got unsure for a moment ...  :shock:


Yeah! :shock: whew... glad i've done nothing wrong.

ANYWAY GUYS... yes, i checked the includes. Apparently we have nothing to worry about doing "too much conversion" regarding wxString functions and operators. And if you have doubts, go to the header files!