Author Topic: wxIsalpha issue in tokenizer  (Read 4099 times)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5280
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
wxIsalpha issue in tokenizer
« on: March 29, 2012, 03:51:47 am »
Code: [Select]
    wxChar a = _T('中');
    if(wxIsalpha(a))
    {
        ....
    }
The if clause will run. We have such code:
Code: [Select]
wxString Tokenizer::DoGetToken()
{
    int start = m_TokenIndex;
    bool needReplace = false;

    wxString str;
    wxChar   c = CurrentChar();

    if (c == '_' || wxIsalpha(c))
    {
        // keywords, identifiers, etc.

        // operator== is cheaper than wxIsalnum, also MoveToNextChar already includes IsEOF
        while (    ( (c == '_') || (wxIsalnum(c)) )
               &&  MoveToNextChar() )
            c = CurrentChar(); // repeat

        if (IsEOF())
            return wxEmptyString;

        needReplace = true;
        str = m_Buffer.Mid(start, m_TokenIndex - start);
    }

I'm not sure the Chinese character entered in code will cause some problem.

Like: Re: character bug
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline MortenMacFly

  • Administrator
  • Lives here!
  • *****
  • Posts: 9606
Re: wxIsalpha issue in tokenizer
« Reply #1 on: March 29, 2012, 06:55:27 am »
I'm not sure the Chinese character entered in code will cause some problem.
Well I believe we had that just recently - IMHO the compiler will compile code that uses non-ASCII characters for variable / function names. No the assumption is correct. Its not nice and actually ugly and evil - but its not an error.
Compiler logging: Settings->Compiler & Debugger->tab "Other"->Compiler logging="Full command line"
C::B Manual: http://www.codeblocks.org/docs/main_codeblocks_en.html
C::B FAQ: http://wiki.codeblocks.org/index.php?title=FAQ