Author Topic: SkipToOneOfChars function issue, it don't handle macro expansion  (Read 8173 times)

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Tokenizer::SkipToOneOfChars() has some issue, it don't involve the macro expansion, to fully solve this problem, the skip should only happens in the ParserThread's level, not Tokenizer's level. (Or maybe another function like: SkipToOneOfTokens() which internally call DoGetToken())

The test case: (WX's event macro)
Code
// an entry from a static event table (simplified version)
struct  wxEventTableEntry
{
public:
    const int& m_eventType;
};

#define BEGIN_EVENT_TABLE(a,b)                         wxBEGIN_EVENT_TABLE(a,b)
#define END_EVENT_TABLE()                              wxEND_EVENT_TABLE()
#define wxEND_EVENT_TABLE() \
        wxDECLARE_EVENT_TABLE_TERMINATOR() };

       
#define wxDECLARE_EVENT_TABLE_ENTRY(type, winid, idLast, fn, obj) \
    wxEventTableEntry(type, winid, idLast, wxNewEventTableFunctor(type, fn), obj)

#define wxDECLARE_EVENT_TABLE_TERMINATOR() \
    wxEventTableEntry(wxEVT_NULL, 0, 0, 0, 0)
   
// #define wxBEGIN_EVENT_TABLE(theClass, baseClass) \
    // const wxEventTable theClass::sm_eventTable = \
        // { &baseClass::sm_eventTable, &theClass::sm_eventTableEntries[0] }; \
    // const wxEventTable *theClass::GetEventTable() const \
        // { return &theClass::sm_eventTable; } \
    // wxEventHashTable theClass::sm_eventHashTable(theClass::sm_eventTable); \
    // wxEventHashTable &theClass::GetEventHashTable() const \
        // { return theClass::sm_eventHashTable; } \
    // const wxEventTableEntry theClass::sm_eventTableEntries[] = { \
 
// a simplified version of wxBEGIN_EVENT_TABLE
#define wxBEGIN_EVENT_TABLE(theClass, baseClass) \
     const wxEventTableEntry theClass::sm_eventTableEntries[] = { \
   
BEGIN_EVENT_TABLE(wx3_hangFrame,wxFrame)
    //(*EventTable(wx3_hangFrame)
    //*)
END_EVENT_TABLE()

int aaa;

//aaa //aaa

Look at the line:
Code
const wxEventTableEntry theClass::sm_eventTableEntries[] = { 
When our Tokenizer see a "=", it will try to find the ending of assignment, see below:
Code

    // skip the following = or ?
    if (m_State & tsSkipEqual)
    {
        if (c == _T('=') && NextChar() != _T('=')) //only skip after single equal sign, not double equals sign
        {
            if (!SkipToOneOfChars(_T(",;}"), true, true, false))
                return false;
        }
    }
So, the macro usage has not chance to expand when we see "END_EVENT_TABLE".

   
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.