Extending the CPP Lexer

[Malc]

I've looked at the code, and it looks as simple as just adding another line to cppWordLists[] in LexCPP.cxx, but I wanted to check first.  I need support for a 3rd set of Keywords.  Using the existing CPP Lexer, I have found I can highlight comments, preprocessor stuff (index 9), primary keywords (index 0) like bool int float etc, and secondary keywords.  I can't get a third set working though within the bounds of the existing lexer.

The purpose of this is for another c-type language that has many global functions, as well as local functions, and I want them highlighted separately.  The language is LPC, in case someone already has support for it  which would be even better.  I want to switch from VIM to CB

Thank you,
Malc

[MortenMacFly]

Did you try menu "Settings" -> "Editor" -> "Syntax Highlighting" -> Button "Keywords" first? Here you can setup additional user keywords for each (!) lexer. I think this is way better than hacking scintilla (the lexer's code) itself.

[Alpha]

I think the easiest (and probably best) way to support this language would be to copy lexer_cpp.xml to lexer_lcp.xml and modify the file masks and keywords.  However, this would still only give you two color sets for highlighted keywords, and if I am understanding you correctly, you want three, right?

[Malc]

Did you try menu "Settings" -> "Editor" -> "Syntax Highlighting" -> Button "Keywords" first? Here you can setup additional user keywords for each (!) lexer. I think this is way better than hacking scintilla (the lexer's code) itself.

Yes, this would be easier, but still does not allow for a 3rd set.  For example, you can enter 1-9.  But if I use '4', as an example, those words are not highlighted.

I think the easiest (and probably best) way to support this language would be to copy lexer_cpp.xml to lexer_lcp.xml and modify the file masks and keywords.  However, this would still only give you two color sets for highlighted keywords, and if I am understanding you correctly, you want three, right?

Correct.  I started out this way, experimenting with indexes and such as well.  But I cannot get 3 sets of keywords

[Alpha]

In that case, modifying LexCPP.cxx might be your only option.  If you go through with this, you also should talk to the Scintilla people to try to get it in their repository.  (Code::Blocks uses a mostly unmodified copy of Scintilla.)

[Malc]

So if I do this, what would need to be modified?  It looks to me like just the wordlist array.  Anything else?

[Folco]

If you want to add a lexer, you have to modify :
src/sdk/wxscintilla/src/scintilla/src/Catalogue.cxx
src/sdk/wxscintilla/Makefile.am
src/sdk/wxscintilla/src/scintilla/include/SciLexer.h
src/sdk/wxscintilla/src/scintilla/include/Scintilla.iface
src/sdk/wxscintilla/include/wx/wxscintilla.h

In your lexer_language.cxx, you must define the entry points of the keyword list that you want to use (indexes are here !). Example :
    WordList &cpuInstruction = *keywordlists[0];
    WordList &registers      = *keywordlists[1];
    WordList &directive      = *keywordlists[2];
    WordList &extInstruction = *keywordlists[3];
    WordList &alert          = *keywordlists[4];
    WordList &doxygenKeyword = *keywordlists[5];

Then you must export that, in the same source file :
static const char * const a68kWordListDesc[] =
{
    "CPU instructions",
    "Registers",
    "Directives",
    "Extended instructions",
    "Alerts",
    "Doxygen keywords",
    0
};

That is the right way to define new indexes and keyword lists.

[MortenMacFly]

... I would seriously suggest you first contact and discuss any changes with the maintainer of scintilla (http://www.scintilla.org/ScintillaToDo.html). He is known to be very restrictive. And what won't make it into scintilla itself most likely also won't make it into C::B.

[Malc]

So Folco, are those first file modifications needed if you are modifying an existing lexer?  Should just be the extension of the wordlists themselves, correct?

Morten, unless the Scintilla maintainer is willing to add the changes in himself, I'm not really concerned about it.  I can just compile it myself and call it good.


Malc

[Alpha]

There are other modifications.  Follow the variable keywords2 around LexCPP.cxx and do a threaded searches for SCE_C_WORD2 and wxSCI_C_WORD2.  That should provide an example, and the locations for modifications.

[Folco]

Malc -> perhaps you have not to modify all these files. Look into them. I don't remember exactly because I rewrote a lexer from scratch for me, and you just want to modify an existing one.

[Malc]

There are other modifications.  Follow the variable keywords2 around LexCPP.cxx and do a threaded searches for SCE_C_WORD2 and wxSCI_C_WORD2.  That should provide an example, and the locations for modifications.

Malc -> perhaps you have not to modify all these files. Look into them. I don't remember exactly because I rewrote a lexer from scratch for me, and you just want to modify an existing one.

Thank you both!
Malc

[Alpha]

Wait!  No programming of the lexer necessary!  Add:

Code

                <Style name="Global classes and typedefs"
                       index="19"
                       fg="190,0,190"
                       bold="1"/>

to lexer_cpp.xml, then list your words in

Code

                        <Set index="3"
                             value="my list of keywords"/>

or in the set labeled "4" inside the highlighting settings.

[carra]

Thanks Alpha! Works like a charm. Very interesting for highlighting domain vocabulary in custom projects.

[Malc]

Wait!  No programming of the lexer necessary!  Add:

AWESOME!  Thank you Alpha, that works awesomely!  I am at a loss why we need to deal with 3 different numbers (19,3,4) for indexing, but nevertheless, very nice


I don't suppose you know to change something like:

#define SOMEVAR(x) somefunc(x)
to just be
#define SOMEVAR(x) somefunc(x)

It seems to want to highlight the whole line.