Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => Topic started by: rickg22 on July 08, 2005, 07:58:46 pm

Title: Custom Lexers for C::B? (Was: XML file types?)
Post by: rickg22 on July 08, 2005, 07:58:46 pm
Recently i've been glimpsing over the scintilla code, and i've seen that there's html support in scintilla. And I think code folding, too!

So, what needs to be done to add XML / HTML support to the codeblocks editor?

Actually, what needs to be done to add other (custom) filetypes to the editor?
Title: XML file types?
Post by: per_eckerdal on July 08, 2005, 08:22:29 pm
Yes, it would be very neat with Python (for SCons) and PHP support :)

I could help developing support for more file types (not until august tho)
/Per Eckerdal
Title: XML file types?
Post by: mandrav on July 08, 2005, 11:12:43 pm
I plan on writing proper documentation for this feature, but here are some *rough* info:

The files that add syntax highlighting support for specific files are found under sd/resources/lexers. They 're simple XML files named as lexer_*.xml.
Let's take lexer_cpp.xml as an example and disect it.

Code
		<Lexer name="C/C++"
index="3"
filemasks="*.c,*.cpp,*.cc,*.cxx,*.h,*.hpp,*.hh,*.hxx,*.inl">

Pretty much self explanatory, except for the "magic" index number (we 'll come to it in a sec).
The name is the lexer's configuration name. This will appear in the editor's configuration dialog, in the languages drop down box (in colors editing page).
The filemasks is a comma separated list of the extensions that this lexer should be used for. This is case-insensitive.
The index corresponds with the wxSTC_LEX_* constants, found in wx/stc/stc.h. In this example, if you look in wx/stc/stc.h, you 'll see that index 3 matches wxSTC_LEX_CPP. The lexer id for C/C++ syntax highlighting :)
If we were building a lexer configuration for, say, XML (random choice ;) ) we would look up the constant wxSTC_LEX_XML which is defined to be number 5. So index=5. Simple :)

Next follows many <Style> tags defining the different styles:
Code
				<Style name="Default"
index="0"
fg="0,0,0"
bg="255,255,255"
bold="0"
italics="0"
underlined="0"/>

Name is the style's name. It appears in the editor's configuration dialog, in the colors editing page.
fg is the foreground color. Comma separated list of three numbers from 0 to 255. In order: red, green and blue (RGB).
bg is the background color.
bold is "0" for disabled, "1" for enabled. Same goes for italics and underlined.
You don't have to define all of these attributes. It's good to define them all for the "default" style (all lexers have a default style), but only the attributes needed should be defined for the rest of the styles.

The index number in the <Style> tags, comes from a different set of constants defined in wx/stc/stc.h. For each language supported by scintilla, there is a set of styles defined (these are what we 're trying to configure with these files). For example, for C/C++ files (wxSTC_LEX_CPP, remember?) the styles are defined as wxSTC_C_*.
For the "default" style shown above, this would be wxSTC_C_DEFAULT which is defined to be 0. Hence index=0 for "default".
Code
				<Style name="Comment (normal)"
index="1,2"
fg="160,160,160"/>

This is the style definition for normal comments. As you can see you can define a single style for more than one style index, in this case two: 1 and 2 (always comma separated).
1 is for wxSTC_C_COMMENT (the C comment /* */) and 2 is for wxSTC_C_COMMENTLINE (the C++ comment to end of line // ).

I just want to add that there are some special styles defined by Code::Blocks and are available to all lexers:

Index -99: the selected text style
Index -98: the active line style (the line the caret is on)
Index -2 : the breakpoint line style
Index -3 : the debugger active line style (while stepping the debugger)
Index -4 : the compiler warning/error line style

Now on to the keywords.
If the language you 're defining a lexer configuration for, has keywords they should be added in the <Keywords> tag. This tag can contain the following tags:
<Language>, <User> and <Documentation>
Language contains the language keywords. These are usually at index 0.
User is not used right now but might be in the future.
Documentation contains the documentation keywords (if any). If you look at the lexer_cpp.xml file, you 'll see that the documentation keywords defined are those of doxygen.


Whew, that was kind of long! I just hope it makes more sense now.

Yiannis.

PS: do you think it should be wiki'ed ?
Title: XML file types?
Post by: rickg22 on July 08, 2005, 11:21:33 pm
Yes, definitely! :D By the way, you didn't mention XML code folding, which is the feature I asked for in the RFE in question. Any more info on this feature in particular, if it exists? (Like, opening and closing the contents of an xml element/tag or comment)
Title: XML file types?
Post by: mandrav on July 08, 2005, 11:27:13 pm
Quote
By the way, you didn't mention XML code folding

Make a lexer configuration for XML files and see if it does :)
I mean, folding is a scintilla feature and Code::Blocks turns it on. It's just a matter of the lexer supporting it (I guess all lexers do).

Yiannis.
Title: XML file types?
Post by: rickg22 on July 08, 2005, 11:50:32 pm
Okay! :) I'll do it when I get time. Maybe tonight, maybe on monday.
Title: It works!
Post by: rickg22 on July 09, 2005, 01:21:11 am
w00t!  :shock: It works! :D
I added in the configuration (a quick hack, won't commit to CVS) in cbeditor.cpp (function SetEditorStyle() ) :

    m_pControl->SetProperty("fold.html", ConfigManager::Get()->Read("/editor/folding/show_folds", 1) ? "1" : "0");

Note: This is a hack, i won't commit to CVS. A new config variable (and its corresponding dialogue) must be added to C::B. I suggest using "/editor/folding/fold_xml".

But it WORKS! :D Now I can fold the sizeritem's in XRC files!

Now, the problem is...

We need to implement a full set of styles etc. But IMO this feature *MUST* be implemented in 1.0. In fact, I'm adding it to RFE's with priority 9  8)

If anyone wants to cooperate with the lexer xml files, he's welcome to contribute!  8)
Title: XML file types?
Post by: squizzz on July 09, 2005, 02:02:39 am
Quote from: rick22
Any more info on this feature in particular, if it exists? (Like, opening and closing the contents of an xml element/tag or comment)


If SciTE derives all features of Scintilla in a straight way, then support for following languages' syntax could be possible :) (* = support for folding)

Quote from: http://scintilla.sourceforge.net/SciTEDoc.html

    - Ada
    - ANS.1 MIB definition files*
    - APDL
    - Assembler (NASM, MASM)
    - AutoIt*
    - Avenue*
    - Batch files (MS-DOS)
    - Baan*
    - Bash*
    - BlitzBasic*
    - Bullant*
    - C/C++/C#*
    - Clarion*
    - conf (Apache)*
    - CSound*
    - CSS*
    - diff files*
    - E-Script*
    - Eiffel*
    - Erlang*
    - Flagship (Clipper / XBase)*
    - Flash (ActionScript)*
    - Fortran*
    - Forth*
    - Haskell
   - HTML*
    - HTML with embedded JavaScript, VBScript, PHP and ASP*

    - Gui4Cli*
    - IDL - both MSIDL and XPIDL*
    - INI, properties* and similar
    - Java*
    - JavaScript*
    - LISP*
    - LOT*
    - Lout*
    - Lua*
    - Make
    - Matlab*
    - Metapost
    - MMIXAL
    - MSSQL
    - nnCron
    - NSIS*
    - Objective Caml*
    - Octave*
    - Pascal/Delphi*
    - Perl, most of it except for some ambiguous cases*
    - PostScript*
    - POV-Ray*
    - PowerBasic*
    - PureBasic*
    - Python*
    - Rebol*
    - Ruby*
    - Scheme*
    - scriptol*
    - Specman E*
    - Smalltalk
    - SQL and PLSQL
    - TADS3*
    - TeX and LaTeX
    - Tcl/Tk - using the cpp lexer*
    - VB and VBScript*
    - Verilog*
    - VHDL*
    - XML*
    - YAML*
Title: XML file types?
Post by: rickg22 on July 09, 2005, 06:53:15 am
Done! :D

Now you guys can edit XRC files inside codeblocks, with the ease of any xml editor! :D
Title: XML file types?
Post by: darklordsatan on July 09, 2005, 07:21:33 am
To the risk of sounding stupid, whats folding?
Title: XML file types?
Post by: per_eckerdal on July 09, 2005, 08:47:19 am
When coding C++ in Code::Blocks, there is a column with pluses to the left of the code. When clicking them you can "fold" the code. :)
/Per Eckerdal
Title: Re: XML file types?
Post by: RShadow on July 09, 2005, 08:47:42 am
Quote from: rickg22
Recently i've been glimpsing over the scintilla code, and i've seen that there's html support in scintilla. And I think code folding, too!

So, what needs to be done to add XML / HTML support to the codeblocks editor?

Actually, what needs to be done to add other (custom) filetypes to the editor?


Speaking of scintilla.. how is this being integrated into C::B? A quick look shows an old control (the name of which I can't recall).  I would recommend (if this is not already the case) to move to wxScintilla.
Title: XML file types?
Post by: rickg22 on July 11, 2005, 04:30:41 am
RShadow: Actually we don't know if wxScintilla REQUIRES wxSTC to be compiled first. If you can provide us directions to replace wxSTC with wxScintilla, we'd appreciate it.
Title: XML file types?
Post by: RShadow on July 11, 2005, 12:31:37 pm
Quote from: rickg22
RShadow: Actually we don't know if wxScintilla REQUIRES wxSTC to be compiled first. If you can provide us directions to replace wxSTC with wxScintilla, we'd appreciate it.


Hmm.. I don't believe so.. I downloaded and compiled wxScintilla without wxSTC.. but let me check..

Nope.. it does not link against wxSTC anywere.. I think this should be a "feature" added to the HEAD cvs branch.
Title: Re: XML file types?
Post by: thomas on August 17, 2005, 03:47:32 pm
...and here is a lexer for nVidia's cg language.

It is derived from the C/C++ lexer and does not only colour the 87 known keywords, but also the 538 different datatypes, register names,  and semantics.
Additionally, all cg 1.2 standard library functions are coloured, too (unless I forgot some?).

This might be interesting in particular because of http://forums.codeblocks.org/index.php/topic,735.0.html.

Revove the .txt extension -- the forum would not otherwise let me post it :)

[attachment deleted by admin]
Title: Re: XML file types?
Post by: rickg22 on August 18, 2005, 01:50:18 am
Thomas, mind posting the lexer on the SF patch requests? That's what they are for, after all ;-)

Anyway, I'm considering adding a lexer for CLIPPER since that's what we use at the job and i'm dying to use C::B instead of the DOS editor we use there :lol:.

Unfortunately that requires Scintilla 1.64, and wxwidgets' is 1.54 :(.

On the good news, i finally found out how to use the wxScintilla control! :D It's a *nearly exact* copy of STC. That means we only have to compile it (gulp :shock: lol) and link to it. Even the constants are the same!  8)

I also consider that a plus, since we would have to spare the author from having to *make* STC separatedly. It would just compile as part of the Code::Blocks package :)
Title: Re: XML file types?
Post by: thomas on August 18, 2005, 02:00:59 pm
Thomas, mind posting the lexer on the SF patch requests? That's what they are for, after all ;-)
Already did last evening :)  RQ 1262336
Quote
Anyway, I'm considering adding a lexer for CLIPPER since that's what we use at the job and i'm dying to use C::B instead of the DOS editor we use there :lol:.
Lol, I would die if I had to use CLIPPER at all.

Quote
I also consider that a plus [...]
Big plus, yes.