Author Topic: Custom Lexers for C::B? (Was: XML file types?)  (Read 15919 times)

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Custom Lexers for C::B? (Was: XML file types?)
« on: July 08, 2005, 07:58:46 pm »
Recently i've been glimpsing over the scintilla code, and i've seen that there's html support in scintilla. And I think code folding, too!

So, what needs to be done to add XML / HTML support to the codeblocks editor?

Actually, what needs to be done to add other (custom) filetypes to the editor?
« Last Edit: August 18, 2005, 05:38:57 pm by rickg22 »

Offline per_eckerdal

  • Multiple posting newcomer
  • *
  • Posts: 49
XML file types?
« Reply #1 on: July 08, 2005, 08:22:29 pm »
Yes, it would be very neat with Python (for SCons) and PHP support :)

I could help developing support for more file types (not until august tho)
/Per Eckerdal

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4291
    • Code::Blocks IDE
XML file types?
« Reply #2 on: July 08, 2005, 11:12:43 pm »
I plan on writing proper documentation for this feature, but here are some *rough* info:

The files that add syntax highlighting support for specific files are found under sd/resources/lexers. They 're simple XML files named as lexer_*.xml.
Let's take lexer_cpp.xml as an example and disect it.

Code: [Select]
<Lexer name="C/C++"
index="3"
filemasks="*.c,*.cpp,*.cc,*.cxx,*.h,*.hpp,*.hh,*.hxx,*.inl">

Pretty much self explanatory, except for the "magic" index number (we 'll come to it in a sec).
The name is the lexer's configuration name. This will appear in the editor's configuration dialog, in the languages drop down box (in colors editing page).
The filemasks is a comma separated list of the extensions that this lexer should be used for. This is case-insensitive.
The index corresponds with the wxSTC_LEX_* constants, found in wx/stc/stc.h. In this example, if you look in wx/stc/stc.h, you 'll see that index 3 matches wxSTC_LEX_CPP. The lexer id for C/C++ syntax highlighting :)
If we were building a lexer configuration for, say, XML (random choice ;) ) we would look up the constant wxSTC_LEX_XML which is defined to be number 5. So index=5. Simple :)

Next follows many <Style> tags defining the different styles:
Code: [Select]
<Style name="Default"
index="0"
fg="0,0,0"
bg="255,255,255"
bold="0"
italics="0"
underlined="0"/>

Name is the style's name. It appears in the editor's configuration dialog, in the colors editing page.
fg is the foreground color. Comma separated list of three numbers from 0 to 255. In order: red, green and blue (RGB).
bg is the background color.
bold is "0" for disabled, "1" for enabled. Same goes for italics and underlined.
You don't have to define all of these attributes. It's good to define them all for the "default" style (all lexers have a default style), but only the attributes needed should be defined for the rest of the styles.

The index number in the <Style> tags, comes from a different set of constants defined in wx/stc/stc.h. For each language supported by scintilla, there is a set of styles defined (these are what we 're trying to configure with these files). For example, for C/C++ files (wxSTC_LEX_CPP, remember?) the styles are defined as wxSTC_C_*.
For the "default" style shown above, this would be wxSTC_C_DEFAULT which is defined to be 0. Hence index=0 for "default".
Code: [Select]
<Style name="Comment (normal)"
index="1,2"
fg="160,160,160"/>

This is the style definition for normal comments. As you can see you can define a single style for more than one style index, in this case two: 1 and 2 (always comma separated).
1 is for wxSTC_C_COMMENT (the C comment /* */) and 2 is for wxSTC_C_COMMENTLINE (the C++ comment to end of line // ).

I just want to add that there are some special styles defined by Code::Blocks and are available to all lexers:

Index -99: the selected text style
Index -98: the active line style (the line the caret is on)
Index -2 : the breakpoint line style
Index -3 : the debugger active line style (while stepping the debugger)
Index -4 : the compiler warning/error line style

Now on to the keywords.
If the language you 're defining a lexer configuration for, has keywords they should be added in the <Keywords> tag. This tag can contain the following tags:
<Language>, <User> and <Documentation>
Language contains the language keywords. These are usually at index 0.
User is not used right now but might be in the future.
Documentation contains the documentation keywords (if any). If you look at the lexer_cpp.xml file, you 'll see that the documentation keywords defined are those of doxygen.


Whew, that was kind of long! I just hope it makes more sense now.

Yiannis.

PS: do you think it should be wiki'ed ?
Be patient!
This bug will be fixed soon...

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
XML file types?
« Reply #3 on: July 08, 2005, 11:21:33 pm »
Yes, definitely! :D By the way, you didn't mention XML code folding, which is the feature I asked for in the RFE in question. Any more info on this feature in particular, if it exists? (Like, opening and closing the contents of an xml element/tag or comment)

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4291
    • Code::Blocks IDE
XML file types?
« Reply #4 on: July 08, 2005, 11:27:13 pm »
Quote
By the way, you didn't mention XML code folding

Make a lexer configuration for XML files and see if it does :)
I mean, folding is a scintilla feature and Code::Blocks turns it on. It's just a matter of the lexer supporting it (I guess all lexers do).

Yiannis.
Be patient!
This bug will be fixed soon...

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
XML file types?
« Reply #5 on: July 08, 2005, 11:50:32 pm »
Okay! :) I'll do it when I get time. Maybe tonight, maybe on monday.

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
It works!
« Reply #6 on: July 09, 2005, 01:21:11 am »
w00t!  :shock: It works! :D
I added in the configuration (a quick hack, won't commit to CVS) in cbeditor.cpp (function SetEditorStyle() ) :

    m_pControl->SetProperty("fold.html", ConfigManager::Get()->Read("/editor/folding/show_folds", 1) ? "1" : "0");

Note: This is a hack, i won't commit to CVS. A new config variable (and its corresponding dialogue) must be added to C::B. I suggest using "/editor/folding/fold_xml".

But it WORKS! :D Now I can fold the sizeritem's in XRC files!

Now, the problem is...

We need to implement a full set of styles etc. But IMO this feature *MUST* be implemented in 1.0. In fact, I'm adding it to RFE's with priority 9  8)

If anyone wants to cooperate with the lexer xml files, he's welcome to contribute!  8)

Offline squizzz

  • Almost regular
  • **
  • Posts: 132
XML file types?
« Reply #7 on: July 09, 2005, 02:02:39 am »
Quote from: rick22
Any more info on this feature in particular, if it exists? (Like, opening and closing the contents of an xml element/tag or comment)


If SciTE derives all features of Scintilla in a straight way, then support for following languages' syntax could be possible :) (* = support for folding)


    - Ada
    - ANS.1 MIB definition files*
    - APDL
    - Assembler (NASM, MASM)
    - AutoIt*
    - Avenue*
    - Batch files (MS-DOS)
    - Baan*
    - Bash*
    - BlitzBasic*
    - Bullant*
    - C/C++/C#*
    - Clarion*
    - conf (Apache)*
    - CSound*
    - CSS*
    - diff files*
    - E-Script*
    - Eiffel*
    - Erlang*
    - Flagship (Clipper / XBase)*
    - Flash (ActionScript)*
    - Fortran*
    - Forth*
    - Haskell
   - HTML*
    - HTML with embedded JavaScript, VBScript, PHP and ASP*

    - Gui4Cli*
    - IDL - both MSIDL and XPIDL*
    - INI, properties* and similar
    - Java*
    - JavaScript*
    - LISP*
    - LOT*
    - Lout*
    - Lua*
    - Make
    - Matlab*
    - Metapost
    - MMIXAL
    - MSSQL
    - nnCron
    - NSIS*
    - Objective Caml*
    - Octave*
    - Pascal/Delphi*
    - Perl, most of it except for some ambiguous cases*
    - PostScript*
    - POV-Ray*
    - PowerBasic*
    - PureBasic*
    - Python*
    - Rebol*
    - Ruby*
    - Scheme*
    - scriptol*
    - Specman E*
    - Smalltalk
    - SQL and PLSQL
    - TADS3*
    - TeX and LaTeX
    - Tcl/Tk - using the cpp lexer*
    - VB and VBScript*
    - Verilog*
    - VHDL*
    - XML*
    - YAML*
this space is for rent

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
XML file types?
« Reply #8 on: July 09, 2005, 06:53:15 am »
Done! :D

Now you guys can edit XRC files inside codeblocks, with the ease of any xml editor! :D

Offline darklordsatan

  • Almost regular
  • **
  • Posts: 132
    • http://eviltrend.sf.net
XML file types?
« Reply #9 on: July 09, 2005, 07:21:33 am »
To the risk of sounding stupid, whats folding?

Offline per_eckerdal

  • Multiple posting newcomer
  • *
  • Posts: 49
XML file types?
« Reply #10 on: July 09, 2005, 08:47:19 am »
When coding C++ in Code::Blocks, there is a column with pluses to the left of the code. When clicking them you can "fold" the code. :)
/Per Eckerdal

Offline RShadow

  • Multiple posting newcomer
  • *
  • Posts: 34
    • http://www.code-universe.com
Re: XML file types?
« Reply #11 on: July 09, 2005, 08:47:42 am »
Quote from: rickg22
Recently i've been glimpsing over the scintilla code, and i've seen that there's html support in scintilla. And I think code folding, too!

So, what needs to be done to add XML / HTML support to the codeblocks editor?

Actually, what needs to be done to add other (custom) filetypes to the editor?


Speaking of scintilla.. how is this being integrated into C::B? A quick look shows an old control (the name of which I can't recall).  I would recommend (if this is not already the case) to move to wxScintilla.
OS: Gentoo Linux (Kernel 2.6.11 (2.6.12 is evil )   x64_x86)
WM: KDE 3.4.1
Compiler: GCC 3.4 (but of course) GLIBC: NPTL wxWidgets: 2.6.1

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
XML file types?
« Reply #12 on: July 11, 2005, 04:30:41 am »
RShadow: Actually we don't know if wxScintilla REQUIRES wxSTC to be compiled first. If you can provide us directions to replace wxSTC with wxScintilla, we'd appreciate it.

Offline RShadow

  • Multiple posting newcomer
  • *
  • Posts: 34
    • http://www.code-universe.com
XML file types?
« Reply #13 on: July 11, 2005, 12:31:37 pm »
Quote from: rickg22
RShadow: Actually we don't know if wxScintilla REQUIRES wxSTC to be compiled first. If you can provide us directions to replace wxSTC with wxScintilla, we'd appreciate it.


Hmm.. I don't believe so.. I downloaded and compiled wxScintilla without wxSTC.. but let me check..

Nope.. it does not link against wxSTC anywere.. I think this should be a "feature" added to the HEAD cvs branch.
OS: Gentoo Linux (Kernel 2.6.11 (2.6.12 is evil )   x64_x86)
WM: KDE 3.4.1
Compiler: GCC 3.4 (but of course) GLIBC: NPTL wxWidgets: 2.6.1

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: XML file types?
« Reply #14 on: August 17, 2005, 03:47:32 pm »
...and here is a lexer for nVidia's cg language.

It is derived from the C/C++ lexer and does not only colour the 87 known keywords, but also the 538 different datatypes, register names,  and semantics.
Additionally, all cg 1.2 standard library functions are coloured, too (unless I forgot some?).

This might be interesting in particular because of http://forums.codeblocks.org/index.php/topic,735.0.html.

Revove the .txt extension -- the forum would not otherwise let me post it :)

[attachment deleted by admin]
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."