Author Topic: Source Formatter  (Read 23170 times)

Offline Ceniza

  • Developer
  • Lives here!
  • *****
  • Posts: 1442
    • CenizaSOFT
Source Formatter
« on: January 14, 2006, 08:37:24 am »
I have ~good and bad news.

The ~good news: I've started another Source Formatter plugin only for C/C++. So far I have a hardcoded lexer, a console test program and a test plugin. It just tests the lexer, so it takes the whole file and modifies it with a (TOKEN, "lexeme") pair from the first to the last token found.

The bad news: I have barely 1 week of vacations left and most of my projects have never been finished (I read it over and over again and it sounds weird, but deep inside it makes sense). If I come with a working version of this plugin, you'll surely flood my e-mail, thru SF, with bug reports and feature requests, plus some posts, surely, in this thread, instead of giving me support, right? Tell me I'm wrong :P

I've had some ideas about the parser (it's a simpler one 'cause it won't need to build an AST or anything like that). I need to write them on paper and check viability.

I'll commit it to my personal SVN or CVS in a few hours, I hope. If it gets to a working state I'll tell you how to get access or beg a developer to put it in contrib.

Offline Michael

  • Lives here!
  • ****
  • Posts: 1608
Re: Source Formatter
« Reply #1 on: January 14, 2006, 05:45:02 pm »
The bad news: I have barely 1 week of vacations left and most of my projects have never been finished (I read it over and over again and it sounds weird, but deep inside it makes sense). If I come with a working version of this plugin, you'll surely flood my e-mail, thru SF, with bug reports and feature requests, plus some posts, surely, in this thread, instead of giving me support, right? Tell me I'm wrong :P

Well, may be wrong is too much :), because I have quite always seen in this forum some people available for help. In my case, I could provide some help, but unfortunately not this month (because of my project and some exams :(). In February, I would be more free. So, if you need something, just ask :D.

Concerning the parser, did you take into consideration rickg22's parser, eranif's parser (with SQLite) and "my" small recursive parser?

Best wishes,
Michael

Offline Ceniza

  • Developer
  • Lives here!
  • *****
  • Posts: 1442
    • CenizaSOFT
Re: Source Formatter
« Reply #2 on: January 14, 2006, 09:32:21 pm »
Quote from: Michael
Concerning the parser, did you take into consideration rickg22's parser, eranif's parser (with SQLite) and "my" small recursive parser?

Hmmm, no? :P

I'll try to get something for the next week. I'll just continue thinking about the implementation and customization issues.

And thank's for offering yourself to help. I'll have you in mind for the next month :)

Offline Michael

  • Lives here!
  • ****
  • Posts: 1608
Re: Source Formatter
« Reply #3 on: January 14, 2006, 09:43:33 pm »
Quote from: Michael
Concerning the parser, did you take into consideration rickg22's parser, eranif's parser (with SQLite) and "my" small recursive parser?

Hmmm, no? :P

I'll try to get something for the next week. I'll just continue thinking about the implementation and customization issues.

I think that it could be useful to look at them and eventually ask questions. It could be helpful to avoid problems, pitfalls and so on.

And thank's for offering yourself to help. I'll have you in mind for the next month :)

Ok. Just ask and if I can, I will gladly help (and not just asking C::B devs for new features, bug fixes and so on :D).

Michael

Offline takeshi miya

  • Lives here!
  • ****
  • Posts: 1487
Re: Source Formatter
« Reply #4 on: January 14, 2006, 10:28:19 pm »
I for one, think it will be way more easy to do things using a centralized parser like the one of eranif, or codestore, in the SDK.

Not only source formatting comes to mind, but a lot of other things (refactoring, code test coverage, documentation generated on the fly, class modeling, etc).

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: Source Formatter
« Reply #5 on: January 15, 2006, 04:10:00 am »
Ceniza:

Please take a look into my "Copy strings" plugin, it's a very basic state-machine parser, and it recognizes C and C++ comments. The good thing about state machines is their simplicity. No need for long comparisons and particular cases, just a bunch of switches. I was amazed at its effectiveness :)

Offline Ceniza

  • Developer
  • Lives here!
  • *****
  • Posts: 1442
    • CenizaSOFT
Re: Source Formatter
« Reply #6 on: January 15, 2006, 07:52:28 am »
rickg22: I took a look at it already, remember? You told me to do so :P

A state-machine is indeed very nice, simple and effective, but I just decided to hard code it instead :)

Anyway, the lexer is almost done (6 more tokens come to my mind, which can be added in just a few minutes (5 of those are keywords which are really simple to add) :D). Haven't had the time to do so though, maybe tomorrow I'll add those (yes, haven't had those few minutes yet).

What Takeshi Miya says is true. How many C/C++ parsers/lexers will be using Code::Blocks (even though some come from external libraries)?: CodeCompletion, Scintilla, AStyle, ¿AngelScript?, my Source Formatter, maybe a Refactoring one, ...

Anyway, those will be there; if anyone needs them, use them. At least my lexer is built as a library that relies only in C++.

One again, I'll tell you when I put the files in SVN or CVS (right, I'ven't done it yet).

Offline Ceniza

  • Developer
  • Lives here!
  • *****
  • Posts: 1442
    • CenizaSOFT
Re: Source Formatter
« Reply #7 on: January 15, 2006, 09:18:26 am »
[edit]
I finally got to enable anonymous access to SVN, so here it is (removed the CVS info here):

Server: cenizasoft.cjb.net
Repository: /svnroot/SourceFormatter

Command line example:
svn co svn://cenizasoft.cjb.net/svnroot/SourceFormatter

Of course you can also use TortoiseSVN.

Right now the plugin just tests the lexer.
[/edit]
« Last Edit: January 15, 2006, 09:42:35 am by Ceniza »

Offline Michael

  • Lives here!
  • ****
  • Posts: 1608
Re: Source Formatter
« Reply #8 on: January 15, 2006, 05:53:17 pm »
A state-machine is indeed very nice, simple and effective, but I just decided to hard code it instead :)

Sorry, but why you want to hard code it :?? IMHO, it is not a very good idea, because of e.g., extensibility, scalability.

Michael

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Source Formatter
« Reply #9 on: January 15, 2006, 06:07:44 pm »
Sorry, but why you want to hard code it :?? IMHO, it is not a very good idea, because of e.g., extensibility, scalability.
Hey, hardcoding is great :)
(For me, that is, since I only use C++)

I'd love to see a better source code formatter for C++. And since hardcoding the lexer saves Ceniza precious time, this means he'll be done with it sooner (and I'll be happy sooner :))
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

Offline Michael

  • Lives here!
  • ****
  • Posts: 1608
Re: Source Formatter
« Reply #10 on: January 15, 2006, 06:28:18 pm »
Sorry, but why you want to hard code it :?? IMHO, it is not a very good idea, because of e.g., extensibility, scalability.
Hey, hardcoding is great :)
(For me, that is, since I only use C++)

Hard coding is useful, I agree :), but not when the code would change oft and/or for extensibility/scalability. In such a case, it is better to avoid hard coding even if the implementation would take some more time. But later, you will save time when modifiying/updating the code.

I'd love to see a better source code formatter for C++. And since hardcoding the lexer saves Ceniza precious time, this means he'll be done with it sooner (and I'll be happy sooner :))

Well, I cannot really disagree with you :D. May be, later it would be possible to re-write the hard-coded part.

Michael

Offline rickg22

  • Lives here!
  • ****
  • Posts: 2283
Re: Source Formatter
« Reply #11 on: January 16, 2006, 07:20:23 pm »
My state machine in the copystrings plugin is hardcoded, my point was the design without special cases   (in a state machine, all cases are special cases - they're handled equally).

If what Ceniza meant by "hardcoding" was to use a series of if's and design of the lexer/parser "by hand", then that's a no-no.

Offline Ceniza

  • Developer
  • Lives here!
  • *****
  • Posts: 1442
    • CenizaSOFT
Re: Source Formatter
« Reply #12 on: January 16, 2006, 07:30:49 pm »
Quote from: thomas
I'd love to see a better source code formatter for C++. And since hardcoding the lexer saves Ceniza precious time, this means he'll be done with it sooner (and I'll be happy sooner :))

Oh, my precious... time.... I would like to get it done soon too :)

There's something I prefer to ask you guys, and girls if any: how to customize it? I mean, how detailed would you like it to be? Any interface idea?

So far I've thought of a series of dialogs, each one to customize specific things (but I also think it would be overkill, even to configure), like the way a "if (expr) stmts", "if (expr) { stmts }", the ones with "else" and "else if" should be formatted. You could then "say", for each thing: the if goes in a new line with current indentation, space + LPAREN, expr, RPAREN, LBRACE in new line with current indentation, stmts in new line with extra indentation, RBRACE in new line un-indenting, ...

That way it would always look like this:

Code: [Select]
if (expr)
{
  stmts
}

Or just use simple options similar to those in AStyle?

Ideas? I still need to add a few more things to the lexer and have a global idea of the parser, including "customization details", before I can start coding it.

rickg22: I think it's a no-no then, sorry :P

Offline Michael

  • Lives here!
  • ****
  • Posts: 1608
Re: Source Formatter
« Reply #13 on: January 16, 2006, 08:03:43 pm »
Or just use simple options similar to those in AStyle?

I do not know what the other devs/users think about, but for my AStyle was quite good :D. I just set the template I wanted to use (ANSI :lol:) and voilà...

The custom option of AStyle is a good idea. So, users could use their own style.

What it would be also good, would be the possibility to define and register new styles. In this was users could define more than one custom style.

IMHO, the code should also thought extensible and scalable to allow integration of e.g., another parser.

Michael

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Source Formatter
« Reply #14 on: January 16, 2006, 08:26:23 pm »
The AStyle way is not so bad actually, it is simple and it works. But there are just a few things which AStyled does not do the way I like it (for example, it compresses white space in comments and around '=', and always puts {} in a new line, so if you spend a lot of time laying out things properly, the code formatter destroys it later). Also you can only configure everything from hand, or nothing, which is annoying (for example, I like the way ANSI style looks, but I don't like spaces for indentation).

Why don't you make a simple "choose one out of n" interface (list box, for example). The actual configuration is read in from a config file, and you just provide 3-4 such configs. If somebody wants something different from ANSI, Gnu, or K&R, then he can write his own config and is not limited to a stiff GUI. Most people will use one of the default sets, so they have a 1-click configuration.
You could use tinyXML for the parsing, so you save an awful lot of time on designing a GUI (designing a GUI takes 90% of your time, but only 10% of the people use it 1% of their time...).
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."