Code::Blocks Forums
Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => CodeCompletion redesign => Topic started by: ollydbg on September 24, 2010, 11:19:10 am
-
As we know, Quex lexer generate some code directed lexer which is much faster then the "table driving lexer" like flex. see:
Lexical analysis - Wikipedia, the free encyclopedia (http://en.wikipedia.org/wiki/Lexical_analysis#Lexical_analyzer_generators)
Quex - A Fast Universal Lexical Analyzer Generator (http://quex.sourceforge.net/)
and the performance compared with flex
SourceForge.net: Lexical Analyzer Generator Quex: Topic: Performance question about Quex (https://sourceforge.net/projects/quex/forums/forum/574343/topic/3791987) post 5
And I just create a lexer grammar here
SourceForge.net: Lexical Analyzer Generator Quex: Modify: 3074664 - a c++ quex lexer code (https://sourceforge.net/tracker/?func=detail&atid=846114&aid=3074664&group_id=168259)
By using a lexer generator, we can remove a lot of hand-craft codes in our currently Tokenizer class, and I hope it will let our CC much faster.
I have a simple demo test for Parserthread using the "TokenID and text infromation" from this lexer, This way, we can avoid a lot of wxString comparing statement in Parserthread's code, instead, we used TokenID comparison, which is only int type comparison or a table driven jump can be used.
but the demo test is quite simple, I hope I can improved it in the future.
-
but the demo test is quite simple, I hope I can improved it in the future.
The dummy demo test project can be found here (with a cbp project, only tested under windows):
http://code.google.com/p/quexparser/
SVN:
http://code.google.com/p/quexparser/source/checkout
A lot of work need to be done. (a lot of function in Parserthread class are stub functions :D).
Any comments are welcome!!!
-
Well done! :D