Author Topic: Major CVS codebase changes  (Read 20608 times)

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Major CVS codebase changes
« on: November 17, 2005, 08:06:13 pm »
Hi all,

Codebase significantly updated.
A little while after 1.0RC2 was released, me and Thomas started working on a separate branch to implement some radical new features (keep reading - we 'll get to them shortly ;)). This was done so that C::B developers wouldn't be disturbed by the every-so-often update of base C::B classes and be forced to rebuild everything so often.
Finally, we managed to pull this off. Here are the major changes:

ConfigManager - completely rewritten from scratch by Thomas.
Configurations are now saved in xml format. There is one config file per personality, so logically separate information is physically separated, too. You can create copies of these files to create new personalities.
wxMSW users: This means you can now delete the registry key used for C::B configuration.
Unfortunately, there is no code to convert your old settings to the new format...

The mere fact that configuration is saved in xml format is not to be understood as an invitation for deliberate editing your configurations by hand. You can of course try, but it is entirely possible that you get unexpected results!

Apart from the actual configuration, the configuration manager offers functions to retrieve "standard paths" such as the installation directory or the data directory. There is a function to locate a Code::Blocks related file in a consistent way, too. For compatibility reasons, you should always use these functions rather than hard-code a path like "share/codeblocks/plugins".

Code::Blocks will load the personality given on the commandline, or the default personality if either none is specified or the specified personality cannot be found or is invalid. If a file named default.conf is found in the same directory as the main executable, this file will be used rather than the one from the config directory. This enables custom installations on read-only media.

You can specify a http:// URL as personality. In this case, Code::Blocks will download the specified URL and parse it as a configuration file. If this fails, the default config file is used (see above). Saving configurations over http:// is not supported.

It is no longer possible to get a pointer to the ConfigManager singleton via ConfigManager::Get(). In fact, ConfigManager no longer is a singleton. Instances of ConfigManager are constructed by a builder class according to the provided namespace.

The correct way to get an instance of ConfigManager is:
Code
Manager::Get()->GetConfigmanager(name_space)
where name_space is a string constant that identifies the namespace which you want to use. Here's an example:
Code
// old way
ConfigManager::Get()->Read("/editor/tab_size", 4);
ConfigManager::Get()->Write("/editor/tab_size", 4);

// new way
Manager::Get()->GetConfigmanager("editor")->ReadInt("/tab_size", 4);
Manager::Get()->GetConfigmanager("editor")->Write("/tab_size", 4);
You are free to do whatever you want within your namespace, all other namespaces will be unaffected by your actions. If your namespace starts with "volatile:" then you get a ConfigManager instance that supports everything as usual but will not save any data to disk. A volatile namespace can be seen pretty much like a static global variable (or a collection of these), except that the data is accessible using the ConfigManager API.

In most cases, it is NOT NECESSARY to have constructs like
  wxString oldPath = cm->GetPath();
  cm->SetPath("somewhere");
  cm->Read("something");
  cm->SetPath(oldPath);
any more, as every namespace has its own proper state. Given the general concept of namespaces, a lot less path mangling needs to be done in most cases, too.

In addition to writing standard primitive types such as int, double, and bool as well as wxString and wxColour, you can implement simple flags using Set(), UnSet(), and Exists() in much the same way as lock files or #defines are typically used.

Complex objects (wxArrayStrings, string maps and sets, serializable objects and maps of serializable objects) can be written to and read from configuration conveniently with one API call.

There are no functions to iterate through subkeys any more, instead you can enumerate all subpaths or subkeys using one convenient function now (if you really need).

Scripting support
The AngelScript script library was added for scripting C::B.
The scripts are written in a C-style syntax so it should be really easy to pick up and start writing scripts :)
Many script bindings have already been added in the SDK with more to come. It's not clear right now what exact parts of C::B will be exposed to scripting, so the scripting API will be changing until a relevant discussion reaches a conclusion.
Current bindings include wxString and wxArrayString from wxWidgets useful types. C::B's exposed types include ProjectManager, ConfigManager, cbProject, ProjectFile, ProjectBuildTarget, cbEditor and EditorManager.
But, as already said, it has not yet been decided which of those stay, which go and which others will be added...
The first step has been taken though ;)

Debugger plugin
This plugin has been redesigned, almost completely. Prepare yourself for what you 're gonna read in the next paragraphs :)

Added "Attach to process" and "Detach" debugger commands.
The format (dec, hex, etc) used for each debugger watched variable can now be specified.
Improved watches support with dialogs to add/edit watches.
Dialogs were also added to edit breakpoints.

On the subject of breakpoints, if you set a breakpoint and right-click on its red-dot in the editor's margin, you can click "Edit breakpoint". What you can set there is:
  • Enable/disable that breakpoint :).
  • Set the ignore-count. This tells the debugger that it should be ignored for X passes before breaking (useful in loops) :).
  • Make it conditional by setting a condition, e.g. x>8 (this would make the breakpoint hit only if x was greater than 8 ) :).

But these wouldn't be possible without the present redesign.
A base DebuggerDriver (debuggerdriver.h) class has been created and GDB_driver derives from it. A base DebuggerCommand (debugger_defs.h) class has also been added, which just sends the passed command to the driver and (optionally) displays its output in the debug log. Every debugger input/output goes to the debugger debug-log.
Already implemented GDB commands can be found in gdb_commands.h. Each command is a small class deriving from DebuggerCommand and, by convenience, their names start with GdbCmd_ (for GDB_driver).

Now for the most important part (for many users): I have already implemented (rudimentary) support for cdb.exe/ntsd.exe. This means C::B can now debug MSVC generated programs :D.
As far as I know, C::B is the very first open source IDE with this very feature ;)
What I mean by "rudimentary support" is it can start the debugging process, set/unset breakpoints, break on set breakpoints and stop the debugging process. It still lacks support for watches, backtrace, disassembly and tooltip evaluation. But all that is needed is to write the relevant commands (driver in cdb_driver.h and commands in cdb_commands.h) :).
But don't try debugging MSVC programs just now. I haven't yet added a way to recognize which debugger to use ;).

After such a rewrite, testing is badly needed. It works fine even for debugging C::B itself but certainly some bugs have creeped in. Those should be identified and fixed. With the new design, it should be easier for people to understand what's going on and help with those bugs. But things are already looking good :)

As a last note, GDB breakpoints are now explicitely pending meaning that you can set a breakpoint on a DLL and it will be set when that DLL is loaded. Also means you can just put breakpoints anywhere in C::B source code and press "Debug->Start" (yes, I renamed it). No more wizard skills are needed to debug C::B :)

Finally, debugging is now much more responsive (aka faster).

Updated tinyXML to version 2.4.2
Better unicode handling and faster string parsing.

wxKeyBinder added in the project
You can find its configuration in "Settings->Keyboard shortcuts". Looks like it's working OK, although I have identified a couple of bugs with it. Its configuration uses the file <config folder>/keys.conf, so if something goes wrong you can always delete the file ;)
By the way, if anyone has a list of menu shortcuts for MSVisualC++ please share them so we can add a second keybinding profile for people migrating over to C::B ;)

Minor yet useful changes
  • Added context menu when right-clicking on editor tabs, ala Firefox (wxMSW only).
  • Close page under mouse when middle-clicking on the tab, ala Firefox (wxMSW only).
  • Control-rightclick in editor, pops-up context menu with web-search options (Google, MSDN) for the keyword under the mouse.
  • Editor's context menu un-cluttered (close/save/etc - exist in tab's context menu)
  • Multifile search is aware of hits in the same line now.
  • Improved time needed for big projects (like C::B) from pressing "Build" to actually start building.
  • Fixed "no handler for wxToolBarAddOn" xrc-loading problem with wx2.6.2/unicode.
  • Moved some functions scattered around in globals.cpp.
  • Fixed loading/saving wxDockit layout in unicode mode.
  • Fixed "auto-complete" text box in editor options to be highlighted always using C/C++
  • Memory-block allocator is used for parser's Token and project's ProjectFile classes (allocates large blocks of memory to avoid memory fragmentation).
  • ~25% speed-up of project loading
  • Show message if trying to open a project file from history which doesn't exist anymore.
  • Fixed bug in cbProject when a template was trying to add a file in a different target.
  • Added attribute in templates to ignore the default compiler (for templates that provide different settings for each compiler)
  • Added projectPath and projectName fields in the new project dialog.
  • Added fix for DigitalMars (wrong static lib linker and added stlport include path). Patch by Lieven de Cock.
  • Don't show "Rename workspace" when right clicking the workspace, if no workspace is open.
  • Fixed automatic filename conversion when changing target type in project options.
  • Added support for Intel compiler (patch by yop).
  • Added global cbSaveToFile() which uses a temporary file before overwriting the target file.
  • Added a few base functions in EditorBase to handle common markers (bookmarks, breakpoints, etc).
  • Added EditorBase::GotoLine(). Moves to specified line and tries to center it vertically on-screen.
  • Added separate history for recent projects.
  • Added network proxy support.
  • Created separate context menu for editor when right-clicking on the margin.
And many other minor changes (quite a few bugs have been fixed).
All the changes were merged just a little while ago to HEAD. So, now you know why you haven't seen any cvs activity by me the last few weeks :)
« Last Edit: November 17, 2005, 08:08:29 pm by mandrav »
Be patient!
This bug will be fixed soon...

Offline killerbot

  • Administrator
  • Lives here!
  • *****
  • Posts: 5490
Re: Major CVS codebase changes
« Reply #1 on: November 17, 2005, 10:24:19 pm »
holy moly, this is GREAT  :shock: :D

When will this be bundled in a not cvs build (RC3 ?) ?

takeshimiya

  • Guest
Re: Major CVS codebase changes
« Reply #2 on: November 17, 2005, 10:28:19 pm »
OMG! Holly S**t! :o

Those are major changes for sure, I'm amazed!

Here are my first impressions:

ConfigManager
Amazing! But I'm not sure if I like breaking interface compatibility with wxConfig.

It is necesary to write
Code
Manager::Get()->GetConfigmanager("editor")->ReadInt("/tab_size", 4);
instead of this?
Code
Manager::Get()->GetConfigManager("editor")->Read("/tab_size", 4);


On where to store the configuration files and data, following the standard rules (like Mozilla apps) is a good idea:
* On Windows XP/2000
    %AppData%\CodeBlocks\
* On Windows 9x/Me (if password protection is disabled)
    C:\WINDOWS\Application Data\CodeBlocks\

* On Windows 9x/Me (if password protection is enabled)
    C:\WINDOWS\Profiles\%USER%\Application Data\CodeBlocks\

* On Windows NT 4.x
    C:\WINNT\Profiles\%USER%\Application Data\CodeBlocks\

* On Linux
    ~/.codeblocks/
* On Mac OS X
    ~/Library/CodeBlocks/
    ~/Library/Application Support/CodeBlocks/

%AppData% is a environmental variable which normally points to
C:\Documents and Settings\%USER%\Application Data on Windows 2000/XP.


About the XML file itself, following a based standard on the tags of the file, like the ones used in GConf would be better.
IMHO Gnome's GConf XML format is one of the best, you can assign schemas to the configuration files, with descriptions on each element, different languages, etc.

A posibility, like GConf does, is to put each major setting (like Editor, Compiler, To-Do, etc) in a separate physical XML file. That way, if you go to another installation of C::B and only wants to import a specific setting (ie. Editor settings) you'll only have to copy that file!

All of that is pretty standard now within the GNOME apps, and is working great :)


Scripting support
Amazing!!

Perhaps you'll want to include wxScript support as well:
It's a set of abstract classes to add a script-interpreter support to your wxWidgets applications/libraries. The implementation of these interfaces for the Lua, Python, UnderC and CINT interpreters (these two are C/C++ intepreters) are provided.


Debugger plugin
 :o!!


wxKeyBinder added in the project


One of my major feature requests was this! :D

I thought some ideas on how to improve even more here. PLEASE, give me your thoughts on that.


Minor yet useful changes
    * Added context menu when right-clicking on editor tabs, ala Firefox (wxMSW only).
           :)
    * Close page under mouse when middle-clicking on the tab, ala Firefox (wxMSW only).
           :)
    * Editor's context menu un-cluttered (close/save/etc - exist in tab's context menu)
           :oops: Great! Now it's un-cluttered, but I'd still would like to see the Cut/Copy/Paste right there instead of a sub-menu.
    * ~25% speed-up of project loading
           :)
    * Added separate history for recent projects.
           My suggestion is to put at the bottom of the lists the "Clear history".
    * Added network proxy support.
           I'd suggest moving the dialog to Settings->Environment.
    * Created separate context menu for editor when right-clicking on the margin.
           :)

I'm amazed with all the changes being made!
I'm pretty sure Code::Blocks together with wxSmith are the best -and most promising- IDE/RADs out there!

Thanks to all devs, keep the good job! :D

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: Major CVS codebase changes
« Reply #3 on: November 17, 2005, 11:09:54 pm »
holy moly, this is GREAT  :shock: :D

When will this be bundled in a not cvs build (RC3 ?) ?

It will need some heavy testing first ;)
But expect it to appear rather soon as a non-official CVS binary (there are a couple of people providing them).

Quote from: Takeshi Miya
ConfigManager
Amazing! But I'm not sure if I like breaking interface compatibility with wxConfig.

To put it simply, we had to.

The reason we have ReadInt(), ReadBool(), Read(), etc. will be better explained by Thomas (he wrote the new ConfigManager). Just a hint though: you can assign to the same key an int, a string, a bool, an array, some base64 encoded binary data (*with* crc), etc ;)

Quote from: Takeshi Miya
On where to store the configuration files and data, following the standard rules (like Mozilla apps) is a good idea:

Again, Thomas will be able to answer this but my tests show:
XP - "Documents And Settings\<username>\Application Data\codeblocks"
Linux - "~/.codeblocks"

Quote from: Takeshi Miya
I'm amazed with all the changes being made!
I'm pretty sure Code::Blocks together with wxSmith are the best -and most promising- IDE/RADs out there!

Well, just today I looked at wxSmith again and I was really amazed too!
Be patient!
This bug will be fixed soon...

Offline hd

  • Multiple posting newcomer
  • *
  • Posts: 45
    • http://www.dynaset.org/dogusanh
Re: Major CVS codebase changes
« Reply #4 on: November 17, 2005, 11:54:48 pm »
Hi Yiannis,

(Tons of improvements again!)

Just curious, did you consider Lua for scripting?


ps. Mentioned changes didn't show up with "cvs update -d". Did I use it wrong again?

--
Regards,
Hakki Dogusan

therion

  • Guest
Re: Major CVS codebase changes
« Reply #5 on: November 18, 2005, 12:01:14 am »
ps. Mentioned changes didn't show up with "cvs update -d". Did I use it wrong again?

me neither :-/

therion

  • Guest
Re: Major CVS codebase changes
« Reply #6 on: November 18, 2005, 12:15:07 am »
found it: its under "head_merged_to_yt" tag... ill compile it and upload the the binaries asap.

great work mandrav, congratulations :-)

takeshimiya

  • Guest
Re: Major CVS codebase changes
« Reply #7 on: November 18, 2005, 12:16:29 am »
Yiannis, can you give me your thoughs on this?


Just curious, did you consider Lua for scripting?
That's why I suggest wxScript which permits multi-scripting languages, providing interfaces to Lua, Python, UnderC and CINT interpreters (these two are C/C++ intepreters).


ps. Mentioned changes didn't show up with "cvs update -d". Did I use it wrong again?
It's because it's not in the HEAD branch, but in another one (I'm not sure what's the difference between "yt" and "head_merged_to_yt").

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: Major CVS codebase changes
« Reply #8 on: November 18, 2005, 12:36:22 am »
ps. Mentioned changes didn't show up with "cvs update -d". Did I use it wrong again?

me neither :-/

It's in the HEAD branch since the announcement was made, but it takes a while for anonymous CVS to catch up, usually 4-6 hours. Be patient ;)
« Last Edit: November 18, 2005, 12:47:32 am by mandrav »
Be patient!
This bug will be fixed soon...

Offline mandrav

  • Project Leader
  • Administrator
  • Lives here!
  • *****
  • Posts: 4315
    • Code::Blocks IDE
Re: Major CVS codebase changes
« Reply #9 on: November 18, 2005, 12:47:09 am »
I forgot to mention that in case you want to build HEAD as soon as anonymous CVS catches up, only the codeblocks-NewBuild.cbp is up to date. It 's using global user vars, a feature introduced right after 1.0rc2 was released.
This means you must have a recent CVS binary to build this.
Be patient!
This bug will be fixed soon...

takeshimiya

  • Guest
Re: Major CVS codebase changes
« Reply #10 on: November 18, 2005, 01:15:48 am »
I forgot to mention that in case you want to build HEAD as soon as anonymous CVS catches up, only the codeblocks-NewBuild.cbp is up to date. It 's using global user vars, a feature introduced right after 1.0rc2 was released.
This means you must have a recent CVS binary to build this.
Funnily, that also means that you need HEAD from yesterday to build HEAD from today. Any other version will not work until the other projects gets updated :lol:

therion

  • Guest
Re: Major CVS codebase changes
« Reply #11 on: November 18, 2005, 01:24:03 am »
Ive just uploaded the files. They were update from head_merged_to_yt tag, but everything looks fine, should be some kind of temp tag, dont know :-)
BTW The KeyBinder is great! Really a great feature!
Ill test the new debug plugin and send a new build from HEAD revision (the real one) tomorrow (quite late here in Brazil, hehehe)

Enjoy the new build. http://paginas.terra.com.br/informatica/mauricio/codeblocks/

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Major CVS codebase changes
« Reply #12 on: November 18, 2005, 01:47:04 am »
Quote
I'm not sure if I like breaking interface compatibility with wxConfig.
It is necesary to write [...]
Yes it is, unluckily. Apart from that, being compatible with wxConfig was never intended (the API misses *many* things of wxConfig and includes *many* things that wxConfig could only dream of).
By using namespaces, we already break compatibility (and all existing code), and when we store maps of serializable objects in the configuration, wxConfig has passed out of sight long ago.
Passing objects as parameters gave me a lot of headaches, the compiler would not easily accept polymorph functions with default arguments of different non-builtin types (such as wxString, wxArrayString and wxColour) either by reference or as pointers. Very often, it would convert an argument to void* and then call the wxString version with a void* argument (which is an awful mess!). Naming the functions differently solved the problem at once. Maybe there is a better workaround, but this is actually not so bad. It works, and it is very explicit.
If you don't like the T ReadT(name) functions, you can still use the Read(name, *T) versions, these are wxConfig compatible and are a few clock cycles faster too (since the other functions use them internally).

Quote
On where to store the configuration files and data, ...
Every decent operating system understands what ~/.codeblocks means, so that is what is used on all platforms except Windows.
Under Windows, SHGetFolderPath is used with CSIDL_APPDATA. Whatever it is, this is what Windows thinks is the right place. Hardcoding anything differently is an extremely bad idea since it is a user-configurable setting. For example on my PC, the location is D:\system\appdata. The way it is, it should work well everywhere.
On the same subject, please note that ConfigManager also offers functions to get the paths to the data folder, the application folder, or the plugins folder, and even a function to locate a file in a defined way. You should always use these functions and never hardcode a path again. For example, if the Code::Blocks devs decide to move the plugin folder to a different location one day, the respective function will be changed, and you don't need to bother. It just works.

Quote
About the XML file itself, following a based standard on the tags of the file
Oh I knew this was going to come... What can I say but these three things:
1. The config does not validate against a DTD, but it is perfectly legal xml (that's a standard too :))
2. The way tags are generated is neither a random choice, nor is it because I don't know any better. The file organisation has been thoroughly planned. The present layout allows O(1) access to child elements as opposed to O(N) in the "standard" schemes which scan through attributes. It also allows to have a key with the same name as a path unambiguously, and it allows several differently typed values tied to one key. Lastly, compound objects are stored in the most efficient and compatible manner.
3. Forget about xml. Use the API to write and read values. The mere fact that the configuration uses xml as a backend does not mean anything. It could as well be Chinese. You don't need to care, you don't want to care, you should not care.

Quote
A posibility, like GConf does, is to put each major setting (like Editor, Compiler, To-Do, etc) in a separate physical XML file
It is not like I did not think about that. And indeed, we do that for personalities. The reason for this design choice is that personalities are a logical unit, and copying namespaces between personalities can easily be offered by a plugin (this is in planning). However, the processing overhead for opening 20 or 30 files at startup is prohibitively high. The initial cost to open a file and parse it is very noticeable. Accessing data stored in a node once the document has been parsed is almost zero (supposed you have laid out your data correctly).

Quote
All of that is pretty standard now within the GNOME apps, and is working great :)
When I log onto GNOME, it takes about 30 seconds to start up on my AMD64 3500. See what I mean?

Quote
~25% speed-up of project loading
Very likely, we can even do a lot better than that in the future. Not promising anything, but the present optimisations were rather conservative (changing one wxFileName function call, omitting some xml parsing, and block-allocating project files). A new class for handling project file paths is in work, and several recurring branches can probably be moved outside loops without affecting the end result. It is uncertain how much this will improve overall performance, but I think it will be quite noticeable, again.

Quote
Funnily, that also means that you need HEAD from yesterday to build HEAD from today.
*big evil grin*  Yes indeed. Sorry, I didn't think about that :lol:
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

takeshimiya

  • Guest
Re: Major CVS codebase changes
« Reply #13 on: November 18, 2005, 02:07:31 am »
I agree with you.
Now that you said about performance of parsing XML files, it's true, like when parsing the lexers for Code::Blocks... it takes a lot (like 3 seconds or more sometimes) and it'll grow more.

About on where to store the configuration files and data, I was saying examples of what would be the %APPDATA% between systems. Of course hardcoding is bad.

PS: I do care what's the backend of the configuration, not that I like it being the windows registry

Now comes the part of updating contrib plugins to the new ConfigManager :)

Offline squizzz

  • Almost regular
  • **
  • Posts: 132
Re: Major CVS codebase changes
« Reply #14 on: November 18, 2005, 02:13:50 am »
Quote from: mandrav
By the way, if anyone has a list of menu shortcuts for MSVisualC++ please share them so we can add a second keybinding profile for people migrating over to C::B ;)
If there is some easy way to export it, I can prepare keyboard scheme of Dev-C++ and Borland Builder. If just a list "action - key(s)" is enough, that's ok too.
this space is for rent