Code::Blocks Forums

Developer forums (C::B DEVELOPMENT STRICTLY!) => Development => Compiler Framework Redesign => Topic started by: Alpha on June 15, 2012, 01:00:36 pm

Title: XML based compilers
Post by: Alpha on June 15, 2012, 01:00:36 pm
It seems there were some plans at one point to convert the compiler system from hard-coded to xml.  The attached patch gives an example of a possible way this could be done.  Before I spend time converting the rest of the compilers, I was wondering what comments or suggestions anyone had on method with which I attempted this (and to check how much interest there currently is in xml based compilers).

This patch only moves the compiler options into the xml file (no regexps and no auto-detect), however I think I could create an additional function to load (custom) pure xml compilers (which would be fully functional except for lacking auto-detect).
Title: Re: XML based compilers
Post by: MortenMacFly on June 15, 2012, 02:39:42 pm
Indeed - this was in internal task, just we didn't have resources so far. This was/is the internal planing of the dev team:

Quote
Task:
- Make compiler options more flexible

The compiler options are added like:
Code
m_Options.AddOption(description, unique_compiler_key, category);

  • Make this available through an XML settings file
  • Design/implement an UI to easily edit these options
  • Probably allow to derive form an other's compiler settings

If you did a first step that would be very nice. I'll certainly look into it - as this would be a #1 priority change. Expect feedback... just my time is very limited atm...
Title: Re: XML based compilers
Post by: MortenMacFly on June 16, 2012, 09:22:46 am
Ok - I finally had a look into it. It goes even further then what I had in mind concerning the options (which is good ;)).

Some notes:

From my side you clearly got the "go", knowing that this is an awful lot of work. But it would be one of the most welcome contributions currently! 8)
Title: Re: XML based compilers
Post by: Alpha on June 16, 2012, 06:06:35 pm
Conversion process started.  I wrote a small text manipulation plugin (which I will release later) to help me, so it might be a bit less work than first estimated.

  • You/we should consider a simple UI in the end where the user can adjust these options as desired so they become really flexible. Just keep in mind if designing interfaces for example.
My current idea is to have a right-click menu with:
Code
New flag...
Modify flag...
Delete flag
I have not tried creating it yet (I am working on converting the compilers first).

  • I would have preferred to see using TinyXML the persistence layer as we do it with any other XML based options files, but this might not be so important (I recall when we decided to go for TinyXML the XML layer in wxWidgets was not that powerful.)
I had selected wxXml because it had enough features for my purposes, and so that I did not have to worry about converting wxStrings.  Is it fine to leave as-is, or should I switch to TinyXML?
Title: Re: XML based compilers
Post by: MortenMacFly on June 17, 2012, 12:28:05 pm
My current idea is to have a right-click menu with:
Code
New flag...
Modify flag...
Delete flag
That's an easy way for a first step, sure.

I had selected wxXml because it had enough features for my purposes, and so that I did not have to worry about converting wxStrings.  Is it fine to leave as-is, or should I switch to TinyXML?
Well as I said - wxXml has changes a lot, so if you have all functions you need, I'm fine with it.
Title: Re: XML based compilers
Post by: Alpha on June 18, 2012, 02:38:12 am
Attached is the initial conversion of all of the compilers.  There is a decent chance (with so much text to convert) that I have missed/messed up something, so I would appreciate usability reports from anyone with time to test this.

My next step is to add user customization features.
Title: Re: XML based compilers
Post by: darmar on June 18, 2012, 09:05:43 am
Thank you Alpha for your efforts.

I would like to add what I am missing in the current C::B implementation of compilers:
Some of compilers require options to be specified in a linker stage. But there is no possibility to have "Linker Flags" just like "Compiler Flags". Currently users can add required options in "Other linker options" window, but this is not as convenient as with flags.

Maybe it could be possible to add "Linker Flags" for "XML based compilers"?
Title: Re: XML based compilers
Post by: thomas on June 18, 2012, 12:28:36 pm
(Some of) the reasons why the XML idea was abandoned back then were

That said, if someone is still willing to work on this beast, great. But please do it properly, don't just tack XML onto the existing system, which frankly would be just shit (no offense intended).
Title: Re: XML based compilers
Post by: Alpha on June 18, 2012, 03:20:55 pm
(Warning: long post.)

Maybe it could be possible to add "Linker Flags" for "XML based compilers"?
This is sort of already available in the current system:
Code
void AddOption(const wxString& name,
               const wxString& option,
               const wxString& category = _("General"),
               const wxString& additionalLibs = wxEmptyString,
               bool doChecks = false,
               const wxString& checkAgainst = wxEmptyString,
               const wxString& checkMessage = wxEmptyString);
Anything in additionalLibs is sent to the linker.  So in this XML system:
Code
<Option name="Strip all symbols from binary (minimizes size)"
        additionalLibs="-s"/>
However, this can lead to confusion because it does not explicitly mention that compiler flags can control the linker.  I am considering the possibility of a "Linker Flags" window, but I am not sure if there are enough common features to warrant the extra space usage (as the functionality is, in essence, already available).

  • Some flags/functions are mutually exclusive on some compilers, sometimes depending on a third one, and sometimes a flag needs to be passed to another stage or requires another one being passed on a different stage
I am afraid your logic in this statement is to complicated for me to follow.  If Code::Blocks current compiler system can do these, then this XML system can as well.  Compilers are currently not completely decoupled into XML; only the contents of the Reset() method: m_Programs, m_Switches, m_Options, and m_Commands.  These items are loaded from the compiler's XML options file (instead of being hard-coded).

  • The build process would have to be reworked (and probably project files as well)
  • Anything else would be the same as we already have, just with a new file format
All changes are currently highly localized; the compilers' interfaces function exactly the same, the only difference is how they populate their options (pure C++ compilers are still completely valid as well).
That said, the compilers I have converted are now a hybrid of C++ (for regexps and auto-detect) and XML (for options/flags).  My final (planned) step would be to write a compilerXML.cpp which would load any pure XML compilers that had been detected during start-up.  (No compilers are, as of yet, dynamically loaded.)

[...]
don't just tack XML onto the existing system
[...]
This may have been exactly what I have done.  However, I think it still holds the advantage of providing a framework to make compiler flags easily configurable.  Have you had a chance to explore my implementation? - I know I am not an experienced programmer, so I would appreciate any critique on if this is even the "right" method to move forwards.


Would anyone like me to write up documentation for the current XML format?  (Otherwise I will wait a bit until I am reasonably sure that it is set.)
Title: Re: XML based compilers
Post by: Alpha on June 18, 2012, 03:34:26 pm
This is not exactly related; in exploring the compilers, I found that the variables
Code
bool doChecks
const wxString& checkAgainst
const wxString& checkMessage
have no effect.  I assume that they were planned, but no one finished the implementation.  If I were to implement this, should I bundle it with my XML compiler patch, or should I make it directly against the trunk?

Edit: Never mind; against trunk (http://developer.berlios.de/patch/?func=detailpatch&patch_id=3295&group_id=5358).
Title: Re: XML based compilers
Post by: MortenMacFly on June 18, 2012, 08:17:21 pm
That said, if someone is still willing to work on this beast, great. But please do it properly, don't just tack XML onto the existing system, which frankly would be just shit (no offense intended).
Well Thomas, I got you point, but please: Don't request a re-build of the core from a single person involved just recently.

And I clearly disagree with you here: What I think completely sucks is that these options are hard-coded inside each compiler. So we basically need to re-build everything if just a single option changes. Also, some compilers are basically the same just they differ in some options. So you could easily go one step further (as Alpha suggested already) and have a "generic" compiler that loads and configures itself with such configuration. This will definitely  work so some extend.
Also, if you change the advanced options of the compiler they are already XML based, just inside the C::B config where they not belong to IMHO. Thus, making all options XML based is just consequent.

So: I agree that there may be the ultimate solution possible, but we don't have time and resources for it ATM. Nevertheless the approach proposed here is a huge step in the right direction, clearly, so we should go for it. Furthermore I don't think the script based approach will be a good idea at the moment. The scripting engine is the show-stopper #1 for the 64 bit build of C::B as SQPlus was abandoned and we have to maintain it ourself. But this is such a low-level stuff that you need an own team of devs just for that. already, meanwhile you cannot even update the underlying squirrel anymore due the massively out-dated SQPlus layer. If we make everything script based we end-up nowhere sooner or later, IMHO.
Title: Re: XML based compilers
Post by: thomas on June 19, 2012, 02:05:30 am
Well, if nothing helps, we can still move back to Angelscript, which is well supported. Ironically, the reason why we abandoned Angelscript back then was that it didn't do 64 bits at that time...  ;D

The thing about scripting is you could not only concatenate some strings, but also do logic inside them. In fact, not only the composition of the options dialog, but the entire build process could be a script which calls some native SDK functions (one would need a well-defined interface, of course). Not much unlike the project wizards (yep, those that I hate so much) work. What I'd wish for is nothing else but the "generic compiler" that you talked about earlier. Except with scripting it would certainly work, as a script can trivially do "just anything", and with XML I see the task of making something that works for all as rather difficult.

Now XML is mighty fine for configuring a few strings, but the logic is still all hardcoded, and the way compiler options are handled is poor at best. You know I'm no friend of "too much darn smartness", but compiler options in Code::Blocks aren't smart at all. You can for example select "do not optimize" together with all levels of optimization, all at the same time (same goes for warnings), you can select a dozen architectures at the same time, and you can generate debug info and strip at the same time. Certainly you can expect a programmer to somewhat know what he's doing, but how often does it happen to you that you accidentially strip your debug info and wonder why the debugger doesn't come up (if it doesn't ever happen to you, you're provably more intelligent than I am, I have this about once every other week). In any case, it wouldn't hurt if the user interface was a bit clearer. Alas, this is darn hard to hardcode.

What I'm fearing is that if this kind of works (even if it works only just a little bit better than what we have now) it will stay like this for the next decade.  ::)

If only I had the time... :(
Title: Re: XML based compilers
Post by: Alpha on June 19, 2012, 04:19:21 am
You know I'm no friend of "too much darn smartness", but compiler options in Code::Blocks aren't smart at all. You can for example select "do not optimize" together with all levels of optimization, all at the same time (same goes for warnings), you can select a dozen architectures at the same time, and you can generate debug info and strip at the same time. Certainly you can expect a programmer to somewhat know what he's doing, but how often does it happen to you that you accidentially strip your debug info and wonder why the debugger doesn't come up (if it doesn't ever happen to you, you're provably more intelligent than I am, I have this about once every other week). In any case, it wouldn't hurt if the user interface was a bit clearer. Alas, this is darn hard to hardcode.
Now that you mention it, I had already created the ability to fix this in my local version with:
Code
struct CompOption
{
// following comments are an example of an option
wxString name; // "Profile code"
wxString option; // "-pg"
wxString additionalLibs;// "-lgmon"
bool enabled; // true/false
wxString category; // "Profiling"
bool doChecks; // true/false
wxString checkAgainst; // "-O -O1 -O2 -O3 -Os" (will check for these options and if any of them is found, will display the following message)
wxString checkMessage; // "You have optimizations enabled. This is Not A Good Thing(tm) when producing debugging symbols..."
wxString supersedes;    // "-O -O1 -O2" (will check for these options and disable any of them that are found)
bool exclusive;         // true/false (will ensure that only one item in this category is ever selected)
};
and
Code
void CompilerOptionsDlg::OnOptionToggled(wxCommandEvent& event)
{
    wxCheckListBox* list = XRCCTRL(*this, "lstCompilerOptions", wxCheckListBox);
    int sel = event.GetInt();
    CompOption* copt = m_Options.GetOptionByName(list->GetString(sel));
    if (copt)
    {
        copt->enabled = list->IsChecked(sel);
        if (copt->enabled)
        {
            if (copt->doChecks)
            {
                wxArrayString check = GetArrayFromString(copt->checkAgainst, _T(" "));
                for (size_t i = 0; i < check.Count(); i++)
                {
                    CompOption* against = m_Options.GetOptionByOption(check[i]);
                    if (against && against->enabled)
                    {
                        AnnoyingDialog dlg(_("Compiler options conflict"),
                                           copt->checkMessage,
                                           wxART_INFORMATION,
                                           AnnoyingDialog::OK,
                                           wxID_OK);
                        dlg.ShowModal();
                        break;
                    }
                }
            }
            if (copt->supersedes != wxEmptyString)
            {
                wxArrayString supersede = GetArrayFromString(copt->supersedes, _T(" "));
                for (size_t i = 0; i < supersede.Count(); i++)
                {
                    for (size_t j = 0; j < m_Options.GetCount(); j++)
                    {
                        if (copt != m_Options.GetOption(j) &&
                            supersede[i] == m_Options.GetOption(j)->option)
                        {
                            list->Check(j, false);
                            m_Options.GetOption(j)->enabled = false;
                        }
                    }
                }
            }
            if (copt->exclusive)
            {
                for (size_t i = 0; i < m_Options.GetCount(); i++)
                {
                    if (copt != m_Options.GetOption(i) &&
                        copt->category == m_Options.GetOption(i)->category)
                    {
                        list->Check(i, false);
                        m_Options.GetOption(i)->enabled = false;
                    }
                }
            }
        }
    }
    m_bDirty = true;
} // OnOptionToggled
(I implemented doChecks and added supersedes and exclusive.  I did not update the current compilers to take advantage of this functionality, however my quick test with the XML compilers worked beautifully - the next patch I upload will contain these and (hopefully) dynamically configurable flags.)

What I am not currently sure about is if it is better (and in which cases) to show a warning dialog, and when to just automatically check/uncheck the conflicting options.

Well, if nothing helps, we can still move back to Angelscript, which is well supported. Ironically, the reason why we abandoned Angelscript back then was that it didn't do 64 bits at that time...  ;D
With respects to scripting, I am curious if anyone has ever discussed the pros/cons of Python?
Title: Re: XML based compilers
Post by: oBFusCATed on June 19, 2012, 10:14:47 am
With respects to scripting, I am curious if anyone has ever discussed the pros/cons of Python?
Python is not made for embedding, it is their agenda that the applications should be written in python, not extended by python.
It is doable but the API is pretty low level or we need to require Boost.Python, which is mighty heavy.
Also shipping python on windows will be problematic.

In my opinion current scripting in C::B is pretty bad, because SQPlus is mighty stupid. I've tried to add some scripting for the debugger and failed quite miserably :(
Also the docs for squirrel are not helping much, as they are written for someone knowing what he is doing, not for someone who is new to it and scripting.
Title: Re: XML based compilers
Post by: MortenMacFly on June 19, 2012, 11:05:25 am
In my opinion current scripting in C::B is pretty bad, because SQPlus is mighty stupid. I've tried to add some scripting for the debugger and failed quite miserably :(
We are becoming slightly off-topic, however...:
I think Scrat (http://scrat.sourceforge.net/) would be another option for a replacement of SQPlus. But as Thomas said, if AngelScript is still alive, actively developed (and supported) and supports 64 bit we might have done a wrong choice the time back... :-\
Title: Re: XML based compilers
Post by: Alpha on June 19, 2012, 03:22:28 pm
OK, compilers can now save their flags back into an XML file.

My current idea is to have a right-click menu with:
Code
New flag...
Modify flag...
Delete flag
I can create this menu, but I am not certain how I am supposed to attach it.  The popup will belong to a wxCheckListBox (XRCID: "lstCompilerOptions").  Does the connection belong in an event table?  Should I be calling Connect()?  (Also, is this (http://wiki.wxwidgets.org/WxMenu) the suggested way to use it?)
The only wxWidgets events I have created before were automatic wxSmith generated ones, so if anyone could point me to the relevant section of documentation/give me a hint, I would be much obliged.
Title: Re: XML based compilers
Post by: Alpha on June 20, 2012, 04:03:26 pm
I have run into even more event related woes, and will be asking some questions in the wxWidgets forum.  So, it may be some time before any more progress is shown here.
Title: Re: XML based compilers
Post by: thomas on June 21, 2012, 06:25:19 pm
I think Scrat would be another option for a replacement of SQPlus.
Sqrat looks nice, it's very straightforward to use, feature-complete, header-only.

That said, I've invested some time today trying it out, and as nice as it is on paper, it crashes even on the simplest possible test case (bind 1 function, compile a 1-line script). Basically the same example like on their website, only even simpler.

Running single-step shows a segfault in sqratScript.h, line 54.

Thing is, the sq tool from the same, identical build works just fine. So I guess it's not something wrong with the Squirrel build, it must really be Sqrat :(

If you want to waste some time in the evening playing with it, I can share the project. Maybe I'm only just too stupid, and you can figure it out.
Title: Re: XML based compilers
Post by: Alpha on June 21, 2012, 07:12:15 pm
Completed patch finally available.  I would assume there are multiple bugs in it that are yet to be found (after all, what is testing for?), but I think it contains a useful set of features.

I will write up some documentation on the XML format soon.
Title: Re: XML based compilers
Post by: Alpha on June 22, 2012, 07:51:38 am
Preliminary documentation (http://wiki.codeblocks.org/index.php?title=Compiler_options_file) is done.

I have begun work on a loader for generic "pure" XML compilers.

Completed patch finally available.
I forgot to mention; this patch includes:
Title: Re: XML based compilers
Post by: MortenMacFly on June 22, 2012, 07:53:22 am
If you want to waste some time in the evening playing with it, I can share the project. Maybe I'm only just too stupid, and you can figure it out.
To be honest I never tried myself, just found this link when looking for a more up-to-date SQPlus. So sure, gimme gimme... (I am back btw...).

And btw:
Well, if nothing helps, we can still move back to Angelscript, which is well supported. Ironically, the reason why we abandoned Angelscript back then was that it didn't do 64 bits at that time...  ;D
This is really fun and honestly I don't get it: Did we ever try 64 bit with scripting at that time seriously? Because then we would have found out easily that we are doomed with SQPlus. I wonder if it is working straight forward on Linux though... on Windows its very bad and it gets even worse as new (more strict) compilers hit the floor.

Having said that, returning to AngelScript is also not that easy: For example, all the script binding would need to be re-done.... :'(
Title: Re: XML based compilers
Post by: MortenMacFly on June 22, 2012, 07:56:49 am
Preliminary documentation (http://wiki.codeblocks.org/index.php?title=Compiler_options_file) is done.
Huh - the first pach provider with a documentation! That's Wohooo! ;-)

BTW: I have tried the patch before the last one during the time I was off - this already seems to work quite well. I didn't find any serious show-stoppers. I'll try version the next days...

Some comments I do have:

1.) I changed the log messages when the XML file cannot be found or alike to cbMessageBoxes. Because if tat happens the compiler is in an "undefined" state and you cannot use it. Newbies won't realise that and otherwise our forums get flooded with such requests. Maybe we should even disable this particular compiler in such a case. But disabling compilers hasn't an API so far IMHO...

2.) I think later on we can/should transfer the regexes and advanced compiler options to the XML file, too. For this, it needs an interface to the configmanager.
Title: Re: XML based compilers
Post by: MortenMacFly on June 22, 2012, 08:14:30 am
...I just saw: Did you realise SVN revision 8060? Is this already integrated? Its not a very wise thing to change the compilers at the moment in SVN. Maybe we should lock the plugin for a while...
Title: Re: XML based compilers
Post by: thomas on June 22, 2012, 10:18:35 am
Sqrat test attached, no external dependencies. Unzip, doubleclick, hit "build".

Builds the libs, the sq interpreter, and a test program which installs error handlers, binds one function, and compiles a 1-line script that calls this function (which is enough to crash).

[attachment deleted by admin]
Title: Re: XML based compilers
Post by: killerbot on June 22, 2012, 10:51:25 am
I added some compiler options, wouldn't lock it at this moment. Important for adding  C++11 related ones.

A for showing message boxes, ensure they don't end up to much during project loading and building. Eg project building for several compilers, but then you use it on a system where some compiler is not present ==> should just be mentioned in text in the build log (as it is now), no message boxes, otherwise automated building is broken.
Title: Re: XML based compilers
Post by: MortenMacFly on June 22, 2012, 11:03:56 am
Sqrat test attached, no external dependencies. Unzip, doubleclick, hit "build".
Same crash here, it seems the VM pointer is zero, therefore compilation fails and throws (in sq_compile)...
Title: Re: XML based compilers
Post by: MortenMacFly on June 22, 2012, 11:15:00 am
Sqrat test attached, no external dependencies. Unzip, doubleclick, hit "build".
Same crash here, it seems the VM pointer is zero, therefore compilation fails and throws (in sq_compile)...
Harhar, funny. Solved it. All you need to do is to set the default VM before you do the table stuff (all what's in your "try..." statements. So just add this line after "sq_seterrorhandler(vm);":
Code
Sqrat::DefaultVM::Set(vm);
Then it works just fine. 8) (Its written in the docs, too btw... ;D)
Title: Re: XML based compilers
Post by: thomas on June 22, 2012, 01:04:14 pm
Wow, what a stupid mistake (the actual mistake was forgetting to pass the VM parameter to the script constructor, though). Hmm good job you can actually make that mistake and it doesn't warn you... but at leat it crashes hard and crashes early, so I guess that's fair enough.

Setting the default VM fixes the issue too, of course, and then you need not pass the VM to the table's constructor or the root-table getter, either. Maybe that's even better, we probably don't want more than one VM in Code::Blocks anyway, do we.

So it works... guess then it should be no biggie to get this running. If I'm not too tired on saturday afternoon, I might just give it a try.
Title: Re: XML based compilers
Post by: Alpha on June 22, 2012, 04:44:46 pm
2.) I think later on we can/should transfer the regexes and advanced compiler options to the XML file, too. For this, it needs an interface to the configmanager.
What are these other advanced compiler options?  I think the XML files currently contain all options except for the regexes.
The options_*.xml files currently only contain settings that could be losslessly transferred from C++.  Looking at the various settings each compiler contains, it appears that regexes are the only remaining items that fit this requirement; the next patch will support these in the options files.
The other main component of a compiler is auto-detect.  I will be creating some (limited) functionality of this in compiler_*.xml files (files I am working on for dynamically detected pure XML compilers).

...I just saw: Did you realise SVN revision 8060? Is this already integrated?
Yes (and it was very fast to do; adding this new flag to options_common_warnings.xml put it in the whole GNU family).

About message boxes/error handling, what I have written so far mostly makes the (incorrect) assumption that it will only be fed properly formed files.  Eventually something more robust will need to be added...

  • Warning messages if multiple compiler options are unwise to use together (but still legal)
  • Automatic disabling of conflicting compiler options (for example, enabling debug symbols automatically disables strip)
(I am no compiler expert, so I only added these interactions where they were obvious.  Feel free to let me know of any flag interactions I have missed/messed up.)

Question: The only condition that <if></if> blocks currently recognize is platform - are there any other conditions that would be made use of during the loading of a compiler?
Title: Re: XML based compilers
Post by: MortenMacFly on June 22, 2012, 05:21:30 pm
What are these other advanced compiler options?
They are for example in default.conf:
<compiler> -> <sets> -> <{compiler's name}> -> <macros>
...and contain everything you setup in the compiler options page under -> Settings -> Compiler -> tab "Other settings" -> Advanced options -> {tabs "Commands", "Output parsing", "Others"}.

Note that this is usually empty, if not changed. You can apply what's written here:
http://wiki.codeblocks.org/index.php?title=Adding_support_for_non_C/C%2B%2B_files_to_the_build_system
...to see any content there.
Title: Re: XML based compilers
Post by: MortenMacFly on June 22, 2012, 05:22:47 pm
Question: The only condition that <if></if> blocks currently recognize is platform - are there any other conditions that would be made use of during the loading of a compiler?
One thing I could think of is an if-statement to check for a compiler version... not sure how to handle this at startp w/o running the compiler though.
Title: Re: XML based compilers
Post by: Alpha on June 24, 2012, 01:53:20 pm
Is it correct for me to assume that the GCC regexes (located in compilerMINGW.cpp) are applicable to all supported (C/C++) GCC variants (ARM, TriCore, ...)?
Title: Re: XML based compilers
Post by: MortenMacFly on June 24, 2012, 05:33:08 pm
Is it correct for me to assume that the GCC regexes (located in compilerMINGW.cpp) are applicable to all supported (C/C++) GCC variants (ARM, TriCore, ...)?
I think it isn't wrong in a first place, but to be sure you'd have to ask the compiler guys developing these. I never worked with ARM/TriCore compilers.

However, from what I see they might override some of the options.
Title: Re: XML based compilers
Post by: MortenMacFly on June 26, 2012, 08:57:17 am
OK, here comes some more from the testing front:

First of all: Also the new patch works great, I didn't have to change a lot (just the very minor stuff I was already referring to).


What didn't work for me is when I add a flag with a different (new) category, the new category does not appear in the "Categories" choicebox of the compiler options, although its present.

The log-term goal should probably be to have the compiler setup (not the project's compiler options setup) unified in a single dialog, including the advanced stuff, reachable only from the compiler settings dialog. For now, you can change the compiler flags even in the project build options, which is kind of mis-leading I'd say. Also, I don't like having the different options scattered in so many dialogs with different access points. But in the end that's "fine-tuning" (maybe not on your list) and I believe we can even re-use the UI resources already created by then.
Title: Re: XML based compilers
Post by: MortenMacFly on June 26, 2012, 09:05:05 am
What didn't work for me is when I add a flag with a different (new) category, the new category does not appear in the "Categories" choicebox of the compiler options, although its present.
Oh well, now I figured out something else: Looking at the modified compiler XML file I realised that if you change a flag the <if> <else> sections are gone, meaning, that the options are no longer cross-platform. Also, the "Common" options are integrated and no longer linked. Is this by design?
Title: Re: XML based compilers
Post by: Alpha on June 26, 2012, 03:08:07 pm
What didn't work for me is when I add a flag with a different (new) category, the new category does not appear in the "Categories" choicebox of the compiler options, although its present.
Oops; I forgot to refresh the categories when a new one is added.

Oh well, now I figured out something else: Looking at the modified compiler XML file I realised that if you change a flag the <if> <else> sections are gone, meaning, that the options are no longer cross-platform. Also, the "Common" options are integrated and no longer linked. Is this by design?
Design (mostly) and laziness (a little).  The modified XML is stored in the user profile, so it is very unlikely to shift platforms.  Also, resolution is not easy: if a flag is added between two flags that are inside the same <if> section, should the new flag be put into the <if>?  Should the <if> be split into two <if>s?
The common options must be unlinked because if a flag was modified/deleted that was loaded from a common location, there is no simple way to resolve; if the changed flag was saved back to the common location, it could unexpectedly alter multiple compilers.

The log-term goal should probably be to have the compiler setup (not the project's compiler options setup) unified in a single dialog, including the advanced stuff, reachable only from the compiler settings dialog.
I think this is the right direction, however I am not going to attempt it (yet).  I do not want to try to be too ambitious, and have to stop with only a half finished product to show for it.

Regexes and advanced compiler options (support for alternate build commands with file extension filters) are now integrated into the XML.  I will soon upload a patch with these and your suggested dialog modifications.
Title: Re: XML based compilers
Post by: MortenMacFly on June 26, 2012, 05:26:52 pm
The modified XML is stored in the user profile, so it is very unlikely to shift platforms.
Ah - OK I didn't know that. If I got it right that means that the default files are distributed with C::B and installed in the "shared" folder of C::B and the modifications are in the user's profile directory and override the default C::B files, right? In that case it should be just fine because then you can easily revert to the default values or - if another user logs on - (s)he gets provided the initial default values, too.

Regexes and advanced compiler options (support for alternate build commands with file extension filters) are now integrated into the XML.  I will soon upload a patch with these and your suggested dialog modifications.
Great... I'll wait patiently... 8)
Title: Re: XML based compilers
Post by: Alpha on June 27, 2012, 04:18:43 am
Regexes and advanced compiler options (support for alternate build commands with file extension filters) are now integrated into the XML.  I will soon upload a patch with these and your suggested dialog modifications.
Great... I'll wait patiently... 8)
Here it is.  It also includes a recursion depth guard (in case some common file decides it is a good idea to recursively load itself, or another common file which references back to it) and the initial code in compilerXML.cpp (disabled because it is not yet functional).

If I got it right that means that the default files are distributed with C::B and installed in the "shared" folder of C::B and the modifications are in the user's profile directory and override the default C::B files, right?
Yes.  (At least, this is what it is supposed to do...)

By the way, does Code::Blocks programming style have a preference between wxT("text") and _T("text")?  (From what I can tell, they mean exactly the same thing.)
Title: Re: XML based compilers
Post by: Alpha on June 27, 2012, 04:51:34 am
Documentation (http://wiki.codeblocks.org/index.php?title=Compiler_options_file) updated.
Title: Re: XML based compilers
Post by: MortenMacFly on June 27, 2012, 07:11:40 am
Here it is.
Nice... will try ASAP...

By the way, does Code::Blocks programming style have a preference between wxT("text") and _T("text")?  (From what I can tell, they mean exactly the same thing.)
Not really, but I personally prefer the wxT macro. The wx guys recommend it, too and _T sometimes conflicts with equal definitions of other frameworks. wxT is safe instead and dedicated for wxWidgets. Remember, that for translation you'll need to use "_".
Title: Re: XML based compilers
Post by: MortenMacFly on June 27, 2012, 08:52:53 am
Some feedback already (just looking at the diff):

- compileroptionsdlg: The global var "menuOption" should become a class member var "m_MenuOption"
- compilers: You can safely remove the options you've commented in the patch btw... that's the overall goal in the end and for reference we have SVN... ;-)
- in the compilers: You are not always using GetID() to obtain the compiler's ID for loading the settings... why?
- I've removed Programs.DBG and all its references because in fact it is nowhere being used anymore since we have merged the debugger branch.
- in compiler.cpp you should add an #include <wx/xml/xml.h> for non-pch compilers

For the autodetection: I could image a core / compiler plugin base method that takes the compiler ID and implements all the different methods we have for auto-detection in one place. This way, you could get a generic compiler w/o specialisation for the auto-detection. But maybe you have a better idea already...
Title: Re: XML based compilers
Post by: Alpha on June 28, 2012, 03:48:07 am
- in the compilers: You are not always using GetID() to obtain the compiler's ID for loading the settings... why?
I plan to use GetID() for all of them, however, when Code::Blocks creates an ID, it removes "-" dash characters.  I used this as a temporary bypass while I explore to see if there is a reason compiler ID's should not contain a dash.

For the autodetection: I could image a core / compiler plugin base method that takes the compiler ID and implements all the different methods we have for auto-detection in one place. This way, you could get a generic compiler w/o specialisation for the auto-detection. But maybe you have a better idea already...
I am not completely certain I understand what you are describing.  My current plan is for the main compiler plugin to scan for compiler_*.xml files after loading all the definitions of other compilers, and for each discovered file, pass it on to compilerXML.cpp (which will contain all auto-detection routines and loading of options/regexes).  Is this the same as what you said?
Or are you saying XML based auto-detection should be part of the sdk?
Or are you saying the sdk should have some generic auto-detection methods that all compilers will call, and compilerXML.cpp will extract the necessary information from an XML file, and pass it on through these methods?
Title: Re: XML based compilers
Post by: MortenMacFly on June 28, 2012, 07:14:29 am
pass it on to compilerXML.cpp (which will contain all auto-detection routines and loading of options/regexes).
That's not what I mean, because this would not allow an "old-school" compiler easily.

Or are you saying the sdk should have some generic auto-detection methods that all compilers will call, and compilerXML.cpp will extract the necessary information from an XML file, and pass it on through these methods?
That's what I meant, but basically not part of the SDK, but the compiler plugin itself as helper functions. Auto-detection of compilers can be very specific, so it might be implemented for a single compiler (not necessarily XML based) differently. So having a "AutoDetector" class with a common interface that you can override if needed sounds most convenient to me. In the end the auto-detection has nothing to do with XML configuration (or am I wrong?!), so putting this into compilerXML doesn't seem logical to me (and only me)....?!

The same applies to the version detection - as you know we call some compilers to obtain their version to adjust the required command line settings accordingly. We didn't talk about this so far btw... any plans for this? What just comes into my mind is that this might be another use case for the <if> statements: You might want to adjust compiler options available by the version of the compiler... 8)

I don't want to make things too complex, just to have it mentioned and discussed... ::)
Title: Re: XML based compilers
Post by: Alpha on June 29, 2012, 03:22:59 am
pass it on to compilerXML.cpp (which will contain all auto-detection routines and loading of options/regexes).
That's not what I mean, because this would not allow an "old-school" compiler easily.
I do not believe I see how any of the changes would disallow "old-school" compilers.
So having a "AutoDetector" class with a common interface that you can override if needed sounds most convenient to me.
However, this idea seems to have more promise.  Forgive my lack of C++ knowledge, but would the way to do this use multiple inheritance:
Code
class CompilerMINGW : public Compiler, public AutoDetector
or static members in AutoDetector or something else?

The same applies to the version detection - as you know we call some compilers to obtain their version to adjust the required command line settings accordingly. We didn't talk about this so far btw... any plans for this?
Yes, I had separated out EvalXMLCondition() (although, I am not certain if it was correct to put it in globals) so more items could be added to the <if> statements with relative ease.  Two items that I will be trying to add are generic run program (true on success, false on failed to launch).  This could be used to add these types of commands (http://wiki.codeblocks.org/index.php?title=Adding_support_for_non_C/C%2B%2B_files_to_the_build_system) only when the relevant program is available.
And version detection/comparison.  There is the easier method of checking the output of a command against a supplied regex.  The other (probably better) way would be to figure out a generic method of parsing for the version string, caching it in m_VersionString, and using some sort of generic comparison.
(One possible issue: what type of start-up cost will launching a series of external commands have?)

I plan to use GetID() for all of them, however, when Code::Blocks creates an ID, it removes "-" dash characters.  I used this as a temporary bypass while I explore to see if there is a reason compiler ID's should not contain a dash.
Upon further inspection, it appears this transformation is to create valid element names; as "-" characters are valid (except at the beginning), I have modified MakeValidID().
Title: Re: XML based compilers
Post by: MortenMacFly on June 29, 2012, 07:14:09 am
Code
class CompilerMINGW : public Compiler, public AutoDetector
Yes, this would look convenient to me. Another (simpler) option is to have the compiler class hold the autodetector and provide a single method like
Code
AutoDetectResult AutoDetect(const wxString& compiler_id /*, something else, a regex to parse the version for example?! */)
{
    CompilerAutoDetector cad(compiler_id /*, regex*/);
    return cad.AutoDetect();
}
Then you don't need multiple inheritance.

There is the easier method of checking the output of a command against a supplied regex.
This is my favourite as you see above - don't make it too complicated.

The other (probably better) way would be to figure out a generic method of parsing for the version string, caching it in m_VersionString, and using some sort of generic comparison.
Nope, I guess it will be very hard to make this generic - compilers are just too different... :-)

(One possible issue: what type of start-up cost will launching a series of external commands have?)
That is an issue, indeed. Already now we call the compilers / frontend tools at a cost. We do this with wxExecute (if I am not mistaken) although way better would be to do this threaded... However, then you have to have a state machine as it is no longer a serialised process. So we shouldn't change it for the moment as this may introduce serious (platform related) issues.

Upon further inspection, it appears this transformation is to create valid element names; as "-" characters are valid (except at the beginning), I have modified MakeValidID().
OK - great. I was wondering anyways why this could have been done... ::)
Title: Re: XML based compilers
Post by: Alpha on June 30, 2012, 04:12:42 am
Another (simpler) option is to have the compiler class hold the autodetector and provide a single method
[...]
OK, I will experiment with this.  Would you happen to know if this technique is used elsewhere in Code::Blocks, so that I could examine it?

I believe I will be busy for a few days, so I am not certain when my next patch will come.
Title: Re: XML based compilers
Post by: MortenMacFly on June 30, 2012, 06:47:16 am
Would you happen to know if this technique is used elsewhere in Code::Blocks, so that I could examine it?
Mmmmh... not to my knowledge. It would be a tooling class. So it its easier for you to derive from - just go that way. In the end from an interface point of view its the same.

I believe I will be busy for a few days, so I am not certain when my next patch will come.
No üroblem, leaves more time for me to test. BTW: already now I have used it a lot w/o issues. Just the patch needs tweaks to compile w/o PCH and on wx 2.9.x.
Title: Re: XML based compilers
Post by: Alpha on July 10, 2012, 03:34:16 am
I believe I will be busy for a few days, so I am not certain when my next patch will come.
... maybe that was a bit more than a few days...

I think I have a working extension to the <if> statements now.  Does anyone have specific example flags that should be hidden if the compiler is detected as a certain version?  (Or a link to where I can find a listing of flags added by compiler version.)
Title: Re: XML based compilers
Post by: Alpha on July 10, 2012, 05:35:29 am
- compileroptionsdlg: The global var "menuOption" should become a class member var "m_MenuOption"
By the way, this change causes the passing on of the event id to mysteriously fail (no errors are raised, and according to gdb, m_MenuOption reverts -1 when OnFlagsPopupClick() returns).

However, it does work if I make it a static class member.  Would this still be the preferred method?
Title: Re: XML based compilers
Post by: MortenMacFly on July 10, 2012, 06:38:10 am
Does anyone have specific example flags that should be hidden if the compiler is detected as a certain version?
Certainly the platform flags usually need attention. Also, -std=c++0x and -std=c++11 and some warnings (like -Weffc++) have not always been available.

The GCC webpage will always tell you what's new, like here:
http://gcc.gnu.org/gcc-4.3/changes.html
...under the section "IA-32/x86-64" you'll see that i.e. -mtune=core2 has been added in this version (4.3).

From my point of view we should consider the last 3.x.x version in contrast to 4.0.x and then 4.5, 4.6 and 4.7 which should cover the main up-to-date compilers. Maybe not all of the flags, but the ones that seem important. If its configurable in the end we can do it step by step anyways...
Title: Re: XML based compilers
Post by: MortenMacFly on July 10, 2012, 06:39:51 am
By the way, this change causes the passing on of the event id to mysteriously fail (no errors are raised, and according to gdb, m_MenuOption reverts -1 when OnFlagsPopupClick() returns).
Huh? How weird is that?! Maybe we are overriding a member variable with the same name of a base class?! I'll have a look.

However, it does work if I make it a static class member.  Would this still be the preferred method?
...in the mean time: Yes. :-)
Title: Re: XML based compilers
Post by: Alpha on July 11, 2012, 02:22:50 pm
OK, thanks.

Now that options are almost all loaded with the same code, is it fine to make Compiler::Reset() and Compiler::LoadDefaultRegExArray() non-pure virtuals?
Title: Re: XML based compilers
Post by: MortenMacFly on July 11, 2012, 02:29:40 pm
Now that options are almost all loaded with the same code, is it fine to make Compiler::Reset() and Compiler::LoadDefaultRegExArray() non-pure virtuals?
Depends on what you mean with almost here...?! If its not the same, shouldn't it remain virtual because in that case a compiler needs to implement it?!
Title: Re: XML based compilers
Post by: MortenMacFly on July 11, 2012, 07:51:40 pm
I've created a branch for you work here:
http://svn.berlios.de/svnroot/repos/codeblocks/branches/xml_compiler
If you have time, please switch to it and provide patches against the branch. This should make things easier and track-able. Notice that I also integrated a few corrections from my side concerning compiler errors and alike (no functional changes). So be careful, I guess the first time you need some clever merging. If you can't do that, just provide patches against the branch, I'll do the rest.
Title: Re: XML based compilers
Post by: Alpha on July 12, 2012, 01:51:10 pm
Now that options are almost all loaded with the same code, is it fine to make Compiler::Reset() and Compiler::LoadDefaultRegExArray() non-pure virtuals?
Depends on what you mean with almost here...?! If its not the same, shouldn't it remain virtual because in that case a compiler needs to implement it?!
They would still remain virtual
Code
virtual void Reset();
just not pure vitrual
Code
virtual void Reset() = 0;

By almost, I mean all of the LoadDefaultRegExArray() look like
Code
void CompilerMSVC8::LoadDefaultRegExArray()
{
    m_RegExes.Clear();
    LoadRegExArray(GetID());
}
and most of the Reset() look like
Code
void CompilerMSVC8::Reset()
{
    m_Options.ClearOptions();
    LoadDefaultOptions(GetID());

    LoadDefaultRegExArray();

    m_CompilerOptions.Clear();
    m_LinkerOptions.Clear();
    m_LinkLibs.Clear();
    m_CmdsBefore.Clear();
    m_CmdsAfter.Clear();
}
The Reset()s that are different include LCC (init member var), DMD (add some link libs), and the main GCC (call SetVersionString()).
In my opinion, it make sense to move this default implementation into the base, so newly written compilers will automatically try to load their XML files.  If a compiler does not want to load from XML (or wants to do additional items), then these methods can still be re-implemented, overriding the default.

I've created a branch for you work here:
Thank you (the patches against the trunk were becoming almost unreadable due to size).  I have finished merging, and will be doing a few tests; the next patch should come soon.
Title: Re: XML based compilers
Post by: Alpha on July 12, 2012, 03:27:09 pm
Patch available (against XML branch).

Changed:
Title: Re: XML based compilers
Post by: Alpha on July 12, 2012, 06:17:42 pm
Documentation (http://wiki.codeblocks.org/index.php?title=Compiler_options_file) updated.
Title: Re: XML based compilers
Post by: MortenMacFly on July 13, 2012, 09:46:48 am
They would still remain virtual
Code
virtual void Reset();
just not pure vitrual
Code
virtual void Reset() = 0;
Ah - that makes sense, sure.

By almost, I mean all of the LoadDefaultRegExArray() look like
[...]
and most of the Reset() look like
Yes, is seems feasible that they are in base now. Maybe we can do it even easier: Reset calls ResetPrivate() which is the virtual method you need to override if you need to do something special and in its default implementation it does nothing (same for LoadXXX). This way, compilers were "forced" to use the default "Reset" (which is good I think) but still can do something special. To soften the requirements we could stick with the virtual Reset method. Although it would be not too bad to make Reset even non-virtual and protected in that case.

What do you think?
Title: Re: XML based compilers
Post by: MortenMacFly on July 13, 2012, 09:58:14 am
Patch available (against XML branch).
For the record: Applies flawlessly in the branch. Nice! (Not compiled in / tested though...)

Edit: I see the compilers individual code really shrinks... nice! :-)
Title: Re: XML based compilers
Post by: MortenMacFly on July 13, 2012, 11:00:43 am
One first feedback:
When running with debug log (and therefore warnings) enabled, now I get tons of warnings, that the command issued to detect the compiler could not be executed.

You should either disable logging for this step (http://docs.wxwidgets.org/2.8/wx_wxlognull.html), or check if the executable really exists before running the command. In the latter case make sure you account for possible macros.
Title: Re: XML based compilers
Post by: Alpha on July 13, 2012, 10:11:27 pm
Maybe we can do it even easier: Reset calls ResetPrivate() which is the virtual method you need to override if you need to do something special and in its default implementation it does nothing (same for LoadXXX). This way, compilers were "forced" to use the default "Reset" (which is good I think) but still can do something special. To soften the requirements we could stick with the virtual Reset method. Although it would be not too bad to make Reset even non-virtual and protected in that case.

What do you think?
Almost everything I know about program design has been self taught through a combination of the internet, and half-finished books; I do not think I know what is the best option here.
Perhaps other developers would like to weigh in?

For the record: Applies flawlessly in the branch. Nice!
Go WinMerge (http://sourceforge.net/projects/winmerge/)! :)

When running with debug log (and therefore warnings) enabled, now I get tons of warnings, that the command issued to detect the compiler could not be executed.
Oops... I will deal with that.
Title: Re: XML based compilers
Post by: Alpha on July 15, 2012, 06:55:45 pm
Quick update (nothing big):
Title: Re: XML based compilers
Post by: Alpha on July 18, 2012, 03:33:42 am
Adding to the previous patch:

(Unrelated, but...)
Code
Index: src/sdk/resources/editor_configuration.xrc
===================================================================
--- src/sdk/resources/editor_configuration.xrc (revision 8138)
+++ src/sdk/resources/editor_configuration.xrc (working copy)
@@ -465,7 +465,7 @@
                                 <flag>wxALL|wxEXPAND|wxALIGN_LEFT|wxALIGN_TOP</flag>
                                 <border>4</border>
                               </object>
-                              <label>Colouring andf highlighting options</label>
+                              <label>Colouring and highlighting options</label>
                               <orient>wxVERTICAL</orient>
                             </object>
                             <flag>wxALL|wxEXPAND|wxALIGN_LEFT|wxALIGN_TOP</flag>
Title: Re: XML based compilers
Post by: MortenMacFly on July 18, 2012, 08:04:51 am
Adding to the previous patch:
[...]
Nice, especially the "welcome" of the XML compiler.

BTW: Something that came into my mind: Did you try what happens if you create a copy of a compiler? Dos that still work? I'll try though... I am working with it permanently and so far I see no regressions... Just to let you know that I am not doing "nothing".

Generally @all: Did anybody already try this branch on Linux?
Title: Re: XML based compilers
Post by: MortenMacFly on July 19, 2012, 08:15:45 am
Looks good so far, I got a question: What Clang compiler do you use? Is there a specific one you use under Windows? (A distro?!)

Also, I've attached another compiler I found in my local copy. Its a specific Fortran compiler, but untested - that's why it is not committed. It should work though, I got this ones from "darmar" - the author of the FortranProjectPlugin. Do you have an easy way/tool to convert/add this? Or do I have to do it the monkey way?
Title: Re: XML based compilers
Post by: Alpha on July 19, 2012, 05:43:44 pm
I use a (slightly old) self compiled version of Clang from their trunk.  The Clang compiler I wrote has only been lightly tested (much if it is written based on pure documentation), so beware of errors.

Do you have an easy way/tool to convert/add this?
The absolute easiest way is to register it in compilergcc.cpp as a normal compiler, then (build and) launch Code::Blocks, open the compiler settings, create (and delete) a new flag for the compiler (to make it think it has changed), and click OK.  This will generate a fully usable file in your user data folder.
However, I usually uses my ListToolbox plugin, because with it, I can make the output look much nicer.  (Using ListToolbox takes about equivalent effort as formatting one of each type of entry because of the way it chains commands.)
However (2), you could just apply this patch ;) (it also contains a fix for the loading of copied compilers whose flags have been changed, and slightly more along the lines of auto-detection routines for pure XML compilers).

Did you try what happens if you create a copy of a compiler? Dos that still work?
It does work except that copied compilers do not load user customized sets of compiler flags; this fixes it.
Title: Re: XML based compilers
Post by: MortenMacFly on July 20, 2012, 08:09:18 pm
It does work except that copied compilers do not load user customized sets of compiler flags; this fixes it.
So... to summarise: It works pretty well and I don't see major drawbacks. It only needs testing on Linux.

What do you think is missing?
Title: Re: XML based compilers
Post by: Alpha on July 20, 2012, 11:31:25 pm
I would like to finish registry support and a couple other tweaks for pure XML compilers.  I also want to allow compilers to specify a 'weight' - which would be used to decide what order to display them in.  There are also some compilers with relatively uncomplicated auto-detect that I think can be swapped for pure XML.

From my perspective: Linux testing, and auto-detect testing (after I rewrite to XML the compilers I can).
Title: Re: XML based compilers
Post by: Jenna on July 20, 2012, 11:51:15 pm
From my perspective: Linux testing,
I just committed fixes to the linux project-file and the automake-system.

It's compilable and seems to work:
I used C::B compiled with automake to compile C::B with C::B and run it from inside C::B.

..., but I did no deeper testing.
Title: Re: XML based compilers
Post by: Alpha on July 21, 2012, 07:02:48 am
This patch completes the remaining features I currently have planned:

I would recommend that testers delete their current default.conf (or launch Code::Blocks with a new profile) because this branch does make a few (minor) changes to how the compilers save (which may, or may not interfere).

I just committed fixes to the linux project-file and the automake-system.
Sorry, I think this brakes the build system again.
Title: Re: XML based compilers
Post by: Jenna on July 21, 2012, 09:30:39 am
I just committed fixes to the linux project-file and the automake-system.
Sorry, I think this brakes the build system again.

No problem !

@MortenMacFly:
Should I do the linux fixes, or are you already working on it ?

Should be trivial, just deleting some files from src/plugins/compilergcc/Makefile.am and fixing the project-file for linux.
The new xml-files are handled automatically as long as they are in src/plugins/compilergcc/resources/compilers/ .
Title: Re: XML based compilers
Post by: Alpha on July 21, 2012, 04:30:17 pm
Maybe this is a slightly tidier way to fix PCH for compilerXML.cpp:
Code
Index: src/plugins/compilergcc/compilerXML.h
===================================================================
--- src/plugins/compilergcc/compilerXML.h (revision 8155)
+++ src/plugins/compilergcc/compilerXML.h (working copy)
@@ -3,7 +3,7 @@
 
 #include <compiler.h>
 
-#include <wx/string.h>
+class wxString;
 
 class CompilerXML : public Compiler
 {
@@ -28,7 +28,7 @@
             none
         };
 
-        bool AddPath(const wxString& pth, int sm, int rmDirs = 0);
+        bool AddPath(const wxString& pth, SearchMode sm, int rmDirs = 0);
 
         wxString m_fileName;
 };
Index: src/plugins/compilergcc/compilerXML.cpp
===================================================================
--- src/plugins/compilergcc/compilerXML.cpp (revision 8155)
+++ src/plugins/compilergcc/compilerXML.cpp (working copy)
@@ -1,10 +1,12 @@
 #include <sdk.h>
 
-#include <wx/arrstr.h>
-#include <wx/filefn.h>
-#include <wx/regex.h>
+#ifndef CB_PRECOMP
+    #include <wx/arrstr.h>
+    #include <wx/filefn.h>
+    #include <wx/regex.h>
+    #include <wx/xml/xml.h>
+#endif // CB_PRECOMP
 #include <wx/textfile.h>
-#include <wx/xml/xml.h>
 #ifdef __WXMSW__ // for wxRegKey
     #include <wx/msw/registry.h>
 #endif // __WXMSW__
@@ -243,7 +245,7 @@
     return adrGuessed;
 }
 
-bool CompilerXML::AddPath(const wxString& pth, int sm, int rmDirs)
+bool CompilerXML::AddPath(const wxString& pth, SearchMode sm, int rmDirs)
 {
     wxFileName fn(pth + wxFILE_SEP_PATH);
     fn.Normalize(wxPATH_NORM_ENV_VARS|wxPATH_NORM_DOTS);
Title: Re: XML based compilers
Post by: MortenMacFly on July 21, 2012, 07:38:33 pm
Maybe this is a slightly tidier way to fix PCH for compilerXML.cpp:
Code
-#include <wx/string.h>
+class wxString;
[...]
         wxString m_fileName;
That is wrong. You cannot use a fwd decl if you are using  an instance of wxString. Its only valid when using only const refs or pointers.

Code
Index: src/plugins/compilergcc/compilerXML.cpp
===================================================================
--- src/plugins/compilergcc/compilerXML.cpp (revision 8155)
+++ src/plugins/compilergcc/compilerXML.cpp (working copy)
@@ -1,10 +1,12 @@
 #include <sdk.h>
 
-#include <wx/arrstr.h>
-#include <wx/filefn.h>
-#include <wx/regex.h>
+#ifndef CB_PRECOMP
+    #include <wx/arrstr.h>
+    #include <wx/filefn.h>
+    #include <wx/regex.h>
+    #include <wx/xml/xml.h>
+#endif // CB_PRECOMP
 #include <wx/textfile.h>
-#include <wx/xml/xml.h>
 #ifdef __WXMSW__ // for wxRegKey
     #include <wx/msw/registry.h>
 #endif // __WXMSW__
@@ -243,7 +245,7 @@
     return adrGuessed;
 }
 
-bool CompilerXML::AddPath(const wxString& pth, int sm, int rmDirs)
+bool CompilerXML::AddPath(const wxString& pth, SearchMode sm, int rmDirs)
 {
     wxFileName fn(pth + wxFILE_SEP_PATH);
     fn.Normalize(wxPATH_NORM_ENV_VARS|wxPATH_NORM_DOTS);
...although this is correct. :-)

Will do.

Should I do the linux fixes, or are you already working on it ?
I would prefer you do it. for me it would be a "Blindflug" as I am not under Linux.
Title: Re: XML based compilers
Post by: oBFusCATed on July 21, 2012, 07:58:08 pm
Its only valid when using only const refs or pointers.
Partially true, because normal (non-const) references are forward declarable, too.
Title: Re: XML based compilers
Post by: Jenna on July 21, 2012, 08:02:57 pm
Should I do the linux fixes, or are you already working on it ?
I would prefer you do it. for me it would be a "Blindflug" as I am not under Linux.

Already done.
Title: Re: XML based compilers
Post by: Alpha on July 23, 2012, 05:10:08 am
Maybe this is a slightly tidier way to fix PCH for compilerXML.cpp:
Code
-#include <wx/string.h>
+class wxString;
[...]
         wxString m_fileName;
That is wrong. You cannot use a fwd decl if you are using  an instance of wxString.
I must have been working without actually thinking... I completely did not see that variable (though I do not know how, as I was the one who wrote it :-[).
Title: Re: XML based compilers
Post by: Alpha on July 28, 2012, 03:43:45 am
In this patch, converted to pure XML:

I cannot think of any major changes/additions (other than possibly converting a few more interfaces to pure XML) from my part to this branch that will occur in the near future, so I think this branch is ready for whatever the next step is (maybe a nightly?) to be considered for integration with the trunk.

(Again, I would recommend to anyone who is testing this that they start with a clean profile.)
Title: Re: XML based compilers
Post by: MortenMacFly on July 28, 2012, 04:03:36 pm
I cannot think of any major changes/additions (other than possibly converting a few more interfaces to pure XML)
I agree. The fact that you converted so many compiler already is also a proof f the concept. 8)
What's needed from my point of view is:
- testing under other platforms
- build test under Linux
- testing the use of macros for path's, executables etc...

I've merged  trunk into the branch so all devs/willing people can try this branch w/o loosing features from trunk.
Title: Re: XML based compilers
Post by: Alpha on July 31, 2012, 03:15:49 am
Compiler logging tweaks:
Code
Index: src/sdk/loggers.cpp
===================================================================
--- src/sdk/loggers.cpp (revision 8175)
+++ src/sdk/loggers.cpp (working copy)
@@ -340,7 +340,7 @@
     control->Thaw();
 }
 
-void ListCtrlLogger::Append(const wxArrayString& colValues, Logger::level lv)
+void ListCtrlLogger::Append(const wxArrayString& colValues, Logger::level lv, int autoSize)
 {
     if (!control)
         return;
@@ -353,6 +353,8 @@
     int idx = control->GetItemCount() - 1;
     for (size_t i = 1; i < colValues.GetCount(); ++i)
         control->SetItem(idx, i, colValues[i]);
+    if (autoSize != -1)
+        control->SetColumnWidth(autoSize, wxLIST_AUTOSIZE);
     control->Thaw();
 }
 
Index: src/plugins/compilergcc/compilergcc.cpp
===================================================================
--- src/plugins/compilergcc/compilergcc.cpp (revision 8175)
+++ src/plugins/compilergcc/compilergcc.cpp (working copy)
@@ -3382,7 +3382,9 @@
     wxArrayString errors;
     errors.Add(filename);
     errors.Add(line);
-    errors.Add(msg);
+    wxString msgFix = msg;
+    msgFix.Replace(wxT("\t"), wxT("    "));
+    errors.Add(msgFix);
 
     Logger::level lv = Logger::info;
     if (lt == cltError)
@@ -3390,8 +3392,7 @@
     else if (lt == cltWarning)
         lv = Logger::warning;
 
-    m_pListLog->Append(errors, lv);
-//    m_pListLog->GetListControl()->SetColumnWidth(2, wxLIST_AUTOSIZE);
+    m_pListLog->Append(errors, lv, 2);
 
     // add to error keeping struct
     m_Errors.AddError(lt, prj, filename, line.IsEmpty() ? 0 : atoi(wxSafeConvertWX2MB(line)), msg);
@@ -3642,6 +3643,8 @@
             LogWarningOrError(cltNormal, 0, wxEmptyString, wxEmptyString,
                               wxString::Format(_("=== Build finished: %s ==="), msg.wx_str()));
             SaveBuildLog();
+            if (!Manager::IsBatchBuild() && m_pLog->progress)
+                m_pLog->progress->SetValue(0);
         }
         else
         {
@@ -3665,9 +3668,6 @@
             Manager::Get()->ProcessEvent(evtSwitch);
 
             m_pListLog->FocusError(m_Errors.GetFirstError());
-            // Build is not completed, so clear the progress bar
-            if (m_pLog->progress)
-                m_pLog->progress->SetValue(0);
         }
         else
         {
Index: src/include/loggers.h
===================================================================
--- src/include/loggers.h (revision 8175)
+++ src/include/loggers.h (working copy)
@@ -136,7 +136,7 @@
     virtual void      CopyContentsToClipboard(bool selectionOnly = false);
     virtual void      UpdateSettings();
     virtual void      Append(const wxString& msg, Logger::level lv = info);
-    virtual void      Append(const wxArrayString& colValues, Logger::level lv = info);
+    virtual void      Append(const wxArrayString& colValues, Logger::level lv = info, int autoSize = -1);
     virtual size_t    GetItemsCount() const;
     virtual void      Clear();
     virtual wxWindow* CreateControl(wxWindow* parent);
Title: Re: XML based compilers
Post by: MortenMacFly on July 31, 2012, 06:44:28 am
@Jens: Do you / Did you try that "beast" under Linux already?
Title: Re: XML based compilers
Post by: Jenna on July 31, 2012, 07:59:40 am
@Jens: Do you / Did you try that "beast" under Linux already?

Yes, but no really deep tests.

At least it works fine to compile and debug C::B and the contrib-plugins and some other (makefile-based) projects (wxWidgets trunk with gtk2 and gtk3 and aui and notebooks samples).
All compiled with gcc, but makefile-based projects should not be changed (much?) anyway, I think.
Title: Re: XML based compilers
Post by: oBFusCATed on July 31, 2012, 08:40:54 am
Compiler logging tweaks:
  • Clear progress bar on abort
  • Remove tabs from build messages (so they do not print as squares)
  • Auto adjust build message size
Keep in mind that changing the interface of public for the SDK class requires an increase of the SDK version (the macros at the top of the cbplugin.h file).
Title: Re: XML based compilers
Post by: Alpha on July 31, 2012, 11:01:12 pm
Keep in mind that changing the interface of public for the SDK class requires an increase of the SDK version (the macros at the top of the cbplugin.h file).
Yes (and I would assume that there are multiple other changes in this branch that would also require an SDK version increment), however, as this is a branch, would it not be better to only have a single increment upon merging with the trunk?

Another logging tweak and the removal of a GCC 4.7 warning:
Code
Index: src/sdk/compiler.cpp
===================================================================
--- src/sdk/compiler.cpp (revision 8176)
+++ src/sdk/compiler.cpp (working copy)
@@ -1013,11 +1013,9 @@
         }
         else if (node->GetName() == wxT("RegEx"))
         {
-            CompilerLineType clt;
             wxString tp = node->GetAttribute(wxT("type"), wxEmptyString);
-            if (tp == wxT("normal"))
-                clt = cltNormal;
-            else if (tp == wxT("warning"))
+            CompilerLineType clt = cltNormal;
+            if (tp == wxT("warning"))
                 clt = cltWarning;
             else if (tp == wxT("error"))
                 clt = cltError;
Index: src/plugins/compilergcc/directcommands.cpp
===================================================================
--- src/plugins/compilergcc/directcommands.cpp (revision 8176)
+++ src/plugins/compilergcc/directcommands.cpp (working copy)
@@ -583,8 +583,8 @@
     if (!fileMissing.IsEmpty())
     {
         wxString warn;
-        warn.Printf(_("WARNING: Target '%s': Unable to resolve %d external dependencies:"),
-                    target->GetFullTitle().wx_str(), fileMissing.Count());
+        warn.Printf(_("WARNING: Target '%s': Unable to resolve %d external dependenc%s:"),
+                    target->GetFullTitle().wx_str(), fileMissing.Count(), wxString(fileMissing.Count() == 1 ? _("y") : _("ies")).wx_str());
         ret.Add(wxString(COMPILER_SIMPLE_LOG) + warn);
         for (size_t i=0; i<fileMissing.Count(); i++)
             ret.Add(wxString(COMPILER_SIMPLE_LOG) + fileMissing[i]);
Title: Re: XML based compilers
Post by: oBFusCATed on August 01, 2012, 12:05:47 am
Yes (and I would assume that there are multiple other changes in this branch that would also require an SDK version increment), however, as this is a branch, would it not be better to only have a single increment upon merging with the trunk?
For me it is better to increment the sdk version number with every break of the api, you can't be sure if there are users running your code and if they have custom plugins or not.
Title: Re: XML based compilers
Post by: MortenMacFly on August 01, 2012, 06:22:14 am
For me it is better to increment the sdk version number with every break of the api, you can't be sure if there are users running your code and if they have custom plugins or not.
Branches are for people knowing what they do. When merged, the SDK major version will increase anyways due to the massive change (as it was with the debugger re-factoring). But in principle you are right... ::)
Title: Re: XML based compilers
Post by: MortenMacFly on August 02, 2012, 11:29:24 am
Al-right, concerning the next movement/discussion between devs, here is something to share. Please continue all discussion on that topic here, not via PM.

Quote from: MortenMacFly
Maybe to clarify a few bits:
You have to differ between compiler settings like flags. Those are in XML files in C::B\share\CodeBlocks\compilers. Those are nicely grouped, i.e. common settings between compilers are shared.

If you change compiler flags (like adding a compiler flag in the compiler options) a copy of that XML file is created in C::B\share\CodeBlocks\compilers which superseeds the old one, but also includes ALL shared (common) parts.

Then you have user settings, like concrete path's which are mapped to the compiler's global settings available but are an "instance". Those are in default.conf and remain there (and belong there, too).

BTW:  Alpha did a very nice WiKi article explaining the concept here:
http://wiki.codeblocks.org/index.php?title=Compiler_options_file

I meanwhile found the following issues: Compiler ID's have indeed changed for:
armelfgcc -> arm_elf_gcc
avrgcc -> avr_gcc
msp430gcc -> msp430_gcc
ppcgcc -> ppc_gcc
tricoregcc -> tricore_gcc

I think I know why, because in the initial algorithm to compile the ID there was a comment stating that to make a compiler ID valid XML all underscores need to be removed. This is wrong (however), so Alpha asked me to remove that check and I said yes. That's why now there is an underscore. Actually this is more correct, but causes trouble. The solution here is, also to check for the old style, when reading, but write the new style. (to project files).
Title: Re: XML based compilers
Post by: MortenMacFly on August 02, 2012, 11:54:21 am
One issue, that is indeed true (and causes missing compiler flags) is the use of "-dumpversion" instead of "--version" for the compiler version detection regex. The result of dumpversion is 4.7 (so only major and minor), while --version shows more, e.g. 4.7.2 (including revision). Thats why the RegEx fails and therefore some compiler flags are missing.

So either we use --version, or make the RegEx just check for major.minor (which should be enough IMHO), or call both - the --version thing as a fallback.

Preferred solution?
Title: Re: XML based compilers
Post by: MortenMacFly on August 02, 2012, 12:15:05 pm
There is indeed a bug: If you make a copy of a compiler, the settings for the new compiler become applied to the last compiler already existing in the list.
I think the reason is here:
Code
void Compiler::ReloadOptions()
{
    if (ConfigManager::LocateDataFile(wxT("compilers/options_") + GetID() + wxT(".xml"), sdDataUser | sdDataGlobal).IsEmpty())
        return; // Do not clear if the options cannot be reloaded
    m_Options.ClearOptions();
    LoadDefaultOptions(GetID());
    LoadDefaultRegExArray();
}
That fails, because there is no XML file with such a name (yet) as it is a user-defined compiler. So this should better check for the parent compiler's ID in addition.
Title: Re: XML based compilers
Post by: Alpha on August 02, 2012, 03:19:03 pm
For the curious, I also started documenting the file format for defining pure XML compilers (http://wiki.codeblocks.org/index.php?title=Compiler_file) (page is only partially complete).

I am looking into these issues, and should be back with solutions soon.
Title: Re: XML based compilers
Post by: killerbot on August 02, 2012, 03:21:58 pm
don't use --version ==> for cross compilers this can become a mess, causing such regexes to fail very quickly.
-dumpversion, and just major/minor seems ok for me.
Title: Re: XML based compilers
Post by: Alpha on August 02, 2012, 11:35:29 pm
This patch should (I cannot be certain, because I only tested it on Windows):

Copied pure XML compilers currently act exactly like copied built-in compilers (or at least, they are supposed to).  The compiler_*.xml files are never duplicated, however, the user data folder is scanned for them in case the user created a local compiler.
In the future, I may create a wizard to allow for (semi-)graphical creation of a fully fledged (XML) compiler (assuming there is any interest in such a feature).
Title: Re: XML based compilers
Post by: MortenMacFly on August 03, 2012, 06:49:26 am
This patch should (I cannot be certain, because I only tested it on Windows): [...]
Thanks for the fast response! I had a a look into it and I see this snippet in compiler.cpp (and compilerfactory.cpp):
     {
        tmp.Replace(wxT("-"), wxEmptyString); // try again using previous id format
        if (!cfg->Exists(tmp + _T("/name")))
            return;
    }

Didn't you mean to replace "_" (underscore) with "" (empty)? Here, you replace "-" (minus), but IMHO that id not present in the IDs anyways...?! (See my previous post about the ID's). Why do you do so?
Title: Re: XML based compilers
Post by: killerbot on August 03, 2012, 09:18:38 am
build is broken :

Code
AutoDetectResult CompilerLCC::AutoDetectInstallationDir()
{
#ifdef __WXMSW__

...

    wxString compiler; compiler << wxFILE_SEP_PATH << _T("bin") << wxFILE_SEP_PATH << m_Programs.C;

...


#endif // __WXMSW__

...

    return wxFileExists(m_MasterPath+compiler) ? adrDetected : adrGuessed;

==> on linux : compiler does NOT exist, since declared in the ifdef part !!!
Title: Re: XML based compilers
Post by: MortenMacFly on August 03, 2012, 09:47:52 am
build is broken :
Try again after SVN update.
Title: Re: XML based compilers
Post by: Jenna on August 03, 2012, 11:26:32 am
Another build fix is in svn (missing symbols in libcompiler.so).

The checkbox to allow non-platform compilers needs a restart to have an effect.
This should be stated in the label, or if possible the compiler should be reloaded automatically.
Title: Re: XML based compilers
Post by: MortenMacFly on August 03, 2012, 11:57:46 am
The checkbox to allow non-platform compilers needs a restart to have an effect.
Maybe to clarify, where this comes from: I recently discussed with Alpha the following:
Quote from: MortenMacFly
Something else in between: I read this:
http://www.orbiterwiki.org/wiki/Free_Compiler_Setup_Under_Linux

And I had an idea: Whats missing for the compiler but would really be useful for cross compiling is that you could setup via Config/API (!) if a compiler is attached/active under the current platform. [...]
But yes, the restart should be clarified.
Edit: Done.
Title: Re: XML based compilers
Post by: Alpha on August 04, 2012, 06:26:02 am
Didn't you mean to replace "_" (underscore) with "" (empty)? Here, you replace "-" (minus), but IMHO that id not present in the IDs anyways...?! (See my previous post about the ID's). Why do you do so?
Although "-" (minus) is valid as an XML element name, TiXml apparently decides that they should be synonymous with, and written as "_" (underscores).  The change I made to the ID creation algorithm allows "-" to be used internally (I mostly did this for the cosmetic purposes of naming the XML files: options_arm-elf-gcc.xml is more recognizable than options_armelfgcc.xml, which is what it would have been under the previous system).

This should be stated in the label, or if possible the compiler should be reloaded automatically.
This patch causes the compilers to be reloaded (if chkNonPlatComp changed) when the settings dialog is dismissed; I did not do this at first because I was worried about possible side effects, however it seemed to function as expected in my few test cases.

(Sorry about the build issues.)
Title: Re: XML based compilers
Post by: Jenna on August 04, 2012, 03:14:46 pm
Applied in the branch, with one little change:
if abuild is active the settings will not be applied, because unloading and loading compilers might lead to problems.
Title: Re: XML based compilers
Post by: Alpha on August 04, 2012, 09:54:56 pm
  • Switch the right-click note to a tool tip (this is less obtrusive, but should still alert the user to the editing capability)
By the way, am I correct in assuming the revert of this means that a visible note is preferred over a tool tip?
Title: Re: XML based compilers
Post by: killerbot on August 05, 2012, 11:16:24 am
when working on the silencing of some warnings, I did the following.

I added for example on the GLOBAL compiler level some directives (-std=c++0x, or -fpremissive, or ...)

Then I rebuild CB with CB, the new directives take effect, but then then generate new unwanted warnings, because then they are also provided to gcc, in case its building c sources (like the depslib), which made me think. Shouldn't those compiler options be categorized (C, C++, both) , and have our system being smart enough that when he sees a C++ directive, but the current project is C, that he does not apply it ?

What do you think ?
Title: Re: XML based compilers
Post by: Alpha on August 05, 2012, 02:30:33 pm
For sorting these flags, I think there are several options:

My personal preference is the first option.  Which option do you (and anyone else who would like to voice their opinion) recommend I pursue?  (Is there another choice I have not considered yet?)
Title: Re: XML based compilers
Post by: killerbot on August 05, 2012, 02:43:02 pm
option 1 is for sure a start. Might create some maintenance problems, but we could start that way.
Title: Re: XML based compilers
Post by: oBFusCATed on August 05, 2012, 08:13:04 pm
Or the best options (three) - separate the c and c++ compilers in different entries/compilers :)
Title: Re: XML based compilers
Post by: MortenMacFly on August 06, 2012, 07:09:36 am
By the way, am I correct in assuming the revert of this means that a visible note is preferred over a tool tip?
For such important things, yes. Most people don'T wait long enough that the tooltip shows. Also, there are platforms where there is no tooltip. So tooltip should really be a "tip", not to worry about if you don't know it.
Title: Re: XML based compilers
Post by: MortenMacFly on August 06, 2012, 07:13:08 am
Or the best options (three) - separate the c and c++ compilers in different entries/compilers :)
Why would you do that? MSVC for example is one compiler for all. We shouldn't split up things like that. I think grouping visually is fair enough. And event hat I believe won't be used by most of the people. To be honest to me (and only me) it sounds a bit like an overkill. But maybe just because I fail to see the benefit for me (again: only for me).,
Title: Re: XML based compilers
Post by: Alpha on August 06, 2012, 01:39:29 pm
I am working on a patch to deal with C-only and C++-only flags; when I complete it, actually seeing its results will probably better explain it than I can by writing a description.

By the way, am I correct in assuming the revert of this means that a visible note is preferred over a tool tip?
For such important things, yes. Most people don'T wait long enough that the tooltip shows. Also, there are platforms where there is no tooltip. So tooltip should really be a "tip", not to worry about if you don't know it.
In that case, the tip is redundant (this also removes the double separator from the context menu when right-clicking on the name of a project).
Code
Index: src/plugins/compilergcc/compilergcc.cpp
===================================================================
--- src/plugins/compilergcc/compilergcc.cpp (revision 8193)
+++ src/plugins/compilergcc/compilergcc.cpp (working copy)
@@ -563,7 +563,9 @@
     else if (data && data->GetKind() == FileTreeData::ftdkProject)
     {
         // popup menu on a project
-        menu->AppendSeparator();
+        wxMenuItem* itm = menu->FindItemByPosition(menu->GetMenuItemCount() - 1);
+        if (itm && !itm->IsSeparator())
+            menu->AppendSeparator();
         menu->Append(idMenuCompileFromProjectManager, _("Build"));
         menu->Append(idMenuRebuildFromProjectManager, _("Rebuild"));
         menu->Append(idMenuCleanFromProjectManager,   _("Clean"));
Index: src/plugins/compilergcc/resources/compiler_options.xrc
===================================================================
--- src/plugins/compilergcc/resources/compiler_options.xrc (revision 8193)
+++ src/plugins/compilergcc/resources/compiler_options.xrc (working copy)
@@ -150,7 +159,6 @@
  </object>
  <object class="sizeritem">
  <object class="wxCheckListBox" name="lstCompilerOptions">
- <tooltip>Right-click to setup or edit compiler flags</tooltip>
  <style>wxLB_HSCROLL</style>
  </object>
  <flag>wxTOP|wxLEFT|wxRIGHT|wxEXPAND|wxALIGN_LEFT|wxALIGN_TOP</flag>
Index: src/plugins/compilergcc/resources/compilers/compiler_clang.xml
===================================================================
--- src/plugins/compilergcc/resources/compilers/compiler_clang.xml (revision 8193)
+++ src/plugins/compilergcc/resources/compilers/compiler_clang.xml (working copy)
@@ -7,6 +7,8 @@
         <Search envVar="PATH"
                 for="C"/>
         <if platform="windows">
+            <Search path="%ProgramFiles%\LLVM"
+                    for="C"/>
             <Fallback path="C:\MinGW"/>
         </if>
         <else>
Title: Re: XML based compilers
Post by: oBFusCATed on August 06, 2012, 02:03:06 pm
Why would you do that?
Because it will be easier in the longer period of time. You won't have to worry if the options is C or C++.
Another benefit is that this will bring us one step closer to multy-language/multi-compiler support in a single project.

MSVC for example is one compiler for all.
In fact MSVC is MSVC++, I doubt there is any support for C provided by Microsoft, and if I remember correctly, they stated that C11 won't be supported by them.

Edit: I'm talking about the options added in "compiler->other options", because  you can't add "-Werror=return-type" in the normal options. So thinking about this again, splitting the compilers in C and C++ ones is the only available option.
Title: Re: XML based compilers
Post by: MortenMacFly on August 06, 2012, 02:16:13 pm
multy-language/multi-compiler support in a single project.
Using different targets and custom build commands we have that already - and I am using it.
But if you believe there is an extra benefit - I won't mind as long as I don't have to open twice as much settings dialogs for a mixed C/C++ project.

I doubt there is any support for C provided by Microsoft,
Oh dear... I thought the Windows core was still plain C - it seems it isn't... or they are using another compiler for it.
Title: Re: XML based compilers
Post by: Alpha on August 07, 2012, 05:30:21 am
Here is a patch with an initial implementation for the first option I described (it is functional, but has poor API and is not user configurable).  This patch also includes the minor changes from my last post and another logging tweak.
Title: Re: XML based compilers
Post by: MortenMacFly on August 07, 2012, 07:36:38 pm
Here is a patch [...]
Giving it a try< leads to tons of errors at startup of C::B, that compiler_options_sort.xml could not be fund. Why? Did I forget to copy / create a file?
Title: Re: XML based compilers
Post by: Alpha on August 07, 2012, 07:54:44 pm
Strange... are you certain that was the name it warned about?  I did add options_common_sort.xml; try running update(.bat) again to see if that fixes it.

(Speaking of which, I forgot to add the file to the .cbp's.)
Title: Re: XML based compilers
Post by: Alpha on August 09, 2012, 06:00:16 am
Added/changed:

Giving it a try< leads to tons of errors at startup of C::B, that compiler_options_sort.xml could not be fund.
By the way, has this been resolved?
Title: Re: XML based compilers
Post by: MortenMacFly on August 09, 2012, 06:15:21 am
Strange... are you certain that was the name it warned about?  I did add options_common_sort.xml; try running update(.bat) again to see if that fixes it.
Oh - I missed hat post. And yes: That was it I figured it out myself when updating from SVN.
Title: Re: XML based compilers
Post by: Alpha on August 10, 2012, 06:57:22 am
On the compiler toolbar, the label "Build target:" takes up a lot of space, and no other toolbar has any such label.  Does anyone have a reason why this text should not be removed (switched to a tooltip)?
Title: Re: XML based compilers
Post by: Alpha on August 14, 2012, 10:23:54 pm
This patch makes first run a little more friendly, and removes the static text from the compiler toolbar.

(By the way, I just got a blue screen of death... in safemode.  So I have decided to try Linux again.  When I use Autotools to build Code::Blocks, the resulting executable can be run by clicking on it.  If I use Code::Blocks to build Code::Blocks, the green run button works, however manual execution of Code::Blocks fails due to not finding libcodeblocks.so.  Is there another solution besides the one in the wiki (http://wiki.codeblocks.org/index.php?title=Installing_Code::Blocks_from_source_on_Linux#Resolving_runtime_issues) - which does not seem to quite apply to me - that I could try?  I am on Ubuntu.)
Title: Re: XML based compilers
Post by: oBFusCATed on August 14, 2012, 10:31:59 pm
There is run.sh script which can be used to start c::b
Title: Re: XML based compilers
Post by: Alpha on August 14, 2012, 10:45:09 pm
Thanks.
Title: Re: XML based compilers
Post by: MortenMacFly on August 22, 2012, 09:29:51 pm
BTW: shall we start another wave with nightly now? I think all complaints should be resolved now.

I'll do another merge from trunk... just in case Lieven has some spare time... ;-)

Edit: BTW: I still would prefer to have one one nightly this time - the one of the branch... stated clearly as major improvement / branch version.
Title: Re: XML based compilers
Post by: Alpha on August 23, 2012, 02:49:36 am
Green light from me :).

(Maybe plus this tiny patch.)
Code
Index: src/plugins/compilergcc/compilergcc.cpp
===================================================================
--- src/plugins/compilergcc/compilergcc.cpp (revision 8246)
+++ src/plugins/compilergcc/compilergcc.cpp (working copy)
@@ -2476,7 +2476,7 @@
                 }
                 wxString msg;
                 msg.Printf(_T("\"%s - %s\": The compiler's setup %sis invalid, so Code::Blocks cannot find/run the compiler.\n")
-                           _T("Probably the toolchain path within the compiler options is not setup correctly?!\n")
+                           _T("Probably the toolchain path within the compiler options is not setup correctly?! (Do you have a compiler installed?)\n")
                            _T("Goto \"Settings->Compiler and debugger...->Global compiler settings->%s->Toolchain executables\"")
                            _T(" and fix the compiler's setup.\n")
                            _T("Skipping..."),
Index: src/plugins/compilergcc/resources/compilers/options_gdc.xml
===================================================================
--- src/plugins/compilergcc/resources/compilers/options_gdc.xml (revision 8246)
+++ src/plugins/compilergcc/resources/compilers/options_gdc.xml (working copy)
@@ -81,7 +81,7 @@
         <Option name="allow deprecated features"
                 option="-fdeprecated"/>
         <Option name="compile in debug code"
-                option="-debug"/>
+                option="-fdebug"/>
         <Option name="inline expand functions"
                 option="-finline-functions"/>
         <Option name="compile release version, which means not generating code for contracts and asserts"
Index: src/plugins/compilergcc/resources/compilers/options_common_re.xml
===================================================================
--- src/plugins/compilergcc/resources/compilers/options_common_re.xml (revision 8246)
+++ src/plugins/compilergcc/resources/compilers/options_common_re.xml (working copy)
@@ -203,4 +203,10 @@
            msg="1">
         <![CDATA[([Ii]nfo:[ \t].*)\(auto-import\)]]>
     </RegEx>
+    <RegEx name="Linker warning (different sized sections)"
+           type="warning"
+           msg="2"
+           file="1">
+        <![CDATA[([][{}() \t#%$~[:alnum:]&_:+/\.-]+):[ \t]+(duplicate section.*has different size)]]>
+    </RegEx>
 </CodeBlocks_compiler_options>
Title: Re: XML based compilers
Post by: killerbot on August 23, 2012, 07:35:00 am
ok for the nightly, but let's be sure it works ok if we send out just one nightly.
Title: Re: XML based compilers
Post by: killerbot on August 28, 2012, 09:16:39 pm
I had a nightly ready, but during testing on linux I found a showstopper, the 2 latest compiler option/directives are still missing.
They are present in some (did not check all of them) gcc derivatives/ports (like gnu arm), but are missing in the main gcc.
I am talking about :
 - std=c++11
 - Wzero-as-null-pointer-constant

How can we be sure not other options have disappeared ?

Next to the issue, I have a question what advice to we want to give wrt "default.conf",
We will ask the user to backup, but then what, do they need to remove something out of it (the compiler section, but for sure not the user sets) ?


EDIT : can the compiler detection at startup be explained, it made no sense to me.

Pre-suported compilers (which were absent) were now user defined ??
Other were grey-ed out.
Some where red ?

I don't get it, for ther majority of them I would have expected the same thing.
Title: Re: XML based compilers
Post by: Jenna on August 28, 2012, 10:12:53 pm
My build is not the newest one (I have 8218), and the two options you have mentioned are there.

I start the xml-branch with an own personality and do not use the default.conf of trunk, maybe that's the reason.
Title: Re: XML based compilers
Post by: Alpha on August 28, 2012, 11:39:48 pm
I had a nightly ready, but during testing on linux I found a showstopper, the 2 latest compiler option/directives are still missing.
In share\CodeBlocks\compilers (user, not global), XML files will be saved for any compilers to which flags have been changed/added/removed.  It is possible options_gcc.xml was generated during a previous test before the version detection was fixed.  These files are loaded preferentially, so either deleting it manually, or resetting the compiler should resolve the problem.


EDIT : can the compiler detection at startup be explained, it made no sense to me.
The ones labeled "user defined" mean that they were not detected, but the master path loaded from default.conf is different than that given by auto-detection (this happens either when the user has set their own path, or when auto-detection has changed defaults).
Grey-ed out means the compiler was simply not detected (but the master path loaded from default.conf is the same as auto-detection).
Red means that the master path was empty before auto-detection began.  The detection dialog will only be shown if at least one compiler has an empty path.  The purpose is to alert the user that the highlighted compiler(s) are invalid and (may) require attention.  (I have another idea to more accurately mark this - currently new compilers are incorrectly marked red as well; I should have a small patch for this soon.)
Title: Re: XML based compilers
Post by: Alpha on August 29, 2012, 01:02:57 am
We will ask the user to backup, but then what, do they need to remove something out of it (the compiler section, but for sure not the user sets) ?
In my opinion, it should be fine to continue working with the same (unmodified) default.conf (of course, it still should be backed up just in case).
Title: Re: XML based compilers
Post by: MortenMacFly on August 29, 2012, 06:26:10 am
My build is not the newest one (I have 8218), and the two options you have mentioned are there.
For me, too - both options are there.
Title: Re: XML based compilers
Post by: killerbot on August 29, 2012, 07:49:20 am
I deleted the "output" folder again, and rerun "update" script, optiotns still don't show up in "options_gcc.xml"

Here's my content of it :
Code
<?xml version="1.0"?>
<!DOCTYPE CodeBlocks_compiler_options>
<CodeBlocks_compiler_options>
    <if platform="windows">
        <Program name="C"         value="mingw32-gcc.exe"/>
        <Program name="CPP"       value="mingw32-g++.exe"/>
        <Program name="LD"        value="mingw32-g++.exe"/>
        <Program name="DBGconfig" value="gdb_debugger:Default"/>
        <Program name="LIB"       value="ar.exe"/>
        <Program name="WINDRES"   value="windres.exe"/>
        <Program name="MAKE"      value="mingw32-make.exe"/>
    </if>
    <else>
        <Program name="C"         value="gcc"/>
        <Program name="CPP"       value="g++"/>
        <Program name="LD"        value="g++"/>
        <Program name="DBGconfig" value="gdb_debugger:Default"/>
        <Program name="LIB"       value="ar"/>
        <Program name="WINDRES"   value=""/>
        <Program name="MAKE"      value="make"/>
    </else>

    <Switch name="includeDirs"             value="-I"/>
    <Switch name="libDirs"                 value="-L"/>
    <Switch name="linkLibs"                value="-l"/>
    <Switch name="defines"                 value="-D"/>
    <Switch name="genericSwitch"           value="-"/>
    <Switch name="objectExtension"         value="o"/>
    <Switch name="needDependencies"        value="true"/>
    <Switch name="forceCompilerUseQuotes"  value="false"/>
    <Switch name="forceLinkerUseQuotes"    value="false"/>
    <Switch name="logging"                 value="default"/>
    <Switch name="libPrefix"               value="lib"/>
    <Switch name="libExtension"            value="a"/>
    <Switch name="linkerNeedsLibPrefix"    value="false"/>
    <Switch name="linkerNeedsLibExtension" value="false"/>
    <Switch name="supportsPCH"             value="true"/>
    <Switch name="PCHExtension"            value="h.gch"/>
    <Switch name="UseFullSourcePaths"      value="true"/>

    <!-- Summary of GCC options: http://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html -->

    <Option name="Produce debugging symbols"
            option="-g"
            category="Debugging"
            checkAgainst="-O -O1 -O2 -O3 -Os"
            checkMessage="You have optimizations enabled. This is Not A Good Thing(tm) when producing debugging symbols..."
            supersedes="-s"/>
    <if platform="windows">
        <Option name="Profile code when executed"
                option="-pg"
                category="Profiling"
                additionalLibs="-pg -lgmon"
                supersedes="-s"/>
    </if>
    <else>
        <Option name="Profile code when executed"
                option="-pg"
                category="Profiling"
                additionalLibs="-pg"
                supersedes="-s"/>
    </else>

    <!-- warnings -->
    <Common name="warnings"/>
    <Category name="Warnings">
        <Option name="Enable Effective-C++ warnings (thanks Scott Meyers)"
                option="-Weffc++"/>
        <Option name="Warn whenever a switch statement does not have a default case"
                option="-Wswitch-default"/>
        <Option name="Warn whenever a switch statement has an index of enumerated type and lacks a case for one or more of the named codes of that enumeration"
                option="-Wswitch-enum"/>
        <if exec="C -dumpversion"
            regex="^[4-9]\.[0-9]"
            default="true">
            <Option name="Warn if a user supplied include directory does not exist"
                    option="-Wmissing-include-dirs"/>
        </if>
        <Option name="Warn if a global function is defined without a previous declaration"
                option="-Wmissing-declarations"/>
        <Option name="Warn if the compiler detects that code will never be executed"
                option="-Wunreachable-code"/>
        <Option name="Warn if a function can not be inlined and it was declared as inline"
                option="-Winline"/>
        <Option name="Warn if floating point values are used in equality comparisons"
                option="-Wfloat-equal"/>
        <Option name="Warn if an undefined identifier is evaluated in an '#if' directive"
                option="-Wundef"/>
        <Option name="Warn whenever a pointer is cast such that the required alignment of the target is increased"
                option="-Wcast-align"/>
        <Option name="Warn if anything is declared more than once in the same scope"
                option="-Wredundant-decls"/>
        <Option name="Warn about unitialized variables which are initialized with themselves"
                option="-Winit-self"/>
        <Option name="Warn whenever a local variable shadows another local variable, parameter or global variable or whenever a built-in function is shadowed"
                option="-Wshadow"/>
    </Category>

    <!-- optimization -->
    <Common name="optimization"/>
    <Option name="Don't keep the frame pointer in a register for functions that don't need one"
            option="-fomit-frame-pointer"
            category="Optimization"
            checkAgainst="-g -ggdb"
            checkMessage="You have debugging symbols enabled. This is Not A Good Thing(tm) when optimizing..."/>

    <!-- machine dependent options - cpu arch -->
    <Common name="architecture"/>

    <Command name="CompileObject"
             value="$compiler $options $includes -c $file -o $object"/>
    <Command name="GenDependencies"
             value="$compiler -MM $options -MF $dep_object -MT $object $includes $file"/>
    <Command name="CompileResource"
             value="$rescomp $res_includes -J rc -O coff -i $file -o $resource_output"/>
    <Command name="LinkConsoleExe"
             value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs"/>
    <if platform="windows">
        <Command name="LinkNative"
                 value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs -Wl,--subsystem,native"/>
        <Command name="LinkExe"
                 value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs -mwindows"/>
        <Command name="LinkDynamic"
                 value="$linker -shared -Wl,--output-def=$def_output -Wl,--out-implib=$static_output -Wl,--dll $libdirs $link_objects $link_resobjects -o $exe_output $link_options $libs"/>
    </if>
    <else>
        <Command name="LinkNative"
                 value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs"/>
        <Command name="LinkExe"
                 value="$linker $libdirs -o $exe_output $link_objects $link_resobjects $link_options $libs"/>
        <Command name="LinkDynamic"
                 value="$linker -shared $libdirs $link_objects $link_resobjects -o $exe_output $link_options $libs"/>
    </else>
    <Command name="LinkStatic"
             value="$lib_linker -r -s $static_output $link_objects"/>
    <Common name="cmds"/>

    <Common name="re"/>

    <Common name="sort"/>
</CodeBlocks_compiler_options>
Title: Re: XML based compilers
Post by: killerbot on August 29, 2012, 07:56:20 am
ok I think I found out why there's a problem . From options_common_warnings.xml :

Code
        <if exec="C -dumpversion"
            regex="^4\.[7-9]|^[5-9]\.[0-9]"
            default="true">
            <Option name="Have g++ follow the C++11 ISO C++ language standard"
                    option="-std=c++11"
                    supersedes="-std=c++98 -std=c++0x"/>
            <Option name="zero as null pointer constant"
                    option="-Wzero-as-null-pointer-constant"/>
        </if>

Now if we do :

Code
killerbot@XIII:~/CodeBlocks/xmlcompiler/xml_compiler/src> gcc -dumpversion
4.6
killerbot@XIII:~/CodeBlocks/xmlcompiler/xml_compiler/src> gcc-4.7 -dumpversion
4.7
killerbot@XIII:~/CodeBlocks/xmlcompiler/xml_compiler/src>
You can see that the system compiler is still gcc.
HOWEVER in my existing CB (so in default.conf), we had/have :
Code
			<gcc>
<NAME>
<str>
<![CDATA[GNU GCC Compiler]]>
</str>
</NAME>
<MASTER_PATH>
<str>
<![CDATA[/usr]]>
</str>
</MASTER_PATH>
<C_COMPILER>
<str>
<![CDATA[gcc-4.7]]>
</str>
</C_COMPILER>
<CPP_COMPILER>
<str>
<![CDATA[g++-4.7]]>
</str>
</CPP_COMPILER>
<LINKER>
<str>
<![CDATA[g++-4.7]]>
</str>
</LINKER>
</gcc>

Meaning that either the info in default.conf get's ignored, or do we just have a one time migration problem, and in the latter case can it be avoided ?
Title: Re: XML based compilers
Post by: Alpha on August 29, 2012, 02:45:29 pm
I deleted the "output" folder again, and rerun "update" script, optiotns still don't show up in "options_gcc.xml"
The folder ~/.codeblocks/share/codeblocks/compilers is the one you want to clean.

Meaning that either the info in default.conf get's ignored, or do we just have a one time migration problem, and in the latter case can it be avoided ?
However, I think you are correct and have found a bug; I will have to double check, but at the point in time when the compiler is executed to retrieve the version number, it *might* not have yet loaded non-default names from the user configuration file.  I will look into this.
Title: Re: XML based compilers
Post by: Alpha on August 30, 2012, 04:27:41 am
This patch should resolve the above issues.
Code
Index: src/sdk/autodetectcompilers.cpp
===================================================================
--- src/sdk/autodetectcompilers.cpp (revision 8275)
+++ src/sdk/autodetectcompilers.cpp (working copy)
@@ -44,25 +44,12 @@
         list->InsertColumn(0, _("Compiler"), wxLIST_FORMAT_LEFT, 380);
         list->InsertColumn(1, _("Status"),   wxLIST_FORMAT_LEFT, 100);
 
-        bool firstRun = true;
         for (size_t i = 0; i < CompilerFactory::GetCompilersCount(); ++i)
         {
             Compiler* compiler = CompilerFactory::GetCompiler(i);
             if (!compiler)
                 continue;
-            if (!compiler->GetMasterPath().IsEmpty())
-            {
-                firstRun = false; // all master paths are empty on first run
-                break;
-            }
-        }
 
-        for (size_t i = 0; i < CompilerFactory::GetCompilersCount(); ++i)
-        {
-            Compiler* compiler = CompilerFactory::GetCompiler(i);
-            if (!compiler)
-                continue;
-
             list->InsertItem(list->GetItemCount(), compiler->GetName());
 
             wxString path = compiler->GetMasterPath();
@@ -71,7 +58,7 @@
 
             int idx = list->GetItemCount() - 1;
             int highlight = 0;
-            if (path.IsEmpty() && !firstRun)
+            if (path.IsEmpty() && Manager::Get()->GetConfigManager(wxT("compiler"))->Exists(wxT("/sets/") + compiler->GetID() + wxT("/name")))
             {
                 // Here, some user-interaction is required not to show this
                 // dialog again on each new start-up of C::B.
Index: src/sdk/compiler.cpp
===================================================================
--- src/sdk/compiler.cpp (revision 8275)
+++ src/sdk/compiler.cpp (working copy)
@@ -814,6 +814,16 @@
     int depth = 0;
     wxString categ;
     bool exclu = false;
+
+    wxString baseKey = GetParentID().IsEmpty() ? wxT("/sets") : wxT("/user_sets");
+    ConfigManager* cfg = Manager::Get()->GetConfigManager(wxT("compiler"));
+    wxString cmpKey;
+    cmpKey.Printf(wxT("%s/set%3.3d"), baseKey.c_str(), CompilerFactory::GetCompilerIndex(this) + 1);
+    if (!cfg->Exists(cmpKey + wxT("/name")))
+        cmpKey.Printf(wxT("%s/%s"), baseKey.c_str(), m_ID.c_str());
+    if (!cfg->Exists(cmpKey + wxT("/name")))
+        cmpKey.Replace(wxT("-"), wxEmptyString);
+
     while (node)
     {
         const wxString value = node->GetAttribute(wxT("value"), wxEmptyString);
@@ -837,19 +847,19 @@
         {
             wxString prog = node->GetAttribute(wxT("name"), wxEmptyString);
             if (prog == wxT("C"))
-                m_Programs.C = value;
+                m_Programs.C       = cfg->Read(cmpKey + wxT("/c_compiler"),   value);
             else if (prog == wxT("CPP"))
-                m_Programs.CPP = value;
+                m_Programs.CPP     = cfg->Read(cmpKey + wxT("/cpp_compiler"), value);
             else if (prog == wxT("LD"))
-                m_Programs.LD = value;
+                m_Programs.LD      = cfg->Read(cmpKey + wxT("/linker"),       value);
             else if (prog == wxT("DBGconfig"))
                 m_Programs.DBGconfig = value;
             else if (prog == wxT("LIB"))
-                m_Programs.LIB = value;
+                m_Programs.LIB     = cfg->Read(cmpKey + wxT("/lib_linker"),   value);
             else if (prog == wxT("WINDRES"))
-                m_Programs.WINDRES = value;
+                m_Programs.WINDRES = cfg->Read(cmpKey + wxT("/res_compiler"), value);
             else if (prog == wxT("MAKE"))
-                m_Programs.MAKE = value;
+                m_Programs.MAKE    = cfg->Read(cmpKey + wxT("/make"),         value);
         }
         else if (node->GetName() == wxT("Switch"))
         {
Title: Re: XML based compilers
Post by: Alpha on September 02, 2012, 09:37:07 pm
I had a nightly ready, but during testing on linux I found a showstopper, the 2 latest compiler option/directives are still missing.
They are present in some (did not check all of them) gcc derivatives/ports (like gnu arm), but are missing in the main gcc.
I am talking about :
 - std=c++11
 - Wzero-as-null-pointer-constant
I forgot to ask, has the last patch fixed this on your computer?
Title: Re: XML based compilers
Post by: killerbot on September 03, 2012, 07:36:21 am
testing this again this evening
Title: Re: XML based compilers
Post by: killerbot on September 03, 2012, 08:18:21 pm
good and bad news.

The 2 compiler options remain in pace.

However it still breaks existing configuration of compilers. I have told CB that gcc, is to be gcc-4.7.
Aka that means in default.conf the following :
Code
			<gcc>
<NAME>
<str>
<![CDATA[GNU GCC Compiler]]>
</str>
</NAME>
<MASTER_PATH>
<str>
<![CDATA[/usr]]>
</str>
</MASTER_PATH>
<C_COMPILER>
<str>
<![CDATA[gcc-4.7]]>
</str>
</C_COMPILER>
<CPP_COMPILER>
<str>
<![CDATA[g++-4.7]]>
</str>
</CPP_COMPILER>
<LINKER>
<str>
<![CDATA[g++-4.7]]>
</str>
</LINKER>
</gcc>

However with the new build this is back to defaults in the default.conf (as also through the GUI)
==> only NAME/MASTER_PATH remain, the other are gone.

Personally I think we should fix this first, since this breaks users existing configurations.
Title: Re: XML based compilers
Post by: Alpha on September 04, 2012, 01:13:46 am
Personally I think we should fix this first, since this breaks users existing configurations.
After four hours of just trying to find the cause of the bug, I would agree, it should be fixed first ;).  This bug was far worse than first apparent.  It actually deletes all renamed executables on the second launch after the change has been made.

My last bug fix patch fixed a small problem, but created this much larger one.  The following patch fixes it (and has been much more thoroughly tested both on Windows and Linux).
Code
Index: src/sdk/compiler.cpp
===================================================================
--- src/sdk/compiler.cpp (revision 8345)
+++ src/sdk/compiler.cpp (working copy)
@@ -843,23 +843,41 @@
                 continue;
             }
         }
-        else if (node->GetName() == wxT("Program"))
+        else if (node->GetName() == wxT("Program")) // configuration is read so execution of renamed programs work, m_Mirror is needed to reset names to their defaults before leaving this function
         {
             wxString prog = node->GetAttribute(wxT("name"), wxEmptyString);
             if (prog == wxT("C"))
-                m_Programs.C       = cfg->Read(cmpKey + wxT("/c_compiler"),   value);
+            {
+                m_Programs.C = cfg->Read(cmpKey + wxT("/c_compiler"), value);
+                m_Mirror.Programs.C = value;
+            }
             else if (prog == wxT("CPP"))
-                m_Programs.CPP     = cfg->Read(cmpKey + wxT("/cpp_compiler"), value);
+            {
+                m_Programs.CPP = cfg->Read(cmpKey + wxT("/cpp_compiler"), value);
+                m_Mirror.Programs.CPP = value;
+            }
             else if (prog == wxT("LD"))
-                m_Programs.LD      = cfg->Read(cmpKey + wxT("/linker"),       value);
+            {
+                m_Programs.LD = cfg->Read(cmpKey + wxT("/linker"), value);
+                m_Mirror.Programs.LD = value;
+            }
             else if (prog == wxT("DBGconfig"))
                 m_Programs.DBGconfig = value;
             else if (prog == wxT("LIB"))
-                m_Programs.LIB     = cfg->Read(cmpKey + wxT("/lib_linker"),   value);
+            {
+                m_Programs.LIB = cfg->Read(cmpKey + wxT("/lib_linker"), value);
+                m_Mirror.Programs.LIB = value;
+            }
             else if (prog == wxT("WINDRES"))
+            {
                 m_Programs.WINDRES = cfg->Read(cmpKey + wxT("/res_compiler"), value);
+                m_Mirror.Programs.WINDRES = value;
+            }
             else if (prog == wxT("MAKE"))
-                m_Programs.MAKE    = cfg->Read(cmpKey + wxT("/make"),         value);
+            {
+                m_Programs.MAKE = cfg->Read(cmpKey + wxT("/make"), value);
+                m_Mirror.Programs.MAKE = value;
+            }
         }
         else if (node->GetName() == wxT("Switch"))
         {
@@ -1000,6 +1018,13 @@
         }
         node = node->GetNext();
     }
+    // reset programs to their actual defaults (customized settings are loaded in a different function)
+    m_Programs.C       = m_Mirror.Programs.C;
+    m_Programs.CPP     = m_Mirror.Programs.CPP;
+    m_Programs.LD      = m_Mirror.Programs.LD;
+    m_Programs.LIB     = m_Mirror.Programs.LIB;
+    m_Programs.WINDRES = m_Mirror.Programs.WINDRES;
+    m_Programs.MAKE    = m_Mirror.Programs.MAKE;
 }
 
 void Compiler::LoadRegExArray(const wxString& name, bool globalPrecedence, int recursion)
Title: Re: XML based compilers
Post by: Alpha on September 04, 2012, 02:03:27 am
The following patch fixes it (and has been much more thoroughly tested both on Windows and Linux).
... but not thoroughly enough.  There is still a small recursion issue that needs to be dealt with.
Title: Re: XML based compilers
Post by: Alpha on September 04, 2012, 04:28:12 am
There is still a small recursion issue that needs to be dealt with.
... and dealt with.
Code
Index: src/sdk/compiler.cpp
===================================================================
--- src/sdk/compiler.cpp (revision 8345)
+++ src/sdk/compiler.cpp (working copy)
@@ -843,23 +843,41 @@
                 continue;
             }
         }
-        else if (node->GetName() == wxT("Program"))
+        else if (node->GetName() == wxT("Program")) // configuration is read so execution of renamed programs work, m_Mirror is needed to reset before leaving this function
         {
             wxString prog = node->GetAttribute(wxT("name"), wxEmptyString);
             if (prog == wxT("C"))
-                m_Programs.C       = cfg->Read(cmpKey + wxT("/c_compiler"),   value);
+            {
+                m_Programs.C = cfg->Read(cmpKey + wxT("/c_compiler"), value);
+                m_Mirror.Programs.C = value;
+            }
             else if (prog == wxT("CPP"))
-                m_Programs.CPP     = cfg->Read(cmpKey + wxT("/cpp_compiler"), value);
+            {
+                m_Programs.CPP = cfg->Read(cmpKey + wxT("/cpp_compiler"), value);
+                m_Mirror.Programs.CPP = value;
+            }
             else if (prog == wxT("LD"))
-                m_Programs.LD      = cfg->Read(cmpKey + wxT("/linker"),       value);
+            {
+                m_Programs.LD = cfg->Read(cmpKey + wxT("/linker"), value);
+                m_Mirror.Programs.LD = value;
+            }
             else if (prog == wxT("DBGconfig"))
                 m_Programs.DBGconfig = value;
             else if (prog == wxT("LIB"))
-                m_Programs.LIB     = cfg->Read(cmpKey + wxT("/lib_linker"),   value);
+            {
+                m_Programs.LIB = cfg->Read(cmpKey + wxT("/lib_linker"), value);
+                m_Mirror.Programs.LIB = value;
+            }
             else if (prog == wxT("WINDRES"))
+            {
                 m_Programs.WINDRES = cfg->Read(cmpKey + wxT("/res_compiler"), value);
+                m_Mirror.Programs.WINDRES = value;
+            }
             else if (prog == wxT("MAKE"))
-                m_Programs.MAKE    = cfg->Read(cmpKey + wxT("/make"),         value);
+            {
+                m_Programs.MAKE = cfg->Read(cmpKey + wxT("/make"), value);
+                m_Mirror.Programs.MAKE = value;
+            }
         }
         else if (node->GetName() == wxT("Switch"))
         {
@@ -1000,6 +1018,15 @@
         }
         node = node->GetNext();
     }
+    if (recursion == 0) // reset programs to their actual defaults (customized settings are loaded in a different function)
+    {
+        m_Programs.C       = m_Mirror.Programs.C;
+        m_Programs.CPP     = m_Mirror.Programs.CPP;
+        m_Programs.LD      = m_Mirror.Programs.LD;
+        m_Programs.LIB     = m_Mirror.Programs.LIB;
+        m_Programs.WINDRES = m_Mirror.Programs.WINDRES;
+        m_Programs.MAKE    = m_Mirror.Programs.MAKE;
+    }
 }
 
 void Compiler::LoadRegExArray(const wxString& name, bool globalPrecedence, int recursion)
Title: Re: XML based compilers
Post by: Alpha on September 13, 2012, 11:30:31 pm
Just wondering, are there any updates on the current status of this branch?
Title: Re: XML based compilers
Post by: killerbot on September 13, 2012, 11:53:26 pm
I am gonna do some tests of your latest updates this Sunday, was completely unable to get to it any sooner :-(
Title: Re: XML based compilers
Post by: Alpha on September 22, 2012, 12:11:56 am
I presume you meant this coming Sunday ;) ...
Title: Re: XML based compilers
Post by: Alpha on September 25, 2012, 11:14:47 pm
Here is a quick patch to prevent compilers from incorrectly being marked as "User-defined" in the auto detection dialog.
Code
Index: src/sdk/autodetectcompilers.cpp
===================================================================
--- src/sdk/autodetectcompilers.cpp (revision 8407)
+++ src/sdk/autodetectcompilers.cpp (working copy)
@@ -96,7 +96,7 @@
                 else if ( !path.IsEmpty() )
                 {
                     // Check, if the master path is valid:
-                    if ( wxFileName::DirExists(path_no_macros) )
+                    if ( wxFileName::DirExists(path_no_macros) && !(path == pathDetected || path_no_macros == pathDetected) )
                     {
                         list->SetItem(idx, 1, _("User-defined")); // OK
                         highlight = 0;
Title: Re: XML based compilers
Post by: killerbot on September 25, 2012, 11:20:21 pm
hmmmm, darn, this weekend for sure, or I owe you some nice belgian beers ;-)
Title: Re: XML based compilers
Post by: MortenMacFly on September 26, 2012, 07:29:55 am
nice belgian beers ;-)
From a German point of view: Isn't this a paradoxon? ;D ;D ;D
Title: Re: XML based compilers
Post by: Alpha on September 27, 2012, 12:55:39 am
hmmmm, darn, this weekend for sure, or I owe you some nice belgian beers ;-)
Not a problem; I am sure you have been very busy.  (Although, if you do not get around to it, that might have to be a rain check; I am currently underage ;).)
Title: Re: XML based compilers
Post by: killerbot on October 01, 2012, 07:57:35 pm
I can confirm the "exe" problem is gone, it now remembered it is gcc-4.7.
One strange things : it claimed the SDCC was user defined. Never touched that one ...

Seems to be ok. I will prepare nightlies.
Title: Re: XML based compilers
Post by: Alpha on October 01, 2012, 09:11:52 pm
One strange things : it claimed the SDCC was user defined. Never touched that one ...
That is an artifact of a fix to its auto-detection routine; as it only will report this oddity once (on the switch from trunk to XML compiler), I did not think it was worthwhile to write a special rule for it.

Seems to be ok. I will prepare nightlies.
Okay.  I will be waiting for the bug reports that only appear after everything is thought to be fixed :).
Title: Re: XML based compilers
Post by: Alpha on October 06, 2012, 09:01:36 pm
I noticed Ubuntu seems to make multiline textboxes default to zero height.  This patch should ensure there is at least reasonable edit room (on any platform).
Code
Index: src/plugins/compilergcc/compileroptionsdlg.cpp
===================================================================
--- src/plugins/compilergcc/compileroptionsdlg.cpp (revision 8435)
+++ src/plugins/compilergcc/compileroptionsdlg.cpp (working copy)
@@ -2777,6 +2777,11 @@
         Compiler* compiler = CompilerFactory::GetCompiler(m_CurrentCompilerIdx);
         wxTextEntryDialog dlg(this, wxT("List flags that will only be used during C compilation"),
                               wxT("C - only flags"), compiler->GetCOnlyFlags(), wxTextEntryDialogStyle|wxTE_MULTILINE|wxRESIZE_BORDER);
+        if (dlg.GetSize().GetHeight() < 220)
+        {
+            dlg.SetSize(dlg.GetPosition().x, dlg.GetPosition().y - (220 - dlg.GetSize().GetHeight()) / 2,
+                        dlg.GetSize().GetWidth(), 220);
+        }
         dlg.ShowModal();
         wxString flags = dlg.GetValue();
         flags.Replace(wxT("\n"), wxT(" "));
@@ -2795,6 +2800,11 @@
         Compiler* compiler = CompilerFactory::GetCompiler(m_CurrentCompilerIdx);
         wxTextEntryDialog dlg(this, wxT("List flags that will only be used during C++ compilation"),
                               wxT("C++ - only flags"), compiler->GetCPPOnlyFlags(), wxTextEntryDialogStyle|wxTE_MULTILINE|wxRESIZE_BORDER);
+        if (dlg.GetSize().GetHeight() < 220)
+        {
+            dlg.SetSize(dlg.GetPosition().x, dlg.GetPosition().y - (220 - dlg.GetSize().GetHeight()) / 2,
+                        dlg.GetSize().GetWidth(), 220);
+        }
         dlg.ShowModal();
         wxString flags = dlg.GetValue();
         flags.Replace(wxT("\n"), wxT(" "));
Index: src/plugins/compilergcc/resources/compilers/options_common_warnings.xml
===================================================================
--- src/plugins/compilergcc/resources/compilers/options_common_warnings.xml (revision 8435)
+++ src/plugins/compilergcc/resources/compilers/options_common_warnings.xml (working copy)
@@ -26,7 +26,7 @@
             <Option name="Have g++ follow the C++11 ISO C++ language standard"
                     option="-std=c++11"
                     supersedes="-std=c++98 -std=c++0x"/>
-            <Option name="zero as null pointer constant"
+            <Option name="Warn if '0' is used as a null pointer constant"
                     option="-Wzero-as-null-pointer-constant"/>
         </if>
         <Option name="Enable warnings demanded by strict ISO C and ISO C++"

Title: Re: XML based compilers
Post by: oBFusCATed on October 06, 2012, 09:19:11 pm
Why don't you specify min size for the control that is made the wrong height?
Title: Re: XML based compilers
Post by: Alpha on October 06, 2012, 09:24:58 pm
In this part of the code I had used a prefabricated wxTextEntryDialog (http://docs.wxwidgets.org/trunk/classwx_text_entry_dialog.html) (for ease of programming), so I do not have (guaranteed) direct access to any of the components (as far as I know).
Title: Re: XML based compilers
Post by: oBFusCATed on October 06, 2012, 09:26:25 pm
So, this is an ubuntu bug and you must file a bug report. No need for such ugly hacks.
Title: Re: XML based compilers
Post by: killerbot on October 06, 2012, 11:09:41 pm
good idea to file it to ubuntu, but keep the hack, and document it for what purpose it is there. The user couldn't care less for technical bla bla, he wants a working product. Just a matter to keep this in mind and check when it is fixed to remove the hack at the appropriate time.
Is it something that can happen often, or only in a very rare case ? If it is rather rare then indeed the hack might not be worth the hassle.
Title: Re: XML based compilers
Post by: oBFusCATed on October 06, 2012, 11:46:44 pm
Is it something that can happen often, or only in a very rare case ? If it is rather rare then indeed the hack might not be worth the hassle.
The hack is already in svn...
Title: Re: XML based compilers
Post by: MortenMacFly on October 07, 2012, 09:03:43 am
The hack is already in svn...
...the description now, too.
I was able to reproduce it with one of my own projects, too btw. But I could swear this came with more recent version of either Ubuntu or wxWidgets. I've never seen such before. So it would either be a wxWidgets or Ubuntu thing. I wonder how a more recent wx294 demo case would behave.
Title: Re: XML based compilers
Post by: killerbot on October 07, 2012, 10:40:36 am
I would refactor this in a little method which name says it is a hack, since now (just looking at the diff), there is code duplication, and will be easier to remove (function calls and implementation), and easier to test if hack is still needed (temporarily make the function empty)

What do you think ?
Title: Re: XML based compilers
Post by: MortenMacFly on October 07, 2012, 10:56:54 am
What do you think ?
Why not? I though in this case, as both hacks are directly after each other it would not be needed. Because if you see the one hack you'll also see the other.
Title: Re: XML based compilers
Post by: Alpha on October 08, 2012, 11:40:02 pm
good idea to file it to ubuntu
Bug reported (https://bugs.launchpad.net/ubuntu/+bug/1064082).
Title: Re: XML based compilers
Post by: MortenMacFly on October 09, 2012, 07:12:38 am
good idea to file it to ubuntu
Bug reported (https://bugs.launchpad.net/ubuntu/+bug/1064082).
Wouldn't the wxWidgets bug-tracker be the right place to report?!
Title: Re: XML based compilers
Post by: Alpha on October 09, 2012, 11:06:47 pm
Wouldn't the wxWidgets bug-tracker be the right place to report?!
I actually was not quite sure where it belonged.  I will submit it there as well.  (It appears that this was reported (http://trac.wxwidgets.org/ticket/1626) a while ago as well.)
Title: Re: XML based compilers
Post by: Alpha on November 12, 2012, 11:36:52 pm
About rev. 8553, would the following be more friendly for translators?  Or should this type of pseudo-intelligent logging be avoided?
Code
Index: src/plugins/compilergcc/compilergcc.cpp
===================================================================
--- src/plugins/compilergcc/compilergcc.cpp (revision 8554)
+++ src/plugins/compilergcc/compilergcc.cpp (working copy)
@@ -3728,8 +3728,9 @@
 
 wxString CompilerGCC::GetErrWarnStr()
 {
-    return wxString::Format(_("%u error(s), %u warning(s)"),
-                            m_Errors.GetCount(cltError), m_Errors.GetCount(cltWarning));
+    return wxString::Format(_("%u %s, %u %s"),
+                            m_Errors.GetCount(cltError),   wxString(m_Errors.GetCount(cltError)   == 1 ? _("error")   : _("errors")).wx_str(),
+                            m_Errors.GetCount(cltWarning), wxString(m_Errors.GetCount(cltWarning) == 1 ? _("warning") : _("warnings")).wx_str());
 }
 
 wxString CompilerGCC::GetMinSecStr()
@@ -3737,6 +3738,7 @@
     long int elapsed = (wxGetLocalTimeMillis() - m_StartTime).ToLong() / 1000;
     int mins =  elapsed / 60;
     int secs = (elapsed % 60);
-    return wxString::Format(_("%d minute(s), %d second(s)"), mins, secs);
-;
+    return wxString::Format(_("%d %s, %d %s"),
+                            mins, wxString(mins == 1 ? _("minute") : _("minutes")).wx_str(),
+                            secs, wxString(secs == 1 ? _("second") : _("seconds")).wx_str());
 }
Index: src/plugins/compilergcc/directcommands.cpp
===================================================================
--- src/plugins/compilergcc/directcommands.cpp (revision 8554)
+++ src/plugins/compilergcc/directcommands.cpp (working copy)
@@ -613,8 +613,8 @@
     if (!fileMissing.IsEmpty())
     {
         wxString warn;
-        warn.Printf(_("WARNING: Target '%s': Unable to resolve %lu external dependenc%s:"),
-                    target->GetFullTitle().wx_str(), static_cast<unsigned long>(fileMissing.Count()), wxString(fileMissing.Count() == 1 ? _("y") : _("ies")).wx_str());
+        warn.Printf(_("WARNING: Target '%s': Unable to resolve %lu external %s:"),
+                    target->GetFullTitle().wx_str(), static_cast<unsigned long>(fileMissing.Count()), wxString(fileMissing.Count() == 1 ? _("dependency") : _("dependencies")).wx_str());
         ret.Add(wxString(COMPILER_SIMPLE_LOG) + warn);
         for (size_t i=0; i<fileMissing.Count(); i++)
             ret.Add(wxString(COMPILER_SIMPLE_LOG) + fileMissing[i]);
Title: Re: XML based compilers
Post by: oBFusCATed on November 12, 2012, 11:53:08 pm
I see no problem with the current version ... KISS for the win.
Title: Re: XML based compilers
Post by: MortenMacFly on November 13, 2012, 08:58:19 am
I see no problem with the current version ... KISS for the win.
I agree. Either we do it like that in all places or we just keep it simple. It has some more advantages: You don't have to translate many units. Already now we have several thousands of units to translate anyways - adding even more might not be the best thing to do you know... ;)
@Alpha: If you want to, make a poEdit session on Code::Blocks. Then you know what I am talking about... ;D

However - you pointed me to a place that I missed to simplify. :P
Title: Re: XML based compilers
Post by: Alpha on November 13, 2012, 11:30:28 pm
I think this might be a side effect from having an overzealous English teacher :).

@Alpha: If you want to, make a poEdit session on Code::Blocks. Then you know what I am talking about... ;D
I can imagine :o (and I have enough trouble as it is when I need to translate just a few sentences).
Title: Re: XML based compilers
Post by: Alpha on November 20, 2012, 12:42:16 am
Do not worry, I am not trying to argue you change your mind ;).  In my reading, I had bumped into the following, and thought I would post some links in case the topic of plural translations was ever revisited in the future.
Title: Re: XML based compilers
Post by: MortenMacFly on November 20, 2012, 06:21:24 am
Do not worry, I am not trying to argue you change your mind ;).  In my reading, I had bumped into the following, and thought I would post some links in case the topic of plural translations was ever revisited in the future.
  • wxPLURAL() (http://docs.wxwidgets.org/trunk/group__group__funcmacro__string.html#gadc7c2f1bab3914af93feb47945003409)
  • wxTranslations::GetString() (http://docs.wxwidgets.org/trunk/classwx_translations.html#a7480771c24a53e0ccf1170a3a70b2ba4)
  • gettext - plural forms (http://www.gnu.org/savannah-checkouts/gnu/gettext/manual/html_node/Plural-forms.html)
  • gettext - translating plural forms (http://www.gnu.org/savannah-checkouts/gnu/gettext/manual/html_node/Translating-plural-forms.html)
I know about these, but they all cause to double the units to translate in the worst case. The question is: In an IDE - is "dependency/ies" enough, or do you really want to be that super correct. I mean its not a scientific thesis we are doing here. And (most important to me) we don't want to scare away translators.
Title: Re: XML based compilers
Post by: oBFusCATed on December 10, 2012, 12:51:58 am
Morten: Can you show us if you can do a pain free svn merge?
Title: Re: XML based compilers
Post by: Alpha on December 10, 2012, 02:50:30 am
I have some changes in my local copy; would you prefer to have them now (some changes are not yet thoroughly tested), or after the merge (assuming it will be soon)?
Title: Re: XML based compilers
Post by: MortenMacFly on December 10, 2012, 08:53:17 am
I have some changes in my local copy; would you prefer to have them now (some changes are not yet thoroughly tested), or after the merge (assuming it will be soon)?
After.

Morten: Can you show us if you can do a pain free svn merge?
Sorry, no time atm, but I'll see what I can do later this or next week.
Title: Re: XML based compilers
Post by: MortenMacFly on December 12, 2012, 08:30:43 pm
Sorry, no time atm, but I'll see what I can do later this or next week.
Done. Branch is successfully merged into trunk now (with only ~10 serious conflicts only, btw. - easy to resolve with the conflict manager of my SVN tool suite ;-)).

Therefore the branch is now obsolete.

A big "THANK YOU" to Alpha for the hard work!
Title: Re: XML based compilers
Post by: oBFusCATed on December 13, 2012, 02:01:41 pm
Any explanation why SDCC compiler is detected as user defined, but I've never change anything in it?
Happened both at work and at home.
Title: Re: XML based compilers
Post by: oBFusCATed on December 13, 2012, 03:26:37 pm
It seems that if there are compile errors and you hit f8, the debugger is not stopped, but continues, no dialog, no nothing.

p.s. this doesn't happen on a simple project :(
Title: Re: XML based compilers
Post by: MortenMacFly on December 13, 2012, 04:50:57 pm
It seems that if there are compile errors and you hit f8, the debugger is not stopped, but continues, no dialog, no nothing.

p.s. this doesn't happen on a simple project :(
Is that related to the compiler?
Title: Re: XML based compilers
Post by: MortenMacFly on December 13, 2012, 04:53:01 pm
Any explanation why SDCC compiler is detected as user defined, but I've never change anything in it?
Happened both at work and at home.
Windows or Linux?
Title: Re: XML based compilers
Post by: oBFusCATed on December 13, 2012, 05:33:58 pm
Is that related to the compiler?
I suppose it is. It happens when there are some major header changes and there are more that one compilation commands.
The error is in a cpp file, not a header. I've never seen this bug with the older builds. I'll try to reproduce this with the C::B's
project.

Windows or Linux?
Both linux.
Title: Re: XML based compilers
Post by: MortenMacFly on December 13, 2012, 05:49:51 pm
I'll try to reproduce this with the C::B's project.
That would be nice because I don't understand currently, how a compiler session could interfere with a debugger session?!
Title: Re: XML based compilers
Post by: oBFusCATed on December 13, 2012, 05:52:35 pm
That would be nice because I don't understand currently, how a compiler session could interfere with a debugger session?!
This is if you have the option "Auto-build project if it is not up-to-date" enabled.
So when you start the debugger, first the compiler is run to build the project.
Normally, when the build fails a message pop-up asking you if you want to debug the project anyway.
This popup doesn't show up with the latest build.
Title: Re: XML based compilers
Post by: MortenMacFly on December 13, 2012, 08:27:05 pm
This is if you have the option "Auto-build project if it is not up-to-date" enabled.
done that, started debugging and it stops at the build error, shows the asks if I want to to debug anyways and both options work for me...?!

What do you mean with:
I suppose it is. It happens when there are some major header changes
?

Is there a safe way to reproduce?
Title: Re: XML based compilers
Post by: oBFusCATed on December 13, 2012, 09:25:35 pm
Is there a safe way to reproduce?
Yes, at least on linux.
1. add space to cbeditor.h
2. add some garbage in src/src/main.cpp
3. press f8
Title: Re: XML based compilers
Post by: Alpha on December 13, 2012, 11:07:55 pm
Any explanation why SDCC compiler is detected as user defined, but I've never change anything in it?
Previously, the default master path for SDCC was incorrectly
This was fixed during the refactor (the trailing "bin" was removed), but has the side-effect of appearing to be different (the previous saved value no longer matches the default fall-back).
Title: Re: XML based compilers
Post by: tomjnx on December 27, 2012, 01:54:38 am
I have now updated my Keil C51 and IAR ICC8051 compiler plugin patch to the new XML compiler framework. It can be found here:
http://www.baycom.org/~tom/cb/keiliarcompilers.patch

(It is also at Berlios Patch #3152, but Berlios seems to eat new files from uploaded patches, so the Berlios version is incomplete).

Keil C51 and IAR ICC8051 are commercial compilers for 8051 type microcontrollers.

The options XML files worked great for me!

I'm not so convinced about the compiler XML files. It seems I cannot use them both for Keil and IAR.

For detecting Keil, I need to read a registry key, but then append another path component to the retrieved path. It seems to me I cannot do this with the XML framework.

For detecting IAR, I need to open a registry "directory", enumerate all "subdirectories", read a registry key out of each subdirectory and check the retrieved paths. This seems not doable with the XML framework either.

So both compilers have option XML files, but "old-style" C++ detection routines.

For Keil, I needed to do additional modifications to the compiler plugin and SDK infrastructure.

For one, Keil Return code 1 also indicates success (but with warnings). So I needed to make the highest return code still considered a success configurable (statusSuccess).

Then, Keil has a very peculiar way to specify include paths and defines to the compiler. Instead of the standard "-Ipath1 -Ipath2", Keil wants paths to be specified as "INCDIR(path1;path2)". There can only be one "INCDIR" command line parameter, so "INCDIR(path1) INCDIR(path2)" does not work. To handle this, special processing is triggered when a compiler "switch" ends with "(". Plus configurable separators (includeDirSeparator).

Third, the Keil Linker is unable to find objects by itself in a list of directories. It needs fully resolved paths as input. Therefore, I needed to add path resolution, which is enabled with linkerNeedsPathresolution.

Title: Re: XML based compilers
Post by: tomjnx on December 27, 2012, 02:00:36 am
A question about SDCC: why has objectExtension been changed from "rel" to "o"? This doesn't work for me,

Also, I need to set linkerNeedsLibPrefix to true, because telling sdcc to -labc makes it try to link abc.lib
Title: Re: XML based compilers
Post by: Alpha on December 27, 2012, 03:48:38 am
What I'm asking for is to make "o" the default for SDCC object files because "rel" has been totally irrelevant for months.
Done in trunk. I guess nobody of us works with that compiler, so hopefully no other users complain that might still use the version that requires "rel"... we will see...
A question about SDCC: why has objectExtension been changed from "rel" to "o"? This doesn't work for me,
Uh-oh...
Do any SDCC users have a version number where this changed?

I have now updated my Keil C51 and IAR ICC8051 compiler plugin patch to the new XML compiler framework. It can be found here:
http://www.baycom.org/~tom/cb/keiliarcompilers.patch

(It is also at Berlios Patch #3152, but Berlios seems to eat new files from uploaded patches, so the Berlios version is incomplete).
Great.  In a quick peek, however, it appears that the baycom.org upload missed the files compilerIAR.cpp, compilerKeilC51.cpp, compilerIAR.h, and compilerKeilC51.h.

I'm not so convinced about the compiler XML files. It seems I cannot use them both for Keil and IAR.
Yes, these auto-detection routines are very basic; only the simpler compilers can make use of them currently.  However, if you happen to have any ideas or suggestions for a better file format, I would appreciate hearing them.
Title: Re: XML based compilers
Post by: tomjnx on December 27, 2012, 05:26:43 am
A question about SDCC: why has objectExtension been changed from "rel" to "o"? This doesn't work for me,
Uh-oh...
Do any SDCC users have a version number where this changed?

I don't think it ever has changed or become irrelevant. The SDCC manual
http://sdcc.sourceforge.net/doc/sdccman.pdf states on p.24 that the object file extension is .rel.

Looking at the sourceforge SVN source code of SDCC, it's slightly more complicated. AVR, DS390, HC08, MCS51, XA51 and Z80 ports use .rel as object extension, only the PIC14 and PIC16 backends use .o as object extension (see sdcc/src/*/main.c, the PORT structs). I'd consider the PIC backends as of the lesser stable/mature backends - so it seems strange to break the stable backends because of a few unstable ones...

So it seems the correct fix would be (apart from convincing SDCC developers to use the same extension for all backends) to select the object file extension based on which architecture switch was selected...

Great.  In a quick peek, however, it appears that the baycom.org upload missed the files compilerIAR.cpp, compilerKeilC51.cpp, compilerIAR.h, and compilerKeilC51.h.

Sorry about that, should be fixed now
Title: Re: XML based compilers
Post by: Alpha on December 28, 2012, 06:23:10 am
Some questions.
Here (compilerKeilC51.cpp):
Code
        wxRegKey key;   // defaults to HKCR
        key.SetName(wxT("HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\Keil \265Vision3"));
        if (key.Exists() && key.Open(wxRegKey::Read)) // found; read it
            key.QueryValue(wxT("LastInstallDir"), m_MasterPath);
did you mean:
Code
        key.SetName(wxT("HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\Keil\\265Vision3"));

Here (options_keilcx51.xml):
Code
    <if platform="windows">
        <Program name="LD"        value="LX51.exe"/>
        <Program name="LIB"       value="LIBX51.exe"/>
    </if>
    <else>
        <Program name="LD"        value="LX51"/>
        <Program name="LIB"       value="LIBX51"/>
    </else>
did you intend to have:
Code
    <if platform="windows">
        <Program name="CPP"       value="CX51.exe"/>
        <Program name="LD"        value="LX51.exe"/>
        <Program name="LIB"       value="LIBX51.exe"/>
    </if>
    <else>
        <Program name="CPP"       value="CX51"/>
        <Program name="LD"        value="LX51"/>
        <Program name="LIB"       value="LIBX51"/>
    </else>
Title: Re: XML based compilers
Post by: Alpha on December 28, 2012, 10:08:05 pm
Attached is a modified version of your patch.  [[Edit: use the version in the later post.]]  There are some changes to formatting, removal of sections that appeared to have no purpose, and tweaks/fixes.  Could you please test and let me know if it still works as intended?

I don't think it ever has changed or become irrelevant. The SDCC manual
http://sdcc.sourceforge.net/doc/sdccman.pdf states on p.24 that the object file extension is .rel.

Looking at the sourceforge SVN source code of SDCC, it's slightly more complicated. AVR, DS390, HC08, MCS51, XA51 and Z80 ports use .rel as object extension, only the PIC14 and PIC16 backends use .o as object extension (see sdcc/src/*/main.c, the PORT structs). I'd consider the PIC backends as of the lesser stable/mature backends - so it seems strange to break the stable backends because of a few unstable ones...

So it seems the correct fix would be (apart from convincing SDCC developers to use the same extension for all backends) to select the object file extension based on which architecture switch was selected...
According to PM with user squalyl, PIC port produces *.o because it uses gpasm and gplink.  The easier solution would be to duplicate SDCC to create SDCC-PIC, however, I would prefer to instead detect the correct option.  Do you have a list of the specific flags that switch it to PIC (is it -mpic14 and -mpic16)?  (I plan to revert the default to *.rel.)

[attachment deleted by admin]
Title: Re: XML based compilers
Post by: tomjnx on December 29, 2012, 12:20:16 am
Thanks for your effort!

did you mean:
Code
        key.SetName(wxT("HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\Keil\\265Vision3"));

Um, no, I meant the lowercase greek letter mu, µ. So the directory component should be "Keil µVision3" (µVision being the name of their IDE)

Here (options_keilcx51.xml):
Code
    <if platform="windows">
        <Program name="LD"        value="LX51.exe"/>
        <Program name="LIB"       value="LIBX51.exe"/>
    </if>
    <else>
        <Program name="LD"        value="LX51"/>
        <Program name="LIB"       value="LIBX51"/>
    </else>
did you intend to have:
Code
    <if platform="windows">
        <Program name="CPP"       value="CX51.exe"/>
        <Program name="LD"        value="LX51.exe"/>
        <Program name="LIB"       value="LIBX51.exe"/>
    </if>
    <else>
        <Program name="CPP"       value="CX51"/>
        <Program name="LD"        value="LX51"/>
        <Program name="LIB"       value="LIBX51"/>
    </else>

Yes. Thanks for the fix. On second thought, the conditional could be removed and only the windows version be retained, as keil doesn't offer their tools for anything other than windows, and apparently the have absolutely no plans to change this.
Title: Re: XML based compilers
Post by: tomjnx on December 29, 2012, 12:26:31 am
Attached is a modified version of your patch.  There are some changes to formatting, removal of sections that appeared to have no purpose, and tweaks/fixes.  Could you please test and let me know if it still works as intended?
I will do that, but it will most likely be after new year.

Do you have a list of the specific flags that switch it to PIC (is it -mpic14 and -mpic16)?  (I plan to revert the default to *.rel.)
Yes, exactly.
Title: Re: XML based compilers
Post by: Alpha on December 29, 2012, 01:14:50 am
Um, no, I meant the lowercase greek letter mu, µ. So the directory component should be "Keil µVision3" (µVision being the name of their IDE)
Ah, okay.
I will do that, but it will most likely be after new year.
Sure; when you do, use the version attached here (it formats a little more, and reverts my change to mu).
Title: Re: XML based compilers
Post by: tomjnx on January 04, 2013, 08:39:38 pm
Sure; when you do, use the version attached here (it formats a little more, and reverts my change to mu).

I've now reviewed and tested this patch. It looks good. You've made some nice improvements!

The only thing I forgot: DBGconfig should be set to the empty string in options_keilc51.xml and options_iar8051.xml; axs_debugger is not yet merged.
Title: Re: XML based compilers
Post by: Alpha on January 04, 2013, 09:09:16 pm
The only thing I forgot: DBGconfig should be set to the empty string in options_keilc51.xml and options_iar8051.xml; axs_debugger is not yet merged.
Okay.
Title: Re: XML based compilers
Post by: tomjnx on January 06, 2013, 01:26:48 pm
Thanks for merging!

Now that these compilers are in, I've posted a patch (#3400 on the berlios queue) to the scripted wizard MCS51 project generator to allow the user to choose IAR or Keil in addition to SDCC.
Title: Re: XML based compilers
Post by: darmar on October 26, 2013, 01:41:10 pm
I have had few problems with ‘xml based compilers’ when I have tried to port Intel Fortran compiler:

1) I can specify, that compiler will run only on ‘windows’, or on ‘linux’, or on ‘macosx’. But how to say ‘non-windows’? Or ‘linux and macosx’?

2) In an options file, I can specify, that one option ‘supersedes’ another. But some of Intel’s options are from two separate words (e.g. "-check bounds"). It seems, that C::B can’t handle such options in ‘supersedes’ list.  Have I missed something?
Title: Re: XML based compilers
Post by: Alpha on October 26, 2013, 11:21:43 pm
Currently, the format does not support those options.  If you have a recommendation on how to extend the format, I will see what I can come up with (when I have time).  Or you could prepare a patch; I would be happy to review it.
Title: Re: XML based compilers
Post by: gd_on on October 27, 2013, 06:21:08 pm
Before the introduction of xml based compilers, it was possible to obtain localized strings for each compiler options. Now, with xml compilers, they only appear in english, even if I force the correct chain and it's translation to be in the .po (and .mo) file. May be the "name" parameter in the xml file should be treated somewhere with a _(...) and not with a _T(...) (or wxT(...)).
Is it possible to reintroduce this localization possibility?
Thanks

gd_on
Title: Re: XML based compilers
Post by: Alpha on October 27, 2013, 07:34:44 pm
Does this patch enable translation?
Code
Index: src/plugins/compilergcc/compilerXML.cpp
===================================================================
--- src/plugins/compilergcc/compilerXML.cpp (revision 9423)
+++ src/plugins/compilergcc/compilerXML.cpp (working copy)
@@ -15,7 +15,7 @@
 #include "compilerXML.h"
 
 CompilerXML::CompilerXML(const wxString& name, const wxString& ID, const wxString& file)
-    : Compiler(name, ID), m_fileName(file)
+    : Compiler(_(name), ID), m_fileName(file)
 {
     wxXmlDocument compiler;
     compiler.Load(m_fileName);
Title: Re: XML based compilers
Post by: gd_on on October 28, 2013, 10:39:07 am
I tried your patch but I obtain a compilation error :
Quote
||=== Build: Compiler in Code::Blocks wx2.8.x (compiler: GNU GCC Compiler) ===|
C:\Users\Gerard_2\Documents\CodeBlocks_SVN\CodeBlocks_src\src\plugins\compilergcc\compilerXML.cpp||In constructor 'CompilerXML::CompilerXML(const wxString&, const wxString&, const wxString&)':|
C:\wxWidgets-2.8.12\include\wx\wxchar.h|235|error: 'Lname' was not declared in this scope|
C:\wxWidgets-2.8.12\include\wx\cpp.h|17|note: in definition of macro 'wxCONCAT_HELPER'|
C:\wxWidgets-2.8.12\include\wx\intl.h|48|note: in expansion of macro 'wxT'|
C:\Users\Gerard_2\Documents\CodeBlocks_SVN\CodeBlocks_src\src\plugins\compilergcc\compilerXML.cpp|18|note: in expansion of macro '_'|
||=== Build failed: 1 error(s), 0 warning(s) (0 minute(s), 4 second(s)) ===|

error at line 235 in wchar.h :
Code
233 #if wxUSE_UNICODE
234      /* use wxCONCAT_HELPER so that x could be expanded if it's a macro */
235     #define wxT(x) wxCONCAT_HELPER(L, x)
236 #else /* !Unicode */
237     #define wxT(x) x
238 #endif /* Unicode/!Unicode */

maybe a missing header.

gd_on
Title: Re: XML based compilers
Post by: osdt on October 28, 2013, 12:43:28 pm
I tried your patch but I obtain a compilation error :
Quote
...
C:\wxWidgets-2.8.12\include\wx\wxchar.h|235|error: 'Lname' was not declared in this scope|
...

The macro '_(name)' expands to 'Lname' for unicode builds. It's designed for quoted strings ("translate me") only.

@gd_on: do you really want to translate compiler names like 'GCC' ?

- osdt

Title: Re: XML based compilers
Post by: gd_on on October 28, 2013, 12:52:28 pm
Quote
do you really want to translate compiler names like 'GCC' ?
No, I don't.
I simply want to translate options as they appear in compiler options, and more precisely in window "Global compiler settings", Compiler settings, Compiler flags tabs.
For example:
the string: Produce debugging symbols (displayed as Produce debugging symbols [-g])
will become in french: Produire des symboles de débogage (displayed as Produire des symboles de débogage [-g])

gd_on
Title: Re: XML based compilers
Post by: Alpha on October 28, 2013, 01:26:27 pm
I tried your patch but I obtain a compilation error : [...]
I wonder why it worked for me then...
Try this:
Code
Index: src/plugins/compilergcc/compilerXML.cpp
===================================================================
--- src/plugins/compilergcc/compilerXML.cpp (revision 9423)
+++ src/plugins/compilergcc/compilerXML.cpp (working copy)
@@ -15,7 +15,7 @@
 #include "compilerXML.h"
 
 CompilerXML::CompilerXML(const wxString& name, const wxString& ID, const wxString& file)
-    : Compiler(name, ID), m_fileName(file)
+    : Compiler(wxGetTranslation(name), ID), m_fileName(file)
 {
     wxXmlDocument compiler;
     compiler.Load(m_fileName);
Index: src/sdk/compiler.cpp
===================================================================
--- src/sdk/compiler.cpp (revision 9423)
+++ src/sdk/compiler.cpp (working copy)
@@ -963,12 +963,12 @@
             wxString exclusive;
             if (!node->GetAttribute(wxT("exclusive"), &exclusive))
                 exclusive = (exclu ? wxT("true") : wxT("false"));
-            m_Options.AddOption(node->GetAttribute(wxT("name"), wxEmptyString),
+            m_Options.AddOption(wxGetTranslation(node->GetAttribute(wxT("name"), wxEmptyString)),
                                 node->GetAttribute(wxT("option"), wxEmptyString),
-                                category,
+                                wxGetTranslation(category),
                                 node->GetAttribute(wxT("additionalLibs"), wxEmptyString),
                                 node->GetAttribute(wxT("checkAgainst"), wxEmptyString),
-                                node->GetAttribute(wxT("checkMessage"), wxEmptyString),
+                                wxGetTranslation(node->GetAttribute(wxT("checkMessage"), wxEmptyString)),
                                 node->GetAttribute(wxT("supersedes"), wxEmptyString),
                                 exclusive == wxT("true"));
         }
@@ -1121,7 +1121,7 @@
             else if (tp == wxT("info"))
                 clt = cltInfo;
             wxArrayString msg = GetArrayFromString(node->GetAttribute(wxT("msg"), wxEmptyString) + wxT(";0;0"));
-            m_RegExes.Add(RegExStruct(node->GetAttribute(wxT("name"), wxEmptyString), clt,
+            m_RegExes.Add(RegExStruct(wxGetTranslation(node->GetAttribute(wxT("name"), wxEmptyString)), clt,
                                       node->GetNodeContent().Trim().Trim(false), wxAtoi(msg[0]),
                                       wxAtoi(node->GetAttribute(wxT("file"), wxT("0"))),
                                       wxAtoi(node->GetAttribute(wxT("line"), wxT("0"))),
Title: Re: XML based compilers
Post by: gd_on on October 28, 2013, 02:19:53 pm
With this last patch, it works  ;) (at least on the string I mentioned in my previous post).
Now, my job is : extract all "name" strings from .xml file and add them in my .po file.
Thanks

gd_on
Title: Re: XML based compilers
Post by: BlueHazzard on October 28, 2013, 06:58:56 pm
This is OT but anyway:
Do you really think localizating the compiler flags is a good idea? I mean, if you learn c/c++ or programming in general, you will be forced to learn and use english. If now a noob, who want learn to code, uses a localized IDE (only the compiler flags here) needs help, he will find it with more difficulties, because of the localization. GCC introduced localization a few releases back. I used it 2 times, and tried then to remove it, because i didn't could find any help in my language...
In short: Is compiler specific translation a goof idea?

greetings
Title: Re: XML based compilers
Post by: gd_on on October 28, 2013, 07:43:18 pm
This is the text accompanying and explaining the flag which is translated, not the flag itself!
So, I think that translating this text may help, a noob or a standard (experienced) user.
Myself, when I doubt on a translation (even on mine  :o), I revert C::B to English, and can see the "official" meaning. Just one or two click and a C::B restart !
I know that many people prefer to use C::B in English and not in their native language. This is their choice.
Myself, I generally prefer to see most of the interface in French, even if in English I understand generally. This is my choice, because I'm more comfortable then.
Both choices have their advantages and disadvantages.

So, yes I like a localized IDE ;). And you are not obliged to use it.

gd_on
Title: Re: XML based compilers
Post by: Alpha on October 30, 2013, 01:03:02 am
With this last patch, it works  ;) (at least on the string I mentioned in my previous post).
Okay, I will commit soon, if no further problems are identified.

Now, my job is : extract all "name" strings from .xml file and add them in my .po file.
Maybe a script could extract the strings?  It would likely be easier in the long run, for when the .xml files are updated.
Title: Re: XML based compilers
Post by: gd_on on October 30, 2013, 10:55:20 am
Quote
Maybe a script could extract the strings? 
It's exactly what I do. I have updated my extracting tool and testing it.
The previous version can be found in the forum (links in Code::Blocks' translation posts).
I have added those lines,
Code
echo ""
echo "************************************************"
echo "* extracting strings from .xml compilers files *"
echo "************************************************"
echo ""
find ../plugins/compilergcc/resources/compilers | grep -F .xml | xargs grep -F "CodeBlocks_compiler name" > src_xml.cpp 2>> log.txt
find ../plugins/compilergcc/resources/compilers | grep -F .xml | xargs grep -F "Option name" >>  src_xml.cpp 2>> log.txt
find ../plugins/compilergcc/resources/compilers | grep -F .xml | xargs grep -F "checkMessage" >> src_xml.cpp 2>> log.txt
grep -v mabi src_xml.cpp | grep -v mno | grep -v apcs | grep -v mtpcs | grep -v mshed | grep -v msoft | grep -v mhard | grep -v mfpe | grep -v msched | grep -v mlong | grep -v mpic | grep -v mcirrus | grep -v mcalle | grep -v mpoke | grep -v mwords > src_xml2.cpp
xgettext -a -o xml.pot src_xml2.cpp
find xml.pot >> files.txt
but I think it could be improved because it still extract some lines that do not need translation.

gd_on
Title: Re: XML based compilers
Post by: Alpha on October 26, 2014, 09:13:04 pm
An afterthought:

Looking back on this, I think I made a major design failure.  Moving default settings and flags to .xml I still believe was a good choice, however, as soon as I started trying to do things like <if platform="windows"> ... </if>, it became a hack on the purpose of XML.  Logic does not belong at all in a data file.  This should have been either left in code, or (better) migrated to a script.

I guess this is another item to add to my todo list, and a lesson on planning ahead  :-\ .
Title: Re: XML based compilers
Post by: gh_origin on November 24, 2020, 09:48:45 am
An afterthought:

Looking back on this, I think I made a major design failure.  Moving default settings and flags to .xml I still believe was a good choice, however, as soon as I started trying to do things like <if platform="windows"> ... </if>, it became a hack on the purpose of XML.  Logic does not belong at all in a data file.  This should have been either left in code, or (better) migrated to a script.

I guess this is another item to add to my todo list, and a lesson on planning ahead  :-\ .

Sorry to post on this old thread. I acknowledged any CodeBlocks developers' works but I have to say your XML based compiler really caused me to suffer for almost a whole day. I don't want to be rude but I had to say you did make a completely design failure and I do not think XML was a good choice for anything. Sorry.
Title: Re: XML based compilers
Post by: oBFusCATed on November 24, 2020, 12:29:23 pm
Sorry to post on this old thread. I acknowledged any CodeBlocks developers' works but I have to say your XML based compiler really caused me to suffer for almost a whole day. I don't want to be rude but I had to say you did make a completely design failure and I do not think XML was a good choice for anything. Sorry.
+1 here. Squirrel would have been the better choice for defining compiler configurations, but now it is already too late.
If you want to design the same system based on squirrel scripts I could help you with the review and guidance :)
Title: Re: XML based compilers
Post by: gh_origin on November 25, 2020, 06:56:41 am
+1 here. Squirrel would have been the better choice for defining compiler configurations, but now it is already too late.
If you want to design the same system based on squirrel scripts I could help you with the review and guidance :)
Thank you but I'm only a high school teacher and I don't have the skills needed to do professional coding. I'm not originally teach programming either but because the school's shortage of teacher I attended a short course and started to teach programming alongside with my own elementary algebra. I only know an outdated subset of C++98 and so do my students but recently some of the curious ones started playing with C++11. I can't help with this, sorry.