Author Topic: cbp2make - makefile generation tool  (Read 203299 times)

Offline blinde

  • Single posting newcomer
  • *
  • Posts: 4
Re: cbp2make - makefile generation tool
« Reply #30 on: March 24, 2011, 05:35:29 pm »
Hi.

In my previous project, all the makefiles are home made.
And dependencies are fully managed.

The makefile can be called with clean / dep / all

dep, will automatically generates some <filename>.dep
it calls gcc with some special options and generate (e.g):

Code
obj/VLX_NoiseMgr.o :   VLX_NoiseMgr.cpp ../../include/CSC_Utils/UTI_Error.h \
 ../../include/CSC_Utils/UTI_types.h \
 /usr/local/include/wx-2.8/wx/wxprec.h \
 /usr/local/include/wx-2.8/wx/defs.h \
 /usr/local/include/wx-2.8/wx/platform.h \

one part of the makefile :


Code
SrcSuf = .cpp
DepSuf = .dep
ObjSuf = .o

OBJDIR = obj

SRCS = $(SRC)
OBJF = $(patsubst %$(SrcSuf),$(OBJDIR)/%$(ObjSuf),$(SRCS))
DEPS = $(patsubst %$(SrcSuf),$(OBJDIR)/%$(DepSuf),$(SRCS))


DEBUGFLAGS = -Wall -g -O2 -D_NO_SHUTDOWN_
CFLAGS     = -D_REENTRANT `$(WXCONFIG) --cxxflags`

INCFLAGS = \
-I../../include/CSC_Spectro \

all:: $(DEPS) $(OBJF)

ifeq ($(MAKECMDGOALS),all)
include $(DEPS)
endif

$(OBJDIR)/%$(DepSuf): %$(SrcSuf)
@echo
@echo '------------------'
@echo -en "\033[30;47;1m"
@echo 'Making $< dependencies...'
@echo -en "\033[30;47;0m\n"
@if [ ! -d $(OBJDIR) ] ; then mkdir $(OBJDIR) ;fi ;
@$(CC) $(DEBUGFLAGS) $(CFLAGS) $(INCFLAGS) $(PROJFLAGS) -MM $< | sed 's/\($*\)\.o[ :]*/$(@D)\/\1.o :   /' >  $@ ;

$(OBJDIR)/%$(ObjSuf): %$(SrcSuf)
@echo '------------------'
@echo -en "\033[30;47;1m"
@echo 'Building $< ...'
@$(CC) $(DEBUGFLAGS) $(CFLAGS) $(INCFLAGS) $(PROJFLAGS) -c $< -o $@ ;
@echo -en "\033[30;47;0m\n"

clean::
@rm -rf $(OBJDIR)

print::
@echo "Dependances"
@echo $(DEPS)
@echo "Fichiers"
@echo $(OBJF)

« Last Edit: March 24, 2011, 06:48:22 pm by blinde »

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: cbp2make - makefile generation tool
« Reply #31 on: March 24, 2011, 06:28:06 pm »
mirai: look at the .depends file; also in the sources you can find a lib that generates these files.
blinde: please use code tags in order to make your posts more readable
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 108
Re: cbp2make - makefile generation tool
« Reply #32 on: March 25, 2011, 02:58:09 pm »
mirai: look at the .depends file; also in the sources you can find a lib that generates these files.
Thanks. I found "depslib" and spent some time digging in its code. According to what I discovered, the only thing that I really need is search for include directives and some magic about file paths to find header files. I think I don't need all of depslib as-is, since half of it is string-related stuff which I already have. And using a full-blown regexp library is an overkill for me here, so I decided to go with a couple of FSA for a couple of languages to parse sources. However, I did not found a piece of code in depslib for processing conditional compilation directives (#define, #ifdef, #endif, #else, etc) and more diffucult to manage #if directive since it may contain expressions. Ignoring these directives will/may result in false includes, and hence, false dependencies. So, my next question/problem is how (or what is the better way) to partially preprocess sources.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: cbp2make - makefile generation tool
« Reply #33 on: March 25, 2011, 03:07:43 pm »
Search the forum, there are some discussion on the topic (and a very recent one in the Code Completion sub forum).
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 108
Re: cbp2make - makefile generation tool
« Reply #34 on: March 25, 2011, 05:04:38 pm »
Search the forum, there are some discussion on the topic (and a very recent one in the Code Completion sub forum).
Well, after reading the topic and code examples I gradually come to a conclusion that I should make some workaround or partial solution, but not the complete one needed for CC. Since the purpose is just to tell 'make' what files it should check for changes before turning a unit into an object file, the procedure of extracting dependencies could be simplified.
I suppose the following algorithm will do the job:

For every unit in a build target:

  • 1) recursively scan for #include directives ignoring #ifdef-s and resolve paths of included files by searching these files in include path.
  • 2) remove duplicate includes.
  • 3) add all successfully found includes to dependencies of current unit's target.
  • 4) forget includes that cannot be found or add them somewhere close to target as remarks.

This way missing dependencies won't trigger 'make' to complain about themselves, but all of existing dependencies will cause a rebuild of certain targets if necessary. There may be a small overhead caused by including false, but existing, dependencies, but this won't break anything.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: cbp2make - makefile generation tool
« Reply #35 on: March 25, 2011, 06:17:39 pm »
So you'll rewrite depslib :lol:
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline Pecan

  • Plugin developer
  • Lives here!
  • ****
  • Posts: 2750
Re: cbp2make - makefile generation tool
« Reply #36 on: March 26, 2011, 03:58:07 pm »
Something like the following might help you get the dependencies that CB already sets up.

Code
//----------------------------------------------------------------------------
void <yourclassname>::WriteMakDependencies(wxFFile& makeFile_fp)
// ----------------------------------------------------------------------------
{
    cbProject* prj = GetProjectManager()->GetActiveCBProject();
    if (not prj) return;

    wxFileName fn(GetProjectManager()->GetActiveCBProjectFilename());
    fn.SetExt(_T("depend"));
    wxString depsFilename = fn.GetFullPath();

    wxArrayString depsarr;
    wxString str;
    // Get array containing lines like "filename;dependfile;dependfile;"
    GetMakeDependencies( depsFilename, depsarr);
    if (depsarr.GetCount() == 0)
        return;

    for (size_t knt=0; knt < depsarr.GetCount(); ++knt)
    {
        wxArrayString adepLine = wxStringTokenize(depsarr[knt], wxT(";"));
        wxString shortFilename = adepLine[0].AfterLast(wxFILE_SEP_PATH);
        wxString filenameBase = shortFilename.BeforeLast(_T('.'));
        if ( not shortFilename.AfterLast(_T('.')).StartsWith(_T("c"))  )
            continue; //not a .cxx file
        if (adepLine.GetCount() > 1) do
        {   //source has dependencies
            wxString depstr = wxEmptyString;
            for (int j=1; j<(int)adepLine.GetCount(); ++j)
            { // append the dependecies;
                adepLine[j].Replace(_T("<"),_T(""));
                adepLine[j].Replace(_T(">"),_T(""));
                depstr.Append( adepLine[j] + _T(" "));
            }
            str.Printf( _T("%s.o: %s\n"), filenameBase.c_str(), depstr.c_str() );
            makeFile_fp.Write( CvtU2C(str), str.Length()); //name.o: dependent files
        }while(0);

        str.Printf( _T("%s.o: %s\n"), filenameBase.c_str(), shortFilename.c_str());
        makeFile_fp.Write( CvtU2C(str), str.Length());  //name.o: cxx filename
        str.Printf( _T("\t$(CC) -c $(CFLAGS) %s\n"), shortFilename.c_str() );
        makeFile_fp.Write( CvtU2C(str), str.Length());
    }

}//WriteMakDependencies
// ----------------------------------------------------------------------------
bool <yourclassname>::GetMakeDependencies(wxString& filepath, wxArrayString& depsarr )
// ----------------------------------------------------------------------------
{

    // Read .depend file into a wxArrayString consisting of lines of
    // sourcefilename;dependfilename;dependfilename;
    // sourcefilename;
    // sourcefilename;dependfilename; etc

    if (not ::wxFileExists(filepath))
        return false;
    wxString str;
FILE *f;
char buf[1024];
//int vmajor, vminor;
wxString h;
int n;
time_t timeval;

depsarr.Clear();

wxFFile file(filepath, _T("r"));

if ( not file.IsOpened() )
    return false;

    f = file.fp();

/* Skip magic */
fgets(buf, sizeof(buf), f);

while (fgets(buf, sizeof (buf), f))
{
buf[strlen(buf) - 1] = '\0'; /* zap newline */

if (!buf[0])
continue;

if (buf[0] == '\t')
{
str.Append( CvtC2U(buf+1) +_T(";"));
continue;
}

sscanf(buf, "%ld %n", &timeval, &n);
if ( not str.empty())
        {
            depsarr.Add(str);
        }
str = CvtC2U(buf+n) + _T(";");
}

    // last buffer
    if ( not str.empty())
    {
        depsarr.Add(str);
    }

//fclose(f); file closed by wxFFile
return true;
}

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 108
Re: cbp2make - makefile generation tool
« Reply #37 on: March 26, 2011, 05:46:15 pm »
Something like the following might help you get the dependencies that CB already sets up.
I would need CB to make this work while one of major requirements is to avoid dependency from anything but .cbp/.workspace file.

p.s. Even just linking with CB SDK is not a solution, this would make me stuck with too large 3rd party code base.
« Last Edit: March 26, 2011, 10:54:31 pm by mirai »

Offline stahta01

  • Lives here!
  • ****
  • Posts: 7582
    • My Best Post
Re: cbp2make - makefile generation tool
« Reply #38 on: March 26, 2011, 10:18:30 pm »
Something like the following might help you get the dependencies that CB already sets up.
I would need CB to make this work while one of major requirements is to avoid dependency from anything but .cbp/.workspace file.


It looks to me that depslib does not depend on Code::Blocks.

Tim S.
C Programmer working to learn more about C++ and Git.
On Windows 7 64 bit and Windows 10 64 bit.
--
When in doubt, read the CB WiKi FAQ. http://wiki.codeblocks.org

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 108
Re: cbp2make - makefile generation tool
« Reply #39 on: March 27, 2011, 12:05:27 pm »
It looks to me that depslib does not depend on Code::Blocks.
I meant that example of WX/CB-based code posted by Pecan.

p.s. Actually I'm already somewhere in the middle of implementing dependency management and will release as soon as it is ready.
« Last Edit: March 27, 2011, 12:09:25 pm by mirai »

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 108
Re: cbp2make - makefile generation tool
« Reply #40 on: April 07, 2011, 03:45:17 am »
Update: (see rev.91)

  • Implemented dependency search for C/C++ source/header files. Use "--with-deps" option to generate makefile with those dependencies for every build unit.
  • Added options to allow use of assembler.
  • Fixed wrong working directory during makefile generation (it should have been changed to the directory of current project/workspace, but it wasn't).
  • Few more docs for relatively stable part of code.


The dependency management feature seem to work although current implementation is mostly experimental. Please, test it with your projects.
p.s. There may be problem with extracting dependencies for projects created by CB on platform different from the one where makefile is created, e.g., extracting dependencies from project of Windows CB may fail in Unix (seems that some paths escaped required conversions).
« Last Edit: April 08, 2011, 11:34:49 am by mirai »

Offline adityagameprogrammer

  • Single posting newcomer
  • *
  • Posts: 8
Re: cbp2make - makefile generation tool
« Reply #41 on: April 14, 2011, 08:54:17 am »
Possible Bug:
A problem in compilation-
Quote
||=== cbp2make, Debug ===|
D:\Aditya\Downloads\cbp2make\cbp2make-stl-rev91\lib\stlfutils.cpp||In function 'CString GetCurrentDir()':
D:\Aditya\Downloads\cbp2make\cbp2make-stl-rev91\lib\stlfutils.cpp|261|error: 'getcwd' was not declared in this scope
D:\Aditya\Downloads\cbp2make\cbp2make-stl-rev91\lib\stlfutils.cpp||In function 'bool ChangeDir(const CString&)':
D:\Aditya\Downloads\cbp2make\cbp2make-stl-rev91\lib\stlfutils.cpp|271|error: 'chdir' was not declared in this scope
||=== Build finished: 2 errors, 0 warnings ===|

i Assume this was due to missing header file. Hence i added it
Code
#include <unistd.h>
in
Code
lib\stlfutils.h 
.It might interest you to know that This was on a windows xp system,with the default gnu gcc compiler. The current version downloaded today 14 April .

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 108
Re: cbp2make - makefile generation tool
« Reply #42 on: April 16, 2011, 12:45:29 am »
Possible Bug:
i Assume this was due to missing header file. Hence i added it
Code
#include <unistd.h>
in
Code
lib\stlfutils.h 
.It might interest you to know that This was on a windows xp system,with the default gnu gcc compiler. The current version downloaded today 14 April .
Hmm... this may happen in Windows if standard libc headers vary among systems (sorry, haven't tested it in Windows yet).
I will check this issue and upload an update ASAP.

Update: (see rev.93) Added missing headers.
« Last Edit: April 16, 2011, 01:40:52 am by mirai »

Offline ripcordjones

  • Single posting newcomer
  • *
  • Posts: 2
Re: cbp2make - makefile generation tool
« Reply #43 on: April 20, 2011, 11:38:14 pm »
Hi,

I love this program.  It has been very useful to me so far.

I am having one problem with it, though.  I have a project under Linux that includes a library where the version number is included in the name of the library.  (It is tcl8.4 if you're curious).  It appears that the program is assuming that the ".4" is a file extension, and so it tries to link without the "-l" flag, as it would when given an object file or the actual file name of a file to link.  Since it is really a library in the library path, the link step fails.

Any suggestions?

Thanks.

Offline mirai

  • Multiple posting newcomer
  • *
  • Posts: 108
Re: cbp2make - makefile generation tool
« Reply #44 on: April 22, 2011, 07:24:28 pm »
It appears that the program is assuming that the ".4" is a file extension, and so it tries to link without the "-l" flag, as it would when given an object file or the actual file name of a file to link. Since it is really a library in the library path, the link step fails.
Well, the program expects that libraries are named as GNU linker expects them to be, i.e., that a static library has file name "lib<something>.a" and corresponding linking option "-l<something>" without prefix "lib" and extension ".a", where <something> can be name or name plus version or anything else. If you have a library that does not follow this naming scheme, cbp2make may fail (and it actually does).

I can try to fix this by adding some logic to check if a library follows active naming scheme and change linking options accordingly.
Another solution is to rename that library file in the way that matches linker expectations or create a well-named symbolic link to the library inside your project tree or somewhere else withing linking path.