We (I) still need more context. I have no idea what mex files are. Do you need to compile them? Is this a output file of matlab?
Because i don't know what mex files are i can not help with steps 1) and 2). Also what is the end result of this? A library or a executable? Do you export functions from matlab and use them in your program, or do you export your own functions to matlab?
Create a project with the corresponding wizard (console application or library, i don't know what your output is)
From there on it should be straight forward:
3) Project->Build options
4) Select the project target on the left->Search directories->Compiler Add the right include folder
5) Select the project target on the left->Compiler settings->#defines add MATLAB_MEX_FILE
Close this dialog
6) Project->Properties->Build targets->Select a target on the left (probably debug)->Change the output filename to the extension .mexw64->Remove thick from "Auto generate filename prefix" and "Auto generate filename extension"
Close this dialog with ok
7) Project->Build options->Select the project target on the left->Linker settings->Add the libraries you need (you can add only the names)
Project->Build options->Select the project target on the left->Search directories->Linker->Add the path to the libraries
NOTE: if you are using the mingw compiler you can NOT use the .lib files but there should be .a files.
NOTE: For simplicity: You have to use the SAME compiler as matlab used to generate the .a files (same version, same fork, same everything)
NOTE: You can use the .lib files, but this is getting difficult:
http://www.mingw.org/wiki/MSVC_and_MinGW_DLLs8 ) I have no idea.... Probably someone else can help you, but as far as i know, if you are using mingw compiler you do not need def files
9) Codeblocks has done this automatically for the debug target