Don't waste your time any further, Biplab. I'll give up. Thank you for trying, though.
The failure is about 20,000 lines of configure and make shit, followed by "compiler can't create executables". This is independent to what combination of --host --target and --build I provide.
I've tried what you said, tried what the abysmal available documentation tells you, tried what the even more abysmal "canadian cross howto" tells you.
It doesn't work with the MinGW32 cross compiler that Ubuntu provides, either (providing /usr as sysroot, as it's in /usr/i586-mingw32msvc). The exact same compiler works just fine for building programs with Code::Blocks, and they run under Windows with no problems.
The GMP build script will also output a message that --target is not appropriate (hidden in 20,000 lines of useless crap that nobody ever wants to read), but it builds fine. If you do as it tells you, it will compile everything using the native compiler... great.
See, this is why GNU and Linux will never be competitive. They have to make everything super configurable and portable for 3500 platforms that nobody uses, but they just can't make it work out of the box with 2-3 simple commands on a plain normal mainstream PC, nor can they provide clear and understandable instructions what to do to get the desired result.
They can't even give you an error message without requiring you to scroll through 30 screen pages of useless information. Bah!