It does not matter an awful lot, you can have a long and happy life either way. Everybody should do what he thinks is best. Personally, I keep headers, libraries, and all that makes up one "unit" together, and separate from the rest. Not only libraries and headers, but also the compiler and the IDE. I believe, this is "best".
This strategy allows me to use a global compiler variable for lib/include configuration and switch between different versions quickly and painlessly when necessary.
It also allows me to replace Code::Blocks with a fresh version without touching anything in MinGW and without worrying about any headers and libraries (and vice versa).