You cannot turn it off. (You could modify the sources, but I won't help on this because it is madness).
This is a feature that ensures that no matter what happens, you never lose all your data (even if you pull the plug while saving files). As such, it cannot and should not be turned off.
Also, it should do something different: it creates a temporary file, writes to it, renames the old file, renames the temporary file, and deletes the old file after a grace period.
The behaviour that you describe is what wxTempFile does, but we don't use that any more, unless we have forgotten to change it somewhere. It is unsafe because it can delete both files under certain (although unlikely) conditions.