One of the most mind-clearing features of C++, C#, VB, and their ilk are exceptions. Rather than constantly checking a cascading series of return values, looking for false, -1, null, and whatever other error-signalling values you made up on the day you were writing the code, you can write relatively clean and neat code knowing that catastrophic errors (we have no more memory, the file you just chose from a list no longer exists, apparently I'm not allowed to write to the hard drive at all) will be handled. In C++, the "unwinding the stack" aspect of exceptions, with the memory cleanup and destructor-calling done for you, is a big part of writing clean code that is also memory-leak-free.
But have you ever thought about what happens to half-constructed things when an exception goes off? That is, when the constructor throws an exception before its work is done? I have had people tell me "oh simple, never throw an exception in a constructor." Wrong! A major motivation for the existence of exceptions is the existence of methods (like constructors) that don't have a return value to check. The rule you're half-remembering is "never throw an exception in a destructor" and the reason for that is to prevent weirdness when an exception triggers an unwinding and some destructors and then one of them throws another exception.
So what does happen to a half constructed object when an exception is thrown in the constructor? Who better to ask than Herb Sutter? He shows how C++, C#, and Java all answer this question... and his commenters toss in some other languages too.