Accumulating inner exception messages
Svick is right

Use the AggregateException class to collect all exceptions you need. With AggregateException everyone can prepare it's own code to handle the multiple exception situation but with your original solution they have to handle a huge string in a stock exception class. How can be this usefull?

Storing exception.Message

It's bad. If you store only the exceptions' messages you will loose a lot of information and the types of the exceptions. How would you handle an exception if only the Message is what you have? Parsing it? And we havent talked about the specific properties of specific exceptions like some validation exception which can contain the unvalid property name and value for an object.