Is creating a own protocol safe or should TLS always be used? The more I start to learn about cryptography the more I hear it is unsafe to 'create' your own cryptographic protocol. With this I mean combining cryptographic primitives like HMAC, AES and RSA

However the only protocol, which is also suited for small embedded devices, I know of is TLS. are there any other protocols similar to TLS which can provide: privacy, authentication and verification (identification and trust are not hard requirements). I use these terms as they are described in this explanation of cryptography (i found this to be the best explanation if I want people to know what I'm talking about)

currently I'm just thinking about combining primitives like mentioned above (AES, HMAC, etc) to create a secured communication for my embedded devices but can this be done or should one really try to avoid this? If this should be avoided are there other protocols like TLS which could do what I want?

EXTRA information:
The embedded devices are (often) placed in secure environments, the communication however, is not going through a secure environment.
specs are: 100mhz mcu and about 20 kb ROM/RAM
Designing your own crypto protocol (using existing primitives) is dangerous if you're not sufficiently familiar with cryptographic protocol design and the ways such protocols might be attacked.

If you wish to gain such familiarity, I'd recommend taking a few introductory crypto courses that focus on protocol design and analysis.*  This won't turn you overnight into an experienced protocol designer, but it should at least give you some idea of how much you actually (don't) know about the subject.

Also, regardless of you own skill level, a critical part of designing a secure communications protocol is to get other experienced cryptographers to review it and look for ways to attack it.  Even the most skilled cryptographers can make mistakes and overlook potential attacks, so it's essential to have as many eyeballs on the system as possible.
To paraphrase Bruce Schneier, anyone can design a cryptosystem which they cannot break &mdash; what's hard is designing something that others cannot break, either.

This is also one of the main reasons to go for existing standard solutions whenever possible: they've already been subject to more scrutiny by skilled cryptographers than your home-brewed protocol is likely to receive in any reasonable time.  Thus, by leveraging existing standard protocols and implementations, you get to enjoy the benefits of this pre-existing third-party scrutiny.

(Of course, there are standards and then there are standards.  Something published in an IETF RFC or an NIST SP is probably fairly safe (except when it isn't), whereas something found in a random article in The International Journal of Computer Engineering and Agricultural Machinery may actually be complete bollocks.  When in doubt, look for credible reviews by well known competent cryptographers.)

As for TLS specifically, I'd say that it's a good choice in general, but hardly perfect; its major weakness is its complexity, which makes it hard to analyze thoroughly, and easy to make mistakes in implementing it.  Using an existing, thoroughly reviewed implementation helps, but even widely used TLS implementations have been known to carry unfixed bugs for a long time.  Especially on constrained platforms, where the complexity and overhead of TLS may be problematic in themselves, a well written and reviewed custom protocol may sometimes be a better choice than TLS &mdash; but it all depends on how carefully the protocol has been designed and implemented.



*) I'm probably not the best person to recommend any specific courses, but in general terms, I've heard good things about Dan Boneh's Crypto I and II on Coursera, for example.  There are also several decent books on the subject, but again, I hesitate to give any specific recommendations for lack of personal familiarity.