Are there asymmetric cryptographic algorithms that are not based on integer factorization and discrete logarithm? In the computer security class (in which cryptography is a big chapter) that I took, I remembered the professor said about current asymmetric cryptography algorithms are based on integer factorization (i.e. prime numbers) and discrete logarithm.

So my question is, are there any asymmetric cryptographic algorithms that are not based on these two mathematical fields? Is it that hard to come up with a strong algorithm without using prime numbers and logarithm?
Yes there are other hard problems you can base asymmetric cryptography on. 

Lattices. The NTRU systems is based on the shortest vector problem in ideal lattices. Lattice-based cryptography is of much interest these days for two reasons: (1) unlike factorization and discrete logarithms, there isn't an efficient algorithm for breaking these problems on a (still theoretic but should happen within our lifetime) quantum computer; (2) you can use lattice-based systems to do arbitrary operations on data while it is still encrypted (not with NTRU though but other lattice-based systems). 

Coding Theory. The McEliece system is based on decoding linear codes. It is a very old system but recently has seen renewed interest for the same post-quantum reasons as lattice-systems.

Misc. There are other more obscure (research-level) cryptosystems based on things like learning parity with noise, learning with error (related to SVP in lattices), subset sum problem (including a famous broken example), solving multivariate or diophantine equations, etc. 

In response to your second question, is it that hard to find alternatives, the answer is it is reasonably hard. For most cryptographic primitives, you need to find examples of functions that are hard to invert, and for asymmetric encryption specifically, you are generally interested in functions that have a specific value that allow them to be inverted (but only if you know that value). See trapdoor functions. It is difficult to find new candidates for trapdoor functions and to establish they are suitably hard.