Partially homomorphic encryption (on addition modulo N) not based on prime factorization
Typical ElGamal is homomorphic with respect to multiplication. You can get an additively-homomorphic variant by encrypting $g^m$ instead of $m$. When you decrypt, however, you will get back $g^m$, not $m$. Therefore, you have to solve a discrete log to get $m$ back. This is feasible if we place certain restrictions on $m$ (e.g., the space from which $m$ is drawn is small enough to brute-force the discrete log). Security of ElGamal relates to solving discrete logs.

You can get an elliptic curve variant of ElGamal by using an elliptic curve group instead of a multiplicative group of integers modulo some number. This will get you an additively homomorphic cipher without the need to solve discrete logs. Security of this cipher would relate to solving discrete logs in the elliptic curve group.

I am not aware of alternatives for XOR.