Diffuse (cos(theta) or Random) Scattering of Particle Collisions
Rough Surface
You could add some roughness to your collision object or to your particles. Fist Subdivide it. Next give it a Displace modifier with some texture. Cloud texture seems to work just fine. Then enable physics for Collision. The order of your Modifier stack is important. Collisions has to be last and Subdivisions has to be first.


Force Field Turbulence
Another option would be to add a Turbulence Force Field next to the wall.
Strength of 100 and Max Distance of 100 cm seem to work well for me.