Mirroring / Copying Constraints over a whole rig
To use the Copy Constraint, you select an object then add a Copy Constraint, then select which other object you want to copy the property from (location, rotation). So no need for paste constraint.
For mirroring the armature, you will find x-mirror property in the tools menu in the edit mode of the armature. Just replace the name of the Armature with the name of your Armature.

UPDATE:

To do what you want it will be easier to use python. I wrote a script that copies constraint from bones with ".L" to corresponding bones with ".R"

import bpy
context = bpy.context
scene = context.scene
MyArmature = bpy.data.objects["Armature"]
bpy.ops.object.select_all(action='DESELECT')
MyArmature.select = True
bpy.context.scene.objects.active = MyArmature
bpy.ops.object.posemode_toggle()
bpy.ops.pose.select_all(action='SELECT')
all_bones = bpy.context.selected_pose_bones
for bone in all_bones:
    bpy.ops.pose.select_all(action='DESELECT')
    if ".L" in bone.name:
        boneL = MyArmature.data.bones[bone.name]
        boneR = MyArmature.data.bones[bone.name.replace('.L','.R')]
        boneR.select = True
        boneL.select = True
        MyArmature.data.bones.active = boneL
        if len(MyArmature.pose.bones[boneL.name].constraints) !=0:
            bpy.ops.object.constraints_copy()
            bpy.ops.pose.constraints_copy()
            for con in MyArmature.pose.bones[boneR.name].constraints:
               con.subtarget = con.subtarget.replace('.L','.R')


This is how it looks before running the script:



And This is how it looks after running the script where the constraints are copied:



Updated .blend file can be found here: