DC motor and encoder for pendulum with low static friction
The problem with motors is that while static friction and dynamic friction can be addressed by selecting the right bearings, many motors have other undesiderable characteristics:  


Brush friction is present in all brushed motors.
Torque ripple affects all electric motors to some degree, with the exception of the impractical homopolar motor. The reluctance of the magnetic circuit changes as the rotor and stator poles step in and out of alignment while the rotor turns. This causes a torque which tries to align the rotor at certain specific angles. 
Cogging torque is a specific case of torque ripple which occurs in all slotted permanent magnet motors and persists even when the motor is at rest and unpowered. Affected motors include permanent magnet brushed DC motors, brushless DC motors and stepper motors. Coreless DC motors, induction motors, reluctance motors and electrically excited (field winding) DC motors are immune.  


Torque ripple can be alleviated by increasing the motor pole count or adding a transmission between the motor and pendulum, but this will obviously cause additional friction and add inertia to the system.  



Given the fairly specific requirements, it might make more sense to use a voice coil rotary actuator (like what is found in hard disk drives) rather than a motor. 

  

Not only does such an actuator have zero torque ripple, but there is no physical contact and the moving part can be made very lightweight to reduce its impact on the system. The variations in torque in response to position can be fairly linear as well, and are easily compensated out:  

  

The downside is a reduced range of motion.