Can PWM affect a brushless DC fan given sufficient time?
DC computer type fans ARE NOT DC motor driven fans, they are BLDC motors, and universally contain either an MCU or logic controller to start and run the fan. Running these fans on a PWM supply is not recommended, and it might well damage the components in some controllers over time.
This question covered the types of controller and may help your understanding.

The click you occasionally hear from your fan is in all probability the controller trying to do the start logic (in other words it thought the power had been cycled). These computer type fans are two phase and the controller has to ensure they will start in only one direction so have stall/kickoff logic. With a 2 wire fan it is close to impossible to sense if the fan has stalled, so their use is very problematic. 

Depending on how advanced the controller is, it's quite possible you could damage one of the capacitors in the circuit supply over time and this might be what you are seeing occur.   

Most motherboards (and I understand you might not be using the fans in a computer) that PWM the fans (typically 3 wire) are semi intelligent too, they start by applying 12V and then back off to a lower voltage to slow the fan. Since they are getting a rotation signal (3 wire) they can sense when the fan slows and increase the voltage. This was common driving methodology on older Server computers, but fan failure was quite common. 

Since the cost differential is low, it is worth going to 4 wire fans where the PWM signal is actually going to an MCU, and it is responsible for controlling the fan speed. The controllers for 4 wire fans are very commonly available.   

If you MUST control the DC voltage, then I'd suggest you do not use PWM, and be aware that the range of control is non-linear, and restricted. Most fans show DC control limited to about 5V DC on a 12V fan.  You might also read this question which covered PWM of a fan for non-computer use.