Can pilots turn off warning alarms? Re: Aeroperu Flight 603
Sometimes they can sometimes they cant, how to do it varies by aircraft but generally alarms can often be silenced by acknowledging they are going off and pressing some kind of button. On my little Piper Archer the alternator under volt it just a light and can't be shut off but if my G430 throws an alarm or warning it can usually be silenced. 

On the 737 some alarms cant be silenced and some can 


  Cockpit aural warnings include the fire bell, take-off configuration
  warning, cabin altitude, landing gear configuration warning,
  mach/airspeed overspeed, stall warning, GPWS and TCAS. External aural
  warnings are: The fire bell in the wheel well and the ground call horn
  in the nose wheel-well for an E &amp; E bay overheat or IRS’s on DC. Only
  certain warnings can be silenced whilst the condition exists.


A lot of aircraft display failures on some kind of annunciator panel or glass cockpit equivalent. These lights typically can not be turned off until the error is addressed (fire suppressed, breaker pulled etc.). Master Caution lights (and audio warnings) can typically be shut off through the use of a Master Caution Reset switch. 


  Once notified, the pilot may cancel the master caution, but a
  dedicated system or component annunciator light stays illuminated
  until the situation that caused the warning is rectified. Cancelling
  resets the master caution lights to warn of a subsequent fault event
  even before the initial fault is corrected.


The FAA's full advice on warning and caution light design can be found here.

They shed more light on the topic in this AC 


  
  Clearing and Recalling Visual Alert Messages. Clearing visual alert messages from the current warning, caution, and advisory display
  allows the flightcrew to remove a potential source of distraction and
  makes it easier for the flightcrew to detect subsequent alerts. a. The
  following guidance should be applied for clearing and recalling or
  storing the visual alert messages: 
  
  
  (1) If a message can be cleared and
  the condition still exists, the system should provide the ability to
  recall any cleared visual alert message that has been acknowledged. 11
  12/13/2010 AC 25.1322-1
  
  (2) Either through a positive indication on the display or through
  normal flightcrew procedures, a means should be provided to identify
  if alert messages are stored (or otherwise not in view). b. The visual
  alert message must be removed from the display when the condition no
  longer exists (§25.1322(a)(3)).


In short, if a message can be cleared you should be able to pull it back up easily and if the problem goes away the alert should clear as well.