Are experimental aircraft exempted from the 2 year transponder and avionics checks?
When you operate in airspace that is under FAA regulations, FAR 91 applies (unless you are flying military)  

FAR 91.413 a) states:


  (a) No persons may use an ATC transponder that is specified in
  91.215(a), 121.345(c), or §135.143(c) of this chapter unless, within the preceding 24 calendar months, the ATC transponder has been tested
  and inspected and found to comply with appendix F of part 43 of this
  chapter; 


So when you haven't performed the two year transponder check, you are not allowed to use the transponder. The way to fly then is to fly VFR in airspace where no transponder is required AND not to use the transponder at all. 

This is not specific to experimental aircraft, there is no exemption.

There is good reason not to have an exemption for experimental aircraft; a malfunctioning transponder can cause many problems to the ATC radar systems.