When was the first rudder on an aircraft used?
Planes as early as the wright glider (1902) had a rudder, it appears from pictures that the 1900 and 1901 variants did not have rudders but its hard to tell from the photos. 



Depending on your definition if you are talking about powered flight the Wright Flyer (1903) had 2 rudders much like its earlier glider predecessor. 

  

In the lighter than air category, the Giffard Dirigible (1852) had a sail like rudder in the aft of the ship. So that may take the cake if you have consider it a true rudder.  



Side Note: Patented in 1930 the V-Tail design could be argued as the first design that did not (as far as I know) employ a rudder. The earlier Beechcraft Bonanza's are the most common plane of that design in commercial production. There is quite a bit of debate as to the safety of ruddervators and the general theme seems to be that they are not perfect and do require some care to fly properly.