Iron Filing Catalyzed Toluene Reaction with Bromine Solution
The iron and bromine must first react to form the catalyst (FeBr3) as the iron does not actually catalyze the reaction (this is known as in situ synthesis of the catalyst).  The FeBr3 catalyzes the reaction between toluene and Br2 to form ortho and para bromotoluene.  Catalysts are NOT consumed in the reaction though, which means the culprit of the rusty color is the catalyst (google iron(iii) bromide to view the color).

Also, ensure that you use a larger excess of the bromine.  The FeBr3 interacts with bromine to make it more electrophillic and able to react with the benzene ring of toluene.  If there is no bromine left after the catalyst has been formed, no reaction will occur.