Washington ( CNN ) -- Federal authorities are investigating a potential case of human trafficking at the suburban Washington home of a Saudi diplomat , officials said Wednesday . Authorities were called Tuesday afternoon to an upscale neighborhood in McLean , Virginia , where they `` removed two potential trafficking victims , '' U.S. Immigration and Customs Enforcement spokesman Brandon Montgomery told CNN . A State Department official said the McLean house is the home of the Saudi military attachÃ© . The case is under investigation by Homeland Security Investigations , a unit within ICE . The two women , who are from the Philippines and currently work at the Saudi Embassy in Washington , claim they were mistreated , according to the State Department official . The women are charging , according to a different State Department official , that the Saudi attachÃ© kept their passports , made them work extremely long hours and did not pay them . The source said they had not seen anything to indicate the women were physically harmed . The official also pointed out these allegations are similar to several other cases they heard from domestic workers who work for diplomats from the Persian Gulf . According to ICE , cases like this are very `` victim-centric , '' meaning that once there is `` confirmation of suspicions , '' authorities immediately go in and remove the victim and then begin the investigation . There was no word on the identity of the Saudi diplomat who lived there . The Saudi Embassy in Washington did not respond to requests for comment . CNN 's Dan Merica contributed to this report .
NEW : Source says women charging Saudi attachÃ© kept their passports , did n't pay them . Authorities investigate possible human trafficking . Official says probe is at home of Saudi military attache . Two women allegedly claim they were mistreated .