IOC President Avery Brundage then told the United States , to either send the two athletes home or withdraw the track and field team . In response to the protest , IOC President Avery Brundage told the United States Olympic Committee ( USOC ) to either send the two athletes home or withdraw the track and field team .