The Dow Jones closed down 122.65 points, or 0.61%, at 19,971.13.
The S&P 500 index dropped 13.8 points, or 0.6%, to 2,280.89, while the Nasdaq fell 47 points, or 0.83%, to 5,613.7.
Airline stocks fell, with American Airlines down 4.4% and United Continental dropping 3.8%.
Michael Hewson, chief market analyst at CMC Markets UK said US markets had taken their lead from weak European markets "as investors wake up to the downside of what a Trump presidency might mean for risky assets".
"The weekend executive order to implement more stringent vetting procedures to visitors to the US from specific destinations, as well as a wider travel ban, could well be the beginning of this realisation.
"It is perhaps no surprise that the tech sector has been the most vocal given that its highly skilled workforce has a higher proportion of overseas workers due to the higher skill set required."
Stocks on Wall Street fell saw their biggest drop so far this year as investors took a cautious stance in the face of uncertainty over Donald Trump's travel and immigration curbs.