The Long Island Railroad train went off the tracks as it arrived on Wednesday at Brooklyn's busy Atlantic Terminal.
New York's fire department said 103 people had reportedly suffered "non-life-threatening" injuries.
Passengers on the train said there had been a loud bang followed by a jolt that made some people on the train fall down.
The New York governor, Andrew Cuomo, said the train was travelling at a "fairly slow speed" when it failed to stop on time and struck a buffer at the station.
The impact was hard enough, however, to smash some of the train's windows and crease its doors.
Some of the injured were taken away on stretchers, while others were seen sitting near the train holding ice packs to their heads.
The cause of the accident, which happened at about 08:20 local time (13:20 GMT), has not been determined.
The US Federal Railroad Administration said its investigators were heading to the scene.
Last September, a woman died and more than 100 other people were injured when a train derailed during rush hour as it entered the station in Hoboken, New Jersey. An investigation into that incident is yet to be concluded.
More than 100 people were injured when a commuter train derailed during rush hour in New York City, officials say.