Mercedes driver Nico Rosberg said parts of the track in the Azerbaijani capital were "really not looking good".
Button added that some corners "don't have any run-off at all".
Rosberg hoped governing body the FIA had stuck to the safety guidelines that govern tracks, but added: "I'm doubting it a little because of those corners."
The main areas of concern on the new track, which joins both the new and historic old parts of the city and is expected to be the fastest street circuit in F1 history, are turns three, seven and 15 and the pit-lane entry.
Button said: "We trust in the FIA and they do a fantastic job in keeping us safe. They have improved pretty much every circuit we go to, including Monaco.
"It's just that on certain corners here, it looks as if there is not a lot of run-off. Hopefully I'm wrong."
He added: "It is fine as long as no-one has a failure, we will go away all happy. We just have to hope that doesn't happen."
Button said the most critical area was turn 15, which the drivers will approach at estimated speeds of about 180mph after a long flat-out section.
He said: "There is run-off but you have to turn right to take it."
Rosberg added: "The track is great, the place is great. I have been welcomed very well so I'm sure all in all it will be a great success.
"But there is a bit of a concern for sure with those run-offs. That's not great. The corner before going down the hill (15) where you have a wall straight on, turn three, and then the pit entry as well. that is not fantastic."
The pit-lane entry is towards the end of a 2.1-kilometre straight, where cars are expected to be travelling in excess of 200mph, and features a tight chicane.
The area governed by the pit-lane speed limit does not start until after the chicane.
Red Bull's Daniel Ricciardo said the pit entry "is going to be quite cool" because, as it is effectively part of the race track, there was "between half-a-second and a second" for a driver to gain in the chicane.
But Rosberg countered: "Yeah but there is also a massive accident to be had if you go a little bit more than that."
McLaren's two-time champion Fernando Alonso, who is the race's official ambassador, played down the issue.
"There are some corners probably we will talk about," he said. "Some others that we need to go on the car first and see how they feel.
"In terms of safety, the FIA ran many simulations in terms of how to make the run-off safe enough. Some of them are safe enough, some of them maybe they look small but it's always better than Monaco, for example. It cannot be any worse than that."
The FIA has yet to answer BBC Sport's request for a response.
Subscribe to the BBC Sport newsletter to get our pick of news, features and video sent to your inbox.
Leading Formula 1 drivers have questioned the safety of the new Baku street circuit that will host Sunday's European Grand Prix.