The US network said it was "not an easy decision" and had "enormous respect" for its creators and cast.
Set in the 1970s, the series told of a charismatic record executive trying to revive his struggling label.
Its ensemble cast included Bobby Cannavale in the lead role, Olivia Wilde as his wife and Sir Mick's son James as a rebellious British singer.
Sir Mick and Scorsese were among the show's executive producers, with the latter directing its pilot episode.
The series premiere in February was watched by 764,000 viewers, though the average audience subsequently dipped to 650,000.
"After careful consideration, we have decided not to proceed with a second season of Vinyl," HBO said in a statement.
"We have enormous respect for the creative team and cast for their hard work and passion on this project."
Ray Romano, another actor in the cast, told an Australian newspaper on Thursday the cancellation felt like "a knife in the guts".
Wilde, meanwhile, thanked her Twitter followers for their "kind words" and said she had been part of "something special".
Sir Mick Jagger and Martin Scorsese's music business-based drama Vinyl has been cancelled by HBO after one season.