The victim was fatally injured outside a property on Daniel Hill Terrace, Upperthorpe, close to the city centre.
People living in the area said the man was either in, or close to, a black Mercedes car he had been driving when he was shot.
Officers, called to the scene at about 13:35 GMT, found the man with a gunshot wound to his chest. Armed police sealed off the area.
The victim was taken to Northern General hospital but died from his injuries.
Local residents said the man was a member of a family who live close to the scene of the shooting.
They said relatives had been gathering for a family celebration when the attack occurred.
Many came out of the house to try to help him after hearing the shot, neighbours said.
There have been a number of shootings in Sheffield in recent weeks, although all the previous incidents have been in the north of the city and not close to the Upperthorpe area.
On Monday, a 25-year-old man was seriously injured when he was shot in the street in the Shiregreen area.
Earlier this month, an elderly couple were left shocked when shots were fired through their living room window in the Southey Green area - an incident police believe was a case of mistaken identity.
In January shots were fired at houses in two separate incidents in the same street in High Green.
A 23-year-old man has died after being shot in a Sheffield street.