Liam Laverick, 25, from Beeford Grove in Hull has pleaded not guilty to the manslaughter of his four-week old son, Tommy Lee.
The baby was taken to Hull Royal Infirmary on 23 September 2014 and died later at Leeds General Infirmary.
Tommy Lee had been left in the sole care of Mr Laverick when the baby became ill, Hull Crown Court heard.
Prosecutor Nicholas Lumley QC said in his opening statement Tommy Lee was left in the care of Mr Laverick at his flat on Linnaeus Street, Hull.
He also told the court Mr Laverick had killed his son by shaking or throwing him.
The jury was shown CCTV footage of Mr Laverick taking Tommy Lee to a relative's house to find the baby's mother, Kelly Whitworth, before the couple ran with the child to the nearby Hull Royal Infirmary.
Mr Laverick gave different accounts of what had happened, including saying that he had fallen over while carrying the child, the prosecution said.
A recording of Mr Laverick speaking to Tommy Lee's mother at a police station and made without their knowledge was also played in court.
In it Mr Laverick described falling over in the hallway of their flat while carrying the baby.
The forensic pathologist who carried out the post mortem examination on the body of Tommy Lee said he had suffered serious bleeding around his brain and the back of his eyes.
The case continues.
A father killed his son by shaking or throwing him when he was just four weeks old, a jury has been told.