Why is my new water heater making a clicking noise?
You probably have sediment buildup in the bottom of the hot water heater.  Pockets of water get trapped in the sediment/scale at the bottom of the hot water heater, turn to steam when it heats up, then "explode" causing the noises you are hearing.

You could try draining the water heater to get some of the sediment out, but honestly it's probably too late for that - might want to have a pro come out and take a look at it.  It's not likely a critical issue, but it would be best to get that scale cleaned off to prolong the life of the unit.