How do I check the functionality of a 4511 x 7 segment display?
You have left some input pins unconnected i.e. "floating". Doing this is a bad idea and can lead to problems, including excessive power consumption i.e. increased IC temperature.

This situation is described in the Wikipedia page for the 4000 series as follows:


  
    An engineer who forgets to 'tie off' (connect the unused gate inputs to VSS or VDD) may find the chip draws excessive current. The problem is caused by biasing in each gate. With the inputs disconnected, the gates may be biased into a mode where the outputs are partially conducting; this leaves the output buffer drawing a great deal of current since it is not fully on or off, creating a low resistance current path between the power supply rails.
  


You will find similar warnings in other databooks and manuals from the era when those ICs were introduced.

With CMOS devices you must ensure that all inputs are driven (or pulled-up/down) to either a valid logic low or a valid logic high.

Although the 4000-series is older than the CMOS IC families mentioned here by TI, their document "Implications of Slow or Floating CMOS Inputs" explains more about the general problem.