Proper use of CD4504BE Level Shifter IC for SPI
Are these connections correct? If not, how should these be connected?


Yes. Except that SELECT pin shall be connected to Ground.  


  I would like to shift this 3.3V signal up to 5V. Is this possible
  with a single IC? My assumption is no


You are right.  


  I would need a second CD4504BE. If it is possible, how should it be
  connected?   


You can opt. for second IC, you have to swap VDD and VCC connection.
But, If you can use single IC solution, you can find many better solutions. As mentioned in the comment the 4 bit bidirectional level translator, TXS0104E,  can do the job of both ICs.    

Here is the block diagram with the voltage rating:
ESP device will be the system controller(block to the left) and Arduino will be the system (block in the right).  Voltage ratings suggest that, it meets your expectation.  

 

for MISO line
you can also consider this and works for sure.  

Low Side - ESP  
High Side - Arduino  




The above circuit can be simply repeated thrice and you do not need the level shifter IC at all.   You just have to swap Low side and High side for MOSI, CS and CLK pins.