How does a convolutional ply differ from an ordinary convolutional network? I am currently working on recreating the results of this paper. 
In the paper they  describe a method for using CNN for features extraction, and have a acoustic model that is Dnn-hmm and pretrained using RBM. 

Section III subsection A states different ways the input data can be represented. I decided to vertically stack the spectrum plots of the static, delta and delta deltas. 

So as such: 


The paper then describes how the network should be.  They state that they use a convolutional network, but nothing about the structure of network?. 
Further more is the network always refered to as a convolutional ply? which i am sure i see any difference in  compared to an ordinary network convolutional neural network (cnn).

The paper states this regarding the difference: 

(from section III subsection B)


  A convolution ply differs from a standard, fully connected hidden
  layer in two important aspects, however. First, each convolutional
  unit receives input only from a local area of the input. This means
  that each unit represents some features of a local region of the
  input. Second, the units of the convolution ply can themselves be
  organized into a number of feature maps, where all units in the same
  feature map share the same weights but receive input from different
  locations of the lower layer


Another thing i was wondering is whether the paper actually states how many output parameter is needed to feed the dnn-hmm acoustic model. I can't seem to decode the number of filters, filters sizes.. in general details of the network?
It seems that a convolutional ply is exactly the same as an ordinary convolutional layer. From their paper, they argue that the term "CNN layer" usually refers to a convolutional layer followed by a pooling layer. In an attempt to reduce confusion, they name the convolutional part a "convolution ply" and a the pooling part a "pooling ply":


  In CNN terminology, a pair of convolution and pooling layers in Fig. 2 in succession is usually referred to as one CNN “layer.” A deep CNN thus consists of two or more of these pairs in succession. To avoid confusion, we will refer to convolution and pooling layers as convolution and pooling plies, respectively.


Ironically, this has increased the confusion, leading to this post. At the time I suppose it was not common to have several convolutional layers in a row before a pooling layer, but we see this often in modern architectures. 

To answer your other question about the network structure; they state the structure of the network that they use in the Experiments section (Section V-B). To hopefully reduce confusion, I've replaced the word "ply" with "layer":


  In these experiments we used one convolution [layer], one pooling [layer] and two fully connected hidden layers on the top. The fully connected layers had 1000 units in each. The convolution and pooling parameters were: pooling size of 6, shift size of 2, filter size of 8, 150 feature maps for FWS, and 80 feature maps per frequency band for LWS.