Implementation of the paper 'Perceptual Generative Adversarial Nets for small object detection'
The generator function is just a CNN that maps an image to a feature map here (well, ROI pooled anyway). 
There is nothing specific to it being a GAN; what makes something a GAN is its use as a density estimator and how it is trained (i.e., adversarially), not its architecture. So don't worry about the lack of familiarity with GANs.

In any case, the complex part here, in my opinion, is the presence of the object detection framework parts. To better understand it, though you probably have done so already, I suggest reading the "RCNN trilogy papers": RCNN, Fast RCNN, and Faster RCNN, or at least the middle one, which is relevant here. See also this post and this one about ROI pooling, which takes the dis-similarly sized ROIs and maps each of them to a small fixed-size feature map.

The rest of the architecture are standard convolutional layers mostly, with which you are presumably familiar. It might be useful to review ResNet though, since ResBlocks are used here.
The core ("backbone") of the pipeline relies on the VGG network; its use is explained in more detail in references 24 and 41 of the paper.

Anyway, here's what's happening. Given an image $I$ with proposals $P$ (where $n=|P|$), we first run conv1 on $I$, giving us the low-level feature map $L$. Two things then happen:


On the "top path", $L$ is run through convi for i=2:5, resulting in a new feature map $L_T$, which is then ROI pooled via $P$. This gives us a stack of proposals $S_1,\ldots,S_{n}$, each of which is a featurized image.
On the "bottom path", $L$ is run through two small conv layers and then ROI pooled as well, giving a similar proposal stack $f_1,\ldots,f_{n}$. These are then processed by $B$ standard ResBlocks, to give us $n$ residuals $r_1,\ldots,r_n$.


Finally, we get the ultimate output of the generator, which is a stack of super-resolved proposals given by $$ Y = (y_1,\ldots,y_n) = ( S_1 \oplus r_1,\ldots, S_n \oplus r_n ) $$
where $\oplus$ here just means element-wise sum (i.e., S_k + r_k in most array-based languages).

Basically, the problem is that the ROI pooled features will "look nice" for big objects but terrible for small ones, due to the fewer number of pixels in the latter. The fix for this here is to do "standard" object detection in the top path, but learn a "residue" $r_j$ in the bottom path for each ROI, so that when this residue is added to a small proposal $S_j$, it is given the details normally only present in large proposals. In other words, we are basically doing super-resolution on the small ROIs to help the detector on small objects. 



As the paper authors write in the caption:


  The generator is a deep residual network which takes
  the features with fine-grained details from lower-level layer as input and passes them to 3 × 3 convolutional filters followed by 1 × 1
  convolutional filters to increase the feature dimension to be aligned with that of “Conv5”. Then B residual blocks each of which consists
  of convolutional layers followed by batch normalization and ReLU activation are employed to learn the residual representation, which is
  used to enhance the pooled features from “Conv5” for small objects to super-resolved representation through element-wise sum operation.


As well as in the main paper:


  As shown in Figure 3, the generator takes the feature
  from the bottom convolutional layer as the input that preserves many low-level details and is informative for feature
  super-resolution. The resulting feature is first passed into
  the 3 × 3 convolution filters followed by the 1 × 1 convolution filters to increase the feature dimension to be the
  same as that of “Conv5”. Then, B residual blocks with the
  identical layout consisting of two 3×3 convolutional filters
  followed by batch-normalization layer and ReLU activation
  layer are introduced to learn the residual representation between the large and the small objects, as a generative model.
  The learned residual representation is then used to enhance
  the feature pooled from “Conv5” for the small object proposal through RoI pooling [11] by element-wise sum operation, producing super-resolved representation.