The reason we copy the THNN.h and THCUNN.h header files into torch/lib is that this is where the generate_nn_wrappers() code expects these files to be located. generate_nn_wrappers() does a few things:

Parses the header files, generating cwrap YAML declarations and writing them to output .cwrap files
Calls cwrap with the appropriate plugins on these .cwrap files to generate source code for each
Parses the headers a second time to generate THNN_generic.h - a library that takes THPP Tensors, PyTorch's "generic" C++ Tensor Library, and calls into the appropriate THNN/THCUNN library function based on the dynamic type of the Tensor

If we take a look into torch/csrc/nn after running generate_nn_wrappers() we can see the output:
(p3) killeent@devgpu047:nn (master)$ ls
THCUNN.cpp  THCUNN.cwrap  THNN.cpp  THNN.cwrap  THNN_generic.cpp  THNN_generic.cwrap  THNN_generic.h  THNN_generic.inc.h

For example, the code generates cwrap like:
```
[[