Method: "Eigen::SpatialConvolutionBackwardKernel" in "tensorflow/core/kernels/eigen_backward_spatial_convolutions.h" is painfully slow !
So, I've tried experimenting. Adding force eval past the patch extraction / reshape on the input tensor:
```
input
.extract_image_patches(
kernelRows, kernelCols, row_stride, col_stride,
row_in_stride, col_in_stride, 1, 1, padding_top,
padding_bottom, padding_left, padding_right, OutScalar(0))
.reshape(pre_contract_dims).eval()
```
makes this method 3x faster, from 9,155 seconds to 3.026 seconds on the previously aformentioned (64,800,800) input and backward output with (64,64,5,5) kernel, which is more in-line with expected performance. 
Please, look into this further. Might have found even better solution...
Switching the order of contraction from output.contract(input) to input.contract(output) without any force eval and with post contraction shuffling, resulted in even faster speeds without possibly higher memory consumption: 2.2 seconds instead of the original 9 seconds or the 3 second with force eval code.
Example of the return statement for col major layout (of course template arguments have to be modified accordingly): 
``` 
input
.extract_image_patches(
kernelRows, kernelCols, row_stride, col_stride,
row_in_stride, col_in_stride, 1, 1, padding_top,
padding_bottom, padding_left, padding_right, OutScalar(0))
.reshape(pre_contract_dims)
.contract(
output_backward.reshape(output_dims),
contract_dims).shuffle(DSizes<TensorIndex, 2>{1,0})
.reshape(kernel_dims)
```
This does work with 3D input/output tensors correctly, ~~didn't test it with higher dimensional outputs, but I have feeling there has to be some additional shuffling to be done.~~
Confirmed that the method behaves correctly with 4D Tensors, it is even up to 5x faster than the original with batch size of 16. > Might have found even better solution... Switching the order of contraction from output.contract(input) to input.contract(output) without any force eval and with post contraction shuffling, resulted in even faster speeds without possibly higher memory consumption: 2.2 seconds instead of the original 9 seconds or the 3 second with force eval code. Example of the return statement for col major layout (of course template arguments have to be modified accordingly):
> 
> ```
> input
> .extract_image_patches(
> kernelRows, kernelCols, row_stride, col_stride,
> row_in_stride, col_in_stride, 1, 1, padding_top,
> padding_bottom, padding_left, padding_right, OutScalar(0))
> .reshape(pre_contract_dims)
> .contract(
> output_backward.reshape(output_dims),
> contract_dims).shuffle(DSizes<TensorIndex, 2>{1,0})
> .reshape(kernel_dims)
> ```
> 
> This does work with 3D input/output tensors correctly, ~didn't test it with higher dimensional outputs, but I have feeling there has to be some additional shuffling to be done.~ Confirmed that the method behaves correctly with 4D Tensors, it is even up to 5x faster than the original with batch size of 16.

Feel free to submit a pull request with your changes. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62920">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62920">No</a>
 I'm just really stupid, I was looking at totally different TF version. Sorry to waste your time. Still old v1.13.1 backward convolution code with my modification is 20% faster than the current version in 2.15.
 Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62920">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62920">No</a>

### Issue type

Performance

### Have you reproduced the bug with TensorFlow Nightly?

No

### Source

source

### TensorFlow version

1.13.1

### Custom code

Yes

### OS platform and distribution

Windows

### Mobile device

---

### Python version

---

### Bazel version

---

### GCC/compiler version

Clang 16

### CUDA/cuDNN version

---

### GPU model and memory

---

### Current behavior?

I've "borrowed" this method in order to use it in own C++ NN project. It is consistently 3x slower than a simple naive nested for loop implementation on moderately sized inputs: eg. convolution with filter(64,64,5,5), stride 1 and no inflation. This implementation takes 10 seconds on 64x800x800 input and 64x800x800 backward output, meanwhile naive loop takes only 3 seconds, so there is a huge difference !
In reference, all the other spatial convolutions (Eigen::SpatialConvolution and Eigen::SpatialConvolutionBackwardInput) easily beat naive implementations by more than 30%. 

### Standalone code to reproduce the issue
Here is a very broad implementation of algorithm that beats your implementation (dimensions, stride and inflation are handled externally), but that is besides the point.
I also know that your implementation has to work with a large range of inputs, but surely there has to be a way to make it faster, at least for most commonly used kernels in image processing. The current contraction method does seem very inefficient.
```shell


using Tensor = Eigen::Tensor<float, 3, Eigen::ColMajor, int>;
using Tenarr = Eigen::Tensor<float, 4, Eigen::ColMajor, int>;
void Conv2D_WGrad(Tenarr& ker, const Tensor& in, const Tensor& out, int pad_w, int pad_h) {
	alignas(64) float tmp[4 * 4096] = {};
	int ker_mem = ker.dimension(0) * sizeof(float);
	for(int i = 0; i < ker.dimension(2); i++) {     
		for(int j = 0; j < ker.dimension(1); j++) {
			memset(tmp, 0, ker_mem);
			int in_i = i - pad_h;
			int in_j = j - pad_w;
			int clip_k = std::max(0, in_i - in.dimension(2) + out.dimension(2));
			int clip_l = std::max(0, in_j - in.dimension(1) + out.dimension(1));
			for(int k = std::max(-in_i, 0); k < out.dimension(2) - clip_k; k++) {
				for(int l = std::max(-in_j, 0); l < out.dimension(1) - clip_l; l++) {
					for(int ich = 0; ich < in.dimension(0); ich++) {
						for(int och = 0; och < out.dimension(0); och++) {
							tmp[och + ich * out.dimension(0)] += in(ich, in_j + l, in_i + k) * out(och, l, k);
						}
					}
				}
			}
			memcpy(&ker(0, 0, j, i), tmp, ker_mem);
		}
	}
}
```
Thank you for your consideration.
```


### Relevant log output

_No response_