CNN - How does backpropagation with weight-sharing work exactly?
I think you're misunderstanding what "weight sharing" means here. A convolutional layer is generally comprised of many "filters", which are usually 2x2 or 3x3. These filters are applied in a "sliding window" across the entire layer's input. The "weight sharing" is using fixed weights for this filter across the entire input. It does not mean that all of the filters are equivalent. 

To be concrete, let's imagine a 2x2 filter $F$ striding a 3x3 input $X$ with padding, so the filter gets applied 4 times. Let's denote the unrolled filter $\beta$.

$$X = \begin{bmatrix}
    x_{11}       &amp; x_{12} &amp; x_{13} \\
    x_{21}       &amp; x_{22} &amp; x_{23} \\
    x_{31}       &amp; x_{32} &amp; x_{33} 
\end{bmatrix}$$

$$F = \begin{bmatrix}
    w_{11}       &amp; w_{12} \\
    w_{21}       &amp; w_{22} 
\end{bmatrix}$$

$$\beta= [w_{11}, w_{12}, w_{21}, w_{22} ]$$

$$F*X = \begin{bmatrix}
\beta \cdot [x_{11}, x_{12}, x_{21}, x_{22}] &amp; \beta \cdot [x_{12}, x_{13}, x_{22}, x_{23}] \\
\beta \cdot [x_{21}, x_{22}, x_{31}, x_{32}] &amp; \beta \cdot [x_{22}, x_{23}, x_{32}, x_{33}]
\end{bmatrix}
$$

"Weight sharing" means when we apply this 2x2 filter to our 3x3 input, we reuse the same four weights given by the filter across the entire input. The alternative would be each filter application having its own set of inputs (which would really be a separate filter for each region of the image), giving a total of 16 weights, or a dense layer with 4 nodes giving 36 weights. 

Sharing weights in this way significantly reduces the number of weights we have to learn, making it easier to learn very deep architectures, and additionally allows us to learn features that are agnostic to what region of the input is being considered.

EDIT: To further motivate this, here's an animation of a 3x3 filter applied to a 5x5 input