How can positional encodings including a sine operation be linearly transformable for any offset?
I elected to ask this question on the Mathematics Stack Exchange and I thought it prudent to add the answer here:

https://math.stackexchange.com/q/3119882

From what I have learned from @Servaes, who was kind enough to answer the question, there is a function 

$\operatorname{PE}(\text{pos}+k,2d)=\operatorname{PE}(\text{pos},2d)\cos(k/c^d)+\operatorname{PE}(\text{pos},2d+1)\sin(k/c^d)$

that allows for the property of transforming any positional encoding into one with a given offset. However, due to the required use of the sin() and cos() functions this is not a linear transformation.