What operation does $\|$ denote in ANSI X9.63 KDF? Can someone tell me in laymens terms what this is?

$$K_i = \mathrm{Hash}(Z \| \mathit{Counter} \| [\mathit{SharedInfo}])$$

What do the double pipes represent?
It means concatenation. Z, Counter, and SharedInfo are three bitstrings which are to be concatenated before hashing. The [ ] around SharedInfo means it may be absent in which case you would use an empty string instead. (Since concatenating an empty string to the end yields the same result as not concatenating anything.)