Speed up position check for embedded length on stream
I agree with the comments that it's not likely to be your position checking code that's slowing you down.  Maybe you could squeeze some performance out with unsafe code, but that's maybe saving a copy from one buffer to another.  It would be nice to put that reading loop into a separate function, something like:

byte[] ReadBytes(SomeType tt1, int bytesToRead);


I can't guess what type tt1 is, but is it possible that retrieving the value for Connected is slow?  What about GetStream?