connecting three 8 gauge wires, wire nut or multi-port?
If you mean these, they are not -- oh wait, if you dig into the data sheet, they are rated for three #8. Things the literature doesn't tell you that the data sheet does. 

Now, you are pushing the limit of the wire nut's range, so it may be difficult to work with.  They work better in the middle of their range.   It also sounds like you are being timid with the wire nut, and you really need to show it who's boss.  

You're worried about the wires twisting, they're supposed to twist. If even the insulated parts of the wires start to coil around each other, that's fine, that's an indicator that that you're tightening it enough. 

Also, do not pre-twist the wires that will be going into the wire nut.  That is just dumb, the wire nuts themselves are about to do a fine job of that.  Make sure the wires are square/even going in, pretwisting makes that harder.  Also you have a chance of twisting in the wrong direction. 

Never tape wire nuts, except for a single wire on a nut (otherwise, those don't hold). Instead, firmly pull on each wire.  They shouldn't even think about coming out of the nut.  If they do, that indicates a bad job of wire nutting.  They are also making bad contact, and will cause arcing and fire.  Unwind it, straighten them out a bit, line them up properly, and do it again.