Can I use Secret Invasion to transfer a trap card in front of me to another player?
No, you can't.

The trap is being kept in front of you as a reminder.  The effect has already been applied.

Arguably you could say that the reminder card moves, the the minus effect won't move with it.

Notice that the trap has no rules about discarding itself.  As currently worded, if it was applying the effect it would stay in front of you forever, giving you -15 until you found a way to discard it with a different card.  This doesn't happen because the card isn't officially in play - it's just a memory tool that you can discard freely once it's done with its purpose.