Slow mac when Terminal have been open for some time
You're going to have to broaden your search. There's nothing in your screenshots that points to Terminal. Nor anything in my experience—I let Terminal run for days with no ill effect.

It's possible that you're starting some task that's running away with the processor. You could set Activity Monitor to show all processes hierarchically, and give the tasks running directly under Terminal closer scrutiny, but I'd be more inclined to think that whatever it is is something you've spawned off as a separate process (using &amp; at the end of a command).

Does quitting Terminal give you your performance back right away? That would indicate that the process is still running as a subtask of Terminal.

Either way, set Activity Monitor to show All Processes, and sort the list by % CPU. See what floats to the top.

Is there anything unusual about your $PROMPT_COMMAND? This is a command that gets run by the shell just before it displays the prompt. Seems to me to be a place where a time-waster could hide.

ADDENDUM: I just noticed your comment "...when I have an ssh session open to one of my servers". Do you also accept incoming ssh sessions? It's possible that you're under attack. Someone is trying to open an ssh session back to you, and is guessing passwords. If they're guessing very fast, they can bring your machine to its knees.

I thought of that earlier, because I've seen that happen when I enable Remote Login and have a publicly visible IP address. Once the bad guys notice that your port 22 is open from the WAN (and they will, within hours), they'll slam it with login attempts. The (temporary) fix is to disable Remote Login for a few minutes. They'll get bored and turn their attention elsewhere, and you have a respite of a few hours until another bad guy notices the open port.

But you say the problem goes away as soon as you quit Terminal, which sounds like a different problem. UNLESS you have some watchdog on your system that automatically opens port 22 (i.e., enables Remote Login) only while Terminal is running, and automatically closes it when Terminal quits. Or I imagine it could be possible to configure a firewall so that it allows incoming connections to port 22 only while there is currently an outgoing connection to port 22. Or only from remote addresses that you have an outgoing connection to, and the server you're connecting to is infected. Quitting Terminal closes all your ssh sessions, which would make the firewall stop accepting ssh incoming sessions, breaking the attack. Just a guess.

To see if you're under this kind of attack, open Activity Monitor and filter the process list for "sshd". If you see lots of very short-lived sshd processes, someone is trying to guess your password. Even if you don't allow password access, they don't know that and will try anyway.