Difference between $Q(s,a)$ ,$V^*(s)$ and $V^\pi(s)$ in Markov Decision Process?
Your confusion seems to come from mixing up between some policy $\pi$ and an optimal policy $\pi^*$. Your summary is generally correct, but missing these extra details.
Let me try go through it again. Starting with the MDP definitions:

First of all, we have the transition probabilities $T(s,a,s') = P(s'|s,a)$ which are conditional probabilities of arriving at state $s'$, given that we've taken action $a$ in state $s$
The (expected) reward is generally associated with the state-action pair $R(s,a)$ - there is some slight variations about it in literature, but that's a subject for a different discussion.
Then we have the policy $\pi(a,s) = P(a|s)$ - the probability of taking an action $a$ in state $s$ for an agent following the policy.

Given all three things above: $T$, $R$ and $\pi$ (plus a discount factor $\gamma$) you can define your value functions as average discounted rewards collected by the agent that follows $\pi$.

$V^\pi(s)$ is the average reward that the agent following $\pi$ will collect, when starting from the state $s$.
$Q^\pi(s,a)$ is the average reward that the agent following $\pi$ will collect, when starting from the state $s$ and taking an action $a$.

Quite often authors drop the index $\pi$ and assume implicitly that we are dealing with some policy $\pi$, but one should keep in mind that some (maybe unspecified) policy is always in the context when we are talking about value functions $V$ or $Q$.
These value functions satisfy the following recursive relationships for any policy $\pi$ (note that there's no $\max_a$ in these):
$$V^\pi(s) = \sum_a\pi(a,s)\left[R(s,a) + \gamma \sum_{s'}T(s',s,a)V^\pi(s')\right]$$
$$Q^\pi(s,a) = R(s,a) + \gamma \sum_{s',a'}T(s',s,a)\pi(a'|s')Q^\pi(s',a')$$
Now, some policies maximize the expected reward - these policies are called &quot;optimal&quot; and standardly denoted with a star: $\pi^*(a,s)$ - optimal policy. The  index $\pi$ for the value functions is usually replaced with a star as well: $V^*$ and $Q^*$. The optimal value functions satisfy Bellman equations (the ones that have $\max_a$ in them):
$$V^*(s) = \max_a\left[R(s,a) + \gamma \sum_{s'}T(s',s,a)V^*(s')\right]$$
$$Q^*(s,a) = R(s,a) + \gamma \max_{a'}\sum_{s'}T(s',s,a)Q^*(s',a')$$
Hope this clarifies it.