Clique-or-almost reduction to clique
Here is a reduction from CLIQUE to CLIQUE-or-almost.

Let $G$ be an arbitrary graph. Form a new graph $G'$ by adding two new vertices $x,y$ connected to all vertices of $G$, but not to each other. If $G$ contains a $k$-clique, then $G'$ contains a $(k+2)$-clique with a missing edge.

Conversely, suppose that $G'$ contains a $(k+2)$-clique $S$, possibly with a missing edge. If the clique doesn't contain a missing edge, then $S \setminus \{x,y\}$ is a clique of size at least $k$ in $G$. If $x,y \in S$ then the missing edge is $(x,y)$, and again $S \setminus \{x,y\}$ is a $k$-clique of $G$. In the remaining case, $|S \setminus \{x,y\}| \geq k+1$. Remove one vertex from the missing edge to obtain a clique of size at least $k$ in $G$.



For a reduction in the other direction, it is well-known that CLIQUE is NP-hard. Since CLIQUE-or-almost is clearly in NP, by definition there is a reduction from CLIQUE-or-almost to CLIQUE. One can construct such a reduction via the proof of the Cook-Levin theorem.