Acidity order of nitrophenols
You have correctly identified that m-nitrophenol should be the weakest acid among the isomers as the resulting anion cannot be stabilised by the $\text{-M}$ effect of nitro group owing to its meta location.
The acidity order of the remaining two nitrophenols can be explained in this way:

Hydrogen bonding is possible in both of them. But note that:

$$\text{H-bonding} \propto \delta^+ \text{charge on $\ce{H}$ atom}$$
Inductive effect (here, $\text{-I}$) is distance dependent. So $\delta^+$ charge on the $\ce{H}$ of ortho-nitrophenol's -$\ce{OH}$ is going to be more than the $\delta^+$ charge the $\ce{H}$ atom of para-nitrophenol's -$\ce{OH}$.

Also,  quoting from this comment:


Usually the $\mathrm{p}K_\mathrm{a}$ of a substance is determined in dilute solution. So just due to dilution, there won't be many/any inter-molecular H bonds, but there can still be plenty of intra-molecular ones.


The answer above the comment relates the strength of these two types of hydrogen bonds to kinetic motion of molecules:


Inter-molecular interaction, such as inter-molecular hydrogen bonding only take place for a short time when two molecules are in each other's vicinity. Shortly after, the molecules separate and the interaction does no longer exist.
Intra-molecular interactions on the other hand are much stronger since the two atoms forming the hydrogen bond remain in each others proximity due to structure of the molecule. In other words, they exhibit strain.

These three factors help us conclude that:

Strength of intra-molecular hydrogen bonding in o-nitrophenol $\gt$ Strength of intermolecular hydrogen bonding in p-nitrophenol

Thus, p-nitrobenzoic acid is more acidic than the o-isomer.

...and since phenol has no ring activating group, it comes out to be least acidic in the given compounds.

This reasoning is incorrect. Phenol has no group that stabilises the negative charge on the resulting phenoxide ion through $\text{-M}$ or $\text{-I}$ effect (a deactivating group). Therefore, it's the weakest acid of all under investigation.