How to calculate the order of the subgroup? Given a curve with points over GF(p), a subgroup of prime order q and a co-factor h. 

How do I calculate the size of q which is also the modulus?

I was thinking q = p/h
Run Schoof's algorithm on the curve parameters to find $qh$, and divide by $h$.

The size $p$ of the coordinate field is only required, by Hasse's theorem, to be near $qh$, within a factor of a square root: $|qh - (p + 1)| \leq 2 \sqrt p$.  Consequently, $p/h$ may be near $q$ but is not equal to $q$ except in anomalous curves in which ECDLP is easily solved by additive transfers as described by Smart (preprint), Araki–Satoh, and Semaev (the ‘Smart-ASS’ attack).

For example, Curve25519's coordinate field is $\operatorname{GF}(2^{255} - 19)$, and its order $2^{255} + 221938542218978828286815502327069187944 = qh$ where $q = 2^{252} + 27742317777372353535851937790883648493$ and $h = 8$.