In Diffie-Hellman, are $g$ and $p$ universal constants or are they chosen by one party according to a set of rules? I can't find any clarity on this. In textbooks, what I'm reading suggests there's already a standardized $g$ and $p$ that everyone uses, but I can't find what they are.

The closest I've come to is this RFC link.
$g$ and $p$ can be viewed as universal constants. These values are public values so the security isn't affected if an attacker knows $p$ and $g$. They also have to be the same values on both sides for a key-exchange to work.

As explained in Wikipedia (section Security):


  [$\ldots$] For this reason, a Sophie Germain prime $q$ is sometimes used to calculate $p = 2q + 1$, called a safe prime, since the order of $G$ is then only divisible by $2$ and $q$. $g$ is then sometimes chosen to generate the order $q$ subgroup of $G$, rather than $G$, so that the Legendre symbol of $g^a$ never reveals the low order bit of $a$. A protocol using such a choice is for example IKEv2.


So these are infact default values, for example the Java documentation (might be outdated for 2019) lists the following values for 512-bit keys:

p = fca682ce 8e12caba 26efccf7 110e526d b078b05e decbcd1e b4a208f3
    ae1617ae 01f35b91 a47e6df6 3413c5e1 2ed0899b cd132acd 50d99151
    bdc43ee7 37592e17
g = 678471b2 7a9cf44e e91a49c5 147db1a9 aaf244f0 5a434d64 86931d2d
    14271b9e 35030b71 fd73da17 9069b32e 2935630e 1c206235 4d0da20a
    6c416e50 be794ca4


Normally such default values are used, but it is also possible to choose new $p$ and $g$ values if you want to.