Can a self-signed certificate use an IP address for an entry in Subject Alternative Name?
Yes, according to the standard (RFC 5280 page 37) you can use an IP address in the SAN (SubjectAltName) field!
The standard defines a number of possibilities for specifying the SAN, including iPAddress, dNSName, and various others.
The most obvious option is to use iPAddress, where the IP address is stored as an octet string. You can also write the IP address in the dNSName field, which is a common string (IA5String to be precise).
In practice, you want to do both at the same time as software support varies. For example it seems that especially Microsoft software (like the Internet Explorer and Edge browsers) look at the dNSName and interprets IP addresses there, whereas browsers like Safari and Chrome look at the iPAddress field.