CiviCRM API and checking if a custom field is empty rather than not null
The answer depends on how you call the API: is this from php or via the ajax or rest interface?

in php, both null and empty string are seen as a boolean false, so

if (!$custom_42)
   echo "it's either null or empty string" 


As for why custom fields behave this way: each custom field belong to a custom group that is stored in a table. if you don't have any of the custom fields in that group, the row isn't created, so all the fields are null, but as soon as you set one of the fields, it creates a row, and the default value of each of the other empty fields depends of their type.

That custom fields that are string become "" instead of null is arguable and it would seem more coherent they stay the same, but as both are seen as false in php (and other languages have the same conversion rules) probably not a big issue.