how can I use $HOME, ~ or environment variable in plist file of LaunchDaemons
Plists, or property lists, are just a file format for storing serialized objects - for example configurations or similar. The file format specifies how strings are stores, but it doesn't specify anything about higher level details - such as how file names are stored, parsed or otherwise dealt with.

This means that it is not possible to answer your question in general for plists.

The interpretation of $HOME, ~ and environment variables in general in plists are thus application specific. For some programs, it could work - for others it wouldn't. Depends entirely on how that program was made.

UPDATE: You have changed your question to be about LaunchDaemons specifically, and not about plists in general.

For /Library/LaunchDaemons/startup.plist it is not possible to refer directly to the environment variables you mention. They're not parsed and replaced with values by launchd. In addition it probably wouldn't have worked like you thought even if they were, because LaunchDaemons are run by root if nothing else is specified.

What you can do is refer a program in the startup.plist that when run can access the environment and do the variable substitution for you. For example like this:

&lt;key&gt;ProgramArguments&lt;/key&gt;
&lt;array&gt;
    &lt;string&gt;/bin/sh&lt;/string&gt;
    &lt;string&gt;-c&lt;/string&gt;
    &lt;string&gt;exec $HOME/startup.sh&lt;/string&gt;
&lt;/array&gt;