Hello world with a twist
Perl - 96
(Pretty good, given the theoretical max 97 score)
s{}[Hel0o, w3$=d!];y&lt;&quot;#%&amp;'*.124578:?@BCDEFGIJKLMNOQ/-9&gt;(
    6PRSTUVWXYZ^_`abfghjmqvxz|~\cAk-u)+print

(Note that second line starts with an actual \t tab character)
The code is 98 characters long and contains every ascii char exactly once, plus an extra -.
98 - 2 = 96
There are no string literals here, but removal of any single character breaks the program.
Deobfuscation
There are 3 statements in this program (well, actually 2 but I abused + as some kind of statement separator)

s{}[Hel0o, w3$=d!]
This is a very stretched case of the sed operator. It is more commonly written as s/a/b/, or s:a:b:, but perl allows much more fantasy here: s(a)(b) s(a)^b^ s(a)[b], and even s qaqbq.
It replaces an empty string (inside {}) with Hel0o w360d! (inside []) ($= interpolates to 60 by default). Because of the lack of an ~= operator, it operates on $_.

y&lt;&quot;#%&amp;'*.124578:?@BCDEFGIJKLMNOQ/-9&gt;(\n\t6PRSTUVWXYZ^_`abfghjmqvxz|~\cAk-u)
This is also a very stretched case, but of a tr operator, which is also called y (y&lt; &gt;( )).
Let's look at the replacement table:
&quot; # %&amp;'*.124578:?@BCDEFGIJKLMNOQ    /-9
\n\t6PRSTUVWXYZ^_`abfghjmqvxz|~\cA  k-u
I have moved some characters around so that existing string is not broken. The only actually working part here is /-9 -&gt; k-u. It replaces 0 with l, 3 with o, and 6 with r.
Again because of lack of the ~= operator, it operates on $_
So now we have the complete phrase in the $_ variable.

print
Lack of arguments makes it print just the $_ variable.