A simple number system
Haskell, 357 306 277 251 228 224 188 185 180 bytes

A token-based parser with an explicit stack. (%) takes a token stack and a character and either pushes (opcode,defaultnumber) or (0,number) for ({[&lt;, or pops topmost numbers and one opcode and pushes the answer for )}]&gt;. Opcodes are encoded by a ascii enumeration hack.

Kudos to @ChristianSievers for his great answer that I borrowed some ideas from.



One-liner!

s%c|elem c"([&lt;{",g&lt;-div(fromEnum c)25=(g,[0,0,1,-1,1]!!g):s|(a,(o,n):b)&lt;-span((==0).fst)s=(0,[foldr1(flip$[(+),quot,(-),(*)]!!(o-1))$snd&lt;$&gt;a,n]!!(0^length a)):b
snd.head.foldl(%)[]


Now with less error-handling!
Usage:

*Main&gt; map (snd.head.foldl(%)[]) ["()","(()())","([][])","({}&lt;&gt;)","({}[])","[]","[[][]]","[()&lt;&gt;]","{()}","{([]&lt;&gt;)}"]
[0,0,-2,2,0,-1,0,-1,0,0]


Thanks @ChristianSievers for saving 14+3 bytes!

Thanks @Zgarb for saving some+4 bytes!