Designing a Variable Set
The only way to do what you are trying to do, and remove the duplication, is really to cast.

What I would do is to use this kind of map:

private final Map&lt;Class&lt;? extends Variable&gt;, Map&lt;String, Variable&gt;&gt; variables;


To add something to the map, you could do:

variables.get(NumberVariable.class).put("key", value);


And to get:

return (NumberVariable) variables.get(NumberVariable.class).get("key");


You could even write a generic method:

private &lt;T&gt; T getVariable(Class&lt;T&gt; clazz, String key) {
    return clazz.cast(variables.get(clazz).get(key));
}


Note that this shouldn't even give any compiler warnings.



Leaking inner variable

Considering your getVariableNames method, imagine if I would call your code like this:

myVariableSet.getVariableNames.clear();


Whoops! I broke it!

The simple fix is this:

public Set&lt;String&gt; getVariableNames() {
    return new HashSet&lt;&gt;(usedVariableNames);
}


That is, return a copy of the data, then I can manipulate it all I want - I won't break anything!

I wouldn't use a usedVariableNames at all though and just do a map.containsKey(key) directly. If you want to make sure that all variable names are unique though, even if they are of different classes, then go ahead and keep this variable.