Scanning an NSArray to produce an NSSet
There is no need to return a copy of the variablesUsed set. It's actually wrong to do so, because naming conventions for your method imply an autoreleased object to be returned. Thus I would rewrite your return as:

return [variablesUsed autorelease];
// or in case of ARC just
return variablesUsed;


See the Advanced memory management programming guide

Also set contains a useful method to filter using a block. In my opinion this results in slightly nicer code. You can convert your incoming array to a NSSet using + setWithArray:. 

The if statement to return nil can be combined in your return.

Combining this results in the following implementation

+ (NSSet *) variablesUsedInProgram:(id)program {
    NSSet *programSet = [NSSet setWithArray:program];
    NSSet *variablesUsed = [NSSet objectsPassingTest:^(id object, BOOL *stop) {
        return !([object isKindOfClass:[NSNumber class]] || [self isOperation:term])
    }];
    return ([variablesUsed count] &gt; 0) ? variablesUsed : nil;
}