Using Image Transform as another custom field
The imageTransform value is actually an object, representing the entire dropdown field.

Update the first line to this...

{% set imageTransform = storyEntry.storyThumbnailTransform.value %}


This will ensure that the selected value is passed on, instead of the full object.

(The reason that you see the proper value when you output it directly is because the object's __toString method is automatically being called internally.)