Is Characterizing an Algorithm's Behavior worth Publication?
It may be thin, but you won't know until you write it up formally and submit it to some appropriate journal. The real feedback will come from reviewers and/or the editor. 

The question here is just a hypothetical. Make it concrete and ask someone (the editor) who is in a position to make a determination. You will have to make your case there. 



In general, to answer the topline question, it may be publishable or not, but the details matter.