What's the App Store refund policy?
Well, in my point of view, if the app works 75% of the time playing AVI files, I would try to contact the developer directly, there might be something required (like a plug-in) or a bug that needs fixing. Like it says in the FAQ, if contacting the developer produces no result, then I guess you could ask for a refund.

I have heard that they gave refunds to people who were unhappy before, but it was a last-resort solution if I remember correctly. Also, I presume the App store staff will fight for you more if you paid 100$ for your app, and a little less if you only paid a buck... :-)