How does Bittrex charge its commission? Assume I want to trade the BTC-BCH pair. I have only one BTC in my exchange wallet and want to buy one BTC worth of BCH.

What currency will the 0.25% commission come out of?
Will I need to trade 1 BTC - 0.25% so there is enough funds in my account to cover the commission?

The same question goes for selling BCH to receive BTC Where I have only 100 BTC in my exchange wallet and want to convert it all to BTC. Will I need to ensure that I only convert 100 BCH - 0.25%?
I can't be "Bittrex specific", but i can provide a general answer to how it would work and have seen it work across a half dozen systems that are crypto specific and dozens of financial systems that are non crypto specific:

When you set up a sell order (or just click sell) it's usually not a "1 and done" situation. You are selling 'x' amount for at least 'y' value. As the system matches up the purchases of your 'x' amount to buy orders, a fee is applied when the match is made and the "trade" occurs. The fee can be taken out of BOTH sides of the trade. You have a .25%.. the buyer may have a .25%. That means if the trade completes, whatever you are "due" they will remove it before handing it over. THEY get BCH. The person buying it get's a fee too.. they paid in BTC. Once the trade completes, they pay their fee in BTC. 

Sorry this isn't specifically a Bittrex answer, it's a "how these things normally work". I can't imagine how else it could be accomplished as it would mess up all normal accounting practices.

In sum, to maintain accounting: The trade is completed. Fee's are assessed against the results.