Is there a way to tone down the flavor of celery in an Oyster Mushroom Chowder?
I'm going to step out on a limb here and declare that you'll need to add more of the other ingredients.  Even then, you're likely to throw off the overall balance of flavors because they have not all cooked together.

Here's a study abstract that suggests there are about 6 compounds that primarily contribute to the flavor and aroma of celery.  However, as a home cook, I have no idea what any of these are or how to suppress them specifically.  It may be possible to do so, but that solution is likely to be impractical for the home chef.

Another problem is that celery has a very flavor-enhancing effect; this is why it shows up so often in traditional vegetable bases like mirepoix and trinity.  Here's another nifty article summarizing a paper that identified the specific compounds responsible for this; the odd thing is that these "phthalides" are largely tasteless.  In short: the balance and overall tastiness of your soup might have been reached specifically by adding so much celery.

I think you're going to have to chalk this one up as a learning experience and adjust your recipe next time.  One thing you could try (if you're not already, which would surprise me a little) is sweating the celery.  I find that this makes the flavor overall a bit milder, with less of the vegetal-bitter flavors while still getting the umami-enhancing effect.