Is it unheard of to pursue writing a relevant master's thesis while enrolled in a capstone-project based masters program?
What's possible is based on what a professor (who has enough authority) will approve. So if you can get the capstone director on board, it is probably possible.

The program may also hesitate to let you NOT do the project that tests and shows off your practical, industry-specific skills. This sounds like a core part of their degree program, including (potentially) the soft and practical skills.

What else should you consider?

For this project to be as good as it can be and valuable academically, you need to identify who has the skill to evaluate your work. To what extent are the faculty of your program industry-based versus academic? For this to help you toward your goal, you would want to work with a professor who has a Ph.D. and actively publishes peer-reviewed work. 

If you stick with the project-based capstone, here are some additional alternatives toward your goal:


Take a specialized class that involves a writing research paper with an academically-oriented professor. It may be easier to wrangle in an extra class into the program than to substitute the major requirement, and it's a smaller ask of that professor, if they do not know you well yet.
Try to find a job doing research work with a business school professor. (This might be while you're studying, during a summer term, or after you graduate.) The practical analytical skills you're learning will probably help you wrangle data for a professor, who in turn can show you what they look for in analysis as an academic, rather than as an industry analyst.


Business school professors are probably going to be more open to this than most other professors, especially if the university has few or no doctoral students in business. At the same time, they may be unused to professional students approaching them about research, and they may be wary or protective of their time.