Is this a good approach to classify tickets which contains description and logs?
First, you should get clear about your non-functional requirements (speed, memory consumption). Make sure you have a clear idea who is going to maintain the software and what the effort will be if you make some changes. Higher accuracy might not be worth it.

The simplest way to combine n models is averaging their predictions. This technique is often used on Kaggle. See chapter 2.6 of https://arxiv.org/pdf/1707.09725

Did you try a simple MLP with tf-idf features? How well does that perform? Usually it executes way faster than SVMs, if you have many classes.