Keep the bevel modifier from creating ngons and tris
I don't think there is any automatic way accompanied with Bevel modifier to operate like that. Some other manual solutions:


If you just want to triangulate, there is a Triangulate modifier for you to use;
You can also select all faces then CtrlT to convert them into Triangles;
If you want to keep quads as many as possible, select all ngon faces only, by Select >  Select Face by Sides, set it greater than 4 (As shown below), then CtrlT to convert all selected faces into triangles, then AltJ to convert them into quads by tweaking Max Angle in F6 panel.