How do i get a transistor to work efficiently?
If, as you say, you are using an op-amp to drive a SMPS MOSFET, that is likely a major problem. You need to drive the gate smartly (hard) in order to have the transistor switch quickly. If it does not switch quickly, it will spend too much time switching, and the power dissipation can be enormous (switching losses are proportional to the switching time, making some assumptions). 

A large MOSFET will also have a lot of gate charge that needs to be stuffed in or sucked out of the gate, so the current required is large. A good MOSFET gate driver might have a current capability in the amperes vs. only tens of mA for a typical op-amp. An op-amp will also slew too slowly in general, even without the gate load. 

Here is a datasheet for a random gate driver (This one is from Microchip). The output current capability is 9 amperes, and it will switch in 135ns with a 47nF load. 

A cheaper driver can be made by complementary emitter-follower BJTs as shown below. It may or may not be adequate, depending on your switching frequency.







simulate this circuit &ndash; Schematic created using CircuitLab

Suggest you look at the gate voltage waveforms in your simulation or prototype and improve the driver until you see MOSFET switching losses that are at least in the same range as the theoretical conduction losses: \${I_{RMS}}^2 \times R_{ds(on)}\$, or \${I_{on}}^2 \times R_{ds(on)} \times \dfrac{t_{on}}{t_{off} + t_{on}}\$ if the on current \$I_{on}\$ is fairly constant. 

Note that building an actual circuit in this power range or higher is non-trivial, and good layout practices have to be followed. International Rectifier has some excellent application notes. 

P.S. Here is an excellent application note that goes into the gory details of MOSFET gate charge.