Can a window A/C unit be installed without the side panels?
In my opinion, no they are not. they are conveniences added to the unit to fill the space of various window sizes. 

Their main thing is to stop the transfer of air into the room

If you remove them, be sure that the openings (louvers) on the sides of the AC unit are not blocked by the thickness of the wall since the fillers used for a wider window opening would maintain some space automatically.