How to hide patch of mud & oil primer under latex paint
Yes, you can and should sand latex if you are painting over it.  You cannot expect the next coat of paint to grip if it doesn't have a roughed surface to grip to.  Painting a glossy or glazed surface is like painting glass - it has no adhesion and will lift right up. 

Fresh latex is hard to sand because it hasn't cured yet.  You have to wait it out.  If it won't sand, it's definitely too soon to overcoat it with anything but itself.  (and on that, check the cans for recoat rules).  You really need to leave decent curing time between dissimilar products - especially oil over latex. 

There is no such thing as paint-primer.  Paint companies tell you that to take your money.  Their marketing claim is based on painting surfaces which are close to ideal - that is to say, well prepared and primed, with no material variations.  

By "material variations" I mean material, color, porosity, chemical differences, anything which might "print through" because the topcoat reacts differently to it.  I'm not referring to physical roughness like brush stipple or high spots. The purpose of primer is to bind to the underlying surfaces no matter what they are, seal them, and make them "equal" to the topcoat, so the topcoat applies uniformly and reflects exactly the physical terrain, without visible glitches for any other reason. 

Done right, you will not see where wood meets marble, an area was previously painted, or where a cat peed on bare drywall. 

You also need to remove surface contaminants which would foul the primer (oil, acids) or prevent adhesion (wax, silicones, linseed oil). In machinery paint, that's the classic 2-cloth wipedown with solvent.  Water is a solvent, but too many contaminants are immune to it, so you need to use a chemical solvent or strong soaps, e.g. the classic TSP.  You need to get the soap residue off. 

Invariably when paint fails, somebody skipped one of these steps. 

Why are the oil brush strokes still visible?  Is this contamination like acid which is fouling each layer? (remove it).  Or is it physical irregularity/bumps (sand it down).  A build primer is specifically meant to be sanded.  It will leave brush stipple marks far more than a topcoat will, but it's easy to sand.