Is it normal for seat stays on new CAAD12 frame to be slightly warped/not straight?
You are concerned that something is wrong with the frame.  You can check the alignment by getting the Park FAG-2 alignment tool or, and it comes to the same thing, measuring to see if the distance to the seat tube is the same on both sides using a string - a string test.  If you feel uncomfortable running a string from the rear dropouts around the head tube and measuring the distance from the string to the seat tube on both sides, then get the Park tool.  Otherwise, run a string and measure.   If the measurements are identical, then the seat stay "warping" is purely cosmetic - the frame is properly aligned.  If the frame is off, then you are warned not to try cold setting thin aluminum tubes.  I have to admit here that, if I got a really good deal on the frame and it was only off a little, I would be tempted to correct it.  See the Park FAG-2 link for other ways of dealing with it in case you find that the frame is off a little.