Poor adhesion only for first couple of inches
I'd recommend using the "skirt" function if you're not already.  

The idea is to print a few perimeter layers around where your part will be, but not actually touching your part.  

Most slicers support this and you can choose how much skirt you want to print.  This addresses the issue you mention, and it purges old filament that has spent too much time in the nozzle.  As an added bonus, it gives you a good indication that your print location, print height, and first layer adhesion are all good.