Which LEGO set did this bags 4 - 10 with castle pieces come from?
Bags 5 till 10 are from 
70404-1: King's Castle




based on Ladder and Staircase 7X4X6 in Reddish Brown. 

It also contain the Plate 4X4 in green, and the door and wall pieces.

Bag 4 is from 9526-1: Palpatine's Arrest



Based on Corner Plate 6X6X45° in Medium Stone Grey and Brick 1X2 M. 2 Holes Ø 4,87 in Bright Yellow.

It also contain the red 2x6 and grey  1x8 technic pieces