'(episode % )def makeq(modelnp array-callable[[np array]np array]"""returns -functionwhich takes state -distribution over actions""return lambda xx dot(modeldef initialize(shapetuple)"""initialize model"" np random normal( shapeq makeq(wreturn wq def train(xnp arrayynp arraywnp array-tuple[np arraycallable]"""train the modelusing solution to ridge regression"" np eye( shape[ ]neww np linalg inv( dot( - idot( dot( ) w_lr neww ( w_lrw makeq(wreturn wq