sure if you could do that or if you would like that. make the inside of the purse with light weight but sturdy materials (with magnets embedded) and the handle you love. then create the book cover portion with fabric (with magnets embedded). then you could switch out book covers and only have 1 or 2 different inner shells.
the only downside i can see is that it might be heavier than doing just the book purse alone and books come in all different shapes and sizes (this could be fixed by attaching fabric around the perimeter of the book or shaving the book cover down).
maybe you could put a pleat 